STANDARD A.1 Content and Pedagogical Knowledge Tatiana Rivadeneyra, - - PowerPoint PPT Presentation

Washington, District of Columbia September 2017

STANDARD A.1

Content and Pedagogical Knowledge

Tatiana Rivadeneyra, Ed.D.

Accreditation Director, Site Visitor Development and EPP Accreditation Procedures Tatiana.Rivadeneyra@caepnet.org

Banhi Bhattacharya, Ph.D.

Accreditation Director Senior Director of Program Review Banhi.Bhattacharya@caepnet.org

Fall 2017 | Washington, D.C.

• This session will focus on the key language and intent of CAEP Standard A.1.
• Content will reference the evidence sufficiency criteria.
• The CAEP Standards for Initial-Level Programs are not covered in this

presentation.

• Please attend the session dedicated to those standards or access the presentation

materials for guidance.

Session Overview

Fall 2017 | Washington, D.C.

EVIDENCE SUFFICIENCY: RESOURCES

CONSULT:

• Evidence Sufficiency Criteria
• Evaluation Criteria for Self-Study Evidence - Standard A.1
• CAEP Guidelines for Plans for phase-in plan content
• SSR submitted through academic year 2018/2019 can include plans for Component A.1.1
• 2019-2020 SSRs can present plan with progress data for Component A.1.1
• Site visits in F22 and beyond are not eligible for phase-in
• Assessment Sufficiency Criteria
• CAEP Evaluation Framework for EPP-Created Assessments

Fall 2017 | Washington, D.C.

Evidence Sufficiency Rules for Standard A.1

General for all Standards

• Key concepts in standard and

components are addressed

• EPP-created assessments meet CAEP’s

assessment sufficiency criteria

• At least three cycles of data that are

sequential and most recent available

• Results disaggregated by specialty field

area (when appropriate)

• Also for main and additional campuses,
• n site and online programs

(if applicable)

• Data/evidence analysis includes

discussion of trends/patterns, comparisons, and/or differences.

Special for Standard 1

• No required components
• All data disaggregated by specialty

licensure area

• Evidence from Standard 1cited in

support of continuous improvement, part of overall review system

• The majority of programs meet the

standards of the selected program review option(s)

Fall 2017 | Washington, D.C.

• The provider ensures that candidates for professional specialties develop a

deep understanding of the critical concepts and principles of their field of preparation [component A.1.1] and, by completion, can use professional specialty practices flexibly to advance the learning of all P-12 students toward attainment of college- and career-readiness standards [component A.1.2].

STANDARD A.1: CONTENT AND PEDAGOGICAL KNOWLEDGE

Standard A.1, Guidance from Component A.1.1

Candidates for advanced preparation demonstrate their proficiencies to understand and apply knowledge and skills appropriate to their professional field of specialization so that learning and development opportunities for P-12 are enhanced, through:

• Application of data literacy;
• Use of research and understanding of qualitative, quantitative and/or mixed methods research

methodologies;

• Use of data analysis and evidence to develop supportive school environments;
• Leading and/or participating in collaborative activities with others such as peers, colleagues, teachers,

administrators, community organizations, and parents;

• Application of appropriate technology for their field of specialization; and
• Application of professional dispositions, laws and policies, codes of ethics and professional standards

appropriate to their field of specialization.

• Evidence of candidate content knowledge appropriate for the professional specialty will be documented

by state licensure test scores or other proficiency measures.

EVIDENCE FOR A.1.1

• Consider: What evidence do you have that would demonstrate

proficiencies in the specialty content and general skills referenced in Component A.1.1 for a specialization?

Fall 2017 | Washington, D.C.

EVIDENCE SUFFICIENCY CRITERIA, A.1.1

SUFFICIENT EVIDENCE

• Demonstrates that most candidates pass state/nationally-benchmarked

content/licensure exams

• Addresses all of the professional skills listed in the component
• Documents proficiency for at least three of the skills for each specialty field
• Utilizes multiple measures to assess each proficiency
• Utilizes measures that meet criteria in CAEP Evaluation Framework for EPP-Created

Assessments

• Phase-In Plans for Component A.1.1 meet the criteria for the CAEP Guidelines for Plans

and are consistent with the Phase-In Schedule.

Fall 2017 | Washington, D.C.

EPP Created- Assessments

Standard A.1, component A 1.1

• Action Research
• Capstones/Portfolios/Thesis
• Dispositional/Professional Responsibility Data
• Problem-based projects with coursework/group

projects

• Problem-based projects with school/district
• Pre- and post-data and reflections on

interpretations and use of data

• End of key-course tests
• Grades, by program field
• Survey Data from Completers/Employers

Advanced Standards

(suggested evidence) Resource: CAEP Evaluation Framework for EPP-Created Assessments

Fall 2017 | Washington, D.C.

Advanced Level Standards

STATE

• Relevant surveys
• Assessments (value added) of completers
• Licensure examinations
• Additional measures utilized toward compliance of
• ther accreditors (e.g., for reporting requirements;

WASC, NASC, HLC (aka NCA), SACS, MSA, NEASC) OTHER

• State Assessments/Surveys
• Other Proficiency Measures

Assessment Test or Section 3.2 Domain—NOTE: proficiency must be met for each domain Group average performance requirements of candidates whose preparation began during the 2016-2017 academic year or earlier GRE “Verbal Reasoning” Reading 150.75** GRE “Quantitative Reasoning” Math 152.75** GRE “Analytical Writing” Writing 3.74**

Standard A.1, Guidance from Component A.1.2

Providers ensure that advanced program completers have opportunities to learn and apply specialized content and discipline knowledge contained in approved state and/or national discipline-specific standards. These specialized standards include, but are not limited to, Specialized Professional Association (SPA) standards, individual state standards, standards of the National Board for Professional Teaching Standards, and standards of other accrediting bodies [e.g., Council for Accreditation of Counseling and Related Educational Programs (CACREP)].

Consider: What evidence do you have that would demonstrate that the program provides candidates the opportunity to both learn and apply content knowledge and skills that are emphasized in professional standards for the specialty area?

Fall 2017 | Washington, D.C.

EVIDENCE SUFFICIENCY CRITERIA, A.1.2

SUFFICIENT EVIDENCE

• Documents that the majority of programs meet the standards of the

selected program review option(s)

• A majority submitted for SPA Review achieved National Recognition
• State Review reports document how well individual programs perform in relation to the

state’s selected standards and that the majority meet the standards

• Program Review with Feedback results show that the state-selected state or national

standards are met for the majority of programs

• Includes a discussion of performance trends and compares across specialty

areas.

• Component A.1.2 is not eligible for Phase-in Plan submission

Fall 2017 | Washington, D.C.

PROGRAM REVIEW AND THE ACCREDITATION PROCESS

COMPONENT A.1.2

VOCABULARY

• 1. EPP: Educator Preparation Provider (previously called “Unit”) that prepares

professionals in various licensure or certification areas to serve in a P-12 setting 2. PROGRAM: A planned sequence of academic courses and experiences

leading to a degree, a recommendation for a state license, or some other credential that entitles the holder to perform professional education services in schools (P-12)

3. CANDIDATES: Pre service educators 4. STUDENTS: P-12 students 5. SPA: Specialized professional associations 6. SPA Program Report: A report submitted at a program level to provide evidence to meet standards developed by SPAs 7. SPA RECOGNITION REPORT/DECISION REPORT: Report providing SPA feedback and recognition decision– used as partial evidence for CAEP Standard 1

BB

Fall 2017 | Washington, D.C.

Program review decisions factor into CAEP Component A.1.2, which says:

“Providers ensure that advanced program completers have opportunities to learn and apply specialized content and discipline knowledge contained in approved state and/or national discipline specific standards. These specialized standards include, but are not limited to, Specialized Professional Association (SPA) standards, individual state standards, standards of the National Board for Professional Teaching Standards (NBPTS), and standards of

• ther accrediting bodies [e.g., Council for Accreditation of Counseling and

Related Educational Programs (CACREP)]

PROGRAM REVIEW: INTEGRAL TO CAEP ACCREDITATION (Advanced Level Program)

Fall 2017 | Washington, D.C.

• CAEP accredits EPPs
• EPP offered programs leading to licensing degrees, certificates, or

endorsements of P-12 professionals fall under the scope

• All endorsements use CAEP Advanced Level Programs Standards
• Programs licensing “Other school professionals” use CAEP Advanced

Level Programs Standards

• Programs accredited by other national accrediting bodies (CACREP,

NASM, etc.):

• EPP may choose to exempt from review by CAEP (will not be recognized as

accredited by CAEP)

• EPP may choose to include in the CAEP accreditation process (will be included in

EPP-wide assessments, annual report, and program review)

CAEP SCOPE AND PROGRAM REVIEW

PROGRAM REVIEW OPTIONS

• CAEP-state agreements determine program review options for EPPs within

state (28 agreements signed to date)

• Available program review options for EPPs in states with agreements:
• SPA review with National Recognition (3 years prior to site visit)
• CAEP program review with feedback (part of self-study report)
• State review of programs (determined by state)
• Available program review options for EPPs in states without agreements:
• SPA review with National Recognition (3 years prior to site visit)
• CAEP program review with feedback (part of self-study report)
• State review of programs (EPP coordinates with state to obtain and provide state

agency report)

Fall 2017 | Washington, D.C.

STATE SPA REVIEW REVIEW WITH FEEDBACK STATE REVIEW ARKANSAS X X X DELAWARE X

• INDIANA

X

• X

KANSAS

• X

NEW JERSEY X X

• EXAMPLES: STATE-SELECTED PROGRAM REVIEW

OPTIONS*

* Information on program review options by state is available on the CAEP website: http://caepnet.org/working-together/state-partners/state-partnership-agreements

Fall 2017 | Washington, D.C.

• What degree of competence in content knowledge do candidates

demonstrate?

• Can candidates successfully develop a conceptual plan for their teaching

and other professional education responsibilities?

• Can candidates implement their conceptual plan with students and

colleagues?

• Are candidates effective in promoting student learning?
• Do candidates meet state licensure requirements?

QUESTIONS THAT PROGRAM REVIEW ADDRESSES

Fall 2017 | Washington, D.C.

CONSIDER:

• 1. Did the EPP update the program review option in AIMS for each program?
• 2. Does the program list match the licensure, certification, or endorsement

programs list on the EPP’s catalog?

• 3. Does the selection of program review option meet CAEP-state agreement (if

applicable)?

• 4. Does the program level evidence (SPA report, state agency report, self-study

addendum) presented on the self-study report match the selected review

• ption?

Remember: 1 licensing program = 1 review option  evidence type

PRESENTING PROGRAM REVIEW EVIDENCE FOR CAEP ACCREDITATION

Fall 2017 | Washington, D.C.

SPA PROGRAM REVIEW OPTION WITH NATIONAL RECOGNITION

Two Steps in CAEP Accreditation Process if Selecting SPA Review Option: Step #1: Initial review report submitted to SPA three years prior to site visit (Program level review) Example: Site Visit in Fall 2020  Initial SPA review in Fall 2017 Step #2: Self-study report submitted to CAEP nine months prior to site visit (Provider level review) Example: Site Visit in Nov. 2020  Self-Study report in Mar. 2020

Fall 2017 | Washington, D.C.

• INITIAL REVIEW DUE DATE: 3 years prior to site visit

Example: site visit in Fall 2020  initial SPA review in Fall 2017

• SPA reports initiated more than three years before = old data

Did the EPP receive an extension to account for older Recognition Reports?

SPA REVIEW EXPECTATIONS: WHAT THE SITE TEAM WILL LOOK FOR

Fall 2017 | Washington, D.C.

• What evidence will the site team look for?
• A SPA Recognition/Decision Report
• 3 year out timing of Initial Review
• How will the site team determine if CAEP expectations are met when an EPP

selects the SPA option?

• Minimum sufficiency criteria: 51% of the total number of programs selecting SPA

review option have full National Recognition from a 3 year out submission

• Which SPA recommendations on the Decision Report will be used?
• Comments in Part E (Areas for consideration)
• Comments in Part F (Additional comments)
• SPA decisions or conditions for the program to address in Part G

PROGRAM REVIEW EVIDENCE: SPA REVIEW

Fall 2017 | Washington, D.C.

• PURPOSE:
• Gather evidence for current accreditation cycle (CAEP Standard 1)
• Initiate process to receive full National Recognition by visit date
• Initiate process to continue prior National Recognition status before expiration
• REVIEW CYCLES: 2 times per year

SPA REVIEW: TIMING AND CYCLES

Spring Cycle Due Date: March 15 Fall Cycle Due Date: September 15 Spring Cycle Decisions: August 1 Fall Cycle Decisions: February 1

Fall 2017 | Washington, D.C.

• INITIAL SUBMISSION: 3 years before site visit
• SHELL REQUESTS BEGIN: 1 year before submission date
• SHELL REQUESTS ENDS (moving forward): 5 days before submission date (March 10 for

spring cycle and September 10 for fall cycle)

• SHELL REQUEST SUBMISSION:
• List all programs preparing P-12 professionals in each specialization area in AIMS to

enable shell request submission

• Submit shell requests through CAEP’s Accreditation Information Management System

(AIMS): http://aims.caepnet.org/AIMS_login.asp)

• Directions requesting shells provided on CAEP website:

http://caepnet.org/accreditation/caep-accreditation/spa-program-review-policies- and-procedur

• CAEP staff creates shells after receiving request

STAGES OF SPA REVIEW PROCESS

Fall 2017 | Washington, D.C.

• The EPP addresses the following questions for programs selecting SPA

Program Review:

• How was the SPA feedback on specialty licensure area used to inform decision making

and improve instruction and candidate learning outcomes?

• What was learned about different specialty licensure areas as a result of the review of

the disaggregated data?

• What trends do the comparison of data across specialty licensure areas indicate and

how do they provide evidence for meeting the CAEP and state expectations and standards?

• Accreditation Decision: Evidence meets CAEP sufficiency criteria, OR,

evidence indicates potential area for improvement (AFI)

SPA REVIEW: DISCUSSION ON SELF-STUDY REPORT

Fall 2017 | Washington, D.C.

• What is CAEP Review with Feedback?
• An alternative option to SPA and state review
• Requires evidence of candidates’ knowledge of content and pedagogical content

knowledge for each licensure area program

• How do programs report evidence for this option?
• Incorporate evidence as part of the self-study report
• Analyze data from state licensure exams and/or other proficiency measures required

by EPP to demonstrate candidates’ content knowledge in the licensure area

• Analyze data to demonstrate candidates’ pedagogical knowledge in the area
• Analyze data from assessment of candidates’ impact on student learning in the area
• Provide assessment description and scoring guide in each case

PROGRAM REVIEW EVIDENCE: CAEP REVIEW WITH FEEDBACK

Fall 2017 | Washington, D.C.

• How do programs report evidence for this option (Continued)?

Address the following questions for each assessment:

• What artifact(s) is used to provide evidence?
• How was the assessment developed?
• How does the assessment provide evidence for meeting standards (next slide)?
• How is the quality of the assessment/evidence determined or assured?
• What criteria of success were established or measured, and how?
• Refer to the Technical Guide for CAEP Program Review with Feedback:

http://caepnet.org/accreditation/caep-accreditation/program-review-

• ptions/caep-program-review-with-feedback

PROGRAM REVIEW EVIDENCE: CAEP REVIEW WITH FEEDBACK

Fall 2017 | Washington, D.C.

Source: Technical Guide for CAEP Program Review with Feedback

Fall 2017 | Washington, D.C.

Source: Technical Guide for CAEP Program Review with Feedback

Fall 2017 | Washington, D.C.

• What standards are used for this option?

As a norm, align the assessments for Advanced Level Programs with the NBPTS Standards in respective areas of specialization

• If a state requires use of other standards for the CAEP Program Review with

Feedback option (state agreement), EPP will align evidence to those standards

PROGRAM REVIEW EVIDENCE: CAEP PROGRAM REVIEW WITH FEEDBACK

Fall 2017 | Washington, D.C.

• EVIDENCE SUBMISSION: Included as part of self-study report
• REVIEWED BY: site team
• PURPOSE:
• Gather program level evidence for current accreditation cycle
• Provide evidence for CAEP Standard A.1 (Advanced-Level Program)
• Receive formative feedback on meeting CAEP Standard A.1
• Feedback used by CAEP’s Accreditation Council to make accreditation

decisions

• Feedback may be used by states to understand if program meets state

expectations

CAEP PROGRAM REVIEW WITH FEEDBACK OPTION: TIMING AND PURPOSE

Fall 2017 | Washington, D.C.

• 3 cycles of data submitted and analyzed as part of self-study report
• Disaggregated data provided on candidates enrolled for main and

branch campuses (Disaggregate data if applicable)

• Cycles of data must be sequential and latest available
• The review is based on guidance provided in the CAEP Evidence

Guide

CAEP PROGRAM REVIEW WITH FEEDBACK OPTION: GENERAL EXPECTATIONS

PROGRAM REVIEW WITH FEEDBACK: DISCUSSION ON SELF-STUDY REPORT

• The EPP addresses the following questions for programs selecting the

Program Review with Feedback Option:

• Based on the analysis of the disaggregated data, how are the results of

specialty licensure area evidence used to inform decision making and improve instruction and candidate learning outcomes?

• Based on the analysis of specialty licensure area data, how have individual

licensure areas used data as the basis for change?

• How do the specialty licensure area data align with and provide evidence

for meeting the state-selected (or InTASC) standards?

Fall 2017 | Washington, D.C.

POTENTIAL ISSUES: STANDARD A.1

AREAS FOR IMPROVEMENT (AFIs) MAY BE CITED WHEN

• Instrument Quality is Poor:
• EPP-created assessments used to collect Standard A.1 data have significant deficiencies

with respect to CAEP’s assessment evaluation framework

• Phase-In Plans for one or more components do not meet CAEP’s guidelines for plans
• Evidence Quantity is Limited:
• Less than three cycles of data are provided
• Less than one cycle of phase-in data collected by academic year 2019/2020

36

Fall 2017 | Washington, D.C.

POTENTIAL ISSUES: STANDARD A.1

AREAS FOR IMPROVEMENT (AFIs) MAY BE CITED WHEN

• Case is Weak:
• Deficiency in evidence that program options foster deep understanding of critical concepts

and skills in the specialty areas

• Deficiency in evidence that knowledge and skills are applied to enhance P-12 settings.
• EPP’s analysis of data/evidence does not identify and discuss trends/patterns, comparisons,

and/or differences between programs or over time.

Fall 2017 | Washington, D.C.

POTENTIAL ISSUES: STANDARD A.1

STIPULATIONS MAY BE CITED WHEN

• Evidence Quality is Low
• Significant aspects/key language of the standard and components are not addressed by

relevant measures

• Majority of measures do not meet assessment sufficiency criteria
• Evidence Quantity is Limited:
• Limited or no evidence for Standard A.1, and (when eligible) no phase-in plan for A.1.1

that meets CAEP’s Guidelines for Plans and phase-in schedule

• Results are not disaggregated by specialty area

Fall 2017 | Washington, D.C.

POTENTIAL ISSUES: STANDARD A.1

STIPULATIONS MAY BE CITED WHEN

• Case is Weak
• Candidate performance is severely below reported standards for content knowledge and

application

• Majority of programs do not meet program review standards
• Limited or no evidence that candidates can apply the professional skills listed in A.1.1 to

enhancing P-12 settings or outcomes

Fall 2017 | Washington, D.C.

Coursework

• Diversity
• Technology

Fieldwork

• Diversity
• Technology

Interpersonal Interactions

• Diversity
• Technology

Cross-Cutting Themes

Embedded in Every Aspect of Educator Preparation

Fall 2017 | Washington, D.C.

• Places in which the cross-cutting themes of diversity and technology must

be explicitly addressed through evidence are identified by the following icons in the CAEP Evidence Tables.

• = diversity

and

• = technology

Cross-Cutting Themes of Diversity and Technology

Fall 2017 | Washington, D.C.

Themes of Diversity and Technology

Diversity

Standard A.1

• Candidates use their professional

specialty practices “flexibly to advance the learning of P-12 students toward attainment of college-and career- readiness standards” to enhance “learning and development

• pportunities” for students.

Technology

Standard A.1

• Candidates apply technology

appropriate to their field of specialization

Fall 2017 | Washington, D.C.

In Summary - The Case for Standard A.1

• Information is provided from several sources and provides evidence of

candidate knowledge, skills, and dispositions.

• Grades, scores, pass rates, and other data are analyzed.
• Differences and similarities across licensure/field areas, comparisons over time, and

demographical data are examined.

• Appropriate interpretations and conclusions are reached.
• Trends or patterns are identified that suggest need for preparation modification.
• Based on the analysis of data, planned or completed actions for change that are

described.