STANDARD 5: PROVIDER QUALITY, CONTINUOUS IMPROVEMENT, AND CAPACITY IN INITIAL- LICENSURE AND ADVANCED-LEVEL PROGRAMS CAEP PRESENTERS: GINA BURKHARDT, Vice President EMERSON J. ELLIOTT, Special projects Washington, District of Columbia September 2017
What will be covered in the next 90 minutes? Point 1 — The highlights of Standard 5 — PPT #3 Point 2 — What Standard 5 is about and why is it part of CAEP standards — PPT #4 Point 3 — Question prompts for the EPP self-study report and Standard 5 — PPT #5-11 Point 4 — Suggestions from the Carnegie Foundation for the Advancement of Teaching — PPT #13-15 Point 5 — Potential issues for AFIs, stipulations and standard not met — PPT #16- 18 Point 6 — Standard 5 and a culture of evidence — PPT # 19-23 Fall 2017 | Washington, D.C. 2
The highlights of Standard 5: 1. The provider maintains a quality assurance system [component 5.1] 2. comprised of valid data from multiple measures ,[components 5.2 and 5.4] including evidence of candidates’ and completers’ positive impact on P -12 student learning and development. [from components 3.5 and 4.1] 3. The provider supports continuous improvement [components 5.3, 5.4, for which evidence is required, and 5.5] that is: sustained and evidence – based, and that evaluates the effectiveness of its completers. The provider uses the results of inquiry and data collection: • to establish priorities, • enhance program elements and capacity, and test innovations • to improve completers’ impact on P -12 student learning and development. Fall 2017 | Washington, D.C. 3
What Standard 5 is about • It is about EPP capability to bring together useful evidence relevant to preparation and its results • It is about useful evidence (that is, information that can help you) • And it is about effective use of that capability in answering faculty questions and continuously improve • Note that Standard 5 is written at the EPP level — It comprehends both initial- licensure and advanced-level programs Fall 2017 | Washington, D.C. 4
Why it is a CAEP Standard • Quality assurance and continuous improvement are means by which an EPP can manage its responsibilities effectively • Accreditation prioritizes continuous improvement — it is not just to focus on a point in time or once each seven years Fall 2017 | Washington, D.C. 5
Guiding Questions
Guiding Questions
• Question prompts for the EPP self-study-report and Standard 5: THE QUALITY ASSURANCE SYSTEM • How well is the quality assurance system working for the EPP and how do you know? [component 5.1]. • Is it able to answer faculty questions about the adequacy of candidate preparation in particular areas (e.g., common core state standards, use of data to monitor student progress, creating assessments appropriate for different instructional purposes)? • What modifications has the faculty identified and carried out to change or increase the capabilities of its quality assurance system? Fall 2017 | Washington, D.C. 8
• Question prompts for the EPP self-study report and Standard 5: DATA IN THE QUALITY ASSURANCE SYSTEM • What strengths and weaknesses in the quality assurance system do faculty find when they use data and analyses from the system? [component 5.2]. • Are the data “relevant, verifiable, representative, cumulative and actionable? • Can findings be triangulated with multiple data so they can be confirmed or found conflicting? • What investigations into the quality of evidence and the validity of their interpretations does the EPP conduct? Fall 2017 | Washington, D.C. 9
• Question prompts for the EPP self-study report and Standard 5: USE OF DATA FOR CONTINUOUS IMPROVEMENT • What is the evidence that the EPP has improved programs in its continuous improvement efforts? [component 5.3] • How have perspectives of faculty and other EPP stakeholders been modified by sharing and reflecting on data from the quality assurance system? [component 5.5] • What “innovations” or purposeful changes has the EPP investigated and what were the results? [component 5.3] Fall 2017 | Washington, D.C. 10
• Question prompts for the EPP self-study and Standard 5: OUTCOME MEASURES • What has the provider learned from reviewing its annual outcome measures over the past three years? These are the measures in component 5.4 (initial- licensure, reported to CAEP annually) and A.5.4 (advanced-level, not in annual report to CAEP): • Licensure rate • Completion rate • Employment rate • Consumer information such as places of employment and initial compensation (including student loan default rates) Fall 2017 | Washington, D.C. 11
Suggestions from Carnegie Foundation for the Advancement of Teaching on improvement research • As you examine the outcomes you currently achieve (i.e., data on the first four standards) and identify gaps between current results and established standards, why is it that these results continue to occur? • How do you understand the problem(s ) you need to solve? • Based on your systematic problem analysis, what is your working theory of improvement? Fall 2017 | Washington, D.C. 12
More from Carnegie • How has this working theory been tested ? • More generally, as you cycle through your processes of continuous improvement (iteratively refining your theories based on the results of the changes made) what are you learning about your instructional system • Remember we often learn most from our failures . So, if relevant, what perhaps might you have tried, found evidence that it did not work as you intended and what did you learn from this about what to try next? Fall 2017 | Washington, D.C. 13
What CAEP site teams will learn about Standard 5 while they review Standards 1-4 • A. The capabilities of the EPP’s quality assurance system • B. Their confidence in EPP efforts to ensure data quality Fall 2017 | Washington, D.C. 14
In addition, site teams will need to find evidence in Standard 5 that: • C. The EPP regularly analyzes and uses data (evidence is required for effective continuous improvement in component 5.3) • D. The EPP reports on the CAEP annual indicators of “impact” and “outcomes” (evidence is required for outcome measures in component 5.4 and “impact” measures in Standard 4) • E. Appropriate stakeholders are regularly involved in decision-making Fall 2017 | Washington, D.C. 15
POTENTIAL ISSUES: STANDARD 5--AFIs AREAS FOR IMPROVEMENT (AFIs) may be cited when: • There are deficiencies in capabilities of the quality assurance system • Data quality is not documented or equal to EPP claims • EPP lacks efforts at continuous improvement • Failures in EPP reporting on the 8 annual indicators • Limited evidence that stakeholders are involved Fall 2017 | Washington, D.C. 16
POTENTIAL ISSUES: STANDARD 5--Stipulations • STIPULATIONS may be cited when: • There is limited evidence of a functioning quality assurance system • There is limited evidence that data are regularly used as a basis for continuous improvement (a required component) • EPP use of the 8 annual measures is deficient (a required component) Fall 2017 | Washington, D.C. 17
POTENTIAL ISSUES: STANDARD 5 — Standard not met • The Accreditation Council decides if AFIs or stipulations will be cited and whether standards are met or unmet • Standard 5 will not be met when two or more stipulations are cited Within a component Across components • If required evidence is not provided for components 5.3 or 5.4, a stipulation is assigned, and the standard may or may not be met (depending on other accreditation findings) • If the standard is met — with one stipulation cited for insufficient evidence on components 5.3 or 5.4, the EPP has 24 months from the decision to provide sufficient evidence to remedy the deficiency. • A document review will be conducted by a site team comprised of 2 or 3 site visitors. Fall 2017 | Washington, D.C. 18
A culture of evidence • The big picture —what CAEP means by a “culture of evidence” “A habit of using evidence in assessment, decision making, planning, resource allocation and other institutional processes that is embedded in and characteristic of an institution’s actions and practices.” Western Association of Schools and colleges (2013) See CAEP Evidence Guide, pp. 5-7 http://caepnet.org/~/media/Files/caep/knowledge-center/caep-evidence- guide.pdf?la=en Fall 2017 | Washington, D.C. 19
More on a culture of evidence • Evidence is NOT Something that an EPP does for the accreditor A “compliance” mechanism • Data are NOT An end in themselves “the answer” for accreditation • WITHIN the EPP, DATA ARE THE MEANS TO SUPPORT A CONTINUING EXAMINATION OF WHAT YOU DO • FOR ACCREDITATION, DATA ARE THE BASIS TO BEGIN A CONVERSATION Fall 2017 | Washington, D.C. 20
More on a culture of evidence • Evidence comes from multiple sources • Validity of evidence is systematically examined • Data are USED by the EPP for purposes of continuous improvement Fall 2017 | Washington, D.C. 21
Essential properties of evidence in a culture of evidence • Evidence is intentional and purposeful • Evidence always entails interpretation and reflection • Good evidence is integrated and holistic • What counts as evidence can be both quantitative and qualitative • Good evidence can be either direct or indirect Fall 2017 | Washington, D.C. 22
Recommend
More recommend
