CREDO Research Request June State Board of Education Meeting June - - PowerPoint PPT Presentation

June State Board of Education Meeting

CREDO Research Request

June 9, 2016

• Requestor: Margaret (Macke) Raymond
• Institution: CREDO at Stanford University
• Study Duration: From date of approval to December 2017.
• Purpose: To update a 2013 quasi-experimental study of how students in

schools affiliated with Charter Management Organizations and charter networks progress academically.

• Funding: The proposed research is funded by the Walton Family Foundation

and the Fisher Fund.

About the Research

2

• Question: Is the research compliant with the Family Educational Rights and Privacy

Act (FERPA)?

• Answer: Yes. Specifically, the disclosure to CREDO would be permitted under the

following FERPA exception: “The disclosure is to organizations conducting studies for,

• r on behalf of, educational agencies or institutions to: (A) Develop, validate, or

administer predictive tests; (B) Administer student aid programs; or (C) Improve instruction.”

• Furthermore, FERPA has additional requirements regarding the need for contractual

agreements with the researcher that covers a number of subjects. If this research is approved, CDE will put in place a Data Sharing Agreement with CREDO that includes all these requirements.

Internal Vetting Process

3

• Question: What is the validity of the research plan (is the researcher qualified)?
• Answer: Dr. Raymond serves as director of the Center for Research on

Education Outcomes (CREDO), which analyzes education reform efforts around the

• country. She has done extensive work in public policy and education reform, and is

currently researching the effectiveness of public charter schools and the creation of reliable data on program performance. Prior to joining Stanford in 2000, she held faculty positions in the political science and economics departments at the University

• f Rochester.

Internal Vetting Process Cont.

4

• Question: What is the significance of research proposal?
• Answer: The aim of the research is to provide education leaders and policy makers

with solid evidence about program and policy performance for their use in decision

• making. Since charter schools and Charter Management Organizations in particular

are dynamic in size and effort, current assessment of whether and how they are effective can have immediate value in policy discussions.

• Question: Is the researcher in good standing on any previous projects?
• Answer: Yes. The researcher has conducted prior research for CDE and has provided

us with the results of that research. An executive summary of the results of the prior research is included in your packets.

Internal Vetting Process Cont.

5

• Question: Is the research of specific benefit to CDE?
• Answer: There is potential to derive benefit from learning how students in charter

schools perform compared to their peers in non-charter schools.

• This research is a continuation of research that has been ongoing in Colorado for a

number of years, and the potential to have data regarding student outcomes in charter and non-charter schools may benefit charter schools and authorizers, as well as families as part of the school choice process.

• Legislators may also be interested in the findings from this research as they make

policy decisions about school governance structures.

Internal Vetting Process Cont.

6

• Question: Does the research align with CDE’s and the Board of Education’s

strategic priorities?

• Answer: Answer unknown as CDE continues to develop its updated strategic
• priorities. However, CREDO supports CDE in developing and meeting

priorities that support the success of all Colorado children and youth.

Internal Vetting Process Cont.

7

• Question: What is the validity of the research plan (are the methods appropriate)?
• Answer: We use a matching methodology to compare the growth of a charter

student to the typical growth they would have had in a traditional public school using the same year/grade/subject tests. The "virtual twin" method used by CREDO has been peer reviewed by other researchers1 and found to be a reliable and valuable method of analysis.

1 Forston, K. and Verbitsky-Savitz, N. et al. (2012). “Using an Experimental Evaluation of Charter Schools to Test Whether

Nonexperimental Comparison Group Methods Can Replicate Experimental Impact Estimates,” NCEE 2012-4019, U.S. Department of Education

Internal Vetting Process Cont.

8

• Question: What are the data elements specified?
• Answer: The following data elements have been requested: state achievement test

scaled scores, proficiency categories, school ID number, race/ethnicity, gender, free- reduced lunch status, special education status, English proficiency, grade level, full student enrollment files, graduation flag, course completion records, charter schools by district and school ID, grade level means and standard deviations for the state reading and math tests, conditional standard error of measurement tables, cut scores for proficiency bands, and teacher-student linkage.

• The only direct identifier is the State Assigned Student ID (SASID) which CREDO has

requested be masked.

Internal Vetting Process Cont.

9

• Question: Do we want to share those data elements with the researcher?

Are any data elements sensitive in any way?

• Answer: CDE’s data owners for each data type have approved the preceding

list to be provided to the researcher.

• The researcher requested a few additional data types (raw test scores and

juvenile detention identification) but we will not provide that data due to its sensitivity.

Internal Vetting Process Cont.

10

• Question: Has IRB approval been provided?
• Answer: Yes. The IRB approval from Stanford University for this research is

contained in your packets.

• Question: Has the data security training documentation been provided?
• Answer: Yes. CREDO provided the following information:
• Any officers or employees of CREDO who have access to student data or teacher
• r principal data have received training on the federal law governing

confidentiality of such data through the Collaborative Institutional Training Initiative (CITI) prior to receiving access to student, teacher or principal data.

Internal Vetting Process Cont.

11

• CDE will work with the researcher to put in place a Research Data Sharing Agreement

that includes the following:

• Strict controls around how the researcher can use data.
• Stringent security and privacy protections that must be in place to protect the data.
• Requirements for the researcher to provide CDE with the results of the research.
• Requirements for the researcher to destroy the data once the research has been completed
• n a schedule specifically listed in the Research Data Sharing Agreement.
• Research Data Sharing Agreements are reviewed on an annual basis, should the

research extend beyond a year.

• Once the term of the Data Sharing Agreement has reached its end, CDE will follow up

with the researcher to obtain the results of the research and gain a confirmation that the data has been destroyed.

Next Steps (Pending Approval)

12

Motion 1: To approve the CREDO Research Proposal. Motion 2: To deny the CREDO Research Proposal.

Approval Request

13