CORE and Its Work in the Area of Measuring Social-Emotional Skills - - PowerPoint PPT Presentation

CORE and It’s Work in the Area of Measuring Social-Emotional Skills Noah Bookman & Rick Miller

March 8, 2017

California submits NCLB waiver, CA denied

CORE Districts’ vision: ✓ Focus on equity and access ✓ Share learning and responsibility for student achievement ✓ Emphasize capacity building

• ver accountability

✓ Support local innovation

Founded in 2010 the CORE Districts joined together with a common interest in collaborating and sharing best practices. By sharing expertise and human capital, the CORE Districts can innovate and bring successful educational practices to scale The CORE Districts are nationally recognized for the innovative school improvement and accountability system created under a federal waiver (2013-future) The CORE Improvement Community connects thousands of educators, integrating the tools and technologies of improvement science with the power of education networks (2017-future)

CORE Districts collaboration since 2010

Traditionally underserved populations make up a large portion of the CORE Districts’ student population. The percentage of these high need students is greater in the CORE Districts than it is in the state of California.

Source: SEF Calculations of NCES Common Core of Data, 2013

Guiding principles: ✓ Information as “flashlight” (and not a “hammer”) ✓ From a narrow focus to a holistic approach ✓ Making all students visible ✓ From just achievement to achievement and growth Goal: College & Career Ready Graduates

Academic Domain Social-Emotional & Culture- Climate Domain

• Achievement and Growth
• Graduation Rate
• High School Readiness Rate

(Gr. 8)

• Chronic Absenteeism
• Student/Staff/Parent Culture-

Climate Surveys

• Suspension/Expulsion Rate
• Social Emotional Skills
• ELL Re-Designation Rate
• Special Education

Disproportionality

Focus: Elimination of Disparity and Disproportionality

All Students Group & Subgroups

Developed through collaboration and partnership: ✓ Led by the CORE Superintendents ✓ Guided by the experts in our districts ✓ With input from hundreds of educators across the CORE districts ✓ With support from our key partners (e.g. Stanford University, Harvard University) ✓ With guidance from our Oversight Panel (e.g. ACSA, CSBA, Ed Trust West, PACE, PTA)

Designing the School Quality Improvement Index:

MAKING ALL STUDENTS VISIBLE:

N size of 20 resulting in

• ver 150,000 additional

students counted!

CORE is part of the national dialogue on including Social Emotional Skills in Multiple Measure approaches to school quality

With almost half a million students participating, our Spring 2015 Field Test of measures

• f social-emotional skills lets us explore how to measure these essential skills at scale.

Social Emotional Skills Cover Four Topics – Including Inter-Personal and Intra-Personal Skills

14

The identified skills include intrapersonal and interpersonal skills Intrapersonal Interpersonal Growth Mindset Self-Efficacy Self- Management Social Awareness

15

These competencies were initially measured through a combination of confidential student self-report and teacher report measures

Student Self-Report* Teacher Report* Key Contributors Grades K-4 Self-Management x Clancy Blair (NYU) Social Awareness x CASEL / AIR Grades 5-12 Growth Mindset x Carol Dweck (Stanford) & Camille Farrington (CCSR) Self-Efficacy x Camille Farrington (CCSR) Self-Management x x Angela Duckworth (UPenn) Social Awareness x x CASEL / AIR

Measures by Grade Level, Competency, and Type

*To improve validity and mitigate the effect of reference bias, self-report and teacher report measures will include anchoring vignettes developed by ETS.

An updated draft of the SE measures will be shared with districts during the week of 3/3

Sample SEL Items To assess social-emotional skills, we ask students about their beliefs and behaviors. Below, for instance, are some self-management items. Please answer how often you did the following during the past 30 days. During the past 30 days…

• I came to class prepared.
• I remembered and followed directions.
• I got my work done right away instead of waiting until the last minute.
• I paid attention, even when there were distractions.
• I worked independently with focus.
• I stayed calm even when others bothered or criticized me.
• I allowed others to speak without interruption.
• I was polite to adults and peers.
• I kept my temper in check.

(Almost Never, Once in a While, Sometimes, Often, Almost All the Time)

Pause, reflect and chat with an elbow partner: What do you find noteworthy about what you have heard so far? What questions are coming up for you?

SEL & Culture Climate: A school’s culture-climate is related to social emotional skills reports, and we see a substantive range in school performance, despite comparable levels of youth in poverty.

Both of these schools have close to 90% of youth in poverty

The larger the dot, the higher the percentage

• f youth in

poverty.

Correlation between overall SEL & overall culture-climate is .47.

Math & SEL: A school’s SEL results are also related to performance on other indicators, such as math.

The larger the dot, the higher the percentage

• f youth in

poverty.

In this graph we see that schools with strong SEL generally performed better on SBAC math.

Means by Grade and Gender

What are your hypothesis about these results? What questions does it raise for you?

Notice that lower performing 11th graders appear to be catching up with their higher performing peers in SE

• skills. Could this

perhaps be because lower performing students with stronger SE skills are more likely to persist to 11th grade?

Whereas, there are ~35,000 students in the 8th grade results, there are ~25,000 students 11th grade results.

Policy Analysis for California Education

SEL & CC and strong predictor of test scores and growth in test scores

After controlling for student demographics

• Self-management, growth mindset, and self-

efficacy most predictive in SEL domain (in order)

• Social awareness has weak negative relationship
• Only safety predictive in CC domain
• Rules/discipline has weak negative relationship
• Patterns similar in relationship of SEL/CC to status

and growth in test scores, but reduced relationship for growth

Policy Analysis for California Education

SEL/CC reports are lower for minority & disadvantaged students

• Students report lower Social Emotional Skills and School

Culture-Climate if:

• They are African American or Hispanic
• They are male
• They are eligible for free or reduced lunch
• They have a disability
• They are classified as EL
• While the exact coefficients vary by construct, the

pattern holds across each of the 4 SEL and 4 CC domains

Policy Analysis for California Education

These gaps remain when we compare students to their peers within schools

• Gaps are smaller but persist if we look at students

within schools vs. in the aggregate

• For SEL, the most significant gaps are between white

and African American/Hispanic students

• This indicates that, for example, AA/Hispanic students report

lower SEL compared to their white peers in the same school

• For CC, the only significant gap is between white and

African American students

• This indicates that AA students feel less safe/supported, for

example, compared to their white peers in the same school

Exploratory Growth Model

 Outcome

 Current year construct (one model per grade)

 Control

 Prior year construct results  Gender  Ethnicity  ELL (yes/no)  SPED (mild/severe)  School fixed effect

With two years of SEL data in several district, we can now look at “growth/impact” in SEL along with “growth/impact” on academic tests.

A B C D A B C D A B C D A B C D

• A. Students are gaining ground in the SE skill relative to their peers, but academic growth is slower than their peers.
• B. Students are gaining ground in the SE skill relative to their peers, and academic growth is faster than their peers.
• C. Students are losing ground on the SE skill relative to their peers, and academic growth is slower than their peers.
• D. Students are losing ground on SE skill relative to their peers, but academic growth is faster than their peers.

Pause, reflect and chat with an elbow partner: What do you find noteworthy about what these findings? What questions are coming up for you?

Starting in 2015-16, our multiple measure school report cards included social- emotional survey results.

Reports support

CONTINUAL IMPROVEMENT

for school leaders and teachers

Our 2015-16 results add academic growth, culture-climate surveys and social emotional skills to the picture.

Illustrative Example

Results include performance by the “all students” group and by subgroups

Illustrative Example

Strategy and Planning:

Aligning Data and the Budget to Create Coherent Action Plans

Future Administrator’s Presentation (Long Beach Unified School District) May 17, 2016

Strategy and Planning

Budget & Action Plan Goals Data Vision

Relationship between Data and the Budget

Budget/ Action Plan Data/Goals

Scores on the Index

• There are results and an index level for each measure
• The index level is a score between 1 and 10, with 10

being the highest level

• The index is color-coded to show performance

– Scores of 8 to 10=Above Average – Scores of 4 to 7=Average – Scores of 1 to 3=Below Average

Sample: Metric Result 2014 Metric Result 2015 Change Index Level 2015

HS Readiness 47%

HS Ready

44%

HS Ready

• 3%

5/10

CORE Report for Sample School A

What are the needs of School A?

School A has $400,000 to Spend on Identified Needs

Which budgeted item(s) should be prioritized?

• a. A Counselor who conducts SSTs based on

attendance and academic needs

• b. A Counselor who manages discipline
• c. A Community Worker who conducts home visits

and makes daily calls regarding absences

• d. After-school tutoring in reading

e. A Librarian How would you prioritize these items?

Your Turn

• 1. Utilize the sample school report provided for

Washington Middle School.

• 2. Identify the 1-2 key needs of the school site.
• 3. Brainstorm some possible uses of targeted

funds to meet the identified needs.

• 4. Prioritize your needs.

Identify Needs Brainstorm Use of Funds Prioritize

CORE has launched an improvement community focused on closing math achievement gaps that will in part be focused on SEL as a root cause of these gaps.

Problem Potential drivers

▪ Integrating social emotional

learning into math instruction Improve math proficiency of African-American and Hispanic/Latino students, especially grades 4-8

▪ Improving the quality of teaching

in math

▪ Aligning curriculum with

assessments

▪ Improving the human capital

pipeline in math

38

Our fishbone exercise in September uncovered a series of preliminary root causes of low math scores among Black and Latino middle school students

Identified by most districts Identified by 2+ districts

Primary root causes Initial hypothesized secondary causes Insufficient social- emotional learning (SEL)

▪ Repeated low achievement fosters low self-efficacy to improve in math ▪ Limited growth mindset among students prevents ownership of learning ▪ SEL is not embedded in learning across other subjects ▪ Particular challenges for students at the transition from elementary to middle school

Human capital challenges

▪ Difficult to hire, select, recruit, and retain quality math teachers – high turnover ▪ Differential sorting of high quality teachers out of high needs schools

Knowledge and skills

• f teachers

▪ Teachers have limited preparation in pedagogical and math content knowledge ▪ Teachers can have fixed mindsets about math ▪ Limited skills of leaders stagnates skill development of teachers

Limited instructional time

▪ Competing subject priorities leads to insufficient time for math instruction ▪ Suspensions, SpEd classifications, and absenteeism disproportionately decrease math

instructional time for Black and Latino students

▪ Weak ELA skills impact math learning and performance

Unconscious bias among teachers

▪ Unconscious biases can lead to low expectations of students’ abilities to learn math,

particularly among Black and Latino students Curricular limitations

▪ Not aligned to measures of student achievement or culturally responsive ▪ Doesn’t allow instruction to be sufficiently differentiated to needs of students

Home/life challenges

• f students

▪ Food, financial, and housing insecurities, especially experienced among Black and

Latino students.

▪ Many parents don’t have the skills or knowledge of how to help students

View Type 1: Slicing scale scores by demographics and

• ther outcomes of interest (A, C and D)

Math performance by avg distance to meeting standards.

View Type 1: Slicing scale scores by demographics and

• ther outcomes of interest (A, C and D)

Symbols correspond to root cause of interest

View Type 1: Slicing scale scores by demographics and

• ther outcomes of interest (A, C and D)

Size relates to population size within each set o f results.

Explore the LBUSD and CORE Math Performance Data by SEL Results and Race: What do you notice? How does it contribute to your understanding of root causes of math performance gaps? What questions or areas of further curiosity does it raise?

SEL and the PK-5 Report Card

Research to Action

44

Overview: Linking SEL in TK-5 Report Cards

• Research & Development
• Outcomes through Teacher & Parent Engagement
• Implications
• Measuring Impact

45

SEL Working Group

Comprised of over 10 SFUSD educators. Expertise of this group includes:

• Practitioners of PAX Good Behavior Game
• Behaviorist and Special Education Coach
• Classroom Teachers
• Early Education Coach
• Stanford Doctoral Student
• Staff members from Research, Planning & Assessment
• Staff members from Pupil Services Office

Research How do Report Card items align with CORE survey items?

Nicole Tirado-Strayer

48

1 Consistently shows effort 2 Respects self and others 3 Listens attentively 4 Meaningfully participates in activities 5 Follows directions 6 Completes tasks and assignments 7 Completes and returns homework 8 Demonstrates organizational skills 9 Works independently Composite

Implication: Refine items to measure SE skills more precisely.

Research What does this section of the report card measure?

4 9

Student Grade Q1 Q2 Q3 Q4 Q5 Q6 Q7 Q8 Q9 Composite

Nicole Tirado-Strayer

Implication: Measure SEL skills using a more precise scale (e.g., 4-pt). Most students received a score of “Satisfactory” (2 on a scale of 1-3) for all 9 items

50

Development Process for SEL Working Group

Revise SEL statements on Report Cards based on 4 rounds of teacher feedback

• Over 60 teachers from 31 school sites

Revise the SEL statements on Report Cards based on parent feedback from parent groups (Parent Advisory Committee (PAC), Family Liaisons, Mission Graduates, Parent for Public Schools (PPS)

• Over 160 parents either responded via survey or during meetings

2014-15 Report Card

• 1. Consistently shows effort
• 2. Respects self and others
• 3. Listens attentively
• 4. Meaningfully participates in

classroom activities and discussion

• 5. Follows directions
• 6. Completes tasks and

assignments

• 7. Completes and returns

homework

• 8. Demonstrates organizational

skills

• 9. Works independently

NEW! 2016-17 Report Card

• 1. Works/plays collaboratively with others
• 2. Regulates emotions and works with focus
• 3. Approaches challenges as learning
• pportunities
• 4. Accomplishes personal and academic

goals

✓ Social Awareness ✓ Self-Management ✓ Growth Mindset ✓ Self-Efficacy

Outcomes Complete Report Card Items

52

Feedback Cycle & Engagement with Educators

Teacher Guides will be provided to support teachers in evaluating student’s SEL skills in a holistic manner across contexts

53

Feedback Cycle & Engagement with Educators

Teachers want a clear guide for providing grades, and a clear progression of learning to help students improve Proficiency levels will now be measured on a 4 point scale progression

54

Feedback Cycle & Engagement with Families

Families want to know how their child is doing, and what they can do to help Parent Guides will now be translated into 5 languages:

• Arabic
• Vietnamese
• Tagalog
• Chinese
• Spanish

55

What Educators & Parents Are Saying…

Helpful, clear, framed in a positive way These are the areas that parents want to know about. The description and rubric for self-management, social awareness, self- efficacy and growth mindset is outstanding! School/Classroom supports – appreciate the suggestions, good reminder Like that the scale is the same as content areas, more clear for parents, provides opportunities for struggling students to receive a 4.

The Assessment Work Group (a collaboration of CASEL, CORE, Transforming Education and other partners) is launching a design challenge for the next generation of measures of social emotional skills.

Our Assessment Work Group is a multiyear effort to make key advancements in student social and emotional (SE) competence. The work group is focused on four main goals: (1) provide practical and timely information to educators about selecting and effectively using currently available SE assessments; (2) demonstrate alignment and distinctions between existing SE frameworks; (3) identify and disseminate key design principles that guide researchers and developers in creating practical and informative SE performance measures; and (4) collaborate with, coordinate with, and learn from ongoing related efforts in the field of SE assessment in order to maximize impact and avoid duplication of efforts.

Stay tuned for more information and a formal Call for Submissions that will be released on March 20, 2017

We want to hear from you. Share any thoughts from this session with us.

SEND A TWEET | POST ON FACEBOOK | EMAIL A COWORKER | TEXT A FRIEND