New Data, New Perspectives: What the New Child Outcomes Data Tell - - PowerPoint PPT Presentation

New Data, New Perspectives: What the New Child Outcomes Data Tell Us (and Don’t Tell Us) about Children and Programs

February 21, 2018 Batya Elbaum, PhD Measuring Outcomes Project

SPP Indicator 7-Child Outcomes

• Indicator 7 addresses three “outcome areas” for

PreK children:

• A. Positive social-emotional skills (including social

relationships)

• B. Acquisition and use of knowledge and skills

(including early language/communication and early literacy)

• C. Use of appropriate behaviors to meet their

needs

What outcomes were achieved by children who exited the PreK Program in 2016-17?

Description of the data

• Number of Records: 10,233
• Range of age at Entry Assessment: 30.0 to 65.8 months,

M = 41.7, SD = 7.7.

• Range of age at Exit Assessment: 39.7 to 86.5 months,

M = 61.5, SD = 3.9.

• Range of time between entry and exit assessments: 3.0

to 43.4 months, M = 19.8, SD = 7.9.

• Percent boys: 70.8%.

The 5 OSEP Outcome Categories

• a. did not improve functioning
• b. improved functioning but not sufficient to move

nearer to functioning comparable to same-aged peers

• c. improved functioning to a level nearer to same-

aged peers but did not reach it

• d. improved functioning to reach a level comparable

to same-aged peers

• e. maintained functioning at a level comparable to

same-aged peers

Distribution of children across the 5 OSEP categories for each outcome

Outcome A

BDI-2 Personal- Social Domain

Outcome B

BDI-2 Communication Domain

Outcome C

BDI-2 Adaptive Domain

OSEP Categories

n % n % n %

• a. did not improve functioning

208 2.0 218 2.1 200 2.0

• b. improved functioning but not sufficient to

move nearer to functioning comparable to same- aged peers

1,154 11.3 2,146 21.0 1,509 14.7

• c. improved functioning to a level nearer to same-

aged peers but did not reach it

521 5.1 990 9.7 499 4.9

• d. improved functioning to reach a level

comparable to same-aged peers

2,527 24.7 3,404 33.2 2,222 21.7

• e. maintained functioning at a level comparable

to same-aged peers

5,823 56.9 3,475 34.0 5,803 56.7

Total N = 10,233

100% 100% 100%

Summary Statements

• SS #1: Percent of preschool children age 3

through 5 with IEPs who made greater than expected growth in each outcome area

(c + d)/(a + b + c + d)

• SS #2: Percent of preschool children age 3

through 5 with IEPs who exited the program within age expectations in each outcome area

(d + e)/(a + b + c + d + e)

Summary Statement Percentages: Results Compared to Targets

Outcome A

BDI-2 Personal-social Domain

Outcome B

BDI-2 Communication Domain

Outcome C

BDI-2 Adaptive Domain

SS#1 SS#2 SS#1 SS#2 SS#1 SS#2 Data 69.1% 81.6% 65.0% 67.2% 61.4% 78.4% Target 68.1% 83.9% 68.4% 73.4% 59.9% 81.0% Target Met? Yes No No No Yes No

SS #1 (“greater than expected progress”) over time

50 55 60 65 70 75 80 85 90

09-10 10-11 11-12 12-13 13-14 14-15 15-16 16-17

Fiscal Year

Outcome A Outcome B Outcome C

SS #2 (“exited within age expectations”) over time

50 55 60 65 70 75 80 85 90

09-10 10-11 11-12 12-13 13-14 14-15 15-16 16-17

Fiscal Year

Outcome A Outcome B Outcome C

Children in the 2016-17 data set, by district size group

5.1% (n = 519) 4.9% (n = 502) 13.0% (n = 1331) 27.9% (n = 2855) 49.1% (n = 5026) 0.0% 10.0% 20.0% 30.0% 40.0% 50.0% 60.0% Small Middle/Small Middle Large Very Large

Summary Statements by Size Group

Size-alike group n SS#1 SS#2 SS#1 SS#2 SS#1 SS#2 Small 519 79.7% 89.6% 75.5% 83.6% 75.7% 89.6% Middle/Small 502 73.3% 86.5% 69.5% 72.1% 72.9% 85.5% Middle 1,331 71.5% 83.5% 72.2% 71.4% 67.7% 83.0% Large 2,855 66.0% 81.3% 62.4% 67.0% 58.2% 77.4% Very Large 5,026 68.9% 79.9% 63.4% 64.1% 59.4% 75.9% STATE 10,233 69.1% 81.6% 65.0% 67.2% 61.4% 78.4% Outcome A BDI-2 Personal-social Domain Outcome A BDI-2 Communication Domain Outcome A BDI-2 Adaptive Domain

Communication: % of children within age expectations (> -1.5 SD) on entry

0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 1 2 3 4 5 6 7 8 9 1011121314151617181920212223242526272829303132333435363738394041424344454647484950515253545556575859606162

Communication outcomes for children with and without a delay on entry

0.0% 10.0% 20.0% 30.0% 40.0% 50.0% 60.0% 70.0% 80.0% 90.0% 100.0%

Delay of < -1.5 SD on entry No delay on entry OSEP Cat A OSEP Cat B OSEP Cat C OSEP Cat D OSEP Cat E

Personal-social outcomes for children with and without a delay on entry

0.0% 10.0% 20.0% 30.0% 40.0% 50.0% 60.0% 70.0% 80.0% 90.0% 100.0%

Delay of < -1.5 SD on entry No delay on entry OSEP Cat A OSEP Cat B OSEP Cat C OSEP Cat D OSEP Cat E

Adaptive outcomes for children with and without a delay on entry

0.0% 10.0% 20.0% 30.0% 40.0% 50.0% 60.0% 70.0% 80.0% 90.0% 100.0%

Delay of < -1.5 SD on entry No delay on entry OSEP Cat A OSEP Cat B OSEP Cat C OSEP Cat D OSEP Cat E

Overall Delay Categories

• Children were grouped into three overall delay

categories based on their standard scores in the 3 domains measured for child outcomes reporting.

• No delay: Scores > -1.5 SD in all three domains.
• Moderate delay: One or more domain scores

between -1.5 SD and -2.0 SD.

• Significant delay: A score ≤-2.0 SD in at least one

domain.

Statewide: % of children in each overall delay category on entry

0.0% 10.0% 20.0% 30.0% 40.0% 50.0% 60.0% 70.0% 80.0% 90.0% 100.0%

State

No delay on entry Moderate delay on entry Significant delay on entry

3 districts: % of children in each overall delay category on entry

0.0% 10.0% 20.0% 30.0% 40.0% 50.0% 60.0% 70.0% 80.0% 90.0% 100.0%

District A District B District C

No delay on entry Moderate delay on entry Significant delay on entry

Statewide: Children within age expectations (> - 1.5 SD) in all 3 domains on entry

0.0% 10.0% 20.0% 30.0% 40.0% 50.0% 60.0% 70.0% 80.0% 90.0% 100.0%

State

No delay on exit Moderate delay on exit Significant delay on exit

3 districts: Children within age expectations (> - 1.5 SD) in all 3 domains on entry

0.0% 10.0% 20.0% 30.0% 40.0% 50.0% 60.0% 70.0% 80.0% 90.0% 100.0%

District A District B District C

No delay on exit Moderate delay on exit Significant delay on exit

Statewide: Children with a significant delay (<=2.0 SD) in at least one domain on entry

0.0% 10.0% 20.0% 30.0% 40.0% 50.0% 60.0% 70.0% 80.0% 90.0% 100.0%

State

No delay on exit Moderate delay on exit Significant delay on exit

3 districts: Children demonstrating a significant delay (<= -2.0 SD) in at least one domain on entry

0.0% 10.0% 20.0% 30.0% 40.0% 50.0% 60.0% 70.0% 80.0% 90.0% 100.0%

District A District B District C

No delay on exit Moderate delay on exit Significant delay on exit