Marshalltown Dual Language Program Evaluation Group 1: DLP: Yes and - - PowerPoint PPT Presentation

▶

Jul 27, 2023 252 likes •414 views

Marshalltown Dual Language Program Evaluation Group 1: DLP: Yes and Ever an ELL: Yes Group 2: DLP: No and Ever an ELL: Yes Group 3: DLP: Yes and Never an ELL Group 4: DLP: No and Never an ELL Guide for Reporting Rigorous

SLIDE 1

Marshalltown

Dual Language Program Evaluation
Group 1: DLP: Yes and Ever an ELL: Yes
Group 2: DLP: No and Ever an ELL: Yes
Group 3: DLP: Yes and Never an ELL
Group 4: DLP: No and Never an ELL

SLIDE 2

Guide for Reporting Rigorous Evaluations

US Department of Education
Equivalent Control Groups and Treatment Groups Guidelines
All Experimental and Control Groups across all cohort groups met the

requirements for having equivalent baseline scores on the Iowa Assessments in third grade in Reading Comprehension.

Groups 1 (N=466) and 2 (N=1,370) across all 8 cohorts
Groups 3 (N= 85) and 4 (N=1,698) across all 8 cohorts

SLIDE 3

Null Hypothesis

There is no significant difference between cohorts of Group 1

students and cohorts of Group 2 students scores on the Iowa Assessment in:

Reading comprehension,
Mathematics, and
Science.
.05 level

SLIDE 4

Iowa Assessment: Reading Group 1 Group 2 Dual Language Yes Ever in an ELL Program Yes Dual Language No Ever in an ELL Program Yes School Year Grade N Mean NCE Standard Deviation Mean NCE Standard Deviation Significant Difference Mean NCE Difference Effect Size 2017-2018 4th Grade 54 34.57 16.16 36.16 15.06 No

1.59

2017-2018 5th Grade 39 46.73 16.70 43.00 18.00 No 3.73 2017-2018 6th Grade 58 40.60 16.97 43.96 18.05 No

3.36

2017-2018 7th Grade 48 48.08 19.02 48.55 20.13 No

0.47

2017-2018 8th Grade 37 46.64 17.47 44.64 18.64 No 2.00 2017-2018 9th Grade 39 53.28 13.70 51.17 15.81 No 2.11 2017-2018 10th Grade 50 57.20 15.56 48.58 14.06 Yes 8.62 0.61 2017-2018 11th Grade 43 43.00 10.64 43.50 13.90 No

0.50

Average difference: 1.31

SLIDE 5

Iowa Assessment: Mathematics Group 1 Group 2 Dual Language Yes Ever in a ELL Program Yes Dual Language No Ever I an ELL Program Yes School Year Grade N Mean NCE Standard Deviation Mean NCE Standard Deviation Significant Difference Mean NCE Difference Effect Size 2017-2018 4th Grade 54 35.77 15.13 37.85 15.02 No

2.08

2017-2018 5th Grade 37 34.54 15.25 36.08 18.18 No

1.54

2017-2018 6th Grade 58 36.36 16.28 42.08 13.91 Yes

5.72

0.39 2017-2018 7th Grade 48 47.79 14.28 46.47 17.38 No 1.32 2017-2018 8th Grade 37 44.70 15.74 43.16 14.74 No 1.54 2017-2018 9th Grade 42 47.11 17.36 49.80 15.67 No

2.69

2017-2018 10th Grade 49 52.11 16.64 45.03 15.75 Yes 7.08 0.45 2017-2018 11th Grade 38 50.18 12.40 48.28 15.04 No 1.90 Average difference: 0.02

SLIDE 6

Iowa Assessments: Science Group 1 Group 2 Dual Language Yes Ever in an ELL Program Yes Dual Language No Ever in an ELL Program Yes School Year Grade N Mean NCE Standard Deviation Mean NCE Standard Deviation Significant Difference Mean NCE Difference Effect Size 2017-2018 4th Grade 54 40.55 14.40 42.29 12.06 No

1.74

2017-2018 5th Grade 37 43.27 12.87 39.24 14.06 No 4.03 2017-2018 6th Grade 58 37.25 17.24 39.08 13.86 No

1.83

2017-2018 7th Grade 48 50.45 13.42 47.77 15.95 No 2.68 2017-2018 8th Grade 37 51.78 18.61 50.75 13.89 No 1.03 2017-2018 9th Grade 42 48.83 15.42 48.83 16.69 No 0.00 2017-2018 10th Grade 50 52.78 16.23 47.32 1.90 Yes 5.46 0.42 2017-2018 11th Grade 39 45.97 17.17 47.56 14.61 No

1.59

Average difference: 1.01

SLIDE 7

Conclusions Group 1 and Group 2

Iowa Assessment scores in Reading:
No Clear Differences or Patterns Across Cohorts
Except 2017-2018 10th Grade Cohort
Iowa Assessment scores in Mathematics
No Clear Differences or Patterns Across Cohorts
Except 2017-2018 10th Grade Cohort
Iowa Assessment scores in Science:
No Clear Differences or Patterns Across Cohorts
Except 2017-2018 10th Grade Cohort

SLIDE 8

Conclusions 2017-2018 10th Grade Group 1 and Group 2

Significant Differences
2017-2018 10th Grade (N=50) Cohort Group 1
Reading: Significant difference
Mean NCE difference= 8.62
Effect size = .61
Mathematics: Significant difference
Mean NCE difference = 5.46
Effect size = .42
Science: Significant difference
Mean NCE difference = 7.08
Effect size = .45

SLIDE 9

Group 1 Group 2 Dual Language Program Yes Ever in an ELL Program Yes Dual Language Program No Ever in an ELL Program YES School Year Grade Significantly Different Percentage Not Significantly Different Percentage Total 2017-2018 11th 0.0% 10 100.0% 10 *2017-2018 10th 7 33.3% 14 66.7% 21 2017-2018 9th 0.0% 8 100.0% 8 2017-2018 8th 1 14.3% 6 85.7% 7 2017-2018 7th 0.0% 6 100.0% 6 2017-2018 6th 1 20.0% 4 80.0% 5 2017-2018 5th 0.0% 4 100.0% 4 2017-2018 4th 0.0% 4 100.0% 4 Across all Grades & Subjects 9 13.8% 56 86.2% 65

SLIDE 10

Null Hypothesis

There is no significant difference between cohorts of Group 3

students and cohorts of Group 4 students scores on the Iowa Assessment in:

Reading comprehension,
Mathematics, and
Science.
.05 level

SLIDE 11

Iowa Assessment: Reading Group 3 Group 4 Dual Language Yes Never an ELL Dual Language No Never an ELL School Year Grade N Mean NCE Standard Deviation Mean NCE Standard Deviation Significant Difference Mean NCE Difference Effect Size 2017-2018 4th Grade 16 45.93 24.39 45.75 23.92 No 0.18 2017-2018 5th Grade 6 55.00 31.36 38.66 25.08 No 16.34 2017-2018 6th Grade 7 58.42 22.75 51.85 19.59 No 6.57 2017-2018 7th Grade 9 64.77 24.65 48.33 14.97 No 16.44 2017-2018 8th Grade 6 49.55 13.01 47.00 11.15 No 2.55 2017-2018 9th Grade 12 61.75 22.65 50.66 17.71 No 11.09 2017-2018 10th Grade 5 61.80 13.66 54.60 16.19 No 7.20 2017-2018 11th Grade 4 74.25 16.64 60.25 14.33 No 14.00 Average difference: 9.3

SLIDE 12

Iowa Assessment: Mathematics Group 3 Group 4 Dual Language Yes Never an ELL Dual Language No Never an ELL School Year Grade N Mean NCE Standard Deviation Mean NCE Standard Deviation Significant Difference Mean NCE Difference Effect Size 2017-2018 4th Grade 16 42.62 21.30 34.68 15.64 No 7.94 2017-2018 5th Grade 6 44.00 18.18 42.33 26.84 No 1.67 2017-2018 6th Grade 7 58.85 19.31 48.57 15.59 No 10.28 2017-2018 7th Grade 9 55.66 17.08 48.33 16.19 Yes 7.33 0.45 2017-2018 8th Grade 6 46.67 11.25 49.00 13.76 No

2.33

2017-2018 9th Grade 12 53.00 17.49 50.41 18.03 No 2.59 2017-2018 10th Grade 5 55.40 27.57 58.60 10.25 No

3.20

2017-2018 11th Grade 4 73.00 11.28 65.75 17.63 No 7.25 Average difference: 3.94

SLIDE 13

Iowa Assessment: Science Experimental Group 3 Control Group 4 Dual Language Yes Never an ELL Dual Language No Never an ELL School Year Grade N Mean NCE Standard Deviation Mean NCE Standard Deviation Significant Difference Mean NCE Difference Effect Size 2017-2018 4th Grade 16 52.18 18.03 45.00 24.62 No 7.18 2017-2018 5th Grade 6 54.00 16.52 44.00 16.66 No 10.00 2017-2018 6th Grade 7 48.57 15.39 51.28 14.27 No

2.71

2017-2018 7th Grade 8 57.00 22.23 50.87 11.38 No 6.13 2017-2018 8th Grade 6 49.17 12.41 61.50 16.48 No

12.33

2017-2018 9th Grade 12 57.00 22.63 46.91 16.90 No 10.09 2017-2018 10th Grade 5 60.00 9.94 57.20 9.98 No 2.80 2017-2018 11th Grade 4 74.25 16.64 60.25 14.33 No 14.00 Average difference: 4.4

SLIDE 14

Group 3: DLP Yes, Never an ELL Group 4: DLP No, Never an ELL

Iowa Assessment scores in Reading:
No Significant Differences Across the 8 Cohorts
Small N Sizes
When combining all cohorts together there is a significant difference favoring DLP
Iowa Assessment scores in Mathematics
Seven No Significant Differences Across the 8 Cohorts
One Significant Difference across the 8 Cohorts favoring DLP
Small N Sizes
When combining all cohorts together there is a significant difference favoring DLP
Iowa Assessment scores in Science:
No Significant Differences Across the 8 Cohorts
Small N Sizes
When combining all cohorts together there is no significance difference

SLIDE 15

Group 3 Group 4 Dual Language Yes Never an ELL Dual Language No Never an ELL School Year Subjects Grades N Mean NCE Standard Deviation Mean NCE Standard Devaition Significant Difference Sig. Mean NCE Difference Effect Size 2017- 2018 Reading 4th - 6th 28 52.64 23.7 46.53 22.91 No 0.109 6.11 2017- 2018 Math 4th - 6th 28 48.03 20.16 41.03 17.32 No 0.091 7 2017- 2018 Science 4th - 6th 28 52.64 16.13 47.96 18.97 No 0.247 4.68 2017- 2018 Reading 7th - 11th 36 60.6 19.67 50.22 14.44 Yes 0.001 10.38 0.72 2017- 2018 Math 7th - 11th 36 55.16 18.14 52.5 16.16 No 0.279 2.66 2017- 2018 Science 7th - 11th 35 58.05 19.29 53.31 14.97 No 0.186 4.74 Average difference: 5.92

SLIDE 16

Conclusions

Combining Cohorts across Groups 3 (DLP but Never and ELL ) and

Combining Cohorts across Groups 4 (Not in the DLP and Never an ELL)

There were two significant differences favoring students in the Dual Language

Program, one in reading and one in mathematics

In the 30 significance tests ran two (7%) revealed a significant difference

between groups 3 and 4, favoring Group 3 while (93%) revealed no significant difference between groups in reading, math and science scores on the Iowa Assessments.