Creating Personalized Education Systems: Growth Modeling and - - PowerPoint PPT Presentation

Xin Wei & Renée Cameto

SRI International

June 19, 2011

Creating Personalized Education Systems: Growth Modeling and Students with Disabilities

• Poor academic performance of students with

disabilities

• Lack of understanding of reading and math growth

trajectories for students with disabilities limit our ability to build sound accountability and assessment systems

• This study using SEELS data aims at building

personalized growth targets to meet high expectations held for students with different types of disabilities

2

Background

• Students with LD had lower levels of reading foundational skills

(Jenkins & O’Connor, 2002) and about 80% of them experience serious problems learning to read (Kavale & Reece, 1992).

• Students with speech/language impairments have lower average

reading scores than students without disabilities (Catts, Bridges, Little, & Tomblin, 2008).

• Students with hearing impairments also have significant challenges

with reading and are reported to have below average performance than typically developing students (Wolk & Allen, 1984).

• Students in every other disability category (e.g., mental retardation,

visual impairments, traumatic brain injury) also have documented reading scores well below their peers without disabilities (Blackorby et al., 2004).

3

Reading Achievement Level of Students with Disabilities

• Large gaps exist between students with disabilities and their peers in the

general population. 30% of students with disabilities score above the 50th percentile in mathematics calculation, whereas 40% score below the 25th percentile (Blackorby et al., 2002).

• Students with learning disabilities performed below students in the general

population but typically outperformed students with mild intellectual disability in math (Caffrey & Fuchs, 2007; Gresham, MacMillan, & Bocian, 1996).

• Students with emotional disturbance had lower academic scores than

students with learning disabilities (Sabornie, Cullinan, Osborne, & Brock, 2005).

• Students with mental retardation or multiple disabilities have the lowest

scores in calculation, with about three-fourths of them receiving scores in the lowest quartile (Blackorby et al., 2002).

4

Math Achievement Level of Students with Disabilities

Two-stage sampling procedure

• 265 LEAs and special schools (size, region, wealth)
• 11,500 students (800-950 per disability category)
• 4,000 schools
• 7,000 teachers

5

SEELS Data

6

Timing of Data Collection

Data Collection Wave 1: 2000-01 Wave 2: 2001-02 Wave 3: 2003-04 Parent interview Summer of 2000 Winter-Spring Winter-Spring Direct assessment/ student interview Winter-Spring Winter-Spring Winter-Spring Language arts teacher survey Winter-Spring Winter-Spring Winter-Spring School program survey Winter-Spring Winter-Spring Winter-Spring School characteristics survey Winter-Spring Winter-Spring Winter-Spring Transcript Winter-Spring Winter-Spring Winter-Spring

• Woodcock-Johnson III letter word identification, passage comprehension,

calculation, and math applied problems.

• W score is appropriate for measuring growth. The W score for each subtest

is centered on a mean of 500 to approximate the average performance of a 10-year-old child (Woodcock et al., 2001).

• A student’s disability categorization was obtained from school district rosters

when the sample was drawn in 1999 and included the 12 disability categories under IDEA.

• Age is measured as years and months when a student took the WJ III test.
• Gender, race, and socioeconomic status (mother’s education, father’s

education, and family income) are covariates.

• General education students growth curves were extracted from WJ III

examiner’s manual (Mather, Shrank, & Woodcock, 2007).

7

Measures

• A quadratic growth curve model
• A two-level HLM model

1. Level-1 HLM model is the within-person model, which included repeated measures of student scores in reading or math across three waves predicted by student’s age and age squared at each wave. 2. Level-2 model is the between-person model, which estimated the differences in scores between students from different disability categories, gender, SES, and race-ethnicity. 3. Intercept, age, and age squared are random effects, and disability category and other covariates are fixed effects. 4. Restricted maximum likelihood estimation with an unstructured covariance structure was specified. 5. Age was centered by mean.

8

Analysis

Reading Growth Trajectories by Disability

Math Growth Trajectories by Disability

400 410 420 430 440 450 460 470 480 490 500 510 520 530 540 7 8 9 10 11 12 13 14 15 16 17

Applied Problems

Age

400 410 420 430 440 450 460 470 480 490 500 510 520 530 540 7 8 9 10 11 12 13 14 15 16 17

Calculation

Age

Learning Disabilities Speech Impairment Intellectual Disability Emotional Disturbances Hearing Impairment Visual Impairment Orthopedic Impairment Other Health Impairment Autism Traumatic Brain Injury Multiple Disabilities General Population

• Reading and math growth rates decrease with age.
• Large variations in achievement levels across different disability categories.

Students with speech/language impairments or visual impairments exhibited performance about 1 standard deviation higher than their peers classified with intellectual disabilities or multiple disabilities.

• The slope and acceleration of growth were similar between students with

learning disabilities and students in most other disabilities categories. When examining the slope of reading achievement across disability categories at age 12.67, we found students with speech/language impairments, hearing impairments, and autism grew significantly more slowly than in students with learning disabilities on both reading measures. Autism grew significantly slower than students with learning disabilities and students with speech/language impairments decelerated significantly faster than students with learning disabilities on both math outcomes.

11

Results

• Normative growth trajectories were plotted on the previous graphs.
• Personalized growth targets take into account students age, the

complexity of the skills tested, and student background characteristics (e.g., disability type, gender, race, socioeconomic status, and ethnicity). The user can input student background characteristics into the system and get an estimated score a child should achievement if she or he is on track.

• This system plots and compares individual trajectories to

personalized growth targets, which provides states, schools and teachers information on whether or not a student is on track and can help students with disabilities to achieve the high expectations set for next generation learners.

12

Personalized Growth Target

An example

400 410 420 430 440 450 460 470 480 490 500 510 520 530 540 7 8 9 10 11 12 13 14 15 16 17

Applied Problems Age

Estimated Growth Trajectories for a White Female Child with LD from an average SES family The Actural Scores This Child Got General Population Trajectories

Additional Information

• Corresponding author: Xin Wei, PhD, Research Analyst

SRI International, 333 Ravenswood Avenue, Menlo Park, CA 94025; E-mail: xin.wei@sri.com

• Presenter: Renée Cameto, PhD, Principal Scientist, SRI

International, renee.cameto@sri.com

• Wei, X., Blackorby, J., & Schiller, E. (2011). Growth in reading

achievement of students with disabilities, ages 7 to 17. Exceptional Children.

• More information about the SEELS data can be found at

www.SEELS.net

14

Thank you!

15

Headquarters: Silicon Valley

SRI International 333 Ravenswood Avenue Menlo Park, CA 94025-3493 650.859.2000

Washington, D.C.

SRI International 1100 Wilson Blvd., Suite 2800 Arlington, VA 22209-3915 703.524.2053

Princeton, New Jersey

SRI International Sarnoff 201 Washington Road Princeton, NJ 08540 609.734.2553

Additional U.S. and international locations www.sri.com