California Assessment of Student Performance and Progress (CAASPP) - - PowerPoint PPT Presentation

▶
california assessment of student
SMART_READER_LITE
LIVE PREVIEW

California Assessment of Student Performance and Progress (CAASPP) - - PowerPoint PPT Presentation

Feb 26, 2023 141 likes •879 views

California Assessment of Student Performance and Progress (CAASPP) Accountability and Reporting Ekaterina P. Forrester Sara Shore Final Scoring For each student, the responses from the PT and CAT portions are merged for final scoring.

slide-1
SLIDE 1

California Assessment of Student Performance and Progress (CAASPP) Accountability and Reporting

Ekaterina P. Forrester Sara Shore

slide-2
SLIDE 2

Final Scoring

  • For each student, the responses from the PT and CAT portions are merged for

final scoring.

  • Resulting ability estimates are based on the specific test questions that a

student answered, not the total number of items answered correctly.

  • Higher ability estimates are associated with test takers who correctly answer

difficult and more discriminating items.

  • Lower ability estimates are associated with test takers who correctly answer

easier and less discriminating items.

  • Two students will have the same ability estimate if they have the same set of

test questions with the same responses.

  • It is possible for students to have the same ability estimate through different

response patterns

slide-3
SLIDE 3

Final Scoring: Contribution of CAT and PT Sections

Number of Items defined by Test Blueprints ELA/Literacy Mathematics Grade CAT PT CAT PT 3–5 38–41 5–6 31–34 2–6 6–8 37–42 5–6 30–34 2–6 11 39–41 5–6 33–36 2–6

slide-4
SLIDE 4

Final Scoring: Contribution of CAT and PT Sections (cont.)

  • Based on the test blueprint, the CAT section is emphasized

because there are more CAT items/points than PT items/points.

  • Claims with more items/points are emphasized.
  • Mathematics: Concepts and Procedures  Problem

Solving/Modeling and Data Analysis  Communicating Reasoning

  • ELA: Reading  Writing  Speaking/Listening  Research
  • Groups of items that are more difficult and discriminating will

have a larger contribution on final scores.

  • Therefore there is no specific weight associated with either PT
  • r CAT Sections

CDE/ETS 2015 Post-Test Workshop: Reporting Summative Assessment Results

slide-5
SLIDE 5

Smarter Balanced Scale Score Ranges by Grade Level

Grade Subject Min Max Subject Min Max 3 ELA 2114 2623 Mathematics 2189 2621 4 ELA 2131 2663 Mathematics 2204 2659 5 ELA 2201 2701 Mathematics 2219 2700 6 ELA 2210 2724 Mathematics 2235 2748 7 ELA 2258 2745 Mathematics 2250 2778 8 ELA 2288 2769 Mathematics 2265 2802 11 ELA 2299 2795 Mathematics 2280 2862

CDE/ETS 2015 Post-Test Workshop: Reporting Summative Assessment Results

slide-6
SLIDE 6

Achievement Levels

  • Achievement level classifications based on overall scores
  • Level 1—Standard Not Met
  • Level 2—Standard Nearly Met
  • Level 3—Standard Met (College and Career Readiness)
  • Level 4—Standard Exceeded

CDE/ETS 2015 Post-Test Workshop: Reporting Summative Assessment Results

slide-7
SLIDE 7

Overall Scores – 2 Areas, 4 Levels

English Language Arts and Mathematics

Nearly Met the Standard Exceeded the Standard Met the Standard Has Not Met the Standard

CDE/ETS 2015 Post-Test Workshop: Reporting Summative Assessment Results

slide-8
SLIDE 8

Achievement Levels by Grade

CDE/ETS 2015 Post-Test Workshop: Reporting Summative Assessment Results

slide-9
SLIDE 9

Smarter Balanced Scale Score Ranges for ELA/Literacy

Grade Level 1 Standard NOT Met Level 2 Standard Nearly Met Level 3 Standard Met Level 4 Standard Exceeded 3 2114–2366 2367–2431 2432–2489 2490–2623 4 2131–2415 2416–2472 2473–2532 2533–2663 5 2201–2441 2442–2501 2502–2581 2582–2701 6 2210–2456 2457–2530 2531–2617 2618–2724 7 2258–2478 2479–2551 2552–2648 2649–2745 8 2288–2486 2487–2566 2567–2667 2668–2769 11 2299–2492 2493–2582 2583–2681 2682–2795

CDE/ETS 2015 Post-Test Workshop: Reporting Summative Assessment Results

slide-10
SLIDE 10

Achievement Levels by Grade

CDE/ETS 2015 Post-Test Workshop: Reporting Summative Assessment Results

slide-11
SLIDE 11

Smarter Balanced Scale Score Ranges for Mathematics

Grade Level 1 Standard NOT Met Level 2 Standard Nearly Met Level 3 Standard Met Level 4 Standard Exceeded 3 2189–2380 2381–2435 2436–2500 2501–2621 4 2204–2410 2411–2484 2485–2548 2549–2659 5 2219–2454 2455–2527 2528–2578 2579–2700 6 2235–2472 2473–2551 2552–2609 2610–2748 7 2250–2483 2484–2566 2567–2634 2635–2778 8 2265–2503 2504–2585 2586–2652 2653–2802 11 2280–2542 2543–2627 2628–2717 2718–2862

CDE/ETS 2015 Post-Test Workshop: Reporting Summative Assessment Results

slide-12
SLIDE 12

Measurement Precision: Error Bands

  • For each scale score estimated for a student, there is

measurement error associated with each score.

  • An error band is a useful tool that describes the measurement error

associated with a reported scale score.

  • The error bands used to construct interval estimates were based
  • n one standard error of measurement.
  • If the same test is given to student multiple times, about 68 percent of the time, the student

will score within this band.

CDE/ETS 2015 Post-Test Workshop: Reporting Summative Assessment Results

slide-13
SLIDE 13

Elements of the Student Score Report

1 2 3 4 5 6 7 8

Front Page Back Page

slide-14
SLIDE 14

3

Elements of the Student Score Report

Front Page

slide-15
SLIDE 15

4

Elements of the Student Score Report

Front Page

slide-16
SLIDE 16

Achievement Levels for Claims

  • Achievement Levels for claims are very similar to subscores.

They provide supplemental information regarding a student’s strengths or weaknesses.

  • No achievement level setting occurred for claims.
  • Only three achievement levels for claims were developed since

there are fewer items within each claim.

  • Achievement levels for claims are based on the distance a

student’s performance on the claim is from the Level 3 proficiency cut.

  • A student must complete all items within a claim to receive an

estimate of his or performance on a claim.

CDE/ETS 2015 Post-Test Workshop: Reporting Summative Assessment Results

slide-17
SLIDE 17

Additional Scores – 3 levels

  • English Language Arts

1.Reading 2.Writing 3.Speaking & Listening 4.Research/Inquiry

  • Mathematics

1.Concepts & Procedures 2.Problem Solving & 4. Modeling & Data Analysis 3.Communicating Reasoning

Below Standard Above Standard At/Near Standard

Above Standard At/Near Standard Below Standard

slide-18
SLIDE 18

5

Elements of the Student Score Report

Back Page

slide-19
SLIDE 19

6

Elements of the Student Score Report

Back Page

slide-20
SLIDE 20

7

Elements of the Student Score Report

Back Page

slide-21
SLIDE 21

Elements of the Student Score Report: Science Grades 5, 8, & 10 only

8

Back Page

slide-22
SLIDE 22

Elements of the Student Score Report: Early Assessment Program Grade 11 only

8

Back Page

slide-23
SLIDE 23

23

42 45 27 41 34 15 40 36 13 36 42 16 28 32 35 27 40 26 22 42 34 30 34 33 19 17 25 20 19 30 26 19 31 25 17 33 11 6 13 12 8 30 12 4 22 9 7 18

10 20 30 40 50 60 70 80 90 100

SBAC 2015 CST 2003 CST 2013 SBAC 2015 CST 2003 CST 2013 SBAC 2015 CST 2003 CST 2013 SBAC 2015 CST 2003 CST 2013 Grade 3 Grade 4 Grade 5 Grade 6

ELA Percent of Students in Each Achivement Level SBAC 2015, CST 2003, 2013

Below Basic Basic Proficient Advanced

slide-24
SLIDE 24

24

37 43 17 33 45 11 28 45 25 25 35 31 34 36 33 30 29 34 30 18 34 28 16 33 30 20 29 8 4 18 5 3 23 12 6 13

10 20 30 40 50 60 70 80 90 100

SBAC 2015 CST 2003 CST 2013 SBAC 2015 CST 2003 CST 2013 SBAC 2015 CST 2003 CST 2013 Grade 7 Grade 8 Grade 11

ELA Percent of Students in Each Achivement Level SBAC 2015, CST 2003, 2013

Below Basic Basic Proficient Advanced

slide-25
SLIDE 25

25

40 35 18 34 35 13 50 50 21 43 41 22 29 26 23 39 30 20 25 27 20 32 33 29 23 25 28 19 24 30 13 19 31 15 20 30 8 14 31 8 11 38 12 5 28 10 6 20

10 20 30 40 50 60 70 80 90 100

SBAC 2015 CST 2003 CST 2013 SBAC 2015 CST 2003 CST 2013 SBAC 2015 CST 2003 CST 2013 SBAC 2015 CST 2003 CST 2013 Grade 3 Grade 4 Grade 5 Grade 6

Mathematics Percent of Students in Each Achivement Level SBAC 2015, CST 2003, 2013

Below Basic Basic Proficient Advanced

slide-26
SLIDE 26

26

45 46 20 48 38 30 53 64 58 30 34 28 29 33 30 27 27 29 16 17 35 14 26 31 17 8 12 8 3 17 9 3 10 3 1 1

10 20 30 40 50 60 70 80 90 100

SBAC 2015 CST 2003 CST 2013 SBAC 2015 CST 2003 CST 2013 SBAC 2015 CST 2003 CST 2013 Grade 7 Grade 8 Grade 11

Mathematics Percent of Students in Each Achivement Level SBAC 2015, CST 2003, 2013

Below Basic Basic Proficient Advanced

slide-27
SLIDE 27

SBAC Claims and Sample Items

New Ways of Assessing Learning

27

slide-28
SLIDE 28

28

slide-29
SLIDE 29

KDS’ SBAC Claims, Targets, Standards Alignment http://www.keydatasys.com/

29

slide-30
SLIDE 30

Finding Claim, Target, DOK, Standards Tables Google: Claims Targets Standards Riverside or go to KeyDataSys.com under Educational Resources

slide-31
SLIDE 31

Assessment Claims for ELA

  • Overall ELA Claim (Grades 3-8)
  • “Students can demonstrate progress toward college

and career readiness in English language arts and literacy.”

  • Overall ELA Claim (High School)
  • “Students can demonstrate college and career

readiness in English language arts and literacy.”

31 http://www.smarterbalanced.org/wordpress/wp-content/uploads/2012/09/Smarter-Balanced-ELA-Literacy- Claims.pdf

slide-32
SLIDE 32

Content and Reporting Claims for ELA

  • Claim #1 Reading – Students can read closely and analytically

to comprehend a range of increasingly complex literary and informational texts.

  • Claim #2 Writing – Students can produce effective and well-

grounded writing for a range of purposes and audiences.

  • Claim #3 Speaking/Listening – Students can employ effective

speaking and listening skills for a range of purposes and audiences.

  • Claim #4 Research– Students engage in research to investigate

topics, and to analyze, integrate, and present information.

32 http://www.smarterbalanced.org/wordpress/wp-content/uploads/2012/09/Smarter-Balanced-ELA- Literacy-Claims.pdf

slide-33
SLIDE 33

Claim 1: Reading

  • Literacy instruction that centers on careful

examination of texts

  • Reading closely and drawing evidence from the text to

support inferences and judgments made

  • Variety of text types
  • Students should be able to
  • Locate key details and summarize central ideas
  • Use textual evidence to analyze and support judgments

made about the ideas presented

  • Refer to details and examples in support of claims,

inferences, and conclusions

slide-34
SLIDE 34

Claim 1: Sample SBAC Items

slide-35
SLIDE 35
slide-36
SLIDE 36

36

slide-37
SLIDE 37

Claim 2: Writing

  • Writing activities and assignments
  • Designed with genuine purposes and audiences in mind
  • from teachers and other students to community groups, local or

national officials, commercial interests, students’ friends and relatives, and other potential readers

  • Students should be able to
  • Work with – compose, revise, and/or edit - a variety of shorter

and longer literary and informational texts for different purposes

  • Narrative, about real or imaginary experiences or events
  • Informational/explanatory texts, writing opinions/arguments

about a topic

  • Opinions/arguments in response to texts read (either fiction or

nonfiction)

slide-38
SLIDE 38

Claim 2: Sample SBAC Items

slide-39
SLIDE 39
slide-40
SLIDE 40

Claim 3: Listening and Speaking

  • Communicate effectively – demonstrating active listening,

interpersonal communication, and the ability to integrate

  • ral/visual/graphic information.
  • Children’s listening comprehension outpaces reading

comprehension until the middle school years (grades 6–8)

  • Sticht and James (1984)
  • Students should be able to
  • Listen to/view a variety of non-print texts, such as following

directions or procedures in a simulation or hands-on task, or view demonstrations, lectures, media messages, speeches, etc.

  • Respond to comprehension- and integration/analysis–type

questions (similar to the selected response and open response questions described for reading Claim #1).

slide-41
SLIDE 41

Claim 3: Sample SBAC Items

slide-42
SLIDE 42
slide-43
SLIDE 43

Claim 4: Research/Inquiry

  • Filter information, evaluate the credibility of

sources, detect and challenge the underlying assumptions, and make thoughtful decisions based on their analysis of what is relevant to the topic, issue, or problem being explored.

  • What students should be able to do
  • Apply literacy skills across content areas -

history/social studies, science, and technical subjects, as well as the language arts

slide-44
SLIDE 44

Claim 4: Sample SBAC Items

slide-45
SLIDE 45
slide-46
SLIDE 46
slide-47
SLIDE 47

Assessment Claims for Math

  • Overall math Claim (Grades 3-8)
  • “Students can demonstrate progress toward college

and career readiness in mathematics.”

  • Overall math Claim (High School)
  • “Students can demonstrate college and career

readiness in mathematics.”

47

http://www.smarterbalanced.org/wordpress/wp-content/uploads/2012/09/Smarter-Balanced-Mathematics- Claims.pdf

slide-48
SLIDE 48

Content and Reporting Claims for Math

  • Claim #1 Concepts & Procedures – Students can explain and

apply mathematical concepts and interpret and carry out mathematical procedure with precision and fluency.

  • Claim #2 Problem Solving– Students can solve a range of

complex and well-posed problems in pure and applied mathematics, making productive use of knowledge and problem solving strategies and Claim #4 Modeling and Data Analysis –Students can analyze complex, real-world scenarios and can construct and use mathematical models to interpret and solve problems.

  • Claim #3 Communicating Reasoning – Students can clearly

and precisely construct viable arguments to support their own reasoning and critique the reasoning of others.

48

http://www.smarterbalanced.org/wordpress/wp-content/uploads/2012/09/Smarter-Balanced- Mathematics-Claims.pdf

slide-49
SLIDE 49

Math Claim 1: Concepts and Procedures

  • Math instruction that is centered on particular

mathematical skills or concepts

  • Explain and carry out mathematical skills fluently
  • Student should be able to
  • Understand how concepts relate to one another and look

for patterns (SMP, 7)

  • Know why math procedures work the way they do (SMP,

8)

  • Fluently (accurately and efficiently) do math (SMP, 6)
  • Attend to precision (SMP, 6)

49

slide-50
SLIDE 50

Claim 1: Sample SBAC Items

50

slide-51
SLIDE 51

51

slide-52
SLIDE 52

52

slide-53
SLIDE 53

Math Claim 2 & 4: Problem Solving and Modeling and Data Analysis

  • Claim 2
  • Number sentences and word problems
  • Students should be able to
  • Come up with a solution to a problem that

is not evident (SMP, 2)

  • Construct their own solution pathway
  • Preserver in solving the problem (SMP, 1)

53

slide-54
SLIDE 54

Math Claim 2 & 4: Problem Solving and Modeling and Data Analysis

  • Claim 4
  • Bridges the gap from “school math” to “real world” math
  • Students are able to
  • Do math when they’re not told what math to do
  • Make assumptions and simplifications (SMP, 2)
  • Select appropriate information
  • Know what data are missing
  • Model with mathematics (SMP, 4)

54

slide-55
SLIDE 55

Claim 2: Sample SBAC Items

55

slide-56
SLIDE 56

56

slide-57
SLIDE 57

57

slide-58
SLIDE 58

58

slide-59
SLIDE 59

Math Claim 3: Communicating Reasoning

  • Instruction is based on allowing students to communicate

their reasoning in math and critic/evaluate the reasoning

  • f others
  • Student are able to
  • Construct and present a clear, logical and convincing

argument

  • Provide justification and explanation, or counter-example
  • Identify flaws or gaps in reasoning
  • Distinguish correct logic or reasoning

59

slide-60
SLIDE 60

Claim 3: Sample SBAC Items

60

slide-61
SLIDE 61

61

slide-62
SLIDE 62

62

slide-63
SLIDE 63

63

slide-64
SLIDE 64

SBAC Claim Level Analysis

64

47 46 29 38 43 37 25 32 42 39 28 25 45 42 24 25 39 43 61 50 44 50 65 54 44 44 61 54 44 46 66 60 14 11 10 12 13 13 10 14 14 17 11 21 11 12 10 15 10 20 30 40 50 60 70 80 90 100

Reading Writing Listening and Speaking Research/Inquiry Reading Writing Listening and Speaking Research/Inquiry Reading Writing Listening and Speaking Research/Inquiry Reading Writing Listening and Speaking Research/Inquiry Grade 3 Grade 4 Grade 5 Grade 6

SBAC 2015 ELA Claim Level Analysis

Below At or Near Above

slide-65
SLIDE 65

SBAC Claim Level Analysis

65

43 36 28 27 41 40 29 30 27 33 24 21 44 46 64 55 45 49 64 57 53 49 64 58 14 18 8 18 14 11 7 13 20 18 12 21

10 20 30 40 50 60 70 80 90 100

Reading Writing Listening and Speaking Research/Inquiry Reading Writing Listening and Speaking Research/Inquiry Reading Writing Listening and Speaking Research/Inquiry Grade 7 Grade 8 Grade 11

SBAC 2015 ELA Claim Level Analysis

Below At or Near Above

slide-66
SLIDE 66

SBAC Claim Level Analysis

66

47 46 33 52 48 43 59 53 46 56 45 41 36 42 55 33 41 45 26 36 43 30 46 49 17 12 12 16 11 12 15 11 11 14 9 10

10 20 30 40 50 60 70 80 90 100

Concepts and Procedures Problem Solving/Data Analysis Communicating Reasoning Concepts and Procedures Problem Solving/Data Analysis Communicating Reasoning Concepts and Procedures Problem Solving/Data Analysis Communicating Reasoning Concepts and Procedures Problem Solving/Data Analysis Communicating Reasoning Grade 3 Grade 4 Grade 5 Grade 6

SBAC 2015 Mathematics Claim Level Analysis

Below At or Near Above

slide-67
SLIDE 67

SBAC Claim Level Analysis

67

55 48 21 54 38 39 54 39 36 30 43 69 33 52 51 37 53 56 15 9 10 13 10 10 9 8 8

10 20 30 40 50 60 70 80 90 100

Concepts and Procedures Problem Solving/Data Analysis Communicating Reasoning Concepts and Procedures Problem Solving/Data Analysis Communicating Reasoning Concepts and Procedures Problem Solving/Data Analysis Communicating Reasoning Grade 7 Grade 8 Grade 11

SBAC 2015 Mathematics Claim Level Analysis

Below At or Near Above

slide-68
SLIDE 68

68

slide-69
SLIDE 69

Additional Resources For Communicating Scores

http://www.cde.ca.gov/ta/tg/ca/communicationskit.asp

69

slide-70
SLIDE 70

Communications Toolkit

  • Short documents, in English and Spanish: key topics such as “Creating a Computer

Adaptive Test” or “Accessibility and Accommodations: Addressing the Needs of all Students”

  • Links to key sites such as the California PTA
  • Brief videos, in English and Spanish: key topics such as What are the Smarter

Balanced Assessments? and Ready. Set. Succeed: Preparing Our Kids for College and Career

http://www.cde.ca.gov/ta/tg/ca/communicationskit.asp

70

slide-71
SLIDE 71

Communications Toolkit (Cont.)

  • Sample parent and guardian letter to accompany the Individual Student Report
  • Reading Your Student Report, in multiple languages, to help parents and guardians read

and interpret the Individual Student Report

  • Documents that include released questions that exemplify items in the Smarter

Balanced assessments to help parents/guardians understand the achievement levels

  • Short video to help parents/guardians understand the Individual Student Report

http://www.cde.ca.gov/ta/tg/ca/communicationskit.asp

  • Understanding your child’s score report

https://www.youtube.com/watch?v=FQi4qlOCrmk&index=1&list=PLgIRGe0- q7SZtZI9b50s4TLpy1_LPf9tP

71

slide-72
SLIDE 72

Ekaterina@KeyDataSys.com Sara@KeyDataSys.com

72