Discussion and Demonstration of New Digitally Based Assessments for - - PowerPoint PPT Presentation

Discussion and Demonstration of New Digitally Based Assessments for the National Assessment of Educational Progress (NAEP)

CCSSO 2019 NCSA June 24-26

1

Presenters

• Dr. Amy Dresher

NAEP Deputy Director Educational Testing Service adresher@ets.org Jeff Ackley Science Director Center for Science & Digital Assessment Design Educational Testing Service jackley@ets.org

2

Shannon Richards NAEP Assessment Development DBA Process Manager Educational Testing Service srichards@ets.org

• Dr. Angela Hemingway

Executive Director Idaho STEM Action Center and STEM Action Center Foundation Executive Office of the Governor Angela.Hemingway@stem.idaho.gov

Agenda

• Introduction to NAEP
• Affordances of Digitally Based Assessments
• Demonstration of Discrete Items Types in Math and Science

Assessments

• Overview of Scenario-based Tasks in Science Assessments
• Affordances of DBA – a State’s Perspective
• Q&A and Discussion

3

Introduction to NAEP

What is NAEP?

• NAEP is an assessment that measures what the student in the

United States know and can do in various subjects across the nation, states, and in some urban districts.

• NAEP administered its first national assessment 50 years ago,

serving as the longest running national assessment available.

• A NAEP assessment is comprised of a variety of forms across

several subjects. Individual students complete only a subset of item in the assessment and each student represents hundreds of students.

5

Digital Transformation and Delivery

• NAEP’s history of digitally based assessments
• 2009: science interactive computer tasks
• 2011: writing
• 2014: technology and engineering literacy (TEL)
• 2015: first pilot of digitally based assessment
• 2017: DBA in mathematics and reading
• 2018: DBA in U.S history, civics, and geography
• 2019: DBA in mathematics, reading, and science

6

NAEP in the Field

7

Contextual Information

• NAEP collects contextual information about students'

demographic characteristics, opportunities to learn in and

• utside of the classroom, and educational experiences.
• Additional questions about technology usage at home and in

schools was collected starting in 2017

• 85% of students have access to computer at home
• 99% of students have access to digital devices in schools

8

NAEP by Numbers

• In total, since NAEP has been administering digital assessments,

we have assessed over 1.78 million students on laptops or tablets

• 2019 was the largest DBA administration in NAEP’s history

9

Affordances of Digitally Based Assessments

Digital Assessments

• Include interactive item types that are engaging and allow for

better measurement of the constructs

• Increase the opportunity to machine score student responses
• Provide assistive technologies not available in paper-based

assessments (PBAs) that enhance accommodations

• Parallel similar learning environments in the classrooms and

digital environments familiar to students outside the classroom

• Collect new and more meaningful data about what students

know and can do, including how students approach problem solving

11

DBA Example

12

Question ID: 2017-8M9 #4 D2357MS

13

Question ID: 2013-8M3 #3 M150001

PBA Example

DBA Tools

• Universal design tools available to all students
• Help
• Change theme
• Zoom
• Read aloud
• Scratchwork
• Other tools available per subject, form, or item type
• Calculator
• Equation editor
• Bilingual toggle

14

DBA Process Data Collected

• Timing data
• How long did students spend per item, item type, section, and

form?

• Tool usage
• Which tools did students use, how often, and for which items?
• Navigation
• Do students start with the first item and then work

sequentially, or do they answer questions out of order? If out

• f order, is there a pattern?
• Cognitive processes
• Do students partially work through the question and then

revise their response?

15

Demonstration of Discrete Item Types in Math and Science Assessments

Released Content

• Information about NAEP DBA can be found at:

https://nces.ed.gov/nationsreportcard/dba/

• To view sample DBA items:
• Go to https://nces.ed.gov/NationsReportCard/nqt/Search/SearchOptions
• Select subjects Mathematics or Reading
• Select a grade(s) of your choice
• Select year 2017 for DBA items
• Select OK
• Example Items
• Matching Item
• Grid Item
• Fill in the Blank Item

17

Calculators

18

Grade 4 Grade 8 and 12

Demonstration of Other Item Types

• Presentation of additional item types available in NAEP DBA.

19