Language Learners in New York City Jimmy J. Han Lisa Lin Doris - - PowerPoint PPT Presentation

Beyond “English Only”: Assessing and Supporting English Language Learners in New York City

Jimmy J. Han Lisa Lin Doris Zahner, PhD June 28, 2018

INTRODUCTION & OVERVIEW

2

Introductions

Moderator: Timothy Boals, PhD, WIDA Consortium Presenters: Jimmy J. Han, New York City Department of Education Doris Zahner, PhD, Council for Aid to Education Lisa Lin, New York City Department of Education

3

Overview

NYCDOE’s Office of Periodic Assessment (OPA) Council for Aid to Education (CAE) Case Studies Q & A

4

About NYCDOE’s OPA

5

OPA’s mission is to provide a portfolio of high-quality assessment tools to support educators in reflecting

• n information about their students' learning, and in

encouraging collaborative conversation about teaching and planning through the use of common grade-level assessments.

• Develop a diverse portfolio of resources that are aligned to a

variety of content area standards and courses

• Provide access to data tools that allow for clear, readily

available information on what students know and are able to do in order to inform reflection, planning, and instruction

• Increase assessment literacy to improve strategic

assessment planning and to limit the testing burden on schools and students

Assessment Framework

6

Strong assessment practice includes formal and informal assessment moments: Baseline assessments gauge students’ starting point at the beginning of a year, course, or unit. Curriculum-embedded assessments and checks for understanding are used on an ongoing basis to provide teachers with immediate feedback on student learning. Benchmark assessments gauge students’ mastery of key concepts and standards over time throughout the school year to inform instructional, curricular, and professional development decisions. End-of-Year assessments gauge students’ ending point at the culmination

• f the school year.

About CAE

7

1952 CAE, a nonprofit organization based in New York City, is founded. 1996 CAE becomes part of the RAND Corporation and begins developing performance-based assessments to measure critical-thinking skills. 2002 CAE launches the Collegiate Learning Assessment (CLA). 2005 CAE becomes an independent organization to focus on assessment. 2015 CAE begins partnership with NYCDOE to develop assessments for ELLs.

CASE STUDY #1: ELL BASELINE ASSESSMENT

8

Context

ELL baseline assessment is used to:

• Assess students’ ability to engage with academic

language that is embedded in content rather than assessing their comprehension of academic content itself

• Provide teachers with detailed information about their

students’ strengths and needs in English language development

9

Assessment Design

10

Booklet 1: Speaking

Theme 1 Theme 2 Items/tasks 1-4 Items/tasks 5-8 Item levels: 2, 3, 4, 5 Item levels: 2, 3, 4, 5

Booklet 2: Listening, Reading, and Writing (Session 1) (Related to Theme 1 in Speaking test)

Listening Reading Writing Stimulus 1 with graphic Items 1-4 Item levels: 1, 1, 2, 3 Stimulus 1 with graphic Items 1-4 Item levels: 1, 1, 2, 3 Stimulus (from Reading) Short constructed response Levels: 1-5 Stimulus 2 with graphic Items 5-8 Item levels: 1, 2, 3, 4 Stimulus 2 with graphic Items 5-8 Item levels: 1, 2, 3, 4

Booklet 3: Listening, Reading, and Writing (Session 2) (Related to Theme 2 in Speaking test)

Listening Reading Writing Stimulus 1 with graphic Items 1-4 Item levels: 2, 2, 3, 4 Stimulus 1 with graphic Items 1-4 Item levels: 2, 2, 3, 4 Stimulus (from Reading) Extended constructed response Levels: 1-5 Stimulus 2 with graphic Items 5-8 Item levels: 3, 4, 5, 5 Stimulus 2 with graphic Items 5-8 Item levels: 3, 4, 5, 5

2016 SPEAKING Performance Level Descriptions – Grades 3–4

GENERAL CLAIM 1

Entering Emerging Transitioning Expanding Commanding

Students can participate in discussions about academic topics.

Anchor 2 Student does not respond or uses words or short phrases to describe

• r convey some

details and narrate a story or process in sequence. Response may contain errors that totally obscure meaning. Student uses phrases and simple sentences to describe or convey some relevant details and partially narrate a story or process in sequence. Response may contain errors in words and structure that mostly obscure meaning. Student uses simple and/or expanded sentences to describe or convey relevant details and narrate a story

• r process in

sequence. Response may contain errors in words and structure that partially obscure meaning. Student uses simple, expanded, and/or complex sentences to describe or convey relevant details and narrate a story

• r process in

sequence. Response may contain some errors in words and structure that minimally obscure meaning. Student uses simple, expanded, and complex sentences and fluid language to describe or convey relevant details and narrate a story

• r process in

sequence. Response contains few or no errors in word choice and structure that

• bscure meaning.

Students can use grade- appropriate language to describe information on a topic or text, with or without graphics. Target of Measurement 2 Students can use grade- appropriate language to describe or convey relevant details and narrate a story

• r process in sequence.

11

Teacher Script Introduction: There are many people in the school cafeteria. Model: A boy gets food. Prompt: What are some of the things you see happening in the cafeteria? Optional Rephrasing: I see a boy getting food. Describe

• ther things that you see

happening in the cafeteria.

12

13

Grades 3-12 ELL Baseline Speaking Rubric

Scoring

• The receptive modalities (Listening and Reading) are

assessed through selected-response (multiple-choice and multiple-select) items and a raw score is provided for both modalities.

• The productive modalities (Speaking and Writing) are

assessed through constructed-response prompts and teachers use rubrics to provide facet scores for both modalities.

14

Psychometrics

Form A NYSESLAT Corr Speaking Degree of Response with NYSESLAT Speaking .45 Speaking Mechanics with NYSESLAT Speaking .45 Speaking Complexity of Language with NYSESLAT Speaking .45 Speaking Degree of Response with NYSESLAT Total .49 Speaking Mechanics with NYSESLAT Total .47 Speaking Complexity of Language with NYSESLAT Total .47 Speaking Degree of Response with NYSESLAT Performance Level .51 Speaking Mechanics with NYSESLAT Performance Level .49 Speaking Complexity of Language with NYSESLAT Performance Level .49

15

Challenges of Development

Typical ELA comprehension items focus on the meaning of the text:

• What is the main

idea of this story?

• Why does the

character take this action?

• What is the tone of

the story? ELL Baseline Assessment items focus on specific pieces of language that convey meaning:

• Which sentence

states a main idea of the story?

• What phrase shows

that the character is happy?

16

2016 LISTENING Performance Level Descriptions – Grades 3-4

GENERAL CLAIM 1

Entering Emerging Transitioning Expanding Commanding

Students can determine information in grade-level spoken discourse. Anchor 2 Student may, with substantial support, identify a few words, short phrases, or predictable sentences that signal or describe key details, sequence, connections, and/or relationships in grade-level spoken discourse. Student can, with moderate support, identify some words, phrases, or a few simple sentences that signal or describe key details, sequence, connections, and/or relationships in grade-level spoken discourse. Student can, with limited support, identify most phrases, simple sentences, or a few expanded or complex sentences that signal or describe key details, sequence, connections, and/or relationships in grade-level spoken discourse. Student can identify most simple or some expanded or complex sentences that signal or describe key details, sequence, connections, and/or relationships in grade-level spoken discourse. Student can identify a variety of simple, expanded, or complex sentences that signal or describe key details, sequence, connections, and/or relationships in grade-level spoken discourse. Students can identify words, phrases, and sentences used to elaborate on and connect ideas in grade-level spoken discourse. Target of Measurement 2 Students can identify words, phrases, or sentences that signal or describe key details, sequence, connections, and/or relationships in grade-level spoken discourse.

17

Example of a Listening Item

18

CASE STUDY #2: ELL BENCHMARK MICRO-ASSESSMENTS BUNDLE

19

Context

ELL Benchmark Micro-Assessments Bundle is used to:

• Evaluate students’ progress toward acquiring and

using academic language when engaging in subject- area tasks

• Provide English as a New Language (ENL) and

content-area teachers with a model for differentiating activities across three bands of English language proficiency

• Offer teachers single-class period, engaging

performance assessments in ELA, Math, Science, and Social Studies

20

Summarize Causes and Effects of Erosion Using Linking Words and Phrases Describe the Formation of Different Types of Rocks Using Correct Regular and Irregular Past- tense Verbs Write Captions for Illustrations

• f a Scientific

Pattern Using Prepositional Phrases

Present Integrated Information from a Scientific Text and Infographic

• r Map Using

Proper Citation Structure

Grade 4 Science Bundle Unit 4.4: Changes on The Earth’s Surface

A B C D

21

3-Part Structure

Part 1 Language

Part 2 Compre- hension

Part 3 Functional Integration Identifying Linking Words and Phrases in a Passage Students read a passage about erosion and underline linking words and phrases, checking comprehension with peers as desired (NLAP RI.4.3 Linguistic Demands). Understanding Causes and Effects Students answer multiple-choice questions about causes and effects related to erosion (RI.4.3). Summarizing Causes and Effects Using Linking Words Students write a short summary of the causes and effects described in the passage, using linking words as appropriate (RI.4.3, W.4.2.c, NLAP RI.4.3 Linguistic Demands).

22

Band A (Entering – Emerging)

Wind also causes erosion. Light wind lifts sand. Strong wind can move rocks and trees. [Drawing: Tree tossing in the wind; wind lifting a broken branch into the air] Erosion can be caused by water, ice, and ________. A. rocks B. wind C. hills D. sand

23

Band B (Transitioning)

Wind also causes erosion. Light wind lifts sand. Over time, even a gentle breeze can move a lot of sand. Strong winds can move rocks and trees. Since the soil that is left is loose, it blows or washes away easily. What happens when strong winds pull up plants and trees? A. Soil gets filled with pebbles. B. Rivers move large rocks. C. Loose soil erodes. D. The soil expands.

24

Band C (Expanding – Commanding)

Wind can also move earth material in different ways and at different rates. A light wind lifts small particles of sand or dirt. Nevertheless, given enough time, even gentle breezes can move large amounts of earth a great distance. Stronger winds can move pebbles and rocks; they can also pull up plants and trees, leaving loose soil that is easily eroded. Which is an example of erosion happening slowly? A. a rainstorm washing away a section of a hill B. light wind moving particles of sand C. winter damage to a road surface D. a strong wind pulling up trees and loosening soil

25

Scoring

• Scoring is based on a 3-point system. Each point

represents both a statement of the amount of student understanding and the amount of future teacher instruction needed for each assessed skill.

26

3-Point System

Points Descriptor 1 Students exhibit little to no understanding

• f the assessed skill and require

additional explicit instruction when they... 2 Students exhibit understanding of the skill but would benefit from additional practice when they… 3 Students exhibit basic proficiency around the assessed skill when they…

27

Linguistic Proficiency

• When scoring students, teachers need to consider

the expected level of linguistic proficiency for each band based on the New York State Performance Level Descriptors (PLDs).

• Teachers use the Linguistic Demands Table to

understand the amount of expected language students will produce when responding at their current level.

28

Linguistic Demands Table

At the specified proficiency band, students should be able to: A Use words and phrases in order to… B Reference words and short or long phrases in order to… C Use their knowledge of the topic and language in order to…

29

Checklist

• The checklist scores the domains of Content

Comprehension, Language, and Functional Integration.

• The checklist highlights which parts assess each skill.

30

Content Comprehension (RI.4.3) Language (NLAP RI.4.3 Linguistic Demands) Writing (W.4.2.c) Student Name Parts 1 & 3 Parts 2 & 3 Part 3

CASE STUDIES #3-4: SPANISH & CHINESE LANGUAGE ARTS ASSESSMENTS

31

Context

32

Spanish and Chinese Language Arts assessments are used to:

• Provide teachers with information about their students’

strengths and needs in Spanish or Chinese academic language literacy skills

• Serve as a resource to help plan individual and group

instruction in Spanish or Chinese bilingual programs

SLA Assessment Design

33

Feature Description Grade K-12 Administration Beginning-of-Year and End-of-Year Alignment CCLS Structure Kindergarten: Read-aloud text, inventory, and informational writing Grades 1-2: Read-aloud text, independent reading, and informational writing Grades 3-5: 2 reading passages, 2 short constructed- response questions, and opinion writing Grades 6-12: 2-3 reading passages, 2 short constructed-response questions, and argumentative writing Stimuli Authentic Spanish texts written in Latin American, Caribbean, and American Spanish dialects.

Sample SLA Grade 11 Assessment

34

Sample Grades 11-12 Rubric

35

Sample Grade 11 Scoring Guide

36

Voices from the Field

• Let’s hear from a K-2 Spanish Dual Language

program…

• Ms. Santos, Assistant Principal
• Ms. Valck, Dual Language Lead Teacher
• Let’s hear from a teacher…
• Ms. Antuñez, HS Science, Spanish Dual

Language Program

37

CLA Assessment Design

38

Feature Description Grade 3-5 Administration Beginning-of-Year and End-of-Year Alignment CCLS Structure Part 1A: Reading passage #1 and 2 multiple-choice questions Part 1B: Graphic organizer Part 2A: Reading passage #2 and 2 multiple-choice questions Part 2B: Graphic organizer Part 3: Opinion writing Text Type Simplified and Traditional Chinese

Sample CLA Grade 5 Assessment

39

Sample CLA Grade 5 Rubric

40

Sample CLA Grade 5 Scoring Guide

41

Additional Support for Teachers

• Teacher Scripts
• Instructional Support Guide
• Grade expectations
• Item types
• Test language / glossary

42

Voices from the Field

• Let’s hear from a teacher…
• Ms. Xie, Grade 3, Elementary Chinese Dual

Language Program

• Let’s hear from a parent…
• Ms. Tang, Grade 3 student’s parent / content

developer

43

Impact Opportunities

• Unifying various instructional resources used across

bilingual programs citywide

• Acknowledging and considering regional language

usages to honor students’ origins

• Providing professional learning opportunities to build

teachers’ and administrators’ instructional capacity

• Enforcing cross-system collaborations for successful

implementation at all levels

44

Q & A

45

Thank you for participating!

Tim Boals, PhD Timothy.boals@wisc.edu Jimmy Han jhan10@schools.nyc.gov Lisa Lin clin2@schools.nyc.gov Doris Zahner, PhD dzahner@cae.org

46