#BIAScapstone Sign up for the Behavioral Buzz ! 2 BIAS Capstone - - PowerPoint PPT Presentation

1

BIAS Capstone Convening | April 27, 2016 | Washington, D.C.

#BIAScapstone Sign up for the Behavioral Buzz!

2

BIAS Domains and Findings: Implications for Policy and Practice

Session 1 | 8:50 – 9:50 AM

• Dr. Lashawn Richburg-Hayes

BIAS Capstone Convening | April 27, 2016 | Washington, D.C.

3

Session 1 overview

Presentation (30 min) Panel Remarks (20 min) Open Q&A (10 min)

4

Identify the problem before creating the solution.

5 insights that motivated BIAS

Small changes in the context can have large effects on

• utcomes.

To understand why a process is not achieving its goals, look at it from the user’s perspective. Incentives are not always aligned. Actions do not always reflect intentions.

5

TX WA OH CA NY IN

Order Modifications Collections Paycheck Plus Meeting Attendance Order Modifications TANF Engagement Recertification & Provider Choice

BIAS Portfolio

3 domains, 7 states, 15 tests

OK

Recertification CHILD SUPPORT CHILD CARE WORK SUPPORT

6

Overview of Findings

• 15 RCTs
• All sites saw a

significant impact on at least one primary

• utcome of interest
• Effect sizes typically

ranged from 2 to 4 percentage points, with some outliers

• 2

2 4 6 8 10 12 14 16 18 20 22 24 26 28 30 32

Impact size (percentage points) Number of Tests

TX WA OH CA NY IN OK OH OH OH OH OH IN IN NY

4 - 8 -

7

Overview of Findings

• 15 RCTs
• All sites saw a

significant impact on at least one primary

• utcome of interest
• Effect sizes typically

ranged from 2 to 4 percentage points with some outliers

• Intervention costs

were typically less than $4 per program group member

$0 $1 $2 $3 $4 $5 $6 $7 $8 $9 $10 $11

Cost per program group member Number of Tests

TX WA OH CA NY IN OK OH OH OH OH OH IN IN NY

4 - 8 -

8

CHILD SUPPORT

• Program was created so

that children receive support from both parents even when the parents are separated

• Parents must often make

complicated decisions with little information in a context where emotions can run high

9

Collections

Franklin County, Ohio

Collections

Cuyahoga County, Ohio

Modifications

Texas

Modifications

Washington

BIAS Child Support Studies

8 of the 15 BIAS Tests

Problem focus #1: Order modifications Problem focus #2: Collections 1 1 6

10

TEXAS

11

Texas Background

Can Texas increase the number of incarcerated NCPs who apply for a child support modification?

• Many incarcerated parents who owe child support have limited

means to make payments.

• Texas has a process for parents to apply for a downward

modification of their child support order.

• Texas Office of the Attorney General sent eligible parents an

application but only about 31 percent applied.

12

Teaser Postcard

Remind parent to take action

Application Packet Reminder Postcard

Texas Intervention

13

Teaser Postcard Application Packet Reminder Postcard

Texas Intervention

Personalize materials

14

Teaser Postcard

Expose parent to process

Application Packet Reminder Postcard

Texas Intervention

15

The revised outreach increased the percentage of parents that submitted complete applications by 11 percentage points

15

27.7% 38.7%

Control group Program group

Texas Findings

11%***

16

OHIO

17

Ohio Background

• Franklin and Cuyahoga Counties (Columbus & Cleveland)

collect over 60 percent of total payments due.

• Payment rates are high when income withholding is in place,

but many orders have no income withholding.

• Parents without withholding must manually make payments

each month, but are sent no reminder or one that could be potentially improved. Can the counties increase the likelihood of payment and the total dollar amount collected?

18

April 2014

Sunday Monday Tuesday Wednesday Thursday Friday Saturday

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30

Ohio Intervention

(Franklin County Test 1: Parents Not Being Sent a Notice)

Without Reminder Payment Made? Payment Made?

19

April 2014

Sunday Monday Tuesday Wednesday Thursday Friday Saturday

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30

Ohio Intervention

(Franklin County Test 1: Parents Not Being Sent a Notice)

With Reminder? Payment Made? Payment Made?

20

20

Franklin County Test 1: Parents Not Being Sent a Notice

48.5

51.5

6 12 18 24 30 36 42 48 Control group Program groups

2.9%***

• Sending parents a low-

cost payment reminder increased the number of parents making a payment (but not the amount paid)

• Effects did not vary by

reminder type

21

CHILD CARE

• Program supports parental

employment or education and furthers children’s development

• Requires low-income parents

facing many challenges to take a series of steps in order to obtain benefits

22

Oklahoma Subsidy renewal Indiana Subsidy renewal and provider choice

BIAS Child Care Studies

3 1

4 of the 15 BIAS Tests

23

OKLAHOMA

24

Oklahoma Background

• Parents need to periodically document their eligibility in order to

renew their child care subsidies.

• In Oklahoma, less than 40 percent of clients renewed their

benefits on time.

• Failing to renew on time creates risks for clients, child care

providers, and Oklahoma. Can Oklahoma increase the percentage of parents who renew their subsidies on time?

25

Client Intervention

Example: Early Alert Postcard

Oklahoma Intervention

Provider Intervention

Example: “Red” Provider List

26

26

Oklahoma Findings

34.4 35.1 36.7

6 12 18 24 30 36 Control group Client Intervention Provider Intervention

2.4%*

• Provider intervention increased the on-time

renewal rate by 2.4 percentage points

• Client intervention did not improve on-time

renewal, but may have helped clients renew by the end of a 30-day grace period

• Combining the client and provider

interventions did not appear to be more effective than either intervention alone

27

WORK SUPPORT

• Programs are designed to

increase participants’ family economic security through employment.

• Participants may not

engage with the program by attending meetings, responding to communications, etc.

28

New York, NY Paycheck Plus engagement Los Angeles, CA TANF engagement

BIAS Work Support Studies

2 1

3 of the 15 BIAS Tests

29

LOS ANGELES, CA

30

Los Angeles Background

• California temporarily exempted some TANF families from the

welfare-to-work program, but that exemption ended.

• In Los Angeles, parents were sent two mailings and two phone

calls, but many parents still did not attend their mandatory appointment or reach other reengagement benchmarks. Can Los Angeles increase the percentage of parents who became positively engaged?

31

Loss Frame

Los Angeles Intervention

Gain Frame

Hi Jane, See you at your appointment next week. We’ll make a plan for work, and make sure you co continue to recei eceive cas e cash benefi

• fits. Call me if you

have any questions. See you then, John Smith (555) 123-4567 Hi Jane, See you at your appointment next week. We’ll make a plan for work, and make sure you don’t lose your ur cash benefi

• fits. Call me if you

have any questions. See you then, John Smith (555) 123-4567

32

32

Los Angeles Findings

4.4%**

• An additional behavioral notice

increased reengagement at 30 days by 3.6 percentage points

• This effect was largely driven by

the loss-framed notice, which increased reengagement by 4.4 percentage points

• No impacts were found for either

notice at 60 or 90 days

28.4 29.9

6 12 18 24 30 Gain frame Loss frame

25.6 29.2

6 12 18 24 30 Control group Behavioral notice

3.6%*

33

Some Take-Aways

• Encouraging one-time behaviors is probably easier

than changing habits

• Many families struggle with the interim tasks critical to

maintaining valuable public benefits

• Nudges can help, but more is needed (e.g., inability to

pay)

34

Sim

Sitkin

Duke University Behavioral Science & Policy Association

Crystal

Hall

University of Washington White House Social and Behavioral Sciences Team

Panelists

Theme How do BIAS findings fit with other areas of behavioral science?

35

Reality-based criteria for judging progress

• Define modest impact (0-4%) as success
• Reporting failures
• Contingent effects are pervasive

Sim

Sitkin

36

Assessing the present with an eye to opportunity

• Client context matters: Process, organizations, staff, family,

community

• Effects: Think cumulative vs. short-term
• Structural and cultural infrastructure: Continuous

experimentation and improvement as a core part of our professional and organizational identity

• BE-all and end-all: Lots of relevant social and behavioral

science, behavioral economics is not the totality

Sim

Sitkin

37

Moving forward: Leveraging additional tools

• More use of administrative data
• Benefit-cost analysis
• Creating relationships across agencies

and organizations

Crystal

Hall

38

Moving forward: Beyond client- centered interventions

• Diagnosis and design for program staff
• Distinct set of touch points
• Wider impact could be possible

Crystal

Hall

39

Up Next

Session 2: Designing SIMPLER Solutions 9:50 A.M. – 10:45 A.M

40

BIAS Capstone Convening | April 27, 2016 | Washington, D.C.

#BIAScapstone Sign up for the Behavioral Buzz!