#BIAScapstone Sign up for the Behavioral Buzz ! 2 BIAS Capstone - PowerPoint PPT Presentation

1 BIAS Capstone Convening | April 27, 2016 | Washington, D.C. #BIAScapstone Sign up for the Behavioral Buzz !

2 BIAS Capstone Convening | April 27, 2016 | Washington, D.C. BIAS Domains and Findings: Implications for Policy and Practice Session 1 | 8:50 – 9:50 AM Dr. Lashawn Richburg-Hayes

3 Session 1 overview Presentation Panel Remarks Open Q&A (30 min) (10 min) (20 min)

4 5 insights that motivated BIAS Incentives are Actions do not not always always reflect aligned. intentions. To understand why a process is not achieving its goals, Small changes in the context look at it from the Identify the can have large user’s perspective. problem before effects on creating the outcomes. solution.

5 BIAS Portfolio 3 domains, 7 states, 15 tests WORK SUPPORT CHILD SUPPORT CHILD CARE OH Collections Recertification OK CA TANF Engagement Paycheck Plus Recertification & NY IN TX Order Modifications Meeting Attendance Provider Choice WA Order Modifications

6 Overview of Findings • 15 RCTs • All sites saw a 8 - CA significant impact on Number of Tests at least one primary OH outcome of interest IN • Effect sizes typically ranged from 2 to 4 OH percentage points, 4 - with some outliers OH OK NY OH IN TX WA OH NY OH IN -2 0 2 4 6 8 10 12 14 16 18 20 22 24 26 28 30 32 Impact size (percentage points)

7 Overview of Findings 8 - • 15 RCTs IN • All sites saw a Number of Tests significant impact on CA at least one primary outcome of interest TX • Effect sizes typically 4 - NY ranged from 2 to 4 percentage points with some outliers OH IN IN • Intervention costs OH NY OH were typically less than $4 per program OH OK OH OH WA group member $0 $1 $2 $3 $4 $5 $6 $7 $8 $9 $10 $11 Cost per program group member

8 CHILD SUPPORT • Program was created so • Parents must often make that children receive complicated decisions with support from both parents little information in a even when the parents are context where emotions separated can run high

9 BIAS Child Support Studies Problem focus #1: Order modifications 1 Texas Modifications 6 Washington Modifications 1 Problem focus #2: Collections Franklin County, Ohio 8 of the 15 BIAS Tests Collections Cuyahoga County, Ohio Collections

10 TEXAS

11 Texas Background • Many incarcerated parents who owe child support have limited means to make payments. • Texas has a process for parents to apply for a downward modification of their child support order. • Texas Office of the Attorney General sent eligible parents an application but only about 31 percent applied. Can Texas increase the number of incarcerated NCPs who apply for a child support modification?

Texas Intervention 12 Teaser Postcard Application Packet Reminder Postcard Remind parent to take action

Texas Intervention 13 Teaser Postcard Application Packet Reminder Postcard Personalize materials

Texas Intervention 14 Teaser Postcard Application Packet Reminder Postcard Expose parent to process

15 Texas Findings The revised outreach increased the percentage of parents that submitted complete applications by 11 percentage points 38.7% 11 %*** 27.7% Control group Program group 15

16 OHIO

17 Ohio Background • Franklin and Cuyahoga Counties (Columbus & Cleveland) collect over 60 percent of total payments due. • Payment rates are high when income withholding is in place, but many orders have no income withholding. • Parents without withholding must manually make payments each month, but are sent no reminder or one that could be potentially improved. Can the counties increase the likelihood of payment and the total dollar amount collected?

Ohio Intervention 18 (Franklin County Test 1: Parents Not Being Sent a Notice) April 2014 Without Reminder Sunday Monday Tuesday Wednesday Thursday Friday Saturday 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 Payment Made? Payment Made?

Ohio Intervention 19 (Franklin County Test 1: Parents Not Being Sent a Notice) April 2014 With Reminder? Sunday Monday Tuesday Wednesday Thursday Friday Saturday 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 Payment Made? Payment Made?

Franklin County Test 1: Parents Not Being Sent a Notice 20 51.5 48.5 2.9%*** 48 • Sending parents a low- 42 cost payment reminder increased the number of 36 parents making a 30 payment (but not the 24 amount paid) 18 • Effects did not vary by 12 reminder type 6 0 Control group Program groups 20

21 CHILD CARE • Program supports parental • Requires low-income parents employment or education facing many challenges to and furthers children’s take a series of steps in order development to obtain benefits

22 BIAS Child Care Studies Indiana Subsidy renewal and provider choice 3 Oklahoma 1 Subsidy renewal 4 of the 15 BIAS Tests

23 OKLAHOMA

24 Oklahoma Background • Parents need to periodically document their eligibility in order to renew their child care subsidies. • In Oklahoma, less than 40 percent of clients renewed their benefits on time. • Failing to renew on time creates risks for clients, child care providers, and Oklahoma. Can Oklahoma increase the percentage of parents who renew their subsidies on time?

Oklahoma Intervention 25 Client Intervention Provider Intervention Example: Early Alert Postcard Example: “Red” Provider List

Oklahoma Findings 26 36.7 35.1 36 34.4 2.4%* • Provider intervention increased the on-time renewal rate by 2.4 percentage points 30 • Client intervention did not improve on-time 24 renewal, but may have helped clients renew by the end of a 30-day grace period 18 • Combining the client and provider interventions did not appear to be more 12 effective than either intervention alone 6 0 Control group Client Provider Intervention Intervention 26

27 WORK SUPPORT • Participants may not • Programs are designed to engage with the program increase participants’ family by attending meetings, economic security through responding to employment. communications, etc.

28 BIAS Work Support Studies 2 Los Angeles, CA TANF engagement 1 New York, NY Paycheck Plus engagement 3 of the 15 BIAS Tests

29 LOS ANGELES, CA

30 Los Angeles Background • California temporarily exempted some TANF families from the welfare-to-work program, but that exemption ended. • In Los Angeles, parents were sent two mailings and two phone calls, but many parents still did not attend their mandatory appointment or reach other reengagement benchmarks. Can Los Angeles increase the percentage of parents who became positively engaged?

Los Angeles Intervention 31 Loss Frame Gain Frame Hi Jane, Hi Jane, See you at your See you at your appointment next week. appointment next week. We’ll make a plan for We’ll make a plan for work, and make sure you work, and make sure you don’t lose your ur cash co continue to recei eceive cas e cash benefi fits. Call me if you benefi fits. Call me if you have any questions. have any questions. See you then, See you then, John Smith John Smith (555) 123-4567 (555) 123-4567

Los Angeles Findings 32 29.9 29.2 30 30 28.4 3.6%* 4.4%** • An additional behavioral notice 25.6 increased reengagement at 30 24 24 days by 3.6 percentage points • This effect was largely driven by 18 18 the loss-framed notice, which increased reengagement by 4.4 12 12 percentage points • No impacts were found for either 6 6 notice at 60 or 90 days 0 0 Control group Behavioral Gain frame Loss frame notice 32

33 Some Take-Aways • Encouraging one-time behaviors is probably easier than changing habits • Many families struggle with the interim tasks critical to maintaining valuable public benefits • Nudges can help, but more is needed (e.g., inability to pay)

34 Panelists University of Washington Sim Crystal Duke University Sitkin Hall White House Social and Behavioral Behavioral Science & Policy Association Sciences Team Theme How do BIAS findings fit with other areas of behavioral science?

Reality-based criteria for Sim 35 Sitkin judging progress • Define modest impact (0-4%) as success • Reporting failures • Contingent effects are pervasive

Assessing the present with an Sim 36 Sitkin eye to opportunity • Client context matters: Process, organizations, staff, family, community • Effects: Think cumulative vs. short-term • Structural and cultural infrastructure: Continuous experimentation and improvement as a core part of our professional and organizational identity • BE-all and end-all: Lots of relevant social and behavioral science, behavioral economics is not the totality

Moving forward: Leveraging 37 Crystal additional tools Hall • More use of administrative data • Benefit-cost analysis • Creating relationships across agencies and organizations