Analytics for Case Outcomes ERICSA 52 nd Annual Training Conference - - PowerPoint PPT Presentation

ERICSA 52nd Annual Training Conference & Exposition ▪ April 26 – 30 Hershey Lodge ▪ Hershey, Pennsylvania

Using the Power of Predictive Analytics for Case Outcomes

ERICSA 52nd Annual Training Conference & Exposition ▪ April 26 – 30 ▪ Hershey Lodge ▪ Hershey, Pennsylvania

Participants

 Presenter – Edward V. Lehman, Jr.

 Director, Case Processing & Data Management  Philadelphia Family Court – Domestic Relations

Division  Presenter – Steven J. Golightly, Ph.D.

 Los Angeles County Child Support

 Moderator – Joyce Match

 Business Analyst Manager  Pennsylvania Bureau of Child Support

Enforcement

ERICSA 52nd Annual Training Conference & Exposition ▪ April 26 – 30 Hershey Lodge ▪ Hershey, Pennsylvania

Predictive Analytics in Action

Edward V. Lehmann, Jr.

ERICSA 52nd Annual Training Conference & Exposition ▪ April 26 – 30 ▪ Hershey Lodge ▪ Hershey, Pennsylvania

Access to the right information is extremely important

What is the requirement?

Increase Visible Results Reduce Information Overload Work the Right Cases Effective Case Worker Management Determine Next Appropriate Action Proactively

ERICSA 52nd Annual Training Conference & Exposition ▪ April 26 – 30 ▪ Hershey Lodge ▪ Hershey, Pennsylvania

Understanding Past, Measuring Present, Predicting the Future

ERICSA 52nd Annual Training Conference & Exposition ▪ April 26 – 30 ▪ Hershey Lodge ▪ Hershey, Pennsylvania

Predictive Analytics in Child Support

• Better estimates of delinquencies, child support collections
• Fewer child support cases in arrears
• Model results can be used to quantify historical process inefficiencies
• Significant marketing value – “We Know Our Citizens”
• More reliable payments for children
• Shift from reactive enforcement to early intervention
• Learning component for new case management approaches
• Assign high risk cases to case workers sooner
• Different approaches for different regions and different case types
• Automate certain research processes
• Reduce user guesswork and “cherry picking”
• Prioritize workload based on high impact
• Incorporate historical experience to drive future activities
• Eliminate a one size fits all approach
• Efficiency of customer service interactions
• Over time reduce future workload

Improved Workforce Effectiveness Increased Resource Allocation Efficiency Improved Outcomes

ERICSA 52nd Annual Training Conference & Exposition ▪ April 26 – 30 ▪ Hershey Lodge ▪ Hershey, Pennsylvania

Predictive Modeling for Child Support

• Child support enforcement has traditionally been a reactive process.
• What if we had a tool that could help us predict which NCPs are

most likely to become in-arrears in the near future?

• We could use such a tool to:
• Prevent arrears
• Decrease custodial parent complaints
• Take the right action on a case at the right time
• Assign the right case workers to the right cases
• Gather information for potential policy changes

Use analytics to make smarter decisions and do more with less.

ERICSA 52nd Annual Training Conference & Exposition ▪ April 26 – 30 ▪ Hershey Lodge ▪ Hershey, Pennsylvania

How do Models Work?

 Statistical models just refine what we do naturally all

the time.

 For example: which of these NCP’s is most/least

likely to pay next month?

Gene: 35 years old $2000 in arrears No other children on case No Salary attachment History of family violence Case is a IV-A assistance Tom: 28 years old $100 in arrears 1 other child on case Salary attachment No history of violence Case is not an assistance case Rick: 39 years old $8000 in arrears No other children on case No salary attachment No history of violence NCP has other child support cases

ERICSA 52nd Annual Training Conference & Exposition ▪ April 26 – 30 ▪ Hershey Lodge ▪ Hershey, Pennsylvania

How do Models Work?

ERICSA 52nd Annual Training Conference & Exposition ▪ April 26 – 30 ▪ Hershey Lodge ▪ Hershey, Pennsylvania

Pennsylvania’s Predictive Analytics solution included updates to the existing Performance Improvement Module (PIM) solution and a new application to calculate a predictive analytics “score.”

• Allows a worker to calculate a score for a case based on 20

variables

• Score is the likelihood of the defendant to pay 80% towards

current support obligation in the next three months

• Information required to generate the score is easily accessed

from the system at the time the support order is created or modified

• Provides a list of upcoming child support establishment

conferences with the cases assigned to the worker to allow the score creation prior to the conference.

Pennsylvania’s Solution

ERICSA 52nd Annual Training Conference & Exposition ▪ April 26 – 30 ▪ Hershey Lodge ▪ Hershey, Pennsylvania

PIM/Predictive Analytics

• Targeted case lists
• Suggested case actions
• Predictive score and

reasons

• Performance metrics
• Projects for targeted,

custom outreach including text messages

ERICSA 52nd Annual Training Conference & Exposition ▪ April 26 – 30 ▪ Hershey Lodge ▪ Hershey, Pennsylvania

Payment Score Calculator Objectives

• Establish a good

payment pattern: Increased quantity and frequency of collections

• More effective

meetings with defendants

• Make available new

methods of reaching

• ut
• Identifying
• pportunities for

proactive enforcement activities

• Maximize

performance metrics

• Minimize costs

related to enforcement

• Improve

cost/effectiveness ratios

• Effective case

assignment based on scores

• Establishing consistent

payment patterns based on successful business process actions for various scores

• Provide the right

services at the right time to encourage compliance with the

• rder
• Let the defendant know

the case is being monitored

ERICSA 52nd Annual Training Conference & Exposition ▪ April 26 – 30 ▪ Hershey Lodge ▪ Hershey, Pennsylvania

Variable Selection

Final model contains 20 variables

36 variables considered for modeling after Exploratory Data Analysis [EDA] Phase 64 candidate variables created during data scrubbing / brainstorming phase Over 400 data elements collected and considered in variable creation phase

The case for which a score is being calculated is compared against the historical behavior of thousands of cases with similar characteristics

ERICSA 52nd Annual Training Conference & Exposition ▪ April 26 – 30 ▪ Hershey Lodge ▪ Hershey, Pennsylvania

Example Predictive Variables

Number

Predictive Variables 1 Collection Indicator 2 High Number of Enforcement Activities Indicator 3 Number of Cases 4 Number of Enforcement Activities 5 Balance of Arrears 6 Defendant Net Income 7 Active Income Attachment Indicator 8 Number of Defendant Member Addresses on MADD 9 Number of Defendant Employers 10 Distance between the Defendant and Plaintiff

These are just a few examples of predictive variables used in

• Pennsylvania. Variables used in each State are different.

ERICSA 52nd Annual Training Conference & Exposition ▪ April 26 – 30 ▪ Hershey Lodge ▪ Hershey, Pennsylvania

Predictive Modeling Scores

Probability of NCP paying 80 percent on current support in the three months after support order issued/modified.

0 – 30% 31 – 50% 51 – 79% 80% +

ERICSA 52nd Annual Training Conference & Exposition ▪ April 26 – 30 ▪ Hershey Lodge ▪ Hershey, Pennsylvania

Confidence in the Score

Payment Score Average Percent of Current Support 1 51% 2 66% 3 77% 4 87%

To validate the accuracy of the payment score, a study of 5,000 PACSES cases was completed For each payment score, the average FYTD percent of current support collected was calculated. A direct relationship between payment score and percent of current paid was found (i.e., the higher the payment score the higher the percent of current paid.

ERICSA 52nd Annual Training Conference & Exposition ▪ April 26 – 30 ▪ Hershey Lodge ▪ Hershey, Pennsylvania

Planned enhancements to the existing functionality of the Payment Score Calculator

• Development of a mechanism for calculating a Payment

Score for all open cases

• Quarterly refresh the Payment Score Calculations on all
• pen cases
• Allow worker to complete a guideline calculation if the

PSC has been updated within the last three months

• Modification of PIM to add several pre-defined filters

which use the Payment Score as one of the factors for consideration

• Implementation planned for June, 2015

PSC Enhancements

ERICSA 52nd Annual Training Conference & Exposition ▪ April 26 – 30 ▪ Hershey Lodge ▪ Hershey, Pennsylvania

After PIM was implemented, Pennsylvania became the first State to achieve above 80% in both percent of current and percent of arrears

PIM/Predictive Analytics Effect in Pennsylvania

PA has been ranked #1 in the United States for both percent of current and percent of arrears, both of which have increased significantly since the implementation of the Performance Improvement Module

ERICSA 52nd Annual Training Conference & Exposition ▪ April 26 – 30 ▪ Hershey Lodge ▪ Hershey, Pennsylvania

• Revenue Impact: Exceeding 80% of collections targets
• Accuracy: “Likely payers” 4x more likely to be above 50% paid than “very unlikely payers”
• Timeliness: Allows taking action at time of support order, before payments are missed
• Actionability: Caseworker gets clear guidance on recommended steps
• Policy Insights: Ability to better segment NCPs to better understand policy impacts
• Workforce effectiveness:
• Shift from reactive enforcement to pro-active education
• Ability to prioritize workload based on high impact
• Efficiency: Increased effectiveness of retention outreach campaign, audits
• Time to production: Scored 17,000 cases within first three months
• Transparency: Unified view across all regions and business processes
• Marketing value: Helps build positive relationships with our clients. Everyone wins when

payments are made

Predictive Analytics Impact in Child Support

ERICSA 52nd Annual Training Conference & Exposition ▪ April 26 – 30 ▪ Hershey Lodge ▪ Hershey, Pennsylvania

Edward V. Lehmann, Jr. Director, Case Processing and Data Management Philadelphia Family Court Domestic Relations Section EdwardLehmann@PACSES.com

Feel free to contact us.

Questions? Need More Information?

ERICSA 52nd Annual Training Conference & Exposition ▪ April 26 – 30 Hershey Lodge ▪ Hershey, Pennsylvania

ERICSA 52nd Annual Training Conference & Exposition ▪ April 26 – 30 ▪ Hershey Lodge ▪ Hershey, Pennsylvania

WHERE DO WE START?

Starting with the Why

Simon Sinek’s Golden Circle

2

ERICSA 52nd Annual Training Conference & Exposition ▪ April 26 – 30 ▪ Hershey Lodge ▪ Hershey, Pennsylvania

WHY

3

ERICSA 52nd Annual Training Conference & Exposition ▪ April 26 – 30 ▪ Hershey Lodge ▪ Hershey, Pennsylvania

TAILORING SERVICES USING PREDICTIVE ANALYTICS

4

Apply a Predictive Analytics lens to Case Management

ERICSA 52nd Annual Training Conference & Exposition ▪ April 26 – 30 ▪ Hershey Lodge ▪ Hershey, Pennsylvania

PREDICTIVE ANALYTICS – CLASSIFICATION MODEL

Forecast (predict) an outcome using multiple predictors TARGET: Predict which cases we can expect a payment of at least 70% of Current Support Amount in subsequent month

Built a Classification Model

• started with 100+ programmatic, demographic, and economic

variables

• narrowed down to 20 statistically significant variables

5

Model accurately predicts target variable for 90% of cases.

ERICSA 52nd Annual Training Conference & Exposition ▪ April 26 – 30 ▪ Hershey Lodge ▪ Hershey, Pennsylvania

CASE SCORING

6 0% 10% 20% 30% 40% 50% 1 2 3 4 5 6 7 8 9

Payment Status (Prior Month) Prediction (Next Month) Confidence Score Score (Propensity Score)

1 Non-Payer Non-Payer .91 1 2 Non-Payer Non-Payer .73 3 3 Payer Payer .91 9 4 Payer Payer .73 7 5 Payer Non-Payer .55 5 6 Non-Payer Payer .55 5

ERICSA 52nd Annual Training Conference & Exposition ▪ April 26 – 30 ▪ Hershey Lodge ▪ Hershey, Pennsylvania

PREDICTIVE ANALYTICS - SEGMENTATION

Built a Segmentation Model

• Started with same

variables as in classification model

• Clustering Analysis

identified natural groupings based on payment history

Group cases using one or more input fields

7

TARGET: Create cluster of cases based on case variables to identify natural grouping of cases

ERICSA 52nd Annual Training Conference & Exposition ▪ April 26 – 30 ▪ Hershey Lodge ▪ Hershey, Pennsylvania

EARLY SUCCESSES Arrears Only Project - January 2014

9,486 cases with an Arrears balance, no minor children, no current support, and no payment within last federal fiscal year

Strategy – Focused Caseload, Locate Experts Results – $2 million in 12 months!

8 47.96% 55.99% Department Arrears Only

FPM 4

ERICSA 52nd Annual Training Conference & Exposition ▪ April 26 – 30 ▪ Hershey Lodge ▪ Hershey, Pennsylvania

PREDICTIVE ANALYTICS FINDINGS

Arrears Only Project Success

• Personalized the client’s experience
• Case Worker a subject matter expert for

case type

9

• Predictive Analytic Model identifies cases that

perform similarly

• PA Model predicts, with a high level of confidence,

how a case will perform

• PA Model identifies characteristics and motivations
• f different customer types

One ne-Si Size e Fit its s One ne

ERICSA 52nd Annual Training Conference & Exposition ▪ April 26 – 30 ▪ Hershey Lodge ▪ Hershey, Pennsylvania

HOW

CREATE NEW BUSINESS MODEL

Case Ownership Case Segmentation

EST Zero Order Non- Paying Occasion al Paying Arrears Only Other Estab. Zero Order Paying Occasional Non- Paying Arrears Only IGR

10

ERICSA 52nd Annual Training Conference & Exposition ▪ April 26 – 30 ▪ Hershey Lodge ▪ Hershey, Pennsylvania

SEGMENTATION MODEL

Intake Establishment Arrears Only IGR Customer Response 11

ENFORCEMENT

• Div. 1
• Div. 3
• Div. 4
• Div. 5
• Div. 6

ERICSA 52nd Annual Training Conference & Exposition ▪ April 26 – 30 ▪ Hershey Lodge ▪ Hershey, Pennsylvania

ENFORCEMENT CASELOAD Definition – Case has an order for $0

OR

Case has a reserved or pending supplemental

• rder OR

Case has a Medical Only

• rder OR

Case does not have an active order, but case has an arrears balance and child is a minor

Occasional Paying Zero Order Non-Paying

12

ZERO ORDER QUADRANT

ERICSA 52nd Annual Training Conference & Exposition ▪ April 26 – 30 ▪ Hershey Lodge ▪ Hershey, Pennsylvania

ENFORCEMENT CASELOAD

Definition – Case has an order AND Current Support Amount is a dollar amount greater than $0 AND NCP pays regularly AND NCP pays over 70% of CS amount

Occasional Paying Zero Order Non-Paying

13

PAYING QUADRANT

ERICSA 52nd Annual Training Conference & Exposition ▪ April 26 – 30 ▪ Hershey Lodge ▪ Hershey, Pennsylvania

ENFORCEMENT CASELOAD

Definition –

Case has an order AND Current Support Amount is a dollar amount greater than $0 AND NCP pays irregularly OR NCP pays less than 70% of CS amount

Occasional Paying Zero Order Non-Paying

14

OCCASIONAL PAY QUADRANT

ERICSA 52nd Annual Training Conference & Exposition ▪ April 26 – 30 ▪ Hershey Lodge ▪ Hershey, Pennsylvania

ENFORCEMENT CASELOAD Definition – Case has an order AND Current Support Amount is a dollar amount greater than $0 AND NCP has not made a payment during the current Federal Fiscal Year

Occasional Paying Zero Order Non-Paying

15

NON-PAY QUADRANT

ERICSA 52nd Annual Training Conference & Exposition ▪ April 26 – 30 ▪ Hershey Lodge ▪ Hershey, Pennsylvania

Occasional

 Send letters of Introduction to new case manager  UIB/DIB/ Mod review (CMT Sort)  Review for SLMS suspension – Manual submission for NCP’s with valid license  Incarcerated Special Mod/closing Project  CMT High TMSO paying less than 90% sort  Review EFO case function cases for order for subsequent child  Work LC005 task for IWOs  Review CMT for SSI benefits

Paying

 Send letters of Introduction to new case manager  Low Order with reported Earnings Mod Project  Admin IWO Project  Incarcerated Special Mod Project  CMT High TMSO paying less than 90% sort  UIB/DIB Mod review (CMT Sort)  Review EFO case function cases for order for subsequent child  New Order clerical contact of NCP (EI)  Work LC005 task for IWOs

Zero Order

 Send letters of Introduction to new case manager  Zero Mod Project  Review EFO case function cases for order for subsequent child  Review Paid in Full cases to confirm emancipated DP’s updated for auto closures.

Non-Paying

 Send letters of Introduction to new case manager  Review for SLMS suspension – Manual submission for NCP’s with valid license  Incarcerated Special Mod Project  Review EFO case function cases for order for subsequent child  Work LC005 task for IWOs  Review CMT for SSI benefits

STRATEGIES BY QUADRANT

16

ERICSA 52nd Annual Training Conference & Exposition ▪ April 26 – 30 ▪ Hershey Lodge ▪ Hershey, Pennsylvania

NEW CSTATS

17 Old CSTATS:

• Performance across the

division

• Compares branch-to-branch

New CSTATS:

• Performance across

the division

• And performance by

quadrants

• Compares branches

by quad

• Month, YTD

ERICSA 52nd Annual Training Conference & Exposition ▪ April 26 – 30 ▪ Hershey Lodge ▪ Hershey, Pennsylvania

IMPLEMENTATION

18

Phased Roll-Out Roadshows Feedback/Workgroups New Performance Measures Reset every FFY

ERICSA 52nd Annual Training Conference & Exposition ▪ April 26 – 30 ▪ Hershey Lodge ▪ Hershey, Pennsylvania

QUESTIONS

19