Seeing the Future: Using Predictive Analytics to Gain an Advantage - - PowerPoint PPT Presentation

Seeing the Future: Using Predictive Analytics to Gain an Advantage in the Changing Environment

Why is this important?

… Proper and proportionate allocation of resources is more important than ever!

Data

Data used for the analysis:

• Allegheny County HealthChoices and ‘Base’ data from 2005 to 2015;

including:

• Claims
• Demographics
• Environmental Variables
• Calculated Variables

What is a High Utilizer?

A client on w hom you spend a disproportionate am ount of resources

There are many ways to define a high utilizer

• Frequent IPMH Stays
• Frequent Re-admissions
• High-cost service users
• High-unit service users
• People frequently in a crisis

For this project, we have chosen to define high utilizers based on their overall annual paid claims amount. This definition allows us to look at individuals from year to year. It also allows us to see if/how they move in and out

• f being a high utilizer over time.

Defining High Utilizers

Pareto principle (80-20 rule)

• 80% of cost is incurred by 20% of the population
• This holds true with the Medicaid BH population

Question?

Although 80-20 m ay apply in a given year, is there m em bership stability in this ratio over tim e (i.e. Are they the sam e people from year to year)?

…NO! They are not the sam e!

Movement over time

The number of high utilizers from year to year remains consistent; however, Each year, 44% of the high utilizers from the previous year drop out of high utilization for the current year and their place is filled by people who were not high utilizers the previous year! So, if you only looked at last year’s High Utilizers to determine allocation of resources, you would be “wrong” 44%

• f the time!

Our analysis of individuals over time has also shown that there are different “types” of high utilizers. We have grouped them as follows:

• Career
• Chronic
• Moderate
• Sporadic
• Never

“Types” of high utilizers

Career – people who have been high utilizers every year of their tenure in the system Chronic – people who have been high utilizers 75% to 99% of their years in the system Moderate – people who have been high utilizers 25% to 74% of their years in the system Sporadic – people who have been high utilizers 1% to 24% of their years in the system Never – people who have had zero high utilizer years up to this point This realization helped inform our definition and our decision to use the Top 5% cut point. This gives us a mix of types of high utilizers in our “target” group. For example, the Top 1% was populated mostly by the “Career” high utilizers, but for our project this was not ideal.

… Yes, with a lot of math Can w e predict w ho is going to be a high utilizer next year?

What is Predictive Analytics?

Predictive analytics uses many techniques from data mining, statistics, modeling, machine learning, and artificial intelligence to analyze current data to make predictions about future. This process can be used to:

• Find relationships between variables
• Exploit patterns found in historical data
• Inform decision making
• Identify opportunities/risks
• Assist in forecasting
• Classifying groups based on common characteristics

Examples you may be familiar with?

• Amazon – What other books you may like.
• Netflix – If you liked this movie, you will probably like

this one also

• FICO score – Will you pay your loan back?

… And many, many more!

Predictive Analytics Process

Model Accuracy

There are many methods for measuring model accuracy.

• Lift
• Gains
• ROI
• Receiver Operating Characteristic (ROC) Curve
• Contingency Table

Methods used vary depending on the purpose of the model. Models are going to be WRONG, the key is to minimize the effects of those errors and to control where you are wrong.

Model Accuracy

Contingency Table

This method can help you identify where you are correct and where you are incorrect. Models can be tweaked to minimize certain types of errors, and the affects can be seen easily through the contingency table.

What if we had no model?

Total People: 12,984 High Util: 1,029 Non-High Util: 11,955 False Positive Rate: 50% True Positive Rate: 50% Correct: 6,493 (50%) Positive Predictive Value: 7.9% Negative Predictive Value: 92.0% Total “Yes” Predictions: 6,493 Total “No” Predictions: 6,493 514 515 5,977 5,978

Our Model

Total People: 12,984 High Util: 1,029 Non-High Util: 11,955 False Positive Rate: 1.9% True Positive Rate: 40.1% Correct: 12,138 (93.48%) Positive Predictive Value: 64.2% Negative Predictive Value: 95.0% Total “Yes” Predictions: 643 Total “No” Predictions: 12,341 616 413 230 11,725

Our Model

ROC Curve This curve allows us to optimize our prediction cut-points to suit our needs. The farther along the x-axis you travel, the more True Positives you get, but you also get more False Positives. AUC – 86.6% 86.6% AUC

We have a good m odel and have generated predictions for each person…now w hat?!

… Decision Theory!!

Informed Decision Making

Decision Theory is concerned with identifying the best decision to make, based on a set of assumptions. Our assumption is that the model is correct in its predictions, and that we are going to act on the predictions generated by the model. There are many strategies that can be used in decision theory, but for our case, we are going to perform some sort of intervention for each person that the model predicts to be a high utilizer in the coming year. A combination of predictive modeling, decision theory, and some math can even tell us for how much money we need to be able to perform that intervention.

Informed Decision Making

616 413 230 11,725 Using our model, we would perform an intervention on 643 people. Of that group, we will be “wasting” effort on 230 of them. For people predicted NOT to be high utilizers in the coming year, we will do nothing differently than we did before. They will continue to receive services as they have in the past.

Informed Decision Making

514 515 5,977 5,978 With no model, we would perform an intervention on 6,493 people. Of that group, we will be “wasting” effort on 5,977 of them. We would also catch 102 more True Positives than with our model. Tweaking the model based on driving factors will help align the decision making process with your goals.

What does this look like w hen you add m oney into the equation?

Here’s an exam ple!

Informed Decision Making

Let’s assume:

• You see 10,000 people annually
• High utilizer ratio is 8% of people incur 60%
• f the cost.
• Those 10,000 people incur $50m in cost

With no model (flipping a coin), you would have 5,000 people that would be predicted to be high utilizers. That leaves you with $5,000 per person per year to perform some sort of intervention

Informed Decision Making

Let’s assume:

• You see 10,000 people annually
• High utilizer ratio is 8% of people incur 60%
• f the cost.
• Those 10,000 people incur $50m in cost

With our model, you would have 495 people that would be predicted to be high utilizers. That leaves you with $25,000 per person per year to perform some sort of intervention

Things to be aware of …

• This is just a tool
• It is not a replacement for clinical expertise
• Accuracy is related to timeliness of original

data

How can this be used?

• Risk Management
• Quality Management
• Outcomes Management
• Case Management
• Population Management

And Others …

Next steps

• Piloting with partners using an automated delivery of longitudinal predictions
• Development of additional models to predict IPMH Readmissions, Suicide Attempts, etc.
• Discussions with partners to find more useful targets to predict