Psychosocial Services The Good Study after study, Study after - - PDF document

heartandsoulofchange.com; pcoms.com February 2016 betteroutcomesnow.com 1

The Partners for The Partners for Change Change Outcome Management System Outcome Management System

Barry Duncan, Barry Duncan, Psy.D Psy.D. . heartandsoulofchange.com heartandsoulofchange.com pcoms.com coms.com betteroutcomesnow.com betteroutcomesnow.com

Eviden Evidence Based Based Practi Practice One One Client Client at a Time at a Time

What’s Up: PCOMS Research, Predictors, Implications What’s Up: PCOMS Research, Predictors, Implications

• 1. Research-proven strategy to

address the problems of psychosocial services

• 2. Incorporates known

predictors of ultimate

• utcome
• 3. Implications for Policy

PCOMS t Gl PCOMS at a Glance

Psychosocial Services The Good…

Study after study, Study after study, and studies of and studies of t di h th t di h th studies show the studies show the average treated average treated client is better off client is better off than 80% of the than 80% of the untreated sample. untreated sample.

Psychosocial Services The Bad…

• Drop out rates

Drop out rates average average 47% 47%

• Providers

Providers vary… a lot vary… a lot

TDCRP: top third psychiatrists giving placebo bested bottom third giving

Provider Differences Provider Differences Incredible Variation Among Providers Incredible Variation Among Providers Provider Differences Provider Differences Incredible Variation Among Providers Incredible Variation Among Providers

g g meds; clients of best counselors improve 50% more & dropped out 50% less; meds useful for clients

• f more effective, not for

less-effective

McKay, K., Imel, Z., & Wampold, B. (2006). Psychiatrist effects in the psychopharmalogical treatment of depression. Journal of Depressive Disorders, 92, 287-290.

heartandsoulofchange.com; pcoms.com February 2016 betteroutcomesnow.com 2

Reported a combined reliable and clinically More Bad: Public Behavioral Health Less than a Third Benefit y significant change rate in everyday clinical practice just 29.1%. Over 70% not helped

Hansen, N., Lambert, M., Forman, E. (2002). The psychotherapy dose-response effect and its implications for treatment delivery services. Clinical Psychology: Science and Practice, 9, 329-343.

And the Ugly Providers Don’t Know

• 20-70% range
• Graded their effectiveness,

A+ to F—67% said A or better; none rated below better; none rated below average; Another study: 85% of their clients improve & 90% saw themselves as above the 75th percentile.

• Providers

Providers don’t know how don’t know how effective they are effective they are

Sapyta, J., Riemer, M., & Bickman, L. Feedback to clinicians: Theory, research, and practice. Journal

• f Clinical Psychology: In Session, 61, 145-153

Walfish, S., McAlister, B., O’Donnell, P., & Lambert,

• M. (2012). An investigation of self-assessment bias

in mental health providers. Psychotherapy Reports, 110(2), 1-6. doi: 10.2466/02.07.17

No More Head in the Sand

• Despite overall

efficacy, dropouts a problem, not everyone benefits, providers vary significantly and don’t have a clue about effectiveness, and

• utcomes in many

settings are dismal.

Problems Spurred Interest in Quality Improvement Strategies

• Primary

Approach Has Been Been Transporting Evidence Based Treatments Problems with Transporting EBTs

• Dodo Bird

Verdict Verdict

• Cost and

Practicality

“Everybody has won and “Everybody has won and all all must have prizes.” must have prizes.”

Cost and Practicality Doesn’t Make Cents

• Cost of Implementing

EBT; VA—20 million

• Practicality Given Staff

Turnover, Fidelity

• Implementing an EBT

for Every Diagnoses…Over 150 EBTs and Over 400 Diagnoses

heartandsoulofchange.com; pcoms.com February 2016 betteroutcomesnow.com 3

No More Head in the Sand

• The Primary

Quality y Improvement Strategy Has Big Problems.

To The Rescue, Another QI Strategy

Continuous Client Feedback: PCOMS

The The ORS ORS The The SRS SRS

Download free working copies at: Download free working copies at:

pcoms.com coms.com or

• r heartandsoulofchange.com

heartandsoulofchange.com

Improving or Not Not Rocket Science

• PCOMS simply identifies

clients who are not responding so that the lack

• f progress can be
• f progress can be

addressed in a proactive way that keeps clients engaged while helpers collaboratively seek new directions…but does it improve outcomes?

• Feedback v TAU;

Both persons reliable or sig. h 50 5% Becoming Better Becoming Better

Anker Anker,

, Duncan

Duncan, &

, & Sparks

Sparks (2009)

(2009) Becoming Better Becoming Better

Anker Anker,

, Duncan

Duncan, &

, & Sparks

Sparks (2009)

(2009) change—50.5% v. 22.6%; ES: .50; 4 xs # of clin. sig. change

• FU: TAU-34.2% v.

18.4% Feedback sep./divorce rate

Anker, M., Duncan, B., & Sparks, J. (2009). Using client feedback to improve couple therapy outcomes: A randomized clinical trial in a naturalistic setting. Journal of Consulting and Clinical Psychology, 77(4), 693-704.

Reese, Norsworthy, & Rowlands (2009)

Reese, Toland, Slone, & Norsworthy, 2010

• N=148: Feedback group

doubled controls (10.4

• vs. 5.1 pts); ES: .48

Reese, R., Norsworthy, L., & Rowlands, S. (2009). Does a continuous feedback model improve psychotherapy

• utcomes? Psychotherapy,46,

418-431.

• Like Norway study,

clients, regardless of risk status, benefit from continuous feedback

• 2010 study is replication
• f Norway Trial

Reese, R., Toland, M., Slone, N., & Norsworthy, L. (2010). Effect

• f client feedback on couple

psychotherapy outcomes. Psychotherapy, 47, 616-630.

A SAMHSA EBP But Different

PCOMS is a- theoretical & therefore additive to therefore additive to any therapeutic

• rientation, including
• ther EBPs; PCOMS

applies to clients of all diagnostic categories

heartandsoulofchange.com; pcoms.com February 2016 betteroutcomesnow.com 4

Meta-analysis by Lambert & Shimokawa (2011)

• f PCOMS (the ORS and SRS)

Those in feedback group had 3.5 higher odds of experiencing reliable change Those in feedback group had less than half the odds of experiencing deterioration Feedback attained .48 ES

Lambert, M. J., & Shimokawa, K. (2011). Collecting client feedback. Psychotherapy, 48, 72-79.

Group, Substance Abuse, & PCOMS Schuman, Slone, Reese, & Duncan, 2014

RCT of 263 returning Afghanistan and Iraq Vets and Soldiers about to be Deployed: improved Deployed: improved

• utcomes on the ORS,

more reliable and clinically significant change, superior clinician and Commander ratings; and reduced drop

• uts

Schuman, D., Slone, N., Reese, R.J. & Duncan, B. (2014). Efficacy of Client Feedback in Group Psychotherapy with Soldiers Referred for Substance Abuse Treatment . Psychotherapy Research.

Cooper, Stewart, Sparks, & Bunting, 2013

• Cohort study of 288 7-11

yr olds; caretaker-228; teacher-249; 11 l 28 h l counselors; 28 schools

• 2 fold advantage over

children not using feedback as measured

• n the SDQ

Cooper, M., Stewart, D., Sparks, J., & Bunting, L. (2013). School-based counseling using systematic feedback: A cohort study evaluating outcomes and predictors of change. Psychotherapy Research, 23, 474–488.

Regarding Provider Variability Feedback Improves Outcomes

• Norway: 9 of 10 got

better outcomes

• Feedback raised

effectiveness of the lower

• nes to their more

successful colleagues.

• Provider in low

effectiveness group became BEST w/feedback!

PCOMS Addresses the Problems Recaptures At Risk Clients

• Feedback tailors services

based on response, provides an early warning provides an early warning system to prevents drop-

• uts & negative outcomes,

& solves provider variability—feedback improves performance as demonstrated in 5 RCTs

But What About the Real World? Benchmarking Studies

• Compares results of

RCTs to outcomes in real world. Real world h h d t much harder to secure good outcomes

• Comparable results

would suggest a viable method of quality improvement

heartandsoulofchange.com; pcoms.com February 2016 betteroutcomesnow.com 5

Southwest Behavioral Health Public Behavioral Health

Achieved outcomes comparable to RCTs of depression

SBHS, Multicultural PBH Center With a $70 Million Budget

p and feedback. N of 5186; 957 with substance abuse How? PCOMS!

Reese, R. J., Duncan, B. L., Bohanske, R. T., Owen, J. J., & Minami, T. (2014). Benchmarking outcomes in a public behavioral health setting: Feedback as a quality improvement strategy. Journal of Consulting and Clinical Psychology, 82, 731–742.

5 RCTs, the largest benchmarking study ever conducted in public behavioral health, & a cohort study h h h CO S d li i h

Delivers

have shown that PCOMS delivers with youth & adults, in individual, couple, & group therapy, with both mental health & substance abuse problems, & with the impoverished and disenfranchised.

• Client’s

Client’s rating of the rating of the alliance alliance the best predictor of the best predictor of engagement and outcome. engagement and outcome.

PCOMS Incorporates the Predictors Early Change and the Alliance

• Client’s

Client’s subjective subjective experience of change early experience of change early in the process the best in the process the best predictor of success for any predictor of success for any particular pairing. particular pairing.

Baldwin, S., Berkeljon, A., Atkins, D., Olsen, J., & Nielsen, S. (2009). Rates of change in naturalistic psychotherapy. Journal of Consulting and Clinical Psychology, 77(2), 203-211.

Some clients do take longer, but the mythology never dies N=4676; 77% attended 8

• r less, and 91% 12 or less

Note that even for the clients who take longer, change starts early…just is flatter Conclusions: Early change is an important factor for the prediction of short- and long-term outcome. Clients who report little or no progress early on will likely show no improvement over the entire course of service or will end up

The Research about Early Change

Is a Gift

The Research about Early Change

Is a Gift

course of service, or will end up

• n the drop-out list—early

change predicts engagement and a good outcome at termination. Provides a tangible way for us to identify folks who are not responding so that we can chart a new course.

Sudden Epiphany? NOT Sudden Epiphany? NOT

Myth: Clients flat-line & then spike. Some clients do take longer but change starts right away Myth: Clients flat-line & then spike. Some clients do take longer but change starts right away starts right away. So the question remains: When do you start getting worried when clients are not responding? I vote for sooner rather than later. starts right away. So the question remains: When do you start getting worried when clients are not responding? I vote for sooner rather than later.

heartandsoulofchange.com; pcoms.com February 2016 betteroutcomesnow.com 6

• Client’s rating of the

rating of the alliance alliance the best predictor of the best predictor of engagement and outcome. engagement and outcome.

Starting With Starting With the Predictors the Predictors Change and the Alliance

• Client’s subjective

subjective experience of change early experience of change early in the process the best in the process the best predictor of success for any predictor of success for any particular pairing. particular pairing.

The Alliance Bordin’s Bordin’s Classic Definition Classic Definition

The Alliance:

• Relational Bond

Relational Bond A t A t l l

Duncan, B. (2014). On Becoming a Better Therapist, 2nd ed. Washington, DC: APA

• Agreement on

Agreement on goals goals

• Agreement on tasks

Agreement on tasks Up to Seven Times the the Impact of Impact of Model/Technique; Model/Technique; Accounts Accounts for Most of for Most of Provider Provider Variance Variance

36-50%

The Alliance: The Alliance: Over Over 1000 Research Findings Research Findings

 Quality of the alliance more

potent predictor of outcome than orientation, , experience, or professional discipline

 Same across modalities and

• rientations; client

populations

The Session Rating Scale Measuring the Alliance The Session Rating Scale Measuring the Alliance

• Give at the end

Give at the end

• f session;
• f session;
• Each line 10 cm

Each line 10 cm in length; in length;

• Reliable, valid,

feasible

• Score in cm to

Score in cm to the nearest mm; the nearest mm;

• Discuss with

Discuss with client anytime client anytime total score falls total score falls below below 36 36

40.0% 40.0% 30 0% 30 0%

Factors Accounting for Factors Accounting for Successful Outcome

Client Client/Life /Life

30.0% 30.0% 15.0% 15.0% 15.0% 15.0%

Lambert, M. J. (2013). The efficacy and effectiveness of psychotherapy. In M. J. Lambert (Ed.), Bergin and Garfield’s handbook of psychotherapy and behavior change (6th ed., pp. 169–218). Hoboken, NJ: Wiley.

Alliance Alliance Hope Hope/Expectancy /Expectancy Models/Techniques Models/Techniques

heartandsoulofchange.com; pcoms.com February 2016 betteroutcomesnow.com 7

Client/Life Factors Bottom Line

• Engage clients

& rally their

“The quality of the participation [emerges] as

& rally their resources to the cause.

[ g ] the most important determinant of

• utcome."

Orlinsky, D. E., Rønnestad, M. H., & Willutzki, U. (2004). Fifty years of process -outcome research: In M. J. Lambert (Ed.), Bergin and Garfield’s handbook of psychotherapy and behavior change (5th ed.,

• pp. 307-390). New York: Wiley.

Until lions have their Until lions have their historians, tales of historians, tales of

Clients Are the Lions of Change Account for Majority of Outcome

, hunting will always hunting will always glorify the hunter. glorify the hunter.

African Proverb

Clients The Heart of Change

Client outcome feedback makes consumers the hi t i f th i historians of their own change Partnering w/clients to monitor outcome engages the most potent factor of change

The Alliance The Soul of Change The Alliance The Soul of Change

Alliance feedback enables a fit between client expectations, f d i preferences, and services Does not leave the alliance to chance—applying over 1000 studies showing the relationship of the alliance to positive outcomes

Client Feedback Engages Consumers in a Partnership Client Feedback Engages Consumers in a Partnership

 Overlaps with & affects all

factors—tie that binds

 Soliciting feedback is a

living process that engages

 Overlaps with & affects all

factors—tie that binds

 Soliciting feedback is a

living process that engages living, process that engages clients in monitoring

• utcome, heightens hope,

fits client preferences, maximizes therapist-client fit, and is itself a core feature of change. living, process that engages clients in monitoring

• utcome, heightens hope,

fits client preferences, maximizes therapist-client fit, and is itself a core feature of change.

So What Does All This Mean in Terms of P li ? So What Does All This Mean in Terms of P li ?

Barry,

Policy? What Can the Data Do for Us? Policy? What Can the Data Do for Us?

heartandsoulofchange.com; pcoms.com February 2016 betteroutcomesnow.com 8

Reliable Change Reliable Change: Exceeds

: Exceeds maturation, chance, and maturation, chance, and measurement error: 6 on measurement error: 6 on

The ORS: Reliable and Clinically Significant Change

the ORS; the ORS; Clinically

Clinically Significant Change Significant Change: A

: A change of 6 plus crossing change of 6 plus crossing the clinical cut off of 25, a the clinical cut off of 25, a score typical of those not in score typical of those not in service service

Using Data At the Individual Level

Becoming Better Linking Outcome to Process

• See clients

See clients more more frequently when the frequently when the slope of change is slope of change is steep. steep. steep. steep.

• Begin to space the

Begin to space the visits as the rate of visits as the rate of change lessens. change lessens.

• See clients as long

See clients as long as there is as there is meaningful meaningful change. change.

Love Your Data Key Progress Indicators Love Your Data Key Progress Indicators

• 1. Average Change:

Is it above 6?

• 1. Average Change:

Is it above 6? Is it above 6?

• 2. % reaching ETR

RCSC

• 3. Effect Size

Is it above 6?

• 2. % reaching ETR

RCSC

• 3. Effect Size

heartandsoulofchange.com; pcoms.com February 2016 betteroutcomesnow.com 9

Stepping Up Return on Investment Stepping Up Return on Investment

 People are afraid the data

will be used against them

• ften called P4P or pay for

performance (punish for

 People are afraid the data

will be used against them

• ften called P4P or pay for

performance (punish for performance (punish for performance

 Better to encourage

participation in data management as a continuous quality improvement strategy performance (punish for performance

 Better to encourage

participation in data management as a continuous quality improvement strategy

Arbitrary Performance Standards Will Bring Out the Dark Side Arbitrary Performance Standards Will Bring Out the Dark Side

 Client welfare invoked as

an explanation—But even the best don’t benefit up to a third of their clients.

 Client welfare invoked as

an explanation—But even the best don’t benefit up to a third of their clients. to a third of their clients. Will promote cheating and ruin the benefits

 Instead encourage to

identify non-responding clients and develop programming to address needs to a third of their clients. Will promote cheating and ruin the benefits

 Instead encourage to

identify non-responding clients and develop programming to address needs

Raise the Bar of Effectiveness and Efficiency Raise the Bar of Effectiveness and Efficiency

 Client feedback improves

both effectiveness and efficiency…engages clients

 Data collection  Client feedback improves

both effectiveness and efficiency…engages clients

 Data collection  Data collection

demonstrates both POV/ROI and serves as a CQI strategy

 Reward for participation,

QI, and creative programming to address non-responders

 Data collection

demonstrates both POV/ROI and serves as a CQI strategy

 Reward for participation,

QI, and creative programming to address non-responders