Objectives of Presentation 1. To determine if it is feasible to - - PDF document

APNA 30th Annual Conference Session 3023: October 21, 2016 Pearson 1

ASSESSING CLINICAL OUTCOMES IN A COMMUNITY BASED OUTPATIENT CHILD AND ADOLESCENT MENTAL HEALTH SERVICE (CAMHS)

American Psychiatric Nursing Association – Hartford, CT October 21, 2016

Ge ra ldine S. Pe a rso n, PHD, APRN, F AAN Unive rsity o f CT Sc ho o l o f Me dic ine The speaker has no conflicts of interest to disclose

NURSING ROLE IN THE CLINIC

• Limited to one psychiatry faculty position – my role as the

Clinic Director

• Previously an APRN/prescriber who left the clinic for

family reasons and was not replaced

• Currently one nursing student in a master’s program at

Yale School of Nursing working with me

• TRADITIONALLY A PHYSICIAN/PSYCHIATRIST

DOMINATED CLINIC

• Please keep this in mind as you listen to my presentation

Purpose and Description of this Presentation

PURPOSE: To describe a community outpatient, developing a database, assessing treatment progress, and using data to inform clinic care. DESCRIPTION: In effective care parents and children stay with the treatment interventions. This research looked specifically at responder status versus non-responder

• status. Responder status was associated with the number
• f appointments in the clinic. Responding to treatment was

greater for every subsequent appointment. The practice changes involve increased training around engagement strategies, improving clarity of realistic treatment expectations.

Objectives of Presentation

• 1. To determine if it is feasible to develop a systematic

measurement‐based and client specific outcomes monitoring methodology under resource‐limited treatment as usual clinic conditions.

• 2. To assess outcomes from a psychotherapeutically

sensitive, evidence‐based, and measurement‐based clinical disease management standard of care

• 3. Identify Implications of this research emphasizing

continuous quality improvement , viewing care as beginning with the first phone call, aiming all clinical activities at client engagement, and educating trainees about measuring progress in treatment

Importance of this Research

• This is an era of expanding “Big Data”

administrative oversight and a payer environment increasingly characterized by “Pay for Performance.”

• It is important to assess the effectiveness of

pediatric mental health treatment delivered under usual conditions.

– Unlikely that ineffective treatments will be supported in the future, i.e reimbursed

APNA 30th Annual Conference Session 3023: October 21, 2016 Pearson 2

Background of Study

• Increasing evidence‐base for pediatric psychotherapy

(James et al., 2015) and psychopharmacology (Cox et al., 2014) treatment efficacy from research RCTs.

• More limited evidence of what works (effectiveness) under

TAU conditions.

– Limited evidence about “what works for whom” in treating children and adolescents (Fonagy et al., 2005; Kennedy, 2015) – In randomized clinical trial studies “evidence based” outpatient mental health treatments for children were generally superior to usual care BUT effect size was small and in some studies TAU was superior to evidence based care (Weisz et al., 2013) – Poor documentation of evidenced based or usual care in child and adolescent psychiatric services in naturalistic outpatient clinic settings (Bearman & Weisz, 2015)

Implications of Research Literature

• Not clear who gets better under TAU conditions
• Not clear what type of treatment is most effective for

specific subgroups (disorder type, age, gender, ethnicity) under TAU conditions

• Not clear what predicts clinical outcomes under TAU

conditions

• Not clear what the role of treatment engagement means

for outcomes under TAU conditions

• Not clear how to make the clinician more effective under

TAU conditions

Multiple Dimensions of the UCONN Child/Adolescent Psychiatry Clinic

Clinical Educational Research

History of UHP Child/Adolescent Psychiatry Clinic

• Began in 2006 at the beginning of the child psychiatry

fellowship training at UCONN

• Developed as an academic training site for psychiatry

where the primary mission of the clinic is educational with a strong clinical component

• Primary child ambulatory and continuity of care training

clinic for child and adolescent psychiatry fellows, general psychiatry residents, psychology interns, and third year medical students (participatory in ambulatory care)

• Serves 6 to 17 year old children and adolescents with

behavioral and mental health disorders of varying severity

• All referrals are triaged by the second year chief resident

in child fellowship and the clinic director

• Referrals come from primary care pediatrics, allied mental

health professionals, child protective services, juvenile court, other mental health clinics, schools and self-referral

• Model of in vivo faculty supervision of psychiatric

evaluations and medication follow-ups

• Psychotherapy services available in clinic and also

collaboratively in community

Model of Treatment

• Combination of disorder-specific medication and/or

individual cognitive-behavioral therapy/parent management

• Delivered within a psychotherapeutically sensitive,

evidence-based, and measurement-based disease management standard of care model (Connor & Banga, 2014)

APNA 30th Annual Conference Session 3023: October 21, 2016 Pearson 3

Clinic Today

• Staffed by 0.4 FTE child psychiatry faculty, 0.6 FTE child

psychology faculty, and 0.4 FTE APRN’s. 4 child psychiatry fellows and 2 general adult psychiatry residents see patient ½ to 1 day/week

• Additional ½ day/week for child fellows of outpatient

psychotherapy experience with a focus on trauma

• 3 days/week of outpatient treatment experiences for

graduate students in psychology

• Systematic collection of data and then a traditional

formulation and treatment plan

Population Demographics and Characteristics

Annual Income

• <20,000: 14.7%
• 20-40,000: 16.9%
• 40-75,000: 22%
• more than 75,000: 22%
• not reported: 17.6%

Commercial Health Insurance: 56.8% Medicaid: 39.3% Self-pay: 1.1% Not reported: 4.3%

Population Demographics and Characteristics

62% 38% 0% 10% 20% 30% 40% 50% 60% 70% Male: Female: Caucasian: Black/AA: Latino/Hispanic: 67.3% 11%

Demographics and Population Characteristics

• Patients between 6-17 years (mean age 12.42 + 3.04)
• 32.2% single parent households
• High school education or greater attained by 79.5% of

mothers (14.5 % unreported) and 69.3% of fathers (24.1% unreported)

• 64% past or current mental health services at intake
• 27% families reported current or past involvement in child

protective services

• 9% reported current or past involvement with juvenile

justice

• 33% history special education services
• 8-12% reported past traumatic stress experiences

including abuse or witnessing domestic violence

Clinic Diagnosis Yearly Prevalence 2014

10 20 30 40 50 60 70 Percent of Visits Total visits 2014: 2002 New Intakes 2014: 133 Average number of diagnoses: 1.8

Study Objectives

• To determine if it is feasible to develop a systematic,

measurement-based, and patient specific outcomes monitoring methodology under resource-limited, treatment-as-usual clinic conditions.

• To assess outcomes from a psychotherapeutically

sensitive, evidence-based, and measurement-based clinical disease management standard-of-care model in pediatric mental health (Connor & Banga, 2014).

• To identify predictors of outcome in a clinic sample

treated under usual conditions.

APNA 30th Annual Conference Session 3023: October 21, 2016 Pearson 4

Study Methodology ‐Retrospective chart review

– All new patients who completed treatment 2006‐2013 and had treatment recommended after evaluation with f‐up in

• ur clinic
• N=444

– 92% returned for recommended follow‐up visit after evaluation

• IRB approved under waiver for de‐identified data
• CGI outcomes data entered at each visit (clinician

assessment)

• CGAS first‐ and last‐visit (clinician completed)

Study Methodology ‐Retrospective chart review

• Apriori definition of responder status:

– A clinician‐rated CGI‐I score of ≤ 2 (very much or much improved) at discharge and a clinician‐rated discharge CGAS score greater than the intake CGAS score, showing improvement in daily impairment.

• Other variables

– Parent CBCL at intake – Demographic, family, treatment history abstracted from chart review – Diagnosis: DSM‐IV‐TR reviewed by supervisor

Outcomes Measurement Feasibility

• Measures need to be in the public domain, efficient to

complete, valid, meaningful for treatment, and sensitive to change

• May be self-reported, observer-reported, and/or clinician-

reported

• Based on above considerations:

– CGI-severity: clinician-rated, at each visit – CGI-improvement: clinician-rated, beginning at visit 2 – CGAS: clinician-rated, first and last visit

Outcomes Measurement Feasibility

• Requires clinic culture of evidence-based empiricism

– Not solely driven by economic considerations – Clinic staff “buy-in”. Clinicians must see usefulness of a feedback

• utcomes measurement system
• Requires continuous quality monitoring

– Yearly training

• Over the past 8 years 83 independent raters have rated 16

written vignettes on the CGI and CGAS. – Clinician agreement on the second vignette (accuracy rating) with a senior rate is 93% for CGI-S – Clinician agreement on the second vignette (accuracy rating) with a senior rate is 84% for CGAS – Discussed at each individual patient clinic supervision – Requires an RA to maintain database – Continuous supervisory oversight required to keep the outcomes monitoring system going as personnel change in the clinic.

Outcomes

(N=434) 39% responder status* at discharge

0.5 1 1.5 2 2.5 3 3.5 4 4.5 5 CGI‐Severity Baseline CGI‐Severity Discharge 50 51 52 53 54 55 56 57 58 59 60 CGAS Baseline CGAS Discharge p < .001 Cohen’s d=.79 p < .001 Cohen’s d=.97

*CGI‐S ≤ 2 AND CGAS discharge > CGAS baseline

Percent Responder X Treatment Type

5 10 15 20 25 30 35 40 45 50 Psychotherapy Only Psychopharmacology Only Combined Therapy *** Difference between psychopharmacology groups vs. psychotherapy alone

APNA 30th Annual Conference Session 3023: October 21, 2016 Pearson 5

Responder status

• Associated with number of appointments with the

probability of responding to treatment greater for every additional appointment.

• Responders attended a mean of 11.84 + 9.02 sessions,

whereas non-responders attended a mean of 8.70 + 10.97 sessions (B = .03, SE = .01, Wald’s χ2 = 8.91, p = .003).

Summary of Results

• Outcomes monitoring feasible under TAU conditions.

– But has a cost:

• Requires an explicitly stated model of treatment.
• Requires continuous supervision, monitoring and

training to sustain over time.

• Requires a clinical culture of empiricism that values

measurement.

• Requires an RA to maintain data.

Summary of Results

• Clinical outcomes: 38% response rate by apriori criteria
• Predictor of good clinical outcomes:

– patient engagement as indexed by number of clinic visits.

• Predictors of poor clinical outcomes:

– Externalizing behavior problems > 95% for age/gender (CBCL) – Juvenile justice involvement – Ethnic status

Discussion

• While a majority of patients treatment in this child

psychiatry outpatient clinic were discharged with at least mild improvement, only approximately 40% could be considered definitive treatment responders

• Consistent with the literature (Murphy et al., 2015)
• Treatment type correlated to positive treatment response:

children receiving pharmacotherapy with or without psychotherapy having twice the likelihood of those receiving psychotherapy only to show a positive treatment response.

Discussion

• A majority of children receiving medication did NOT show

a positive treatment response (i.e., 44% responders; 56% non-responders

• Treatment response rate was highest for children

receiving a combination of medication and psychotherapy with 48% being identified as responders.

• Incremental benefit associated with the addition of

psychotherapy to pharmacotherapy with this child

• utpatient sample

Limitations of Study

• Missing data in some patients
• Retrospective methodology
• Single-site data with implications for generalizability of

findings

APNA 30th Annual Conference Session 3023: October 21, 2016 Pearson 6

Implications for Care

• More multi-disciplinary clinical treatment resources

needed for: – Children with JJ involvement – Children with symptomatic externalizing behavior symptoms

• A better understanding required of the needs of referred

minority children and families.

Implications for Care

• Discussing parental and client expectations of care early

in the process to prevent dropout

• Process of continuous quality improvement of care
• Viewing care as beginning with the phone call to the

patient care representative

• All patient activities aimed at engagement of client and

parent/guardian

• Educating trainees about the value of quantitatively

measuring progress in treatment

In Summary

• This study provides evidence that individual patient-specific
• utcome measurement is achievable in routine clinic pediatric

mental health care.

• Identifying variables associated with clinical outcomes can be

utilized to direct clinical resources and refine care planning.

• Routine pediatric mental health care delivered under usual and

naturalistic conditions is effective for many children and adolescents

• The challenge comes in making effective treatment available to

a larger population of children and adolescents in need.

REFERENCES

Bearman, S., K., Weisz, J. R. (2015). Review: Comprehensive treatment for youth comorbidity- evidence guided approaches to a complicated problem. Child and Adolescent Mental Health, 20, 131-41. Connor, D. F., Banga, A. (2014). A model of pediatric psychopharmacology-psychotherapy treatment integration in the ambulatory clinical setting. Journal of Child and Family Studies, 23, 686-703. Cox, G. R., Callahan, P., Churchil, R., et al. (2014). Psychological therapies versus antidepressant medications, alone and in combination for depression in children and

• adolescents. The Cochrane Database of Systematic Reviews, 11, CD008324.

Fonagy, P., Cottrell, D., Phillips, J., et al. (2015). What Works for Whom: A Critical Review of Treatments for Children and Adolescents. 2nd Edition. New York, NY: Guilford, Press. James, A. C., James, G., Cowdrey, F. A., et al. (2015). Cognitive behavioural therapy for anxiety disorders in children and adolescents. The Cochrane Database of Systematic Reviews, 2, CD004690 Kennedy, E. (2015). Developing interventions in child and adolescent mental health services: Do we really know what works for whom? Clinical Child Psychology and Psychiatry, 20, 529- 31. Murphy, J. M., Blais, M., Baer, L., et al (2015). Measuring outcomes in outpatient child psychiatry: reliable improvement, deterioration, and clinically significant improvement. Clinical Child Psychology and Psychiatry, 20, 39-52. Weisz, J. R., Kuppens, S., Eckshtain, D., et al. (2013). Performance of evidence-based youth psychotherapies compared with usual clinical care: a multilevel meta-analysis. JAMA Psychiatry, 70, 750-761.