Decisions and Disparities: Disentangling Sources of Inequity John - - PowerPoint PPT Presentation
Decisions and Disparities: Disentangling Sources of Inequity John D. Fluke, Kempe Center for Children, University of Colorado School of Medicine February 7, 2013 Shubert Center for Child Studies, Case Western Reserve University Content
John D. Fluke, Kempe Center for Children, University of Colorado School of Medicine
February 7, 2013 Shubert Center for Child Studies, Case Western Reserve University
with the Canadian Incidence Study of Reported Child Abuse and Neglect (CIS)
Donald J. Baumann, Ph.D. Saint Edwards University Len Dalgleish University of Sterling, UK John D. Fluke, Ph.D. Kempe Center for the Prevention and Treatment of Child Abuse and Neglect, University of Colorado School of Medicine
(GADM) and Thresholds
ASSESSMENT Child protection Decisions/Actions Screening Assessment Placement Reunification
points along the child protection continuum
decision and action
Decision Making
Consequences
How do we make progress in integrating and improving clinical/professional judgment in the assessment process? What and whose consequences are we actually most concerned about? What are the best ways to influence decision actions?
External Factors Decision Maker Factors Organizational Factors Decision Making Outcomes Influences Decisions Outcomes Case Factors
(Baumann, Dalgleish, Fluke &Kern, 2011)
External Factors Decision Maker Factors Organizational Factors Decision Making Outcomes Influences Decisions Outcomes Case Factors
External Factors Decision Maker Factors Organizational Factors Decision Making Outcomes Influences Decisions Outcomes Case Factors
External Factors Decision Maker Factors Organizational Factors Decision Making Outcomes Influences Decisions Outcomes Case Factors
External Factors Decision Maker Factors Organizational Factors Decision Making Outcomes Influences Decisions Outcomes Case Factors
External Factors Decision Maker Factors Organizational Factors Decision Making Outcomes Influences Decisions Outcomes Case Factors
used as a way to summarise the case information.
decision about a course of action?
sufficient to warrant taking protective action.
decisions are based on information that is
and uncertain.
under time pressure in a highly emotional atmosphere.
and resource constraints, media interest, unpredictability of
Decision making under uncertainty.
The general model for assessment and decision making.
Separates: The assessment of the situation. From: The decision to something about it. – Qualitatively different factors influence assessment and decision making. Distinguishes: The person’s ability to detect the need to take action (how good they are). From: The person’s willingness to take action (their threshold).
– Prior History of Maltreatment – Child Disability – Type(s) of Maltreatment – Severity of Maltreatment – Substance Abuse – Younger children – Domestic Violence – Family Stress – Lacking Social Supports – Inability to Use Resources – Provision of Services
Sensitivity (true positive)
N = 210,642
High threshold Low threshold
The assessment has an Area Under the Receiver Operator Curve = 63%: Prevalence assumed to be 10%: Applied to 100,000 children
HIGHER THRESHOLD LOW THRESHOLD
derives from case information on the Child, the Family and the nature
concerns.
into operationally defined factors. E.g. A comprehensive system
Dalgleish and Drew (1989)
for Action derives from the experiences and history of the worker. – Possible consequences for the different stakeholders. – How the worker values the consequences.
If the Assessment is ABOVE the Threshold, then ACTION is taken. If the Assessment is BELOW the Threshold, then NO ACTION is taken.
Threshold
Factors Influencing Threshold for Action
Information from Experiences and Organizational Factors)
HIGH LOW
Assessment Dimension: e.g. Risk or ‘Level of Concern’ Assessment
Factors Influencing Assessment.
Information from Current situation being Assessed. The Case Factors.
needs little evidence before taking action.
Assessed level of risk or need Low High Threshold W1 Threshold W2 W2 Assessment. W1 Assessment.
If threshold high, W2 needs
much evidence before taking action.
Even if they agree on the
assessment,
they disagree about taking
action.
Yes No
*From Len Dalgleish, 2000
External Factors Decision Maker Factors Organizational Factors Decision Making Outcomes Influences Decisions Outcomes Case Factors
– Safety – Permanence – Well-Being
– Satisfaction – Turnover – Corrective Actions – Reorganization – Redefinition of Functions
– Public Anxiety – Media Scrutiny – Legislative Scrutiny
ASSESSMENT Child protection Decisions & Actions Screening Assessment Services Placement Reunification
Incidence?
33
– Based on data from one child welfare decision (e.g., new placements/population) – Easiest to obtain
– Based on data from at least two child welfare decisions (e.g., new placements/opened cases)
Population Based Denominator Ratio= e( ∑ ln(Decision Based Denominator Ratio))
34
2.34 2.42 2.16 2.69 2.34 1.03 0.89 1.35
Population Denominator Decision Denominator
Comparison of Population and Decision Based Disparity Ratios - Colorado 2009 African American Children with Respect to White Children
Decision Points
2.34 2.42 2.16 2.69 2.34 1.03 0.89 1.35 0.86 0.89 0.79 1.07 Population Denominator Decision Denominator Poverty Adjusted Population Denominator
Comparison of Population and Decision Based Disparity Ratios - Colorado 2009 African American Children with Respect to White Children
Decision Points
– What are the source of Disparities?
Each Decision Point
(Baumann, et al., 2011) and the Explanatory Factors to Frame Research Questions
– Are Decision Making Errors (False Positives, False Negatives) Disparate? – Some examples
37
Alan J. Dettlaff, PhD, MSW Jane Addams College of Social Work University of Illinois at Chicago Stephanie L. Rivaux, PhD Texas Department of Family and Protective Services Donald J. Baumann, Ph.D. Texas Department of Family & Protective Services John D. Fluke, Ph.D. Child Protection Research Center American Humane Association Joan R. Rycraft, Ph.D. University of Texas at Arlington
Disentangling Substantiation: The Influence of Race, Income, and Risk
Child Welfare
making points along the child welfare pathway is essential to understanding and addressing racial disproportionality.
risk factors that are controlled for, including poverty, the less likely it is that studies examining racial disparities will find evidence of racial bias.
welfare systems.
identify the source of disparities found at the substantiation decision, including measures of family income and caseworker perceptions of risk.
Protective Services (DFPS) between 09/1/2003 and 02/28/2005
– 38.7% Hispanic, 19.7% African American, 38.9% White, 2.8% Other
– About 2/3 of sample had a family income less than $20,550.
– Less than $10,150 – $10,150 - $20,549 – $20,550 - $40,549 – $40,550 and greater
concern rated on 5-point Likert-type scale, high scores = higher risk
age group of youngest child, number of children
Risk Assessment Scores by Race and Income
5 10 15 20 25 Low- income Hig her- Low- income Hig her- Low- income Hig her- Low- income Hig her- Low- income Hig her- Low- income Hig her- Low- income Hig her- Low- income Hig her- African American Hispanic Other White African American Hispanic Other White Substantiated Not Substantiated Mean Risk Score
Category means Test Statistic p Risk by Race F(3) = 85.92 <.001 African American 15.53 Hispanic 15.22 Other 15.36 Anglo 15.80 Risk by Income F(3)=1344.01 <.001 Less than $10,149 16.76 $10,150-$20,549 15.12 $20,550-$40,549 14.48 $40,550 and greater 14.19 Income by Race x2(9) = 7979 <.001 Substantiation by Race x2(3) = 93.01 <.001 Substantiation by Income x2(3) = 3,034 <.001 Substantiation by Risk OR = 1.197 <.001
Race OR CI p African American 1.006 (0.974-1.039) 0.710 Hispanic 0.911 (0.885-0.938) <.001 Other 0.948 (0.883-1.018) 0.143 White* Income Less than $10,150 1.956 (1.858-2.060) <.001 $10,150-20,549 1.290 (1.226-1.357) <.001 $20,550-40,549 1.103 (1.048-1.161) <.001 $40,550 and greater*
* Reference
Race OR CI p African American 1.148 (1.104-1.193) <.001 Hispanic 1.209 (1.167-1.252) <.001 Other 1.231 (1.127-1.346) <.001 White* Income Less than $10,150 0.936 (0.878-0.998) 0.042 $10,150-20,549 0.850 (0.799-0.905) <.001 $20,550-40,549 0.896 (0.841-0.954) 0.001 $40,550 and greater* Risk Assessment 1.205 (1.201-1.209) <.001
* Reference
assessing the role of race as an independent causal factor in decisions made by child welfare systems.
non-racial explanations that are included in studies examining decision-making, the less likely it is that race will emerge as an explanatory factor.
this contention, suggesting that race interacts with other variables in a complicated manner that varies depending on the factors that are included in statistical models.
relationship between race, income, and caseworkers’ assessment of risk.
If the Assessment is ABOVE the Threshold, then ACTION is taken. If the Assessment is BELOW the Threshold, then NO ACTION is taken.
Threshold
Factors Influencing Threshold for Action
Information from Experience & History
(Race) HIGH LOW
Assessment Dimension: e.g. Risk or ‘Level of Concern’ Assessment
Factors Influencing Assessment
Information from Current situation being assessed. The Case Factors (Income)
A General Model for Assessing the Situation and Deciding what to do about it (Dalgleish)
assessment scores.
unsubstantiated cases were assessed by caseworkers as having lower risk than White families.
predictor of substantiation, while race was a significant predictor.
it is not a factor that influences the threshold for decision.
threshold used by caseworkers in making the substantiation decision.
higher for Whites than it is for African Americans.
John Fluke, Ph.D. Child Protection Research Center American Humane Association Barbara Fallon. Factor-Inwentash Faculty of Social Work, University of Toronto, Ontario, Canada Bruce MacLaurin Faculty of Social Work University of Calgary, Calgary, Canada Martin Chabot Centre for Research on Children and Families, McGill University, Montreal, Canada Cindy Blackstock First Nations Child and Family Caring Society, Ottawa, Canada
families who come into contact with child welfare authorities.
data on First Nations, Métis and Inuit children
Reported Child Abuse and Neglect
– 1998 – 2003
non-Aboriginal child welfare organizations and are funded by the provinces/territories
First Nations child and family service agencies (there are 125 in Canada). These agencies receive statutory authority from the provinces but are funded by the federal government
– 55 child welfare sites was selected from 327 child welfare service areas – October 1, 1998 to December 31, 1998
– Worker forms completed
– Organizational Questionnaire
– Investigations that remained open for
(versus no formal placement)
– Two Levels
Model 2 (Parsimonious case factors and Parsimonious organizational factors)
Estimate S.E. Est./S.E. P-Value Odds ratio 95 % C.I. Variables Child and Family Characteristics - Level One (report child pair) Type of Maltreatment (presence of type) Emotional maltreatment
0.255
0.000 0.355 0.215 0.586 Mental or Emotional Harm (present) 1.021 0.174 5.881 0.000 2.776 1.974 3.904 Number of Moves Two or more moves 1.067 0.246 4.329 0.000 2.907 1.795 4.708 Caregiver Functioning Presence of Three or more Concerns 0.900 0.174 5.174 0.000 2.460 1.749 3.459 Cooperation (present)
0.232
0.012 0.560 0.355 0.882 R-squared 0.195 0.033 5.975 0.000 Organizational Characteristics – Level Two (Local CPS Agency) Aboriginal Investigations (20% investigations are aboriginal caregivers) 1.124 0.328 3.425 0.001 3.077 1.618 5.853 R-squared 0.327 0.131 2.492 0.013 Direct explained variation 19.76% m.a.e. 0.25324
Slide 57
External Factors Decision Maker Factors Organizational Factors Place Child Out of Home Outcomes Case Factors
Maltreatment
Functioning
Investigation Caseload
Original 1998 cut point Replication 1998 & 2003 cut point
in out of home care not due to differential decision making regarding specific children or families
– Independent assessments indicate that resources in tribal serving agencies is disparate – Our 2003 replication suggests that social work education may moderate the effect of aboriginal caseload
proportion of Aboriginal families on the caseload means and implies
situation from the decision to take action is useful in thinking about the range of decision improvement strategies.
types of factors and some factors may be modifiable.
well.
evidence to parse out the sources of disparities at each decision point in each CW system.
disparities will be based on:
– Identifying the decisions where disparities emerge – Focusing on the factors that are most important for each decision.
American Humane Association.
M.J., James, J.Kromrei, L., Craig, S., Capouch, D., Sheets, J., Ward, D., Breidenbach, R., Hardaway, A., Boudreau, B., & Brown, N.(2010).Disproportionality in child protective services: The preliminary results of statewide reform efforts.Texas Department of Family and Protective Services.March.
H., Baumann, D.J., & Fluke, J.(Eds.).Worker Improvements to the Decision and Outcome Model (WISDOM): The child welfare decision enhancement project. The Children’s Bureau, Washington, D.C.
detection theory.In B.Brehmer & C.R.B.Joyce (Eds) Human Judgment: The SJT view.North Holland Elsevier.
comprehensive guide to frameworks and their use. (pp.86-99).Dorset, UK: Russell House Publishing.
11th International Congress on Child Abuse and Neglect, Dublin, August.
influence of race, risk and poverty on the substantiation decision in child welfare. Children and Youth Services Review .
among aboriginal groups: An application of the decision-making ecology through multi-level analysis.Child Abuse and Neglect. 34, 57-69.
K.(2001).The dynamics of unsubstantiated reports: A multi-state study – final report.Denver, CO: American Humane Association.
children in foster care: A structural equation model.
in health and medicine: Integrating evidence and values.” Cambridge University Press.
142-145.
Prize Lecture, December 8.
crisis of confidence in child protection. Children & Youth Services Review, 33(11), 2076-2086. doi:10.1016/j.childyouth.2011.04.019.
driving risk assurance levels.Social Policy Journal of New Zealand, 28, 97-132.
communication in mental health law.American Psychologist, September, 931-938.
Review, 27, 375-391.
intersection of race, poverty and risk: Understanding the decision to provide services to clients and to remove children.Child Welfare, Special Issue on Disproportionality.87, 151-168.
Review, 23(6) 579-598.
138.
Economic Review, Vol XLIX, 3,” in Radner and Radner (Eds.), Decision and Organization.
Welfare League of America.New York.
301-340.
Applications: Some Challenges.International Journal of Industrial Engineering: Applications and Practice, Vol.2, No.1, pp.35-44, 1995.
1131.2001.
medical decision-making.Artificial Intelligence, Issues 1-2, pp.145-172.
agencies, in Children and Youth Services Review.17(4) pp.523-535.
factors and interactions with child race.
64
John Fluke John.fluke@ucdenver.edu