Elevating Lived Experience in Coordinated Entry Evaluation THE ROLE OF QUALITATIVE DATA 1
Elevating Lived Experience in CES Evaluation Understand how to Consider tech options to Become familiar with include qualitative data in streamline feedback qualitative data collection the CE eval framework processes & analysis strategies 2
Elevating Lived Experience in CES Evaluation 3
Agenda 4
Rationale HUD Honoring diversity, Better & more actionable requirements equity & data inclusion 5
Elevating Lived Experience in CES Evaluation HUD requirements Coordinated Entry Notice Section II.B.15 6
Elevating Lived Experience in CES Evaluation and HUD Guidance 7
Elevating Lived Experience in CES Evaluation “When we consult the experts, service Better, implementations are made more relevant and more responsive.” actionable data 8
Elevating Lived Experience in CES Evaluation Honoring diversity, equity, and inclusion 9
Let’s Go 10
11
Qualitative Data Strategies 12
Qualitative Data Strategies 13
Qualitative Data Strategies 14
Qualitative Data Strategies 15
Qualitative Data Strategies | Valence 16
Qualitative Data Strategies | Multipart Qs 17
Qualitative Data Strategies 18
Qualitative Data Strategies Qualitative data Mixed methods HMIS data 19
20
Data Collection Considerations 21
Data Collection Considerations 22
Data Collection Considerations 23
Data Collection Considerations|Being Concrete 24
Data Collection Considerations 25
Data Collection Considerations * aka retrospective cued recall 26
Qualitative Data Strategies 27
Data Collection Considerations 28
Notes from the Field 29
Using Participant, Provider, and Community Leader Focus Groups for CE Evaluation 30
Perspective Learning Who are the customers? • Still Homeless • Formerly Homeless • Intake/Case Manager/Outreach staff • Program Managers and Executive Directors • Community Champions and Visionaries 31
Formerly Intake Workers/ Homeless Case Managers/ Outreach Staff Still Homeless Program Managers Community and Executive Champions and Directors Visionaries 32
Focus Group Design Previously Housed Still Homeless • Organizations to whom referrals • Names were randomly selected from were made for permanent housing by-name list resources • Shelter staff talked with potential • Each organization identified 1-3 participants to gauge willingness to potential participants participate • Used a flyer to standardize • Provided lunch as participation information about focus groups incentive • Offered payment for client time, via • Offered payment for client time, via gift cards gift cards 33
Focus Group Design Intake/Case Manager/ Outreach and Program Manager/ Executive Director • Blanket invitation for each organization to voluntarily participate • Organizations chose representatives • Organization representatives limited to 1 Visionaries and Champions • Completely self-identified and balanced by the community 34
Implementation: Still Homeless and Formerly Homeless • Encourage storytelling to understanding client involvement in CE process • Use language appropriate to the group participants • Always pay for participant time • Use a facilitator who is a person the clients don’t associate with influencing their ability to get housing. 35
Implementation: Other Focus Groups • Ensure a common understanding of CE process • Use information from prior groups to stimulate discussion and clarify challenges • Get participants to articulate their idea of where they fit in the process and where they provide best value 36
Conduct focus groups in order: still homeless to champions Use information from prior focus groups to refine questions for following focus groups Use an objective person to take notes and document the discussion 37
Let still homeless/formerly homeless tell their stories and identify qualitative data through their storytelling Be completely non-judgmental Know the CE system design and participating agencies before conducting the focus groups Community responsibility for recruiting and balancing participants 38
Do not use a local person to conduct still homeless focus group, to minimize expectations for participation. Make sure you have a really good notetaker. Don’t try to facilitate AND take notes. Always strive to use good research and eval techniques, such as random selection and participant balancing. 39
Useful Resources Guidelines for Conducting a Focus Group https://irep.olemiss.edu/wp- content/uploads/sites/98/2016/05/Trinity_Duke_How_to_Conduct_a_Focu s_Group.pdf Center for Community Health and Development Community Toolbox, Chapter 3, Section 6 https://ctb.ku.edu/en/table-of-contents/assessment/assessing- community-needs-and-resources/conduct-focus-groups/main 40
? 41
42
Qualitative Data Analysis is Content Analysis 43
Qualitative Data Analysis is linked to… • An inductive, bottom-up approach to the collection and description of the data • Results in a qualitative interpretation • Often communicated visually, e.g. through a diagram 44
Content Analysis Steps Close reading Group codes And categories Analyze to develop into into & codes (tags) categories… themes interpret 45
Content Analysis Steps • Read, read, reread text • Label chunks of text with key words and phrases (codes) • Iterate: you can re-read and Close reading apply multiple codes to the to develop same passages codes (tags) • Use highlighting, underlines, comments etc. as needed 46
Content Analysis Steps Close reading to develop codes (tags) example 47
Content Analysis Steps • Group your tags into categories -e.g. ‘felt angry at staff’ • If you’ve color -coded or used key words, create your themes Organize tagged based on this text • Move text around, rearrange it, into etc. to facilitate the process themes • For rigor, have a partner who does the same 48
QDA Steps • Group your categories into themes -e.g. ‘feelings about staff’ • Move text around, rearrange Organize themes it, etc. to facilitate the process into categories • Iterate until you have a cohesive set of 4-8 themes 49
Content Analysis Steps • Interpret the data by using frequency analyses to describe categories and themes and • Stratify by positive and Analyze negative sentiment, if useful and interpret and using frequencies • Highlight key words/ key phrases 50
QDA utilizing AI 51
QDA Using AI • Sentiment analysis • Emotion detection • Keyword extraction 52
QDA Semantic Analysis with AI Open source, low cost/no cost option 53
QDA Sentiment Analysis with AI pasted text 54
QDA Sentiment Analysis with AI 55
Key words https://www.jasondavies. com/wordcloud/ 56
QDA Tools Quick Guide 57
58
Streamlining Tools and Resources Lean Data Efficiency and automation and Listen4Good model through tech 59
Lean Data & Listen4Good Model • Emphasize rigorous but efficient methods for soliciting feedback from a large number of clients • The methods are streamlined through a variety of tech options • The emphasis is on listening to, and acting upon, the voices of those who receive services 60
Lean Data & Listen4Good Model 5 core survey questions underlie the model 1. How likely is it that you would recommend […] to a friend or family member? 2. What is […] good at? 3. What could […] do better? 4. Overall, how well has [….] met your needs? 5. How often do staff at […] treat you with respect? 61
Lean Data & Listen4Good Model The answers to these questions, especially the question #3: “What could […] do better,” allow clients to suggest ways to promulgate change. Profiling their input and its influence on program delivery and on clients’ own self -advocacy changes the story lens, putting client agency at the center. 62
Lean Data & Listen4Good Model Perceptual Feedback: What’s it all about? 63
Closing the Loop 64
Tech Options Tools to implement efficient & effective QDA • Internet-based technologies* • Mobile Apps • Text messaging (SMS) * Including computer transcription 65
Tech Options Lean Data Field Guide 66
Tech Options Enhancing evaluation through technology 67
Tech Options Quick Guide 68
Notes from the Field 69
The Power of Feedback A homeless provider in Little Rock, AR closes the loop vimeo.com/316157380
The Power of Feedback A homeless services organization in Salt Lake City collects direct feedback daily with kiosks https://www.youtube.com/w atch?v=F_zlIShnxrI
Privacy, Confidentiality and Informed Consent 72
? 73
Recommend
More recommend
