Latinos in Oregon: Studio to School Initiative Trends and - - PowerPoint PPT Presentation

Latinos in Oregon:

Trends and Opportunities in a Changing State

THE OREGON COMMUNITY FOUNDATION

Studio to School Initiative Evaluation Jam Session # 3

April 12th, 2017

Logistics

• Thank you for patience!
• We are recording
• If you can’t hear us,

check your speaker or switch to the phone. Stay on your computer to see the slides!

Logistics

• Everyone is muted, we’ll

unmute for Q&A if feasible

• Please mute your

microphones as well

• Use chat box for

questions, etc.

Upcoming Evaluation Deadlines

When Topic May 22 10am to 11am Assessing Arts Learning: Measuring student learning in and through the arts. Remember that slides and recordings for Jam Sessions 1, 2 & 3 are on the Studio to School website!

• Logic models due April 14th
• Email to Madeline: mbaars@oregoncf.org
• Next eJournal post due May 1
• Spring interviews with project teams

From anecdotes to useful information

“To the professionally trained and attuned ear, an anecdote is scientific data— a note in a symphony of human

• experience. Of course, you have to know how to listen.”
• Michael Quinn Patton

Quantitative versus Qualitative

Quantitative data: things you count, quantify, and measure numerically. Examples:

• Attendance
• Test scores
• # of performances
• # of instruments

Quantitative versus Qualitative

Qualitative data: descriptions of reasons, opinions, motivations, context.

Qualitative data can help you:

• Understand

program’s effects;

• Explore successes

and challenges;

• Communicate what

the program is doing.

Basics of qualitative data

If you want to change attitudes toward or increase the likelihood

• f certain behaviors,

you need qualitative data about attitudes and feelings toward these behaviors.

Use qualitative methods to communicate on an emotional level.

Examples of qualitative data

• Answers to
• pen-ended

survey questions

• Information

from an interview / focus group

• Observations

from a site visit

• Journal entries

Student interviews: RACC Hillsboro

https://www.youtube.com/watch?v=FDw3O8HXGms

Questions?

Remember your logic model?

• Use your logic model to

guide the questions you ask and data you gather

• Look at the output and
• utcomes columns to

formulate evaluation questions

• Output column = raw

data

Anecdotes versus data

A collection of anecdotes can become data when systematically, intentionally, and carefully recorded and thoughtfully analyzed.

Qualitative data collection

• Collect multiple anecdotes
• Look for patterns and common themes that

emerge across anecdotes

• Check the accuracy with other sources

Example: gather anecdotes to capture the experience of a theater performance from the perspective of the audience.

What kind of data do you need?

• Impact data:

program’s effect on participants

• Process data: how

the program works

Determining the right method

Depends on many factors, including:

• Staff capacity
• Program and community culture
• Data sharing plans
• Purpose for collecting data

Qualitative data collection

Method Characteristics Observation Program activities/operations, context. Interviews & focus groups Personal stories and context. Lots of detail. Open-ended survey questions Provide stories and context. Generally less detail. Case studies Produce detailed data of context, challenges, successes and outcomes. Often a combination of qualitative and quantitative sources.

Case study

• Detailed examination of a program,

initiative, project or individual.

• Highlight context, challenges, successes

and outcomes.

• Combine quantitative and qualitative data,

producing detailed information

• Include data from many sources:

interviews, observations, surveys, document reviews.

Case studies: RACC Hillsboro

Questions?

Storytelling with data

Combines quantitative and qualitative data. Participatory form of evaluation.

Storytelling with data

Giving a human face to evaluation strengthens program messages. Incorporate perspectives that speak to the concerns of your target audience. Highlight program progress.

Best practices in story gathering

• Ask permission to

record

• Explain how stories

will be used

• Seek approval

before sharing

• Change names &

distinguishing characteristics

Story circles

A simple, focus group-style method.

At the end of a meeting, ask everyone for a story. Example prompts:

• “What have you learned through being a part of [program]?”
• “How has participating in [program] changed your life?”

Ask the group to summarize the most important lessons from their stories. Record this! Repeat every 3-6 months to assess progress.

Vignettes

Short stories that illustrate and contextualize facts/numbers.

For example, a written report might include:

“We held four concerts, with fifty attendees at each.”

Add an personal account from a teacher:

“We have never had performances like this at our school

• before. So many members of the community came – even

those who don’t have kids! I was amazed. We’re planning to make the performances even bigger next year.”

That’s qualitative info supporting quantitative data– personal statements giving data life!

Visual storytelling methods

• Photovoice: https://photovoice.org/
• Scrapbooking:
• Collect artifacts of program activities, outcomes
• Document challenges, accomplishments, growth
• Story-quilting:
• Each participant makes a square of a personal reflection
• n the program, process, or outcomes.
• Create a quilt of individual/collective experiences.
• Story Theater:
• Participants to role-play program process, challenges,

accomplishments, and outcomes.

Qualitative data analysis

Remember that collecting and sharing stories is not the same thing as analysis. You have data from interviews,

• bservations, written

documents or journals,

• pen-ended survey

questions…

Qualitative data analysis

Some accounts stand alone, providing information about how the program is working. In most cases, you must analyze qualitative data in systematic ways.

Qualitative data analysis methods

• Check circumstances, background, source(s)
• Categorization and coding
• Identify themes, patterns
• Participatory analysis
• Subjects interpret their interview transcripts
• Apply rubrics
• Help with rubric creation:

http://www.studiotoschool.org/evaluation_handbook_arts-2/ http://www.studiotoschool.org/evaluation_guide_arts_based/

Considerations with qualitative data

Start with the data, let it tell you the story

• Don’t only include success stories!

Considerations with qualitative data

Be aware of stories becoming “stale”

Considerations with qualitative data

Complement stories with other data.

Using many forms of data, and multiple perspectives, enhances evaluation quality and impact. This turns anecdotes into accurate, useful information.

Questions?

Reminders

• Slides and recording available on the website
• http://www.studiotoschool.org/info-for-grantees/project-

evaluation-worksheets/

• Upcoming jam session: May 22, 10am
• Logic models due April 14th
• Email to Madeline: mbaars@oregoncf.org
• eJournal post due May 1st
• Spring interviews coming
• Rendezvous Aug 1 – 3 at Oregon Gardens in Silverton