USER FOCUS GROUPS Poll Question 1 How would you rate your libraries - - PowerPoint PPT Presentation

GETTING STARTED WITH USER FOCUS GROUPS

Charmaine Scott

Training Officer Staff Development

Washington University Libraries

• St. Louis, Missouri

Poll Question 1

How would you rate your libraries’ experiences with focus groups?

 None  A little  Somewhat experienced  Much experience

Focus groups in marketing

Which one tasted better?

“Steve” (nods, agreement all around)

Steve was delicious…

“Yes. Steve had just eaten…”

Steve sees the sharks when he enters…

Smiling shark: “Eat both squares please”

PROCESS:

1. Purpose 2. Define 3. Questions 4. Recruiting 5. Conducting 6. Analyzing

Poll Question 2

What are you hoping to get from today’s session?

Choose the most important for you.

 When to use focus groups  Writing questions  Skills needed by a facilitator  Recruiting participants  Steps in data analysis

• 1. Purpose

Clarifying your purpose

Choosing the right tool

Assessment method Pros Cons Sample use Survey via paper, online, phone; one-on-one interviews Private, anonymous, everyone has same questions, control the answers, quick to get results; shorten prep time by using previous surveys; easy to analyze Can’t ask follow up questions; self- reporting may be unreliable; hard to get people to respond (return rate); sample size may be hard to figure LibQual; Indirect Costs Survey Case study Only need a few participants; have a holistic view of those participants Analysis takes time; may not be applied to whole population University of Rochester daily movement study Content Analysis/Analyze existing data Data already exists; fast to gather it; representative of what is actually happening Doesn’t cover potential; could be

• utdated; info might not exist; not

sure if reliable methods were used to gather the data Analysis of circulation statistics Observation Results are reliable; don’t need to seek participants Creep factor—hard to be watched; the motivation for the observed behavior is not apparent; no demographics—not know who you are observing Observation of Art library grad study room Focus group Open discussion’ ideas-free form; can use to get ideas for survey; unexpected feedback occurs Difficulty recruiting participants; easy to get off topic; difficult to analyze data; time consuming to set up and implement Follow up a survey; start with focus groups to decide what to put

• n a survey

Tool: Survey

Assessment method Pros Cons Sample use

Survey via paper, online,

• ne-on-one

interviews, phone Private, anonymous, same questions for everyone, control the answers, quick to get results; shorten prep time by using previous surveys; easy to analyze Can’t ask follow up questions; self- reporting may be unreliable; hard to get people to respond (return rate); sample size may be hard to figure LibQual; Indirect Costs Survey

Tool: Case Study

Assessment method Pros Cons Sample use

Case study Only need a few participants; have a holistic view of those participants Analysis takes time; may not be applied to whole population University of Rochester daily movement study

Tool: Content Analysis

Assessment method Pros Cons Sample use

Content Analysis/ Analyze existing data Data already exists; fast to gather it; representative of what is actually happening Doesn’t cover potential; could be outdated; info might not exist; not sure if reliable methods were used to gather the data Analysis of circulation statistics; blogosphere studies

Tool: observation

Assessment method Pros Cons Sample use Observation Results are reliable; don’t need to seek participants Creep factor—hard to be watched; the motivation for the observed behavior is not apparent; no demographics— not know who you are observing Observation

• f Art

Library graduate student study room

Tool: Focus Groups

Assessment method Pros Cons Sample use Focus group Open discussion ideas free-form; can use to get ideas for survey; unexpected feedback occurs-- cascading Difficulty recruiting participants; easy to get off topic; difficult to analyze data; time consuming to set up and implement Follow up a survey; start with focus groups to decide what to put

• n a survey

Use a focus group:

When you want to….

 understand motivations and attitudes  provide insights about how people perceive a

situation

Legitimate technique?

 Quantitative vs. Qualitative data  Process of disciplined inquiry  Systematic and verifiable

Data Interpretation

 Purpose is to understand perceptions  Can not generalize to whole population

INFER

• 2. Define Focus Groups

Functions:

• 1. Size
• 2. Duration
• 3. Atmosphere
• 4. Common factor

Size

http://www.cnelson.com/gallery/cart-photos/small-person-or-big-chair

Duration

Scott Kerber, Enterprise Publications at http://www.enterprisepub.com/sports/lady-eagles-soar-to-win-at-logan-view/article_80a31860-66c5- 11e0-98bf-001a4bcf6878.html?mode=image

Open Atmosphere

http://www.ocregister.com/articles/county-499134-human-relations.html?pic=2

Common factor

http://ingesidee.de/page.php?pgid=74&lang=en

Functions:

• 1. Size
• 2. Duration
• 3. Atmosphere
• 4. Common factor

• 3. Questions

Stimulate discussion

Question Characteristics

 Conversational  One idea per question  Non-threatening or non-embarrassing  Free of bias

From surface to the deep

Easy to answer More thought needed Deeper attitudes and perceptions

surface below

Open ended

• What is your reaction to…?
• How do you feel about…?
• What is your thinking on…?

Introduce a Subject

 “Please think back…”  “Please think about…”  “When…”  “Let’s talk about…”

Test Your Questions

1.

• ne or two people

2.

typify the audience

3.

test for understanding

Write Test Rewrite Test Toss or keep?

Example: Focus Group Guiding Questions on eLearning

Let’s talk about your Current Usage

• What is your level of expertise with eLearning?
• What do you use eLearning for?
• What do you like about eLearning?

Let’s talk about Usability

• What do you think about eLearning’s interface?
• What improvements do you need?

Let’s talk about Tools in a Learning Management System?

• What do you like about the tools you are

currently using in eLearning

• What do you not like about these tools?
• Which tools are essential?

Library Service Quality Survey (LibQUAL 2004) Follow up Project Graduate Students American History Focus Group Questions:

• 1. First, we’re interested in finding out about the strengths & weaknesses of

the library collections in American History.

• a. How would you say the collections have changed since you started your

program?

• i. Have they improved? Remained the same? Weakened?
• ii. Does anyone else have a different take on it?’
• b. Which collections or resources are indispensable to you? (Why?)
• c. Which major resources are we missing that you would use regularly?
• i. Do you use Mobius often?
• ii. For what kinds of materials do you end up going to Mobius?

Example:

Library Service Quality Survey (LibQUAL 2004) Follow up Project Graduate Students American History Focus Group Questions:

II. We’re also interested in how we interact with you.

• a. What can we do to better support your teaching?
• i. Are there any projects in your classes …?
• ii. Have you ever considered … ?
• iii. How well is the reserve process working for you?
• b. What can we do to better support your research?

…Continued from previous slide

Focus group for Domino’s Pizza

“How many of you believe Domino’s cheese is real cheese made from milk?”

No! No! No!

The “Reveal”

cows farmer

• 4. Recruiting

Poll Question 3

If you have participated in a focus group, why did you do it?

 Curious  Incentive was offered  Wanted to give my opinion  Went along with a friend  Never participated

Recruiting – target audience

 Homogeneity  Gender  Age  Status  Social connections

Recruiting- finding them

 Nomination  Self-selection  Volunteer bank  Class rolls  Past events  Campus groups

 Office of Undergraduate Research  Writing Center  Student government

Recruiting Follow-up

 Over-invite – 10-20%  Respond a.s.a.p. to each inquiry  Give a contact name  Send reminders

Incentives

 Food at session  Gift cards  Class assignment/credit from faculty

Location and schedule

 Accessible venue  Comfortable temperature, well lit  Seating allows participants to see each other  Food is permitted  Directions/signage  Time of day  Day of week

 Participants Form

 Name  Phone  Email address  Program/Major  Year in program  Expected date of graduation  How you would like to be contacted?  Interested in giving feedback in future?

Demographics

Example: participant form

Sign in sheet Focus Group Participants Date _Feb 4, 2010 __________ Name Program/Major Expected Date

• f graduation

Email Phone Interested in future sessions of library testing? Y /N How do you prefer to be contacted-- Email or Phone?

1. Candace xxxxx Undergrad WGSS Anthro May 2011 y 2. Daniel xxxxxxx Undergrad History, Political Science May 2011 Y 3. “Christal”xxxxxx Undergrad Econ 2012 y 4. Lisa xxxxxx Grad/History 2014 y 5. Jenny xxxxxx Grad/History 2013 y 6. Xiaoyan xxxxxx Grad/East Asian 2011 y 7. Zach xxxxxx Grad/Philsophy 2011 Y 8. Nora xxxxxx Grad/Soc. Work 2013 y 9. 10.

Saturation

Questions about Recruiting?

• 5. Conducting a session

Poll question 4

How have you been involved with conducting focus groups?

a.

No involvement

b.

Participant

c.

Moderated

d.

Took notes

e.

Publicity/recruiting/other role

3 Roles

Host Scribe Facilitator

Host

 Name tent/tag  Participant number  Consent form  Demographic form

Scribe/s

 Tape record/video  Write (by hand or PC)  Be neutral  Note body language  Ask questions for clarity

Online focus groups--scribe needed?

Facilitator Characteristics

 Neutral attitude  Relatable  Active Listener

Characteristics

Facilitator Duties

Guide group discussion to keep it on track  Group dynamics  Process  Content

Manages the Content

Probing questions

 Can you talk about that more?  Would you explain further?  Help me understand what you mean.  Can you give an example?

Pause Wait 5 seconds for response before rephrasing

Rephrase to clarify

What are the benefits of garlic? Please tell me about the benefits of garlic. How does garlic help you? What do you like best about garlic?

http://www.focusgrouptips.com/marketing-research-surveys.html (FocusGroupTips.com)

Manages the group

 Experts  Dominant talkers  Shy participants  Ramblers

Redirection

 Does anyone think the opposite is true?  Joe, we’ve heard from you. What do you think Sue?  Let’s hear from someone who hasn’t contributed yet.

Scripting

Welcome Introductions Purpose Ground Rules Icebreaker Discussion Windup

Debrief

 Facilitator  Scribe  Did they see the same

event? Hear the same comments?

 Preliminary insights?

• 6. Analyzing the results

Prep data

 Transcribe/Type notes  Combine  Clean  Number

Sort by cards

Analysis of Qualitative Data

 Coding  Sort  Thematic  Patterns

Synthesize data

 Sort categories again  Identify patterns  Insights  Gut reaction

Reporting

 Include:

 Focus group description:

 Goal  audience  demographics  dates

 Results  Recommendations  Question set

 Present results

Summary

Participants will be able to:

 Evaluate whether a focus group is the best source

for the data they seek.

 Identify questions to ask for the focus groups.  List skills needed by a facilitator.  Describe three phases to use for data analysis.

Poll Question 5 (2 again)

What did you learn from today’s session?

Choose all that are true for you.

 When to use focus groups  Writing questions  Skills needed by a facilitator  Recruiting participants  Steps in data analysis

Poll responses

Charmaine Scott cgscott@wustl.edu 314-935-7277

QUESTIONS?

Bibliography



Conducting Focus Groups, Center for the Study of Student Life at Ohio State University, (http://youtu.be/-27WjZVDlbk)



Ingrid Bens, Facilitation at a Glance! A pocket guide of tools and techniques for effective meeting facilitation, 2nd ed. GOAL QPC, 2008



Ingrid Bens website (http://www.facilitationtutor.com/)



Richard A. Krueger & Mary Anne Casey, Focus Groups: A Practical Guide for Applied Research, 3rd ed. Sage Publications, 2000



Richard A. Krueger website (http://www.tc.umn.edu/~rkrueger/focus.html)



Video stills from Sharks for Snickers ,

(http://youtu.be/8fvCPvzy-kM)



Video stills from Real Cheese in Domino’ s Pizza ,

(http://youtu.be/IgwPw59iqhw)

Richard A. Krueger & Mary Anne Casey, Focus Groups, Sage Publications, 2000 Richard A. Krueger & Mary Anne Casey, Focus Groups, Sage Publications, 2000