A Step-by-Step Guide to Planning and Conducting Your First Nominal - - PowerPoint PPT Presentation

A Step-by-Step Guide to Planning and Conducting Your First Nominal Group Technique Session

• The benefits & limitations of

Nominal Group Technique (NGT)

• When to use NGT
• How to prepare for and conduct

your first NGT session

NGT is a structured group brainstorming and prioritization technique used in boardrooms, classrooms, and research studies.

• You’re interested in identifying

and prioritizing issues.

• The issues you’re exploring are

controversial.

• The group does not easily

generate quantities of ideas.

• Some group members are difficult

to engage.

• Some group members are more

vocal than others.

• Some group members think better

in silence.

Highly structured technique can limit discussion

01

Requires significant planning & preparation

02

Data collection requires more time & staffing than traditional focus groups

03

Requires all participants read & write in the same

language

04

PREPARING FOR AN NGT SESSION

• Team Roles & Job Descriptions
• Facilitation Guide Development

& Piloting

• Materials Checklist

• Facilitator: Guides process
• Note Taker(s): Documents process
• “Floater(s)”: Helps participants as-needed:
• Manages arrival tasks (consent forms, table tents)
• Assists as-needed during group
• Collects score sheets
• Manages closing tasks (incentives, parking

vouchers)

• Optional: Childcare Provider(s)

Facilitator should be:

• Seen by group as a

neutral and trustworthy.

• Skilled in managing

group dynamics.

• Good at reading body

language and anticipating needs.

• Comfortable improvising.
• Skilled at working with a

team.

Note Taker should be:

• Comfortable recording information on a flipchart
• r projection in front of group.
• Comfortable asking clarifying questions when-

needed.

• Skilled at working with a team.

“Floater” should be:

• Organized.
• Skilled at reading body language and anticipating

needs.

• Skilled at working with a team.

Childcare Provider should be:

• Comfortable supervising a small group of children.
• Experienced working with a range of ages.
• Skilled at managing conflict.
• Comfortable improvising.
• Cleared by all relevant state/federal agencies and

trained in CPR.

Script an opening statement that includes “housekeeping” information about the space and process, and group norms. Script a closing statement that

• utlines next steps (if any)

participants can expect and check-

• ut logistics.

Because the brainstorming and prioritizing group process is time consuming, limit the number of questions in your guide. Avoid “double-barreled” questions (two questions in one).

Walk through the guide with someone familiar with your target population. This person should be comfortable providing constructive feedback regarding:

• Whether the instructions for each

step are clear.

• Whether the purpose of each

question is clear.

• Whether the language is socially

appropriate, age appropriate, and culturally appropriate.

• Consent Forms
• Demographic Sheets
• Participant Packets with NGT instructions

& space for writing and prioritizing lists (include page numbers)

• Table Tents
• Markers for writing names on table tents
• Pen for each participant
• Audio Recorders (2-3 depending on size of

group)

• Flipchart or computer and projector
• Incentives / Parking vouchers
• Meal / Snacks

CONDUCTING AN NGT SESSION

• Room Set-up
• Session Structure
• Facilitation Tips

Welcome table at entry

Sign-in Sheet Table Tents & Markers Consent Forms Demographic Forms

Assure all participants have a writing surface and pen Assure all participants can easily see flipchart / projection

• Sign-in & paperwork as individuals

arrive

• Food Available
• Welcome from facilitator and/or host
• Location of restrooms
• Overview of session
• Setting group norms for the session
• Brainstorm and prioritize issues
• Closing, including any future follow-up
• Incentives and related paperwork

processing

• Ask the group to silently

brainstorm as many responses as possible to a single question. For example: “What makes accessing healthcare difficult?”

• Give participants 3-5

minutes to complete their list

Participant 101

Unreliable transportation Lack of insurance Lack of childcare Language barriers Work schedules Long wait-times

The facilitator invites the group to go around the circle and each share

• ne item from their list

until everyone’s items have been shared. The note taker writes the items on the flipchart in the order that they are named.

The facilitator explains to participants that:

• Once an item has been named, there’s no need to

name it again – participants should only share new items.

• When a participant runs out of new items they can

“pass” when it’s their turn.

• If a participant thinks of a new item during the

group share, they don’t have to limit themselves to the items on their written list.

The notetaker:

• Writes participants’ items in the order in which

they are named.

• Writes the items verbatim. Where necessary, works

with the participant to summarize their item into a single word or phrase.

• Uses A) B) C) … As the bullet points for each item.

This will make referring to items during the discussion easier.

• A. Tight finances
• B. Lack of health insurance
• C. Unreliable transportation
• D. Difficult to schedule appointments
• E. Long wait-times
• F. Hard to get time off work
• G. Lack of childcare
• H. Language barriers

I . Work schedules

• J. Shift work

After the list is complete, the facilitator:

• Explains that this is not yet the time for debate – that

will come later.

• Reads the list aloud and asks the group if everyone

understands the meaning of each item.

• Calls the group’s attention to items that need

clarification.

• For example, are there two very similar items on the

list? If so, the facilitator can ask the individuals who named the items to help the group distinguish between them. The group may then decide to keep both items, merge the items, or eliminate one of the items.

• A. Tight finances
• B. Lack of health insurance
• C. Unreliable transportation
• D. Difficult to schedule appointments
• E. Long wait-times
• F. Hard to get time off work
• G. Lack of childcare
• H. Language barriers

I . Work schedules

• J. Shift work

For example:

• On this list, the facilitator noted that four items

were similar:

• D. Difficult to schedule appointments
• F. Hard to get time off work
• I. Work schedules
• J. Shift work

• A. Tight finances
• B. Lack of health insurance
• C. Unreliable transportation
• D. Difficult to schedule appointments
• E. Long wait-times
• F. Hard to get time off work (esp. shift

work)

• G. Lack of childcare
• H. Language barriers

I . Work Schedules

• J. Shift work

In this example, as a group, participants decided to:

• Merge “shift work” and “hard to get time off

work”

• Keep “difficult to schedule appointments”
• Eliminate “work schedules”

When the group is satisfied with the list, the facilitator asks participants to:

• Silently choose their five highest priority items (or in

this case, the items that participants think pose the greatest challenges) selecting only from the items

• n the group’s list.
• Prioritize the list by writing the number 5 next to

their highest priority, 4 next to their second highest priority, and so on. *this can be confusing, make sure participants understand that the higher the priority the higher the number.

• Hand their lists to the note taker to tally.

Participant 101 5 Lack of health insurance 4 Language barriers 3 Unreliable transportation 2 Hard to get time off work (esp. shift work) 1 Long wait-times

Notetakers / Floaters:

• Tally each item on the flipchart showing the

group’s consensus on highest priority items.

• Write the final number of votes next to each item.
• Circle or otherwise denote the items that received

at least one tally mark.

• A. Tight finances 5
• B. Lack of health insurance 13
• C. Unreliable transportation 10
• D. Difficult to schedule appointments 5
• E. Long wait-times 6
• F. Hard to get time off work 11

(esp. shift work)

• G. Lack of childcare
• H. Language barriers 9

Rinse and Repeat (almost). Ask participants to:

• Silently choose their three highest priority items,

selecting only from the items on the group’s list.

• Prioritize the list by writing the number 3 next to

their highest priority item, 2 next to their second highest priority, and 1 next to their third priority.

• Hand their list to the note taker to tally.

Participant 101

3 Lack of health insurance 2 Unreliable transportation 1 Hard to get time off work (esp. shift work)

Notetakers / Floaters:

• Tally each item on the flipchart showing the

group’s consensus on highest priority items.

• Write the final number of votes next to each item.
• Circle or otherwise denote the top 1, 2, or 3 items,

depending on how you plan to use the data.

• A. Tight finances
• B. Lack of health insurance

9

• C. Unreliable transportation 8
• D. Difficult to schedule appointments
• E. Long wait-times 1
• F. Hard to get time off work

(esp. shift work) 3

• G. Language barriers 2

You’ve reached the end of your structured brainstorming process. Now, you can pose other questions

• f interest. For example:
• What ideas do you have for

addressing the barriers we listed?

• How do you think this list might

have looked different if this group included doctors? Social workers?

• It’s easy to overwhelm

participants with details / process – speak slowly, and share instructions one step at a time.

• Watch for signs of confusion and

be prepared to repeat instructions.

• Rely on your team members –

each person has a role.

DATA MANAGEMENT & ANALYSIS

• Storing NGT data
• Analyzing NGT data

• If you used flipcharts to display your lists / tally

priorities during the session, take photos of each list to store the raw data electronically.

• If your participants completed demographic forms
• r any other paperwork on paper, enter this

information into a secure database in accordance with your IRB’s standards.

• If you recorded your sessions, consider having

them transcribed by a professional transcriptionist. Store data with identifying information securely and in accordance with your IRB’s standards.

• If you’re comparing NGT data from multiple focus

groups, be prepared to lump and distinguish between “like items” – different groups of participants may use slightly different language to name very similar items.

• Transcripts can be used to develop definitions for

items – where possible, use participants’ own words to define items.

• Nominal group technique is reliable for deciding research priorities. Brit

Med J. Apr 8 2000;320(7240):E.

• Allen J, Dyas J, Jones M. Building consensus in health care: a guide to using

the nominal group technique. Br J Community Nurs. Mar 2004;9(3):110-114.

• Sarre G, Cooke J. Developing indicators for measuring research capacity

development in primary care organizations: A consensus approach using a nominal group technique. Health Soc Care Community. May 2009;17(3):244- 253.

• Van de Ven AH, Delbecq AL. The nominal group as a research instrument for

exploratory health studies. Am J Public Health. Mar 1972;62(3):337-342.

• The engagement project described here was funded through a Patient-

Centered Outcomes Research Institute (PCORI) Eugene Washington PCORI Engagement Award (#3988-TTUP).

• PCORI is an independent, nonprofit organization authorized by Congress in
• 2010. Its mission is to fund research that will provide patients, their caregivers,

and clinicians with the evidence-based information needed to make better- informed healthcare decisions. PCORI is committed to continually seeking input from a broad range of stakeholders to guide its work.

• The views presented in this presentation are solely the responsibility of the

author(s) and do not necessarily represent the views of the Patient-Centered Outcomes Research Institute (PCORI), its Board of Governors or Methodology Committee.

• Much appreciation for the efforts of Lisa Jacobs in the production of these

materials.

• Dr. Peter F. Cronholm

Peter.Cronhnolm@pennmedicine.upenn.edu

• Dr. Wendell E. Kellum

Wendell.Kellum@pennmedicine.upenn.edu