Conducting Qualitative Research Remotely: Investigating Conceptual - - PowerPoint PPT Presentation

Conducting Qualitative Research Remotely:

Investigating Conceptual Understanding of Chemistry Outreach Practitioners

Justin M. Pratt & Ellen J. Yezierski

Methods in Chemistry Education Research - Session 2 June 11, 2020

Getting on the Same Page – Let’s Talk about Terms

• Outreach
• Science Outreach
• Informal Science Education
• Chemistry Outreach
• Informal Chemistry Education
• Communicating Chemistry in Informal

Environments

• Working definition: Chemistry Outreach is any event that occurs outside of a normal

classroom setting designed to increase interest, understanding, and/or involvement in

• chemistry. Targeted audiences are wide ranging and can include the general public,

children/students, and/or teachers.

2

Wang, L. Highlights from National Chemistry Week 2019. Chemical & Engineering News, 2019, 97(48). https://cen.acs.org/acs-news/programs/Highlights-National- Chemistry-Week-2019/97/i48

Examples of Chemistry Outreach

• Public Lectures
• Science Festivals
• Informational Videos
• Demonstration Shows (sometimes

referred to as “magic shows”)

• Afterschool Activities

3

Wang, L. Highlights from National Chemistry Week 2019. Chemical & Engineering News, 2019, 97(48). https://cen.acs.org/acs-news/programs/Highlights-National- Chemistry-Week-2019/97/i48

Unique Population of Outreach Practitioners

• College students involved in extracurricular student organizations that conduct
• utreach voluntarily during personal time
• Two major collegiate chemistry organizations:
• Student Chapters of the American Chemical Society
• Collegiate Chapters of the Alpha Chi Sigma Fraternity
• Reach approximately 1 million outreach event attendees each year
• National Survey (N = 206)
• Goal: Audience Learning
• Common Activities: Elephant Toothpaste, Liquid Nitrogen Ice Cream, Making Slime
• Unanswered Question: How accurate and in-depth is their chemistry content knowledge?

Connelly, T. M. (2015) Why Is Chapter Community Outreach So Important? inChemistry, 24 (1), 3. Pratt, J. M. (2017) Alpha Chi Sigma and Chemistry Outreach: Promoting the Second Object. The HEXAGON, 108 (1), 8–9. Pratt, J. M., & Yezierski, E. J. (2018). Characterizing the landscape: Collegiate organizations’ chemistry outreach practices. Journal of Chemical Education, 95(1), 7-16.

4

Qualitative Data Collection in an Online Environment

• Need access to research participants
• Gatekeepers
• Need to elicit desired data
• Need to develop rapport with participants

Pratt, J. M., & Yezierski, E. J. (2018). A novel qualitative method to improve access, elicitation, and sample diversification for enhanced transferability applied to studying chemistry outreach. Chemistry Education Research and Practice, 19(2), 410-430.

Access Elicitation

Rapport

5

Accessing Our Research Participants – Story of Flexibility

• Goal of Maximum Variation Sampling
• Multiple recruitment techniques with varied success
• Gatekeepers were primary contacts
• Email Faculty Advisor to student organization
• Email Department Chair at institution
• Testing for data saturation with targeted recruitment

from awarded chapters

Patton, M. Q. Qualitative Research and Evaluation Methods, 3rd ed.; Sage Publications, Inc.: Thousand Oaks, CA, 2002. Pratt, J. M., & Yezierski, E. J. (2018). A novel qualitative method to improve access, elicitation, and sample diversification for enhanced transferability applied to studying chemistry outreach. Chemistry Education Research and Practice, 19(2), 410-430.

6

Diversity of Sample

Pratt, J. M., & Yezierski, E. J. (2018). A novel qualitative method to improve access, elicitation, and sample diversification for enhanced transferability applied to studying chemistry outreach. Chemistry Education Research and Practice, 19(2), 410-430.

Year in School n Major n Sophomore (Second Year) 5 Chemistry or Biochemistry 22 Junior (Third Year) 11 Science (Non-Chemistry) 11 Senior (≥ Fourth Year) 19 Non-Science 2 Graduate Student 2 Chemistry Graduate Student 2 Sex n Males 17 Females 20 School Type n Public School 17 Private School 5

22 different universities

7

Eliciting Desired Data

• Multimedia-based semi-structured interviews
• Average duration = 1.5 hours
• Transcribed verbatim

Pratt, J. M., & Yezierski, E. J. (2018). A novel qualitative method to improve access, elicitation, and sample diversification for enhanced transferability applied to studying chemistry outreach. Chemistry Education Research and Practice, 19(2), 410-430.

8

Eliciting Conceptual Understanding

Research Question: For this sample of college student outreach practitioners, how accurate and in-depth is their chemistry content knowledge? Elephant Toothpaste

9

What is the Elephant Toothpaste Reaction?

Elephant Toothpaste – Tulsa City County Library. Tulsa Children’s Museum Discovery Lab. https://events.tulsalibrary.org/event/83253

10

What is the Elephant Toothpaste Reaction? 30% H2O2 KI Dish Soap

11

What is the Elephant Toothpaste Reaction? 30% H2O2 KI Dish Soap

https://www.youtube.com/watch?v=p1eG2y2mn54

12

Pilot Study Results

Participants had prior experience facilitating the demonstration with an audience prior to the interview. What chemicals go into elephant toothpaste? “I don't remember exactly.” Barbara (Senior/PUI/Chem) “Hydrogen peroxide and…something else that you mix together” Marie (Senior/Large/Bio) “I don't remember that one.” Lois (Senior/PUI/Chem)

Pratt, J. M., & Yezierski, E. J. (2018). A novel qualitative method to improve access, elicitation, and sample diversification for enhanced transferability applied to studying chemistry outreach. Chemistry Education Research and Practice, 19(2), 410-430.

13

Developing a Novel Critiquing Task

Oral Prompt:

We’ve received different explanations from other students, and they contain a mixture of accurate and inaccurate ideas. We want you to go line by line and critique these explanations for: 1) accuracy of the content, is it correct or incorrect? 2) age appropriateness for intended audience, is it appropriate or not?

Pratt, J. M., & Yezierski, E. J. (2018). A novel qualitative method to improve access, elicitation, and sample diversification for enhanced transferability applied to studying chemistry outreach. Chemistry Education Research and Practice, 19(2), 410-430.

14

General Chemistry Prompt: Elephant Toothpaste

This reaction involves the catalytic decomposition of hydrogen peroxide into water and hydrogen gas. This acid-base reaction is an exothermic reaction because bonds are broken and heat is released. A catalyst is used because the decomposition is not spontaneous. The catalyst allows the reaction rate to increase because the mechanistic pathway changes. The catalyzed mechanism has two steps with higher activation energies. Overall, the catalyst decreases the overall enthalpy change of the reaction. The reaction starts off slow because the first step is the rate limiting step. Soap is used to help break down the hydrogen peroxide. Once all of the catalyst is converted to the intermediate, the reaction dramatically speeds up as noted by the increase in foam being

• produced. Since the products are gas, the foam expands as the gas molecules

inside the foam spread out.

Pratt, J. M., & Yezierski, E. J. (2018). A novel qualitative method to improve access, elicitation, and sample diversification for enhanced transferability applied to studying chemistry outreach. Chemistry Education Research and Practice, 19(2), 410-430.

15

General Chemistry Prompt: Elephant Toothpaste

This reaction involves the catalytic decomposition of hydrogen peroxide into water and hydrogen gas. This acid-base reaction is an exothermic reaction because bonds are broken and heat is released. A catalyst is used because the decomposition is not spontaneous. The catalyst allows the reaction rate to increase because the mechanistic pathway changes. The catalyzed mechanism has two steps with higher activation energies. Overall, the catalyst decreases the overall enthalpy change of the reaction. The reaction starts

• ff slow because the first step is the rate limiting step. Soap is used to help

break down the hydrogen peroxide. Once all of the catalyst is converted to the intermediate, the reaction dramatically speeds up as noted by the increase in foam being produced. Since the products are gas, the foam expands as the gas molecules inside the foam spread out.

Pratt, J. M., & Yezierski, E. J. (2018). A novel qualitative method to improve access, elicitation, and sample diversification for enhanced transferability applied to studying chemistry outreach. Chemistry Education Research and Practice, 19(2), 410-430.

16

General Chemistry Prompt: Elephant Toothpaste

This reaction involves the catalytic decomposition of hydrogen peroxide into water and hydrogen gas. This acid-base reaction is an exothermic reaction because bonds are broken and heat is released. A catalyst is used because the decomposition is not spontaneous. The catalyst allows the reaction rate to increase because the mechanistic pathway changes. The catalyzed mechanism has two steps with higher activation energies. Overall, the catalyst decreases the overall enthalpy change of the reaction. The reaction starts

• ff slow because the first step is the rate limiting step. Soap is used to help

break down the hydrogen peroxide. Once all of the catalyst is converted to the intermediate, the reaction dramatically speeds up as noted by the increase in foam being produced. Since the products are gas, the foam expands as the gas molecules inside the foam spread out.

Pilot study Goal of very obvious inaccuracies Boo, H. W. (1998) Students' understandings of chemical bonds and the energetics of chemical reactions. J. Res. Sci. Teach. 35(5), 569. Taştan Ö, Yalçinkaya E. and Boz Y., (2010), Pre-service chemistry teachers’ ideas about reaction mechanism, J. Turk. Sci. Educ., 7, 47.

17

General Chemistry Prompt: Elephant Toothpaste

This reaction involves the catalytic decomposition of hydrogen peroxide into water and hydrogen gas. This acid-base reaction is an exothermic reaction because bonds are broken and heat is released. A catalyst is used because the decomposition is not spontaneous. The catalyst allows the reaction rate to increase because the mechanistic pathway changes. The catalyzed mechanism has two steps with higher activation energies. Overall, the catalyst decreases the overall enthalpy change of the reaction. The reaction starts

• ff slow because the first step is the rate limiting step. Soap is used to help

break down the hydrogen peroxide. Once all of the catalyst is converted to the intermediate, the reaction dramatically speeds up as noted by the increase in foam being produced. Since the products are gas, the foam expands as the gas molecules inside the foam spread out.

Pilot study Goal of very obvious inaccuracies Boo, H. W. (1998) Students' understandings of chemical bonds and the energetics of chemical reactions. J. Res. Sci. Teach. 35(5), 569. Taştan Ö, Yalçinkaya E. and Boz Y., (2010), Pre-service chemistry teachers’ ideas about reaction mechanism, J. Turk. Sci. Educ., 7, 47.

18

Eliciting Research Participant Ideas

• Initial ideas about elephant toothpaste:

‘‘I don’t know what’s goin’ on’’ – Shayera

• Response to critiquing task:

‘‘Uhm...‘overall the catalyst decreases the overall enthalpy change of the reaction’...yes? Yes, but only [as a] result...so...the catalyst decreases the activation energy which leads to a decrease in overall enthalpy, but a catalyst does not directly change enthalpy. The enthalpy changes as a result of the decreased activation energy.’’ – Shayera

Pratt, J. M., & Yezierski, E. J. (2018). A novel qualitative method to improve access, elicitation, and sample diversification for enhanced transferability applied to studying chemistry outreach. Chemistry Education Research and Practice, 19(2), 410-430.

19

General Chemistry Prompt: Elephant Toothpaste

This reaction involves the catalytic decomposition of hydrogen peroxide into water and hydrogen gas. This acid-base reaction is an exothermic reaction because bonds are broken and heat is released. A catalyst is used because the decomposition is not spontaneous. The catalyst allows the reaction rate to increase because the mechanistic pathway changes. The catalyzed mechanism has two steps with higher activation energies. Overall, the catalyst decreases the overall enthalpy change of the reaction. The reaction starts

• ff slow because the first step is the rate limiting step. Soap is used to help

break down the hydrogen peroxide. Once all of the catalyst is converted to the intermediate, the reaction dramatically speeds up as noted by the increase in foam being produced. Since the products are gas, the foam expands as the gas molecules inside the foam spread out.

Pilot study Goal of very obvious inaccuracies Boo, H. W. (1998) Students' understandings of chemical bonds and the energetics of chemical reactions. J. Res. Sci. Teach. 35(5), 569. Taştan Ö, Yalçinkaya E. and Boz Y., (2010), Pre-service chemistry teachers’ ideas about reaction mechanism, J. Turk. Sci. Educ., 7, 47.

20

Eliciting Research Participant Ideas

• Initial ideas about elephant toothpaste:

‘‘It’s been a long time since chem two!...It’s not knowledge I deem important enough to keep in my brain on a daily basis.’’ – Sue

• Response to critiquing task:

‘‘The line a couple of sentences from the end ‘once all of the catalyst is converted to the intermediate’ uhm…catalysts don’t change so that’s super wrong.”– Sue

Pratt, J. M., & Yezierski, E. J. (2018). A novel qualitative method to improve access, elicitation, and sample diversification for enhanced transferability applied to studying chemistry outreach. Chemistry Education Research and Practice, 19(2), 410-430.

21

Summary of Method

• Access
• Requires flexibility and multiple strategies
• Leads to greater sample diversity, adds to transferability
• Elicitation
• Requires flexibility and multiple strategies
• Data is commensurate with in-person interviews
• Rapport
• Evidence that rapport can successfully

be established online

“If we’ve learned anything today is that...[Max] doesn’t remember chemical reactions’!’ – Max “Obviously I’m a girl, a female in science so…[laughter] I totally get that one!’’ – Lois

22

Pratt, J. M., & Yezierski, E. J. (2018). A novel qualitative method to improve access, elicitation, and sample diversification for enhanced transferability applied to studying chemistry outreach. Chemistry Education Research and Practice, 19(2), 410-430.

Technology Limitations and Considerations

• Freezing and/or dropped connections (n = 12 / 37)
• Audio/video skipping (N = 37)
• No evidence of hindered rapport
• Flexibility and participant choice
• Skype, Google Hangouts, Facebook Video

Pratt, J. M., & Yezierski, E. J. (2018). A novel qualitative method to improve access, elicitation, and sample diversification for enhanced transferability applied to studying chemistry outreach. Chemistry Education Research and Practice, 19(2), 410-430.

23

Audience Participation!

Breakout Rooms

1.

Introduce yourself (Name & Institution).

2.

Open chat window to see discussion questions.

3.

Pick a discussion question and think about access, elicitation, and/or rapport for remote qualitative research:

a.

What types of research questions align with these methods?

b.

What is important to consider if using these methods during the COVID-19 pandemic? 4.

Identify 1 person to summarize your group’s conversation into 1-2 sentences (or a question) to share when we get back to main Zoom room.

5.

Discuss!! Participants will have ~10 minutes to discuss

24

Report Out!

• Group Summarizers: Please summarize your group's discussion or pose a

question in the chat. Please keep it short!

• Think about access, elicitation, and/or rapport for remote qualitative research:

a.

What types of research questions align with these methods?

b.

What is important to consider if using these methods during the COVID-19 pandemic?

25

Acknowledgments

• Michael Seery and Aishling Flaherty
• Research participants from ACS and ΑΧΣ

chapters

• ACS Undergraduate Programs Office
• ΑΧΣ Supreme Council
• Yezierski & Bretz Research Groups at Miami

University

• Raker Research Group at the University of

South Florida

26

1.

How accurate and in-depth are collegiate students’ chemistry content knowledge as related to common chemistry outreach activities?

2.

How well do collegiate students adjust their explanations of chemistry outreach activities/demonstrations based on the audience?

3.

How do these college students develop skills and expertise to facilitate chemistry outreach events for varying audiences?

Conceptual Understanding

Pratt, J. M., Yezierski, E. J. (2018).

• J. Chem. Educ. 95(12), 2091-2102.

Pratt, J. M., Yezierski, E. J. (2019).

• J. Chem. Educ. 96(2), 203-212.

Pratt, J. M., Yezierski, E. J. (2019).

• J. Chem. Educ. 96(3), 414-422.

27

Findings for Chemistry Outreach Study