Assessing course design and student outcomes in an inquiry-based - - PowerPoint PPT Presentation

Assessing course design and student outcomes in an inquiry-based lab course

BPK 420: “Cell Physiology Lab”

Megan Barker, Ciara Morgan-Feir, Damon Poburko, Nadine Wicks, Tom Claydon

megan.barker@sfu.ca | twitter/wordpress: @meganbarkerase

BPK 420: Cell Physiology Laboratory

Developed jointly by the depts. of Biological Sciences and Biomedical Physiology & Kinesiology Four units: 1) Introduction to cells 2) Inducing and measuring gene expression in mammalian cells 3) Earthworm physiology 4) Independent projects

Timeline: Week 4 – discuss possible projects Week 7 – submit top 2 ideas/picks Week 9 – submit project proposal Week 10 – in-lab consultation, submit protocol, “order” reagents Weeks 11/12 – do experiment(s) Week 13 – poster session Week 14 – submit project report

How/what to assess?

1)Develop/refine (non-content) learning outcomes 2)Find or build assessments (mixed methods) 3)Prioritize! 4)Deploy 5)Analyze 6)Do it again!

Workflow of developing LOs & tools…

à 4 quantitative tools chosen/built

Quantitative Assessment

Attitudes about biology Content knowledge Tool: Diagnostic test Multiple choice test, compiled from concept inventories. Deployed pre- &post-course. Tool: CLASS-Bio

(Semsar, Knight, Birol, Smith 2011)

Likert-scale questions to measure novice-to-expert-like perceptions about biology.

Quantitative Assessment

Experimental Design Skills Confidence in lab skills Tool: Self-assessment of confidence in lab skills Likert-scale responses to “rate your confidence in each lab activity” Tool: EDAT

(Experimental Design Ability Test, Sirum & Humberg 2011)

Students given prompt, design a related experiment, their designs scored on a rubric. Deployed “pre” (before independent projects), and

• n last day of term

Qualitative assessment

Weekly feedback At end of term

• Prior to developing your

project ideas, what aspect(s) of our course helped you understand how to design a (good) experiment?

• From doing your project,

what have you learned about experimental design so far?

• What were the

most challenging, and the most interesting parts

• f your work this week?

Preliminary data…

Content knowledge Experimental design

Week 7 Week 13 0.0 0.2 0.4 0.6 0.8 1.0 1.2

EDAT Score

**

W e e k 1 W e e k 1 3 0.0 0.2 0.4 0.6 0.8 1.0

Content knowledge ched pairs t-test

*

P = 0.0083; Two-tailed; non-parametric; Paired, Wilcoxon matched pairs t-test P = 0.0179; Two-tailed; non-parametric; Paired, Wilcoxon matched pairs t-test

Comments from students…

Designing my own experiment was really great because obtaining results from an experiment that you made is a lot more satisfying than following someone else’s instructions. I learned that designing your own protocol actually gives you a much deeper understanding about how the procedure affects your final results. I have a greater appreciation for people who do research and the amount of time, dedication, planning, lit reviews, and commitment that goes into working in science. I think that no matter how frustrating an experiment can seem at the time, one can feel accomplished and satisfied with results even if they aren’t what were expected.

Authentic work: Characterizing the level of inquiry based on what is provided to students

• Level 0—Confirmation: An activ
• Level ½—Structured inquiry: The
• Level 1—Guided inquiry: The labo
• Level 2—Open inquiry: The prob
• Level 3—Authentic inquiry: The

TABLE 2 A rubric to characterize inquiry in the undergraduate laboratory.

Characteristic Level 0: Confirmation Level ½: Structured inquiry Level 1: Guided inquiry Level 2: Open inquiry Level 3: Authentic inquiry Problem/Question Provided Provided Provided Provided Not provided Theory/Background Provided Provided Provided Provided Not provided Procedures/Design Provided Provided Provided Not provided Not provided Results analysis Provided Provided Not provided Not provided Not provided Results communication Provided Not provided Not provided Not provided Not provided Conclusions Provided Not provided Not provided Not provided Not provided More structure Less structure

From Buck et al, JCST, 2013

BPK 420: Level ???

Not provided Somewhat provided Somewhat provided Somewhat provided Provided Not provided

More structured Less structured

Anecdotally, what we learned…

Getting them prepared is key

• Saw much stronger understanding with pre-class quizzes

versus pre-lab quizzes

Build in-class/lab time for theory & especially analysis

• Especially important in a non-cookbook lab

Scale down the scope of the experiments

• Pushing them to focus improves their conclusions

Backwards design -> more straightforward assessment

• Lots of data, so having clear questions was helpful

Connect with student undergraduate research journal

• Pride and ownership in work – authentic experience