SLIDE 1
Variable Questions and Small Worlds Variable Questions and Small - - PowerPoint PPT Presentation
Variable Questions and Small Worlds Variable Questions and Small - - PowerPoint PPT Presentation
Variable Questions and Small Worlds Variable Questions and Small Worlds Colin Melville Doug Quinney Problem Problem Training doctors to become experts ICD-10 disease classification >6500 diseases Training time reducing Hours 1980
SLIDE 2
SLIDE 3
Model of Expertise Model of Expertise
>10,000 hours deliberate practice (Ericsson, Road to Excelence) Small worlds (Patel)
Experts are only experts within their own small domains
SLIDE 4
Question Question
How can we mimic that prolonged experience? How can we do it cheaply with no danger to patients? Can we give tailored feedback?
SLIDE 5
Potential Solution Potential Solution
Computer-based simulation Text questions +/- media +/- calculations Multiple variables Never tried (to our knowledge) in medicine
SLIDE 6
Choose area requiring Choose area requiring overlearning
- verlearning
APLS
'Simple' Problems Some calculation Multiple variations on theme Important area
SLIDE 7
Progress Progress
Experimentation with various systems
OS
Moodle + webwork
Proprietary
WebCT EDU
EDU chosen
DQ familiar with it Had all functionality RAD
SLIDE 8
Questions for development Questions for development
How much variation needed? How much variation while preserving realism? What sort of areas best to try out? Any ground rules for further development?
SLIDE 9
Development Development
6 areas
Airway Breathing Circulation Disability Severity scenarios x 12 Arrhythmias x 6
SLIDE 10
Scenario development Scenario development
Each case consists of:
Scenario development
Written scenario Plausible variables (up to 6) Plausible variations Error checking
Encoding into TeX
Error checking
Encoding into EDU
Error checking
SLIDE 11
Scenario testing Scenario testing
Field testing
Experts
Error checking Incorporating new ideas
Novices (not yet done)
SLIDE 12
Field testing results Field testing results
Rating of each question in 3 areas
Realism (validity) Accuracy (reliability) Utility (teaching effect)
Overall average 4.27/5
Realism 4.3 Accuracy 4.45 Utility 4.1
Comments
Much useful on implementation Simpler questions may be most effective Questions requiring memory e.g. of drug doses felt to be trivial Questions requiring application e.g. of scales much appreciated.
SLIDE 13