External Validity March 25 1 / 16 Definition How do we define - - PowerPoint PPT Presentation

External Validity

March 25

1 / 16

Definition

How do we define external validity? Mundane realism (surface similarity) Generalizability of results 2 / 16

Discussion

Experiments have strong internal validity. If that is why we do experiments, why do we care about the external validity of experiments? 3 / 16

SUTO framework

Setting Units Treatments Outcome 4 / 16

Assessing external validity

How do we assess the external validity of a single study? 5 / 16

Assessing external validity

The most common multi-study method of assessing external validity is the review

• 1. Qualitative reviews
• 2. Quantitative/systematic reviews (meta-analysis)

6 / 16

Assessing external validity

Other strategies of assessing external validity include: Formal/exact replication Approximate replication Statistical generalization Parallel experimentation 7 / 16

Exact replication

Recreate a previous experiment exactly Requires a complete protocol SUTO: What can we replicate? 8 / 16

Approximate replication

Model a new study off a previous study Possibly intentional changes in SUTO characteristics 9 / 16

Statistical Generalization

Hedges is interested in the formal, statistical generalization of experiments How does this work? 10 / 16

Parallel Experiments

Execute the same experiment with variations in some experimental feature Compare results across experiments SUTO: What can we replicate? 11 / 16

Discussion

Are these types of replication useful? Why or why not? 12 / 16

Discussion

How important is it for an experiment to have high external validity? 13 / 16

Discussion

What is the point of a single experiment if it does not generalize? 14 / 16

Presentations

What are you thinking for your exam paper? 15 / 16

Preview

No class next week In 2 weeks: Survey Experiments Presentation by Rune Slothuus Share a one-page synopsis of your project Due via email to me: Monday April 7 12:00 16 / 16