Introduction to Experiments February 4 1 / 42 Outline for today 1. - - PowerPoint PPT Presentation

▶
introduction to experiments
SMART_READER_LITE
LIVE PREVIEW

Introduction to Experiments February 4 1 / 42 Outline for today 1. - - PowerPoint PPT Presentation

Sep 17, 2022 198 likes •629 views

Introduction to Experiments February 4 1 / 42 Outline for today 1. Introductions 2. Overview of course 3. Introduction to experiments 4. Preview of next week 5. In-class exercise 2 / 42 Introductions Name tags Go-around Who are you?

slide-1
SLIDE 1

Introduction to Experiments

February 4

1 / 42

slide-2
SLIDE 2

Outline for today

  • 1. Introductions
  • 2. Overview of course
  • 3. Introduction to experiments
  • 4. Preview of next week
  • 5. In-class exercise

2 / 42

slide-3
SLIDE 3

Introductions

Name tags Go-around Who are you? What do you want to do after your education? 3 / 42

slide-4
SLIDE 4

Outline for today

  • 1. Introductions
  • 2. Overview of course
  • 3. Introduction to experiments
  • 4. Preview of next week
  • 5. In-class exercise

4 / 42

slide-5
SLIDE 5

Overview

Meet for 10 weeks Small assignments on some weeks (presentations, etc.) Synopsis presentations on: Mar 25, Apr 8, Apr 15 Individual meetings with me after April 15 Light reading load 5 / 42

slide-6
SLIDE 6

Overview Exam

Propose an experimental study on a relevant topic from any area of political science Topic is completely up to you May be useful preparation for a masters thesis Assume 400 pages of individual reading for the exam 6 / 42

slide-7
SLIDE 7

Overview Exam

Contents: Question, theory, and hypotheses Design Stimulus/treatment materials All measures Complete "protocol" Planned statistical analysis Accounts for possible data challenges Discuss feasibility and ethics Discuss external validity and contribution 7 / 42

slide-8
SLIDE 8

Overview Exam Schedule

Part 1 4.1 Introduction to Political Science Experiments (Feb 4) 4.2 Concepts, Research Questions, and Hypotheses (Feb 11) 4.3 Internal Validity and Experimental Design (Feb 18) 4.4 Analysis of Experiments (Feb 25) 4.5 Practical Issues and Challenges (Mar 4) 8 / 42

slide-9
SLIDE 9

Overview Exam Schedule

Part 2 4.6 Examples: Laboratory Experiments (Mar 11) 4.7 Examples: Field Experiments (Mar 18) 4.8 Examples: Survey Experiments (Mar 25) Presentations start on Mar 25 9 / 42

slide-10
SLIDE 10

Overview Exam Schedule

Part 3 No class (Apr 1) 4.9 External Validity (Apr 8) 4.10 Effect Sizes, Meta-Analysis, Decision Making (Apr 15) Presentations on Apr 8 and Apr 15 10 / 42

slide-11
SLIDE 11

Outline for today

  • 1. Introductions
  • 2. Overview of course
  • 3. Introduction to experiments
  • 4. Preview of next week
  • 5. In-class exercise

11 / 42

slide-12
SLIDE 12

History of experiments

American Political Science Association president A. Lawrence Lowell: `We are limited by the impossibility of

  • experiment. Politics is an observational, not

an experimental science..." Experiments prominent in psychology, natural sciences King, Keohane, and Verba (1994) only mentions experiments once Since ~2000, "credibility revolution" 12 / 42

slide-13
SLIDE 13

Uses of Experiments

Alvin Roth, Stanford, 2012 Nobel Prize winner Searching for facts Speaking to theorists Whispering in the ears of princes 13 / 42

slide-14
SLIDE 14

Types of Experiments

Lab: treat in a controlled research environment Field: treatment occurs in course of everyday life Survey: treatment occurs outside of the control of the research 14 / 42

slide-15
SLIDE 15

Causality

Correlation 15 / 42

slide-16
SLIDE 16

16 / 42

slide-17
SLIDE 17

Causality

Correlation Physical causality Philsophical perspectives 17 / 42

slide-18
SLIDE 18

Hume

Three tenents

  • 1. Spatial/temporal contiguity
  • 2. Temporal succession
  • 3. Constant conjunction

18 / 42

slide-19
SLIDE 19

Four (or five) principles of causality

A more modern take involves 4-5 principles: Relationship Direction (temporality) Nonconfounding Mechanism Appropriate level of analysis 19 / 42

slide-20
SLIDE 20

Agreement If two or more instances of the phenomenon under investigation have only one circumstance in common, the circumstance in which alone all the instances agree, is the cause (or effect) of the given phenomenon.

Mill's Methods

20 / 42

slide-21
SLIDE 21

Agreement Difference

If an instance in which the phenomenon under investigation

  • ccurs, and an instance in which it

does not occur, have every circumstance save one in common, that one occurring only in the former; the circumstance in which alone the two instances differ, is the effect, or cause, or an necessary part of the cause, of the phenomenon.

Mill's Methods

21 / 42

slide-22
SLIDE 22

Agreement Difference Agree & Diff

If two or more instances in which the phenomenon occurs have only

  • ne circumstance in common,

while two or more instances in which it does not occur have nothing in common save the absence of that circumstance; the circumstance in which alone the two sets of instances differ, is the effect, or cause, or a necessary part

  • f the cause, of the phenomenon.

Mill's Methods

22 / 42

slide-23
SLIDE 23

Agreement Difference Agree & Diff Residue

Subduct from any phenomenon such part as is known by previous inductions to be the effect of certain antecedents, and the residue of the phenomenon is the effect of the remaining antecedents.

Mill's Methods

23 / 42

slide-24
SLIDE 24

Agreement Difference Agree & Diff Residue Concomitant variations

Whatever phenomenon varies in any manner whenever another phenomenon varies in some particular manner, is either a cause or an effect of that phenomenon, or is connected with it through some fact of causation.

Mill's Methods

24 / 42

slide-25
SLIDE 25

Causal Terminology

Unit: A physical object at a particular point in time Treatment: An intervention, whose effects we wish to assess relative to some other (non-)intervention Potential outcomes: The outcome for each unit that we would observe if that unit received each treatment Multiple potential outcomes for each unit, but we

  • nly observe one of them

Causal effect: The comparisons between the unit- level potential outcomes under each intervention Average causal effect 25 / 42

slide-26
SLIDE 26

Potential Outcomes

Causal inference is about estimating what would have happened in a counterfactual reality 26 / 42

slide-27
SLIDE 27

Potential Outcomes

Causal inference is about estimating what would have happened in a counterfactual reality Has anyone read or seen A Christmas Carol? 27 / 42

slide-28
SLIDE 28

Fundamental problem of causal inference

But we can only observe any given unit in one reality! 28 / 42

slide-29
SLIDE 29

Scientific solution

Used in physical sciences (e.g., agriculture) Two strategies: Take the same unit and it expose it to both treatments at different points in time Take two similar units and expose to the two treatments at the same Requires constant effect assumption: The past does not matter Also requires homogeneity of units assumption Units are identical (or differences are irrelevant) 29 / 42

slide-30
SLIDE 30

Statistical solution

Random assignment Observation of average causal effects 30 / 42

slide-31
SLIDE 31

Causal inference in political science

Traditional observational research approach: The observation of one or more units. 31 / 42

slide-32
SLIDE 32

Causal inference in political science

Traditional observational research approach: The observation of one or more units. Experimental approach: Observation plus intervention 32 / 42

slide-33
SLIDE 33

"Perfect Doctor"

True potential outcomes (unobservable in reality) Unit Y(0) Y(1) 1 13 14 2 6 3 4 1 4 5 2 5 6 3 6 6 1 7 8 10 8 8 9 Mean 7 5 33 / 42

slide-34
SLIDE 34

"Perfect Doctor"

How observational data can mislead Unit Y(0) Y(1) 1 ? 14 2 6 ? 3 4 ? 4 5 ? 5 6 ? 6 6 ? 7 ? 10 8 ? 9 Mean 5.4 11 34 / 42

slide-35
SLIDE 35

Definition of an experiment

Minimum definition The observation of one or more units after an intervention in a controlled setting. More complete definition The observation of units after, and possibly before, a randomly assigned intervention in a controlled setting, which tests one or more precise causal expectations. 35 / 42

slide-36
SLIDE 36

Elements an experiment

  • 1. Physical intervention
  • 2. Control
  • 3. Treatment assignment independent of potential
  • utcomes
  • 4. Treatment assignment independent of all

confounding variables 36 / 42

slide-37
SLIDE 37

Outline for today

  • 1. Introductions
  • 2. Overview of course
  • 3. Introduction to experiments
  • 4. Preview of next week
  • 5. In-class exercise

37 / 42

slide-38
SLIDE 38

Next week: Readings

Shadish, Cook, and Campbell on research design Chapter from Gerring (I will send this to you via email) A short article by me explaining what goes into an experimental protocol Gives you a sense of details for the exam An example experiment by Druckman and Nelson 38 / 42

slide-39
SLIDE 39

Next week: Assignment

Complete a summary of the experiment by Druckman and Nelson 39 / 42

slide-40
SLIDE 40

Outline for today

  • 1. Introductions
  • 2. Overview of course
  • 3. Introduction to experiments
  • 4. Preview of next week
  • 5. In-class exercise

40 / 42

slide-41
SLIDE 41

In-class exercise

How do we read experimental literature? Research question Theory/hypotheses Variables Design Data collection/protocol Analysis Results/findings 41 / 42

slide-42
SLIDE 42

Kahneman and Tversky

Try to summarize Kahneman and Tversky in this way Research question Theory/hypotheses Variables Design Data collection/protocol Analysis Results/findings 42 / 42