SLIDE 1
H Y P OT H E S I S T E S T I N G
MPA 630: Data Science for Public Management November 15, 2018
Fill out your reading report
- n Learning Suite
P L A N F O R T O D A Y Why are we even doing this?
(again!)
H Y P OT H E S I S T E S T I N G MPA 630: Data Science for Public - - PowerPoint PPT Presentation
H Y P OT H E S I S T E S T I N G MPA 630: Data Science for Public - - PowerPoint PPT Presentation
H Y P OT H E S I S T E S T I N G MPA 630: Data Science for Public Management November 15, 2018 Fill out your reading report on Learning Suite P L A N F O R T O D A Y Randomness, repetition, and replicability Why are we even doing this?
SLIDE 2
SLIDE 3
R A N D O M N E S S , R E P E T I T I O N , & R E P L I C A B I L I T Y
SLIDE 4
SLIDE 5
SLIDE 6
P O W E R P O S I N G
Increases individual perception of power
She made a guess at a population parameter and published it This is the process
- f science!
Increases testosterone and decreases cortisol
SLIDE 7
R U H R O H
SLIDE 8
B U T W A I T
Increases individual perception of power Increases testosterone and decreases cortisol
SLIDE 9
M O R A L O F T H E S T O R Y
Randomness is weird Capturing true population parameters is hard Replication and repetition are needed to check the net
SLIDE 10
W H Y A R E W E E V E N D O I N G T H I S ?
Round 2!
SLIDE 11
P O P U L A T I O N P A R A M E T E R S
Key assumption in the flavor of statistics we’re doing:
There are true, fixed population parameters out in the world
SLIDE 12
P O P U L A T I O N V S . S A M P L E
Proportion Mean Difference between proportions Difference between means Intercept ! " !# − !% "# − "% &' ̂ ! ̅ * ̅ *# − ̅ *% ̂ !# − ̂ !% + &'
,
SLIDE 13
W H Y W E S I M U L A T E
Is ̅ " an accurate guess of #? Is ̅ "$ − ̅ "& or ̂ ($ − ̂ (& real? Does it matter? Is it accurate? Is it real? Is it substantive?
Width of confidence interval Important numbers included in confidence interval
SLIDE 14
B U R D E N S O F P R O O F
SLIDE 15
A M E R I C A N L E G A L S Y S T E M
Accused must be judged Accuser has burden of proving guilt Presumption of innocence
We never prove innocence; we try (and fail) to reject innocence
Judge/jury decide guilt based
- n amount of evidence
SLIDE 16
L E G A L E V I D E N T I A R Y S T A N D A R D S Preponderance of evidence Beyond reasonable doubt Clear and convincing evidence
Why do we have these different levels? We’re afraid of locking up an innocent person
SLIDE 17
Actual truth Guilty Not guilty Jury decision Not guilty Guilty
Yay!
True positive
Yay!
True negative
Oh no!
False positive (I)
Oh no!
False negative (II)
SLIDE 18
SLIDE 19
S T A T I S T I C A L “ L E G A L” S Y S T E M
You have burden of proving effect Presumption of no effect (null)
We never prove that the null is true; we try (and fail) to reject the null
You decide “guiltiness” of effect based on amount of evidence Sample statistic (δ) must be judged
SLIDE 20
Actual truth Yes effect No effect Result of hypothesis test No effect Yes effect
Yay!
True positive
Yay!
True negative
Oh no!
False positive (I)
Oh no!
False negative (II)
!
0.10 0.05 0.01
SLIDE 21
S T A T I S T I C A L S I G N I F I C A N C E
There’s enough evidence to safely reject the null hypothesis
SLIDE 22
P - V A L U E S
The probability of observing an effect at least that large when no effect exists
SLIDE 23
N O B O D Y U N D E R S T A N D S T H E S E
http://fivethirtyeight.com/pvalue
SLIDE 24
H O W TO T E S T A N Y H Y P OT H E S I S
SLIDE 25
SLIDE 26
SLIDE 27
SLIDE 28
S I M U L A T I O N S A N D H Y P O T H E S E S
SLIDE 29
Find δ
The sample statistic: diff in means, mean, diff in props, etc.
Invent world where δ is null
Simulate what the world would look like if there was no effect.
Look at δ in the null world
Is it big and extraordinary, or is it a normal thing?
Calculate probability that δ could exist in the null world
This is your p-value!