Statistical-Significance Background & Goal Shortcuts - - PDF document

▶
statistical significance
SMART_READER_LITE
LIVE PREVIEW

Statistical-Significance Background & Goal Shortcuts - - PDF document

Aug 17, 2022 133 likes •284 views

Statistical-Significance Shortcuts 9 Mar 2015 V0F V0F V0F 2015 Schield SS Shortcuts 1 2015 Schield SS Shortcuts 2 Statistical-Significance Background & Goal Shortcuts Statistical significance is one of statistics big ideas. by

slide-1
SLIDE 1

Statistical-Significance Shortcuts 9 Mar 2015 V0F 2015-Schield-StatChat-Slides.pdf 1

2015 Schield SS Shortcuts

V0F 1

by Milo Schield StatChat Feb 24, 2015 Slides at: www.StatLit.org/pdf/ 2015-Schield-StatChat-Slides.pdf

Statistical-Significance Shortcuts

2015 Schield SS Shortcuts

V0F 2

Background & Goal

Statistical significance is one of statistics’ big ideas. For Z-scores, statistical significance is a single value. For Chi-squared, student-T, the F-statistic, correlation and relative risk, statistical significance is complex. To better understand statistical significance, students need to see it in different contexts. Goal: To create “shortcut” formulas for statistical significance that are sufficient, memorable and apply to a wide variety of statistics.

2015 Schield SS Shortcuts

V0F

If |p2 - p1| > 1/Sqrt(n), then that difference is statistically significant

  • Q. Has anyone seen this shortcut? Where?

Yes! Seeing Through Statistics, Jessica Utts Statistics: Art+Science of Data, Agresti/Franklin

  • Q. Anywhere else?

3

#1: Proportions Shortcut (SS)

2015 Schield SS Shortcuts

V0F 4

#2: Chi-Squared

Has anyone seen this shortcut anywhere?

5 10 15 20

1 3 5 7 9

Chi‐squared Degrees of Freedom

Chi‐Squared Shortcut

Statistically‐Significant

Chi‐squared > 2(DF+1)

Actual Cutoffs

2015 Schield SS Shortcuts

V0F 5

Has anyone seen this shortcut anywhere? #3: Correlation

0.1 0.2 0.3 0.4 0.5 0.6 50 100 150 200 250 300

Sample Size (# of Pairs)

Correlation Shortcut (S/S)

Model (2 tail): r > 2/Sqrt(N) for #Pairs > 10 r > 2/Sqrt(N‐1) for #Pairs > 4

Error < 5%

Solid is the Model

Dashed is Actual

2015 Schield SS Shortcuts

V0F 6

Consider two groups each of size n. Relative Risk: RR = p2/p1 RR>1 is statistically significant if RR-1 = 2/sqrt(k1) where k1 = n*p1 >4 Has anyone seen this shortcut anywhere?

#4: Relative-Risk

slide-2
SLIDE 2

Statistical-Significance Shortcuts 9 Mar 2015 V0F 2015-Schield-StatChat-Slides.pdf 2

2015 Schield SS Shortcuts

V0F 7

#5: T - Z

Has anyone seen this shortcut anywhere?

1.5 2.0 2.5 3.0 3.5 4.0 2 4 6 8 10

T

Degrees of Freedom

Shortcut for 2‐tail T

Actual Model: T > 1.645 + 2/(df‐1)

2015 Schield SS Shortcuts

V0F

.

8

#6a: F-statistic

4.00 4.50 5.00 5.50 5 10 15 20 25 30 35 40 45 50

F‐statistic Denominator Degrees of Freedom

F vs DDF (NDF=1)

4 + 11/DDF 4 + 4/sqrt(DDF) Actual n=sample size; k=groups DDF = n‐k; NDF = k‐1

2015 Schield SS Shortcuts

V0F

.

9

#6b: F-statistic DDF = N-k; NDF = k-1

3.0 3.5 4.0 4.5 5.0 5.5

5 10 15 20 25 30

Numerator Degrees of Freedom

F‐Statistic (DDF=1) versus NDF

Model = 2.1 + 2 /NDF Actual

2015 Schield SS Shortcuts

V0F

N = sample size; K = # of groups If 7 < DDF < 100 and 0 < NDF < 30, then Fcritical value (sufficient) = 2.1 + (11 / DDF) + (2 / NDF) Error in this region (Model vs. actual): Min 1.5%, Max 31%. If n = 13 and k = 2, then ddf=11, ndf = 1, and

  • Fsuff. = 2.1+1+2 = 5.1

10

#6c: F-statistic Model DDF = n-k; NDF = k-1

2015 Schield SS Shortcuts

V0F

High: k/n > p*n + 2*Sqrt[p*(1-p)*n] Low: k/n < p*n - 2*Sqrt[p*(1-p)*n]

11

#7: Binomial Distribution If p*n > 5

‐0.6 ‐0.4 ‐0.2 0.0 0.2 0.4 0.6 0.8 1.0 5 10 15 20 25 30

Binomial Distribution: p = 0.2 Cutoffs: Actual vs. Model

High Actual Low Model High Model Low

2015 Schield SS Shortcuts

V0F

1. |p2-p1| > 1/sqrt(n)

  • 2. Chi-squared: Χ2 > 2(df+1)
  • 3. Correlation: r > 2/sqrt(n-1) for n > 4
  • 4. RRisk > 1+ 2/sqrt(k1): k1=n*p1, p1<p2
  • 5. t-stat (2-tail): t > 1.645 + 2/sqrt(df-1)
  • 6. F > 2.1 + 11/(n-k) + 2/(k-1)
  • 7. Binomial: k/n > p+2√[p(1-p)/n] if p*n>5

12

Why don’t we teach these shortcuts?

slide-3
SLIDE 3

2015 Schield SS Shortcuts

V0F

by Milo Schield StatChat Feb 24, 2015 Slides at: www.StatLit.org/pdf/ 2015-Schield-StatChat-Slides.pdf

Statistical-Significance Shortcuts

1

slide-4
SLIDE 4

2015 Schield SS Shortcuts

V0F 2

Background & Goal

Statistical significance is one of statistics’ big ideas. For Z-scores, statistical significance is a single value. For Chi-squared, student-T, the F-statistic, correlation and relative risk, statistical significance is complex. To better understand statistical significance, students need to see it in different contexts. Goal: To create “shortcut” formulas for statistical significance that are sufficient, memorable and apply to a wide variety of statistics.

slide-5
SLIDE 5

2015 Schield SS Shortcuts

V0F

If |p2 - p1| > 1/Sqrt(n), then that difference is statistically significant

  • Q. Has anyone seen this shortcut? Where?

Yes! Seeing Through Statistics, Jessica Utts Statistics: Art+Science of Data, Agresti/Franklin

  • Q. Anywhere else?

3

#1: Proportions Shortcut (SS)

slide-6
SLIDE 6

2015 Schield SS Shortcuts

V0F 4

#2: Chi-Squared

Has anyone seen this shortcut anywhere?

5 10 15 20

1 3 5 7 9

Chi-squared Degrees of Freedom

Chi-Squared Shortcut

Statistically-Significant

Chi-squared > 2(DF+1)

Actual Cutoffs

slide-7
SLIDE 7

2015 Schield SS Shortcuts

V0F 5

Has anyone seen this shortcut anywhere? #3: Correlation

0.1 0.2 0.3 0.4 0.5 0.6 50 100 150 200 250 300

Sample Size (# of Pairs)

Correlation Shortcut (S/S)

Model (2 tail): r > 2/Sqrt(N) for #Pairs > 10 r > 2/Sqrt(N-1) for #Pairs > 4

Error < 5%

Solid is the Model

Dashed is Actual

slide-8
SLIDE 8

2015 Schield SS Shortcuts

V0F 6

Consider two groups each of size n. Relative Risk: RR = p2/p1 RR>1 is statistically significant if RR-1 = 2/sqrt(k1) where k1 = n*p1 >4 Has anyone seen this shortcut anywhere?

#4: Relative-Risk

slide-9
SLIDE 9

2015 Schield SS Shortcuts

V0F 7

#5: T - Z

Has anyone seen this shortcut anywhere?

1.5 2.0 2.5 3.0 3.5 4.0 2 4 6 8 10

T

Degrees of Freedom

Shortcut for 2-tail T

Actual Model: T > 1.645 + 2/(df-1)

slide-10
SLIDE 10

2015 Schield SS Shortcuts

V0F

.

8

#6a: F-statistic

4.00 4.50 5.00 5.50 5 10 15 20 25 30 35 40 45 50

F-statistic Denominator Degrees of Freedom

F vs DDF (NDF=1)

4 + 11/DDF 4 + 4/sqrt(DDF) Actual n=sample size; k=groups DDF = n-k > 7; NDF = k-1 =1

slide-11
SLIDE 11

2015 Schield SS Shortcuts

V0F

.

9

#6b: F-statistic DDF = N-k; NDF = k-1

3.0 3.5 4.0 4.5 5.0 5.5

5 10 15 20 25 30

Numerator Degrees of Freedom

F-Statistic (DDF=8) versus NDF

Model = 2.1 + 2 /NDF Actual

slide-12
SLIDE 12

2015 Schield SS Shortcuts

V0F

N = sample size; K = # of groups If 7 < DDF < 100 and 0 < NDF < 30, then Fcritical value (sufficient) = 2.1 + (11 / DDF) + (2 / NDF) Error in this region (Model vs. actual): Min 1.5%, Max 31%. If n = 10 and k = 2, then Fcrit = 5.5

10

#6c: F-statistic Model DDF = n-k; NDF = k-1

slide-13
SLIDE 13

2015 Schield SS Shortcuts

V0F

High: k/n > p*n + 2*Sqrt[p*(1-p)*n] Low: k/n < p*n - 2*Sqrt[p*(1-p)*n]

11

#7: Binomial Distribution If p*n > 5

  • 0.6
  • 0.4
  • 0.2

0.0 0.2 0.4 0.6 0.8 1.0 5 10 15 20 25 30

Binomial Distribution: p = 0.2 Cutoffs: Actual vs. Model

High Actual Low Model High Model Low

slide-14
SLIDE 14

2015 Schield SS Shortcuts

V0F

1. |p2-p1| > 1/sqrt(n)

  • 2. Chi-squared: Χ2 > 2(df+1)
  • 3. Correlation: r > 2/sqrt(n-1) for n > 4
  • 4. RRisk > 1+ 2/sqrt(k1): k1=n*p1, p1<p2
  • 5. t-stat (2-tail): t > 1.645 + 2/sqrt(df-1)
  • 6. F > 2.1 + 11/(n-k) + 2/(k-1)
  • 7. Binomial: k/n > p+2√[p(1-p)/n] if p*n>5

12

Why don’t we teach these shortcuts?