V0D 14 Nov 2018 Why 25% of Voter Polls Are Wrong V0D 2018 CTC 1 - - PDF document

▶
v0d 14 nov 2018 why 25 of voter polls are wrong
SMART_READER_LITE
LIVE PREVIEW

V0D 14 Nov 2018 Why 25% of Voter Polls Are Wrong V0D 2018 CTC 1 - - PDF document

Jul 18, 2023 110 likes •264 views

V0D 14 Nov 2018 Why 25% of Voter Polls Are Wrong V0D 2018 CTC 1 V0D 2018 CTC 2 Why 25% of Voter Polls 25% of Voter Polls are Wrong are Wrong! Polls are short surveys. by Less demographic information. Milo Schield, Augsburg

slide-1
SLIDE 1

Why 25% of Voter Polls Are Wrong V0D 14 Nov 2018 www.StatLit.org/pdf/2018-Schield-CTC-Slides.pdf Page 1

2018 CTC

V0D 1

by Milo Schield, Augsburg University Fellow: American Statistical Association US Rep: International Statistical Literacy Project December 2, 2018

Twin Cities Critical Thinking Club (CTC) Slides: www.StatLit.org/pdf/2018-Schield-CTC-Slides.pdf

Why 25% of Voter Polls are Wrong

2018 CTC

V0D 2

Polls are short surveys.

  • Less demographic information.
  • Fewer choices for answers.

Polls typically involve sampling: sampling error. Voter polls are of wide-spread interest. 25%* of voter polls in the final 21 days are wrong. Wrong: actual result outside 95% confidence interval In 2016, 29%** of those in final 21 days were wrong.

* https://fivethirtyeight.com/features/how-the-fivethirtyeight-senate-forecast-model-works/ ** https://fivethirtyeight.com/features/the-polls-are-all-right/

25% of Voter Polls are Wrong!

2018 CTC

V0D 3

Surveys and polls often interview people by phone. Plus: Cheap; random sample. Minus: Non-response bias. People who respond are not representative of the entire population. The lower the response rate, the greater the risk of bias.

http://www.pewresearch.org/2017/05/15/what-low-response-rates-mean-for-telephone-surveys/pm-05-15-2017_rddnonresponse-00-14/

Random Sampling: Non-Response Bias

2018 CTC

V0D 4

For a given election, voting polls are either

  • before the election (opinion polls) or
  • after the election (exit polls).
  • Opinion polls forecast: use hypothetical questions
  • Exit polls explain: use factual questions

Explanatory polls just tabulate – like sports statistics Forecast polls use models – like weather forecasts

Voter Polls: Before or After

2018 CTC

V0D 5

Election (forecast) polls are harder to interpret! Consider two candidates in different contests Both have 52% of their respective votes.

  • Candidate #1 has 60% chance of winning.
  • Candidate #2 has a 75% chance of winning.
  • Q. How can they have different chances of winning?
  • A. It depends on the size of the polling error!

Chance of Winning: “It depends”

2018 CTC

V0D 6

Forecast polls have sampling error (theoretical):

  • Difference b/t sample and population statistic
  • Entirely due to random sampling
  • Decreases as sample size increases

Forecast polls also have modeling error (empirical)

  • Difference between the forecast and actual result
  • Due to different models of undecided
  • Independent of sample size

Error in Forecast Polls: Sampling and Forecast

slide-2
SLIDE 2

Why 25% of Voter Polls Are Wrong V0D 14 Nov 2018 www.StatLit.org/pdf/2018-Schield-CTC-Slides.pdf Page 2

2018 CTC

V0D 7

.

Average Opinion Polling Error 1972-2016

2018 CTC

V0D 8

Average popular-vote polling error* in final 21 days:

  • 4 points: Presidential
  • 5.4 points: US Senate and Governor
  • 6.2 points: US House
  • 8.9 points: Primary

* D-R margin error. https://fivethirtyeight.com/features/the-polls-are-all-right/

95% sampling error by sample size:

  • 3.3 points: n = 900
  • 3.0 points: n = 1,024 [Most common sample size]
  • 2.8 points: n = 1,225

Opinion Polling Error Bigger than Sampling Error

2018 CTC

V0D 9

.

Low Response Rates: Non-Response Bias ???

2018 CTC

V0D 10

Causes of error unique to opinion (forecast) polls:

  • 1. Respondents undecided about whether to vote
  • 2. Respondents undecided about who to vote for
  • 3. Respondents change their mind before voting
  • 4. Voter opinion polls must forecast
  • which undecided will vote
  • who undecided voters will vote for
  • which third-party voters will change their mind
  • 5. Different polls allocate undecided differently
  • 6. Polls allocate using demographic/historical data

Causes of Error in Opinion Polls

2018 CTC

V0D 11

Undecided, Third-Party & Other: last three Presidential elections 2016 ….. * Biggest peak (16%) * Slowest to decide * Hardest to model * Biggest problem for forecasters

Drew Linzer tweet (11/5/2016) Permission granted.

www.hot1045.net/decodedc/yes-undecided-voters-still-exist

Main Cause of Polling Error: Undecided

2018 CTC

V0D 12

To interpret polls, know the difference:

  • 1. between an opinion poll, an exit poll and a survey
  • 2. between polling error and sampling error

To evaluate polls, know that

  • 1. the chance of winning depends on polling error
  • 2. polling error is bigger than sampling error
  • 3. polling error exists because opinion polls are forecasts
  • 4. the size of polling error depends on the contest
  • Q. How can pollsters educate readers on polling error?

Conclusions

slide-3
SLIDE 3

2018 CTC

V0D 1

by Milo Schield, Augsburg University Fellow: American Statistical Association US Rep: International Statistical Literacy Project December 2, 2018

Twin Cities Critical Thinking Club (CTC) Slides: www.StatLit.org/pdf/2018-Schield-CTC-Slides.pdf

Why 25% of Voter Polls are Wrong

slide-4
SLIDE 4

2018 CTC

V0D 2

Polls are short surveys.

  • Less demographic information.
  • Fewer choices for answers.

Polls typically involve sampling: sampling error. Voter polls are of wide-spread interest. 25%* of voter polls in the final 21 days are wrong. Wrong: actual result outside 95% confidence interval In 2016, 29%** of those in final 21 days were wrong.

* https://fivethirtyeight.com/features/how-the-fivethirtyeight-senate-forecast-model-works/ ** https://fivethirtyeight.com/features/the-polls-are-all-right/

25% of Voter Polls are Wrong!

slide-5
SLIDE 5

2018 CTC

V0D 3

Surveys and polls often interview people by phone. Plus: Cheap; random sample. Minus: Non-response bias. People who respond are not representative of the entire population. The lower the response rate, the greater the risk of bias.

http://www.pewresearch.org/2017/05/15/what-low-response-rates-mean-for-telephone-surveys/pm-05-15-2017_rddnonresponse-00-14/

Random Sampling: Non-Response Bias

slide-6
SLIDE 6

2018 CTC

V0D 4

For a given election, voting polls are either

  • before the election (opinion polls) or
  • after the election (exit polls).
  • Opinion polls forecast: use hypothetical questions
  • Exit polls explain: use factual questions

Explanatory polls just tabulate – like sports statistics Forecast polls use models – like weather forecasts

Voter Polls: Before or After

slide-7
SLIDE 7

2018 CTC

V0D 5

Election (forecast) polls are harder to interpret! Consider two candidates in different contests Both have 52% of their respective votes.

  • Candidate #1 has 60% chance of winning.
  • Candidate #2 has a 75% chance of winning.
  • Q. How can they have different chances of winning?
  • A. It depends on the size of the polling error!

Chance of Winning: “It depends”

slide-8
SLIDE 8

2018 CTC

V0D 6

Forecast polls have sampling error (theoretical):

  • Difference b/t sample and population statistic
  • Entirely due to random sampling
  • Decreases as sample size increases

Forecast polls also have modeling error (empirical)

  • Difference between the forecast and actual result
  • Due to different models of undecided
  • Independent of sample size

Error in Forecast Polls: Sampling and Forecast

slide-9
SLIDE 9

2018 CTC

V0D 7

.

Average Opinion Polling Error 1972-2016

slide-10
SLIDE 10

2018 CTC

V0D 8

Average popular-vote polling error* in final 21 days:

  • 4 points: Presidential
  • 5.4 points: US Senate and Governor
  • 6.2 points: US House
  • 8.9 points: Primary

* D-R margin error. https://fivethirtyeight.com/features/the-polls-are-all-right/

95% sampling error by sample size:

  • 3.3 points: n = 900
  • 3.0 points: n = 1,024 [Most common sample size]
  • 2.8 points: n = 1,225

Opinion Polling Error Bigger than Sampling Error

slide-11
SLIDE 11

2018 CTC

V0D 9

.

Low Response Rates: Non-Response Bias ???

slide-12
SLIDE 12

2018 CTC

V0D 10

Causes of error unique to opinion (forecast) polls:

  • 1. Respondents undecided about whether to vote
  • 2. Respondents undecided about who to vote for
  • 3. Respondents change their mind before voting
  • 4. Voter opinion polls must forecast
  • which undecided will vote
  • who undecided voters will vote for
  • which third-party voters will change their mind
  • 5. Different polls allocate undecided differently
  • 6. Polls allocate using demographic/historical data

Causes of Error in Opinion Polls

slide-13
SLIDE 13

2018 CTC

V0D 11

Undecided, Third-Party & Other: last three Presidential elections 2016 ….. * Biggest peak (16%) * Slowest to decide * Hardest to model * Biggest problem for forecasters

Drew Linzer tweet (11/5/2016) Permission granted.

www.hot1045.net/decodedc/yes-undecided-voters-still-exist

Main Cause of Polling Error: Undecided

slide-14
SLIDE 14

2018 CTC

V0D 12

To interpret polls, know the difference:

  • 1. between an opinion poll, an exit poll and a survey
  • 2. between polling error and sampling error

To evaluate polls, know that

  • 1. the chance of winning depends on polling error
  • 2. polling error is bigger than sampling error
  • 3. polling error exists because opinion polls are forecasts
  • 4. the size of polling error depends on the contest
  • Q. How can pollsters educate readers on polling error?

Conclusions