impact of methodology John Curtice Strathclyde University - - PowerPoint PPT Presentation

Are the polls right? A look at the impact of methodology

John Curtice Strathclyde University

Questions

• Are different companies telling the same or a

different story?

• Why?

– Similarity/differences in raw data (achieved samples) – Weighting – Turnout – Spiral of Silence

Method

• Calculate each company’s average ratings

between November 2014 and January 2015

• Look for clues in details of last 2 polls

conducted by each company in this period

• Replicates similar exercise undertaken on polls

conducted between June and September 2014

• Exercise is confined to British polls

Average Poll Ratings

Con Lab Lib Dem UKIP Green (N) Survation * 29 32 7 23 3 (3) TNS* 29 33 6 18 6 (4) Opinium* 30 33 7 18 5 (7) ComRes * 32 34 8 18 3 (3) ComRes 29 31 10 17 6 (3) Ashcroft 32 35 8 16 5 (9) Populus* 32 35 8 16 5 (25) YouGov* 32 33 7 15 7 (61) ICM 30 33 12 13 7 (3) Ipsos MORI 32 31 9 13 8 (3) * = Internet Polls. Remainder by Phone

Points To Note

• ‘House’ differences in estimate of Con/Lab

lead are small (Ipsos MORI somewhat apart)

• But systematic differences in UKIP Estimate
• ICM noticeably favourable to Lib Dems
• Not a simple phone/internet divide

Weighting Etc. Is Helping To Achieve Convergence

(Lab lead over Con) Raw Reported ComRes 6.5 1 ICM 5.5 4 Ashcroft 5

• 0.5

Opinium* 4.5 3 YouGov* 2.5 1.5 Populus* 2.5 1 Survation* 1 1.5 ComRes* 0.5 1 TNS* 0.5 3.5 Ipsos MORI

• 1.5
• 1

Turnout Weighting/Filtering Usually Reduces Labour Lead

Average Impact on Lab Lead Ashcroft

• 1.5

Populus*

• 1.5

ICM

• 1.25

Opinium*

• 0.5

Survation*

Ipsos MORI +1.5

‘Spiral of Silence’ Adjustments

ICM (1) ICM (2) A (1) A (2) SV (1) SV (2) Con

• 1

+1 +1 +1 Lab

• 1
• 2

Lib Dem +3 +3 +1 +1 +1 UKIP

• 1
• 1
• 1
• 1

Grn

UKIP Differences Mostly in Raw Data

Reported Raw Survation* 23 22.5 TNS* 17 18 Opinium* 19 16 ComRes* 18 19 ComRes 16.5 17 Ashcroft 15 15 Populus* 13.5 20 YouGov* 14.5 17 ICM 13 14 Ipsos MORI 12 10.5

YouGov PartyID Weighting

Target Poll 1 Poll 2 Con 24 24 26 Lab 32 30 31 LD 9 7 8 Nat 2 3 3 Other 5 9 9 None 24 27 23

Populus Party ID Weighting

Target Poll 1 Poll 2 Con 28 29 29 Lab 29 30 31 Lib Dem 10 7 7 UKIP 4 12 11 Other 5 9 9 None etc 24 13 13

Conclusions

• Estimate of Lab lead is relatively consistent between

companies (they are either all right or all wrong!)

• Widely used turnout filtering/weighting is usually

helping to reduce that lead as are less commonly applied ‘spiral of silence’ adjustments

• But are substantial differences in UKIP estimates that

mostly reflect differences in achieved samples & correlates with a internet/phone divide

• Is ICM’s ‘spiral of silence’ adjustment too kind to the

Lib Dems?

Demographic Weighting

(Ratio Weighted:Unweighted 18-24 C2DE Opinium* 0.79+ 1.38 YouGov* 1.36 1.28 ComRes* 1.68 1.22 ICM 1.93 1.19 Ashcroft 1.38 1.15 Ipsos MORI 1.45 1.14 Survation* 1.12+ 1.13 ComRes 1.44 1.08 Populus* 0.88 1.07 TNS* 0.89 1.04 + Based on 18-34 year olds. * Internet poll – rest by phone