impact of methodology John Curtice Strathclyde University - PowerPoint PPT Presentation

Are the polls right? A look at the impact of methodology John Curtice Strathclyde University

Questions • Are different companies telling the same or a different story? • Why? – Similarity/differences in raw data (achieved samples) – Weighting – Turnout – Spiral of Silence

Method • Calculate each company’s average ratings between November 2014 and January 2015 • Look for clues in details of last 2 polls conducted by each company in this period • Replicates similar exercise undertaken on polls conducted between June and September 2014 • Exercise is confined to British polls

Average Poll Ratings Con Lab Lib Dem UKIP Green (N) Survation 29 32 7 23 3 (3) * TNS* 29 33 6 18 6 (4) Opinium* 30 33 7 18 5 (7) ComRes * 32 34 8 18 3 (3) ComRes 29 31 10 17 6 (3) Ashcroft 32 35 8 16 5 (9) Populus* 32 35 8 16 5 (25) YouGov* 32 33 7 15 7 (61) ICM 30 33 12 13 7 (3) Ipsos 32 31 9 13 8 (3) MORI * = Internet Polls. Remainder by Phone

Points To Note • ‘House’ differences in estimate of Con/Lab lead are small (Ipsos MORI somewhat apart) • But systematic differences in UKIP Estimate • ICM noticeably favourable to Lib Dems • Not a simple phone/internet divide

Weighting Etc. Is Helping To Achieve Convergence (Lab lead over Con) Raw Reported ComRes 6.5 1 ICM 5.5 4 Ashcroft 5 -0.5 Opinium* 4.5 3 YouGov* 2.5 1.5 Populus* 2.5 1 Survation* 1 1.5 ComRes* 0.5 1 TNS* 0.5 3.5 Ipsos MORI -1.5 -1

Turnout Weighting/Filtering Usually Reduces Labour Lead Average Impact on Lab Lead Ashcroft -1.5 Populus* -1.5 ICM -1.25 Opinium* -0.5 Survation* -0 Ipsos MORI +1.5

‘Spiral of Silence’ Adjustments ICM (1) ICM (2) A (1) A (2) SV (1) SV (2) Con 0 -1 +1 +1 +1 0 Lab -1 -2 0 0 0 0 Lib Dem +3 +3 +1 0 +1 +1 UKIP 0 -1 0 -1 -1 -1 Grn 0 0 0 0 0 0

UKIP Differences Mostly in Raw Data Reported Raw Survation* 23 22.5 TNS* 17 18 Opinium* 19 16 ComRes* 18 19 ComRes 16.5 17 Ashcroft 15 15 Populus* 13.5 20 YouGov* 14.5 17 ICM 13 14 Ipsos MORI 12 10.5

YouGov PartyID Weighting Target Poll 1 Poll 2 Con 24 24 26 Lab 32 30 31 LD 9 7 8 Nat 2 3 3 Other 5 9 9 None 24 27 23

Populus Party ID Weighting Target Poll 1 Poll 2 Con 28 29 29 Lab 29 30 31 Lib Dem 10 7 7 UKIP 4 12 11 Other 5 9 9 None etc 24 13 13

Conclusions • Estimate of Lab lead is relatively consistent between companies (they are either all right or all wrong!) • Widely used turnout filtering/weighting is usually helping to reduce that lead as are less commonly applied ‘spiral of silence’ adjustments • But are substantial differences in UKIP estimates that mostly reflect differences in achieved samples & correlates with a internet/phone divide • Is ICM’s ‘spiral of silence’ adjustment too kind to the Lib Dems?

Demographic Weighting (Ratio 18-24 C2DE Weighted:Unweighted Opinium* 0.79+ 1.38 YouGov* 1.36 1.28 ComRes* 1.68 1.22 ICM 1.93 1.19 Ashcroft 1.38 1.15 Ipsos MORI 1.45 1.14 Survation* 1.12+ 1.13 ComRes 1.44 1.08 Populus* 0.88 1.07 TNS* 0.89 1.04 + Based on 18-34 year olds. * Internet poll – rest by phone