From offline to online: Retaining quality while transferring an - PowerPoint PPT Presentation

From offline to online: Retaining quality while transferring an established survey ASC - Smart research technology for cost- conscious times 27 th April 2012

Background Shared\Execs\Tim Barber\My Documents\Powerpoint\Ofcom\J11426 – Online Pilot\MRS conference presentation – 12.01.12 v3

Ofcom’s Media Tracker survey - background • Run since 1991 Methodology • Monitors public’s opinions and attitudes: • TV programme standards UK nat rep, adults 16+ • Protection of Children n = c.1,700 – 2 waves per year • News – sources, impartiality Long interview (c. 35 minutes) • Take up and usage of devices / services F2F • Purpose Oversample and monitor certain demographic groups • Methodology 3 Shared\Execs\Tim Barber\My Documents\Powerpoint\Ofcom\J11426 – Online Pilot\MRS conference presentation – 12.01.12 v3

Why did Ofcom want to explore possibility of transferring Media Tracker online? 4 Shared\Execs\Tim Barber\My Documents\Powerpoint\Ofcom\J11426 – Online Pilot\MRS conference presentation – 12.01.12 v3

Why did Ofcom want to explore possibility of transferring Media Tracker online? 5 Shared\Execs\Tim Barber\My Documents\Powerpoint\Ofcom\J11426 – Online Pilot\MRS conference presentation – 12.01.12 v3

MR spend Seeking to obtain enhanced value for money… 2.16 2.08 2.06 2005 2006 2007 2008 2009 2010 2011 UK Internet penetration UK MR spend (£bn) Sources: Ofcom Tech Tracker; MRS 6 Shared\Execs\Tim Barber\My Documents\Powerpoint\Ofcom\J11426 – Online Pilot\MRS conference presentation – 12.01.12 v3

MR spend vs. internet penetration Seeking to obtain enhanced value for money - at the same time viability of a natrep online survey has potentially increased 78% 2.16 75% 73% 67% 65% 2.08 61% 2.06 58% 2005 2006 2007 2008 2009 2010 2011 UK Internet penetration UK MR spend (£bn) Sources: Ofcom Tech Tracker; MRS 7 Shared\Execs\Tim Barber\My Documents\Powerpoint\Ofcom\J11426 – Online Pilot\MRS conference presentation – 12.01.12 v3

Why did Ofcom want to explore possibility of transferring Media Tracker online? Maximise value Economic climate Internet penetration Explore feasibility and practicalities 8 Shared\Execs\Tim Barber\My Documents\Powerpoint\Ofcom\J11426 – Online Pilot\MRS conference presentation – 12.01.12 v3

Key benefits from transferring to online Speed Reduce elapse time of project Cost-effectiveness Reduce cost and thus increase value Ability to use visual stimuli Engages respondents and makes the survey more interesting And others! 9 Shared\Execs\Tim Barber\My Documents\Powerpoint\Ofcom\J11426 – Online Pilot\MRS conference presentation – 12.01.12 v3

Key concerns - representivity Everyone People who use the internet are not the same as people that don’t use the internet Usage of technology Internet users Usage of media Internet users that are on online panels are not the same as people that aren't on online panels Attitudes towards media Internet users not on a panel Internet usage Internet users that aren't on online panels but fill in surveys, are not the same as internet users that aren't on online panels but don’t fill in surveys 10 Shared\Execs\Tim Barber\My Documents\Powerpoint\Ofcom\J11426 – Online Pilot\MRS conference presentation – 12.01.12 v3

Key concerns – Internet penetration varies between key groups Internet users are younger Ever use internet 96% 94% 89% 89% 87% 82% 76% 73% 70% 45% All Adults 16-34 35-44 45-54 55-64 65+ ABC1 C2DE Freeview Pay TV only Source: Ofcom Media Tracker 2011 11 Shared\Execs\Tim Barber\My Documents\Powerpoint\Ofcom\J11426 – Online Pilot\MRS conference presentation – 12.01.12 v3

Key concerns – Internet penetration varies between key groups Internet users are younger and watch TV differently Ever use internet 96% 94% 89% 89% 87% 82% 76% 73% 70% 45% All Adults 16-34 35-44 45-54 55-64 65+ ABC1 C2DE Freeview Pay TV only Source: Ofcom Media Tracker 2011 12 Shared\Execs\Tim Barber\My Documents\Powerpoint\Ofcom\J11426 – Online Pilot\MRS conference presentation – 12.01.12 v3

Key concerns – behavioural and attitudinal representivity Internet users behave differently…. All adults Internet users 86% 82% 69% 61% 55% 50% 42% 38% Ever listen to radio Sexually explicit programmes Aware TV channel blocking Aware Ofcom responsible should never be shown on TV regualting TV (DISAGREE) Source: Ofcom Media Tracker 2011 13 Shared\Execs\Tim Barber\My Documents\Powerpoint\Ofcom\J11426 – Online Pilot\MRS conference presentation – 12.01.12 v3

Key concerns – behavioural and attitudinal representivity Internet users behave differently and have different attitudes…. All adults Internet users 86% 82% 69% 61% 55% 50% 42% 38% Ever listen to radio Sexually explicit programmes Aware TV channel blocking Aware Ofcom responsible should never be shown on TV regualting TV (DISAGREE) Source: Ofcom Media Tracker 2011 14 Shared\Execs\Tim Barber\My Documents\Powerpoint\Ofcom\J11426 – Online Pilot\MRS conference presentation – 12.01.12 v3

Key concerns – behavioural and attitudinal representivity Internet users behave differently and have different attitudes…. All adults Internet users 86% 82% 69% 61% 55% 50% 42% 38% Ever listen to radio Sexually explicit programmes Aware TV channel blocking Aware Ofcom responsible should never be shown on TV regualting TV (DISAGREE) Source: Ofcom Media Tracker 2011 15 Shared\Execs\Tim Barber\My Documents\Powerpoint\Ofcom\J11426 – Online Pilot\MRS conference presentation – 12.01.12 v3

Key concerns – behavioural and attitudinal representivity Internet users behave differently and have different attitudes and knowledge All adults Internet users 86% 82% 69% 61% 55% 50% 42% 38% Ever listen to radio Sexually explicit programmes Aware TV channel blocking Aware Ofcom responsible should never be shown on TV regualting TV (DISAGREE) Source: Ofcom Media Tracker 2011 16 Shared\Execs\Tim Barber\My Documents\Powerpoint\Ofcom\J11426 – Online Pilot\MRS conference presentation – 12.01.12 v3

Key concerns – behavioural and attitudinal representivity Internet users behave differently and have different attitudes and knowledge All adults Internet users 86% 82% 69% 61% 55% 50% 42% 38% Ever listen to radio Sexually explicit programmes Aware TV channel blocking Aware Ofcom responsible should never be shown on TV regulating TV (DISAGREE) Source: Ofcom Media Tracker 2011 17 Shared\Execs\Tim Barber\My Documents\Powerpoint\Ofcom\J11426 – Online Pilot\MRS conference presentation – 12.01.12 v3

Key concerns – behavioural and attitudinal representivity And of course – they use the internet more frequently 80% All adults Internet users 56% 20% 17% 5% 5% 5% 2% 1% 0% Every day Several times a week At least once a month Several times a year Never Source: Ofcom Media Tracker 2011 18 Shared\Execs\Tim Barber\My Documents\Powerpoint\Ofcom\J11426 – Online Pilot\MRS conference presentation – 12.01.12 v3

How to ensure representative (as far as possible) Media Tracker particularly prone to highlights differences Set quotas to ensure demographic match in answers concerning: Sources of news Sample from multiple sources Attitudes towards news sources Include specific additional attitudinal and behavioral questions TV reception method Weight the data Devices used for watching TV Internet usage 19 Shared\Execs\Tim Barber\My Documents\Powerpoint\Ofcom\J11426 – Online Pilot\MRS conference presentation – 12.01.12 v3

How to ensure representative (as far as possible) Media Tracker particularly prone to highlights differences Set quotas to ensure demographic match in answers concerning: Sources of news Sample from multiple sources Attitudes towards news sources Include specific additional attitudinal and behavioral questions TV reception method Weight the data Devices used for watching TV Internet usage 20 Shared\Execs\Tim Barber\My Documents\Powerpoint\Ofcom\J11426 – Online Pilot\MRS conference presentation – 12.01.12 v3

Other concerns - quality Data checked for: Speeding Not as regulated as F2F or telephone Flat lining Self-completion Contradictory answers Participation is incentivised Sensible open-ended responses Self-screening Responses compared to information already held Multiple responses from same IP address 21 Shared\Execs\Tim Barber\My Documents\Powerpoint\Ofcom\J11426 – Online Pilot\MRS conference presentation – 12.01.12 v3

Other concerns - quality Data checked for: Speeding Not as regulated as F2F or telephone Flat lining Self-completion Contradictory answers Participation is incentivised Sensible open-ended responses Self-screening Responses compared to information already held Multiple responses from same IP address 22 Shared\Execs\Tim Barber\My Documents\Powerpoint\Ofcom\J11426 – Online Pilot\MRS conference presentation – 12.01.12 v3

Other concerns - acceptance TV and Radio broadcasters Not as established as F2F or telephone Advertisers Non-acceptance that it is as valid or comparable as offline methods News suppliers If weighting is required, it can be difficult to Lobbying groups explain to audiences what has been done and why it is necessary The general public! 23 Shared\Execs\Tim Barber\My Documents\Powerpoint\Ofcom\J11426 – Online Pilot\MRS conference presentation – 12.01.12 v3