Implementing a Big Data strategy in the European Statistical System - - PowerPoint PPT Presentation

Implementing a Big Data strategy in the European Statistical System

Pieter Everaers,

Director Cooperation in the European Statistical System; International Cooperation, Eurostat SPECIAL SESSION ON SHOWCASING BIG DATA Bangkok, Thailand

Eurostat / ESS experience

• High level actionable commitment
• Engage stakeholders within the European Commission
• Raising awareness within Eurostat and the ESS

ESS

• Scheveningen Memorandum on Big Data
• Eurostat Task Force Big Data and ESS task force Big Data
• Big Data Roadmap and Action Plan 1.0
• Big data Vision Implementation Project
• Legal, ethical aspects, communication, skills and training,

information exchange

• ESS Pilots

Engage stakeholders

• European Commission Communication
• "Towards a thriving data driven economy"

Implementation of ESS Vision 2020

• Big Data roadmap and action plan integral part of portfolio

Current state

• Examine the potential of Big Data sources for official statistics
• Official Statistics Big Data strategy as part of wider government

strategy

• Address privacy and data protection
• Collaboration at European and global level
• Address need for skills
• Partnerships

between different stakeholders (government, academics, private sector)

• Developments in methodology, quality assessment and IT
• Adopt action plan and roadmap for the European Statistical

System

Scheveningen Memorandum

Roadmap

"As is" versus "To be"

Long term Vision Medium term aims Short term objectives

BY 2016 BY 2020 > 2020

Short-term objectives

• Official statistics integrated in the Commission big data strategy
• Identification and analysis of output portfolio of big data

sources

• Pilot projects
• Skills and ESS professional training programmes are established
• Research lines on big data included into the 2016-2017 Work

Programme of the Horizon 2020 Research Framework Programme

• Communication to the general public on big data activities
• Exchange of information with stakeholders within the

statistical system and the research community

Medium-term objectives

• Official Statistics integrated in governmental big data

strategies at national levels across the ESS

• Pilots are finalised
• Adequate IT infrastructures and methodological and quality

frameworks are available, data science skills integral part of

• fficial statistics education
• Public Private Partnerships are in place on big data and

Official Statistics.

• Ethical guidelines and communication strategy

Long-term objective

• Big data sources integrated in ESS official statistics

production

Experience sharing Policy Legislation Quality Skills Methodology IT Infrastructure Ethics / Communication P I L O T S

The ESS Big Data Action Plan

Communication

Mobile phone data Social Media

WWW

Web Searches Businesses' Websites E-commerce websites Job advertisements Real estate websites

Sensors

Traffic loops Smart meters Vessel Identification Satellite Images

Process generated data

Flight Booking transactions Supermarket Cashier Data Financial transactions

Crow d sourcing

Volunteered geographic information (VGI) Community pictures collection

The ESS Big Data Action Plan

Blending of Sources and multipurpose Statistics

Mobile Phone Data

Tourism Statistics Population Statistics Migration Statistics Traffic Statistics Commuting Statistics

Population Statisics

Mobile phone data Smart Meters VGI w ebsites Satellite Images

• Risks identified in the Action Plan and Roadmap
• Difficult access to data
• Negative public perception
• Skill shortages
• …
• Eurostat suggested approach
• Structuring risks and solutions
• Analysis based on a quality framework
• Collection of additional risks via survey among experts
• Likelihood and Impact of risks by source

Risks in the use of big data for producing official statistics

• Engage the stakeholders within the European Commission
• Cooperation at International level
• Competition (Nowcasting)
• Hackathons

Raising awareness

Engage stakeholders within the European Commission

• Data, Information and Knowledge Management at the

European Commission

• Active cooperation with several policy DGs
• Recognising a central role for the ESS
• Towards a common Big Data Strategy supported by

appropriate IT infrastructure and skills

• Data4Policy inter-departmental group
• Applications oriented interdepartmental group
• Fostering collaboration with policy departments and the

Joint Research Center (JRC) on concrete applications of using big data for policy.

• High-Level Group for the Modernisation of Statistical Production

and Services, Task Teams working on:

• Pilots
• Indicators for monitoring Sustainable Development Goals
• Advocacy and Communication
• Access and Partnerships
• Quality, Methodology, Taxonomy
• Inventory
• Collaboration with the UNECE
• Sandbox environment

Global Working Group on Big Data for Official Statistics

Format

• Collect predictions before official release is out. Repeat for

several periods and finally compare performance. Purpose

• Provides an evaluation forum for methods and sources and

their applicability for statistical purposes

• Raise awareness within the scientific community of problems

faced by the statistical system Application domains

• Any statistical indicator which is frequent and stable enough

can be used including SDG related ones Expected launch: Autumn 2015

Competition (Nowcasting)

Hackathons

Format and purpose

• Get together for a day (or a night) or two, work intensely
• n a problem
• Get to know about specific problems, get to know people,

come up with innovative solutions

• Traditionally done in computer science

Application domains

• For Eurostat it was nowcasting unemployment with data

from Google Trends

• Many other possibilities exist including SDG related

indicators

• Data flows instead of surveys and censuses
• Data customer instead of data provider
• Product designers instead of data collection designers
• New answers related to
• Quality and transparency
• Privacy and confidentiality
• Access to third party data sources / data sharing
• Scientific standards and methodology
• Professional ethics
• Skills
• Accreditation and certification instead of production
• Embedded in data flow – statistics 'everywhere'

The statistical office of the future