Data exploration at the speed of thought Lessons learned from - PowerPoint PPT Presentation

Data exploration at the speed of thought Lessons learned from inside Google Nico G o Gaviol ola Head ead o of H Heal ealthcar are e an and Li Lifes escien ences es U UKIE nicoga ogaviola@goo googl gle.com om

Goog oogle’s mission on is t t o or o organize t t he world ld’s in informat io ion and make it it univ iversally lly accessib ible le and useful. l. Sundar Pichai CEO, Google

Google computing scale 500h 500hrs uploads per minute 1B+ B+ users 100P 100PB+ search index 0. 0.25s 25s query response time

Hitting the limits, early on... The Anatomy of a Large-Scale Hypertextual Web Search Engine 1996, Sergey Brin and Lawrence Page Computer Science Department, Stanford University, Stanford, CA 94305

Single Node to Cluster MapReduce GFS BigTable 2002 2002 2004 2004 2006 2006 2008 2008 2010 2010 2012 2012 2013 2013 Google Research Publications referenced are available here: http://research.google.com/pubs/papers.html The Datacenter as a Computer: An Introduction to the Design of Warehouse-Scale Machines, 2009 http://research.google.com/pubs/pub35290.html

Google’s Data Research Flume MapR educe Dremel Millwheel TensorFlow G FS Megastore PubSub BigTable C olossus Spanner F1 2002 2004 2006 2008 2010 2012 2014 2016

Google’s Data Products DataFlow DataProc BigQ uery DataFlow ML C loud Storage DataStore PubSub BigTable C loud Storage 2002 2004 2006 2008 2010 2012 2014 2016

Moni onitor oring ng Pro rogramm mming Performa mance Res esource e tuni ning ng provisioni oning ng Typical Big Data Jobs Utiliz ilizatio ion Handl ndling ng impro rovements grow owing ng scale Depl ploy oyment nt & & Relia liabilit ility conf onfiguration on

Pro rogramm mming Big Data with Google Unde nderstandi nding ng Focus on insights. Not infrastructure.

Google’s Big Data Vision Pay $5 per TB

Open Source & APIs Active contributor to numerous OSS projects Make migrations easier with open APIs Customers should use us because they love us, not because they are unable to move off 12

Google Security Model & You! You own your data and You can delete or Google does not share Strict Internal Policies : Internal data access remain Data Controller remove your data at your content or all accesses to auditing tracks any time personal information customer or consumer Googlers google.com/privacy data applications are logged Google Cloud Platform Confidential & Proprietary 13

Example

“Right at the start of the partnership we were able to reduce tim e to insight from 96 hours to 30 m inutes by using BigQuery” Gar ary S San anders Head of Digital Analytics

What’s Next?

“Machine learning is a core, transformative way by which we’re re- thinking how we’re doing everything” Sundar Pichai CEO, Google

15% reduction in PUE

Fully trained, easy to use Machine Learning models Cloud Cloud Cloud Cloud Stay tuned… Translate Vision Speech Natural Language

Use your own data to train models Cloud Machine Learning Develop, Model, Cloud Storage BigQuery Cloud Datalab Train, Test

One more thing

Free training courses coming near you!

Thank you!