Data exploration at the speed of thought Lessons learned from - - PowerPoint PPT Presentation

▶
data exploration at the speed of thought
SMART_READER_LITE
LIVE PREVIEW

Data exploration at the speed of thought Lessons learned from - - PowerPoint PPT Presentation

Apr 06, 2024 128 likes •369 views

Data exploration at the speed of thought Lessons learned from inside Google Nico G o Gaviol ola Head ead o of H Heal ealthcar are e an and Li Lifes escien ences es U UKIE nicoga ogaviola@goo googl gle.com om Goog oogles

slide-1
SLIDE 1

Nico G

  • Gaviol
  • la

Head ead o

  • f H

Heal ealthcar are e an and Li Lifes escien ences es U UKIE nicoga

  • gaviola@goo

googl gle.com

  • m

Data exploration at the speed of thought

Lessons learned from inside Google

slide-2
SLIDE 2

Goog

  • ogle’s mission
  • n is t

t o or

  • organize t

t he world ld’s in informat io ion and make it it univ iversally lly accessib ible le and useful. l.

Sundar Pichai CEO, Google

slide-3
SLIDE 3
slide-4
SLIDE 4

uploads per minute users search index query response time

500h 500hrs 1B+ B+ 100P 100PB+ 0. 0.25s 25s

Google computing scale

slide-5
SLIDE 5

Hitting the limits, early on...

The Anatomy of a Large-Scale Hypertextual Web Search Engine 1996, Sergey Brin and Lawrence Page Computer Science Department, Stanford University, Stanford, CA 94305

slide-6
SLIDE 6

2012 2012 2013 2013 2002 2002 2004 2004 2006 2006 2008 2008 2010 2010

Google Research Publications referenced are available here: http://research.google.com/pubs/papers.html The Datacenter as a Computer: An Introduction to the Design of Warehouse-Scale Machines, 2009 http://research.google.com/pubs/pub35290.html

GFS MapReduce BigTable

Single Node to Cluster

slide-7
SLIDE 7

Google’s Data Research

2002 2004 2006 2008 2010 2012 2014 2016 G FS MapR educe TensorFlow BigTable Dremel C

  • lossus

Flume Megastore Spanner Millwheel PubSub F1

slide-8
SLIDE 8

Google’s Data Products

2002 2004 2006 2008 2010 2012 2014 2016 ML PubSub DataFlow DataStore DataFlow C loud Storage BigQ uery BigTable DataProc C loud Storage

slide-9
SLIDE 9

Pro rogramm mming Res esource e provisioni

  • ning

ng Performa mance tuni ning ng Moni

  • nitor
  • ring

ng Relia liabilit ility Depl ploy

  • yment

nt & & conf

  • nfiguration
  • n

Handl ndling ng grow

  • wing

ng scale Utiliz ilizatio ion impro rovements

Typical Big Data Jobs

slide-10
SLIDE 10

Big Data with Google

Focus on insights. Not infrastructure.

Pro rogramm mming

Unde nderstandi nding ng

slide-11
SLIDE 11

Google’s Big Data Vision

Pay $5 per TB

slide-12
SLIDE 12

Active contributor to numerous OSS projects Make migrations easier with open APIs Customers should use us because they love us, not because they are unable to move off

Open Source & APIs

12

slide-13
SLIDE 13

Confidential & Proprietary Google Cloud Platform 13

You own your data and remain Data Controller You can delete or remove your data at any time Google does not share your content or personal information google.com/privacy Strict Internal Policies : all accesses to customer or consumer data applications are logged Internal data access auditing tracks Googlers

Google Security Model & You!

slide-14
SLIDE 14

Example

slide-15
SLIDE 15

“Right at the start of the partnership we were able to reduce tim e to insight from 96 hours to 30 m inutes by using BigQuery”

Gar ary S San anders Head of Digital Analytics

slide-16
SLIDE 16

What’s Next?

slide-17
SLIDE 17

“Machine learning is a core, transformative way by which we’re re- thinking how we’re doing everything”

Sundar Pichai CEO, Google

slide-18
SLIDE 18

15% reduction in PUE

slide-19
SLIDE 19

Fully trained, easy to use Machine Learning models

Cloud Translate Cloud Vision Cloud Speech Cloud Natural Language Stay tuned…

slide-20
SLIDE 20

Use your own data to train models

Cloud Storage BigQuery Cloud Datalab Cloud Machine Learning Develop, Model, Train, Test

slide-21
SLIDE 21

One more thing

slide-22
SLIDE 22

Free training courses coming near you!

slide-23
SLIDE 23

Thank you!