Modernizing Data Estates with Presto Ken Seier, Chief Architect | - - PowerPoint PPT Presentation

Modernizing Data Estates with Presto

Ken Seier, Chief Architect | Data & AI ken.seier@insight.com

Insight fast facts

1988

GLOBAL REACH DEEP PORTFOLIO & RELATIONSHIPS ENGAGED WORKFORCE BROAD EXPERTISE FINANCIAL STABILITY FOUNDED IN

3,500+ 7,500+

19 countries

$9B+ 11,000+

Insight teammates worldwide Fortune 500 company with long legacy and knowledge serving clients around the globe in revenue in 2018 Hardware, software and cloud partners Sales and service delivery professionals

Presto today

• Targeted query federation for

line-of-business applications or reporting

• Ad hoc analytics enablement
• Tech and Retail verticals, with

some FinServ

https://db-engines.com/en/ranking_trend/system/Presto

Federated queries and data aggregation

Presto doing what we know its good at, and a little more.

Challenge

• Global technical services company
• 500,000+ customers
• 300,000+ events/second
• End-user investigation tool with

cumbersome Java query tier

Detail events in Amazon S3 Event aggregates in Elasticsearch Simplified Java/SQL services Custom insights UX Starburst Presto query fabric

Federated query solution

Pre-aggregation ETL solution

Amazon Elasticsearch

Outcomes

• Rationalized Java query tier to single

Presto SQL source

• Implemented pre-aggregation ETL in

same AWS/Java/Presto toolset

• Elasticsearch queries through Presto
• ver 1 million documents return in

<2 seconds

Big Data 2.0

Presto is a lighter replacement for aging big SQL tools.

Challenge

• Global software-as-a-service

company

• 15,000,000+ customers
• Ad-hoc queries over 100 terabytes of

cleansed data

• Aging on-prem big-data-SQL

implementation challenged to scale

Hive to Presto

ANSI SQL queries Hive QL queries

Data lake Data lake

Starburst Presto

Outcomes

• Data-in-place replacement for Hive
• Migrate from HiveQL to ANSI SQL
• Many-X concurrency improvement
• ver Hive
• 10X performance over Spark

benchmarks

Unified Query Plane

Using Presto to simplify and de-risk legacy data management

Challenge

• Global manufacturer/retailer
• $20,000,000,000+ globally
• Rich operational ecosystem
• Aggressively working toward

comprehensive stack rationalization

Legacy data estate

Presto data fabric

Outcomes

• Many, many Presto sources and consumers
• Supporting data science and line-of-business on

isolated clusters

• Presto abstraction over legacy systems enables

table-by-table migrations

• Row and column level RBAC enabled in Ranger
• End-to-end automation for registering and

managing data definitions: metadata, stats and security

• Query-grain costbacks enabled with log listener

Trends

Where Presto may be headed

Presto today

• Targeted query federation for

applications or reporting

• Ad hoc analytics enablement
• Tech and Retail verticals, with

some FinServ

https://db-engines.com/en/ranking_trend/system/Presto

Presto going forward

• Adoption driven by data science value
• Drafting with Kubernetes adoption
• Awareness in new industries
• Data estate rationalization
• Blue/green migration abstraction
• New data tier and estate patterns

Presto going forward Core data value cases

Historical Reporting

Line of business reporting for defined historical period using defined metrics and performance indicators

Data warehouse Data mart Operational Data Store

Line of business reporting for making real-time course corrections in day to day

• perations

Small, disk-bound or in-memory store Analytics Discovery

Line of business reporting for making real-time course corrections in day to day

• perations

Data lake Lab environment

Presto going forward Conceptual data architecture

ANSI SQL data & insight

Data directly from source systems Data staged from source systems Direct event and transactional data Insight enrichment

Three fully- decoupled, horizontally scalable, single-tool tiers

Data lake

• f choice

Questions?

Ken Seier, Chief Architect | Data & AI ken.seier@insight.com