VLDB Challenges VLDB Challenges in in Very Large Very Large - - PowerPoint PPT Presentation

▶

Feb 29, 2024 216 likes •330 views

VLDB Challenges VLDB Challenges in in Very Large Very Large Enterprises Enterprises Panelists Chair Chair Dr. Michael L. Brodie, Chief Scientist, Verizon Problem Owners Problem Owners Dr. Hans-Peter Steiert, Research &

SLIDE 1

VLDB Challenges VLDB Challenges in in Very Large Very Large Enterprises Enterprises

SLIDE 2

Chair Chair

Dr. Michael L. Brodie, Chief Scientist, Verizon

Problem Owners Problem Owners

Dr. Hans-Peter Steiert, Research & Technology,

DaimlerChrysler AG

Solution Owners Solution Owners

Adam Bosworth, VP, Engineering, BEA Systems Inc
James Hamilton, Architect, Microsoft SQL Server, Microsoft
Pat Selinger, IBM Fellow and VP, Data Management

Architecture and Technology, IBM

Panelists

SLIDE 3

Very Large Enterprises

Petabyte

248 $ 6 7 1 1 2 6 Verizon 373 $ 1 3 6 N/ A 7

Daim ler Chrysler

Data

Em ployees ( 1 ,0 0 0 s)

Revenues ( $ B US)

Fortune 5 0 0 Global 5 0 0 Very Large Enterprise

Problem Drivers Problem Drivers

Data Growth
Data Life Cycle

SLIDE 4

OLTP W orkload Grow th

1,000 2,000 3,000 Jan-01 Dec-02 Dec-04

OLTP/sec Triples 2001-2004

Projected average workload

0% 40% 80% 120% Two Years 1/01 - 12/02 Four Years 1/01 - 12/04

OLTP Doubles by 12/04

Projected Workload Growth Rate (average)

SLIDE 5

DSS W orkload Grow th

25 50 75 100 125 150 On 1/2001 On 12/2002 On 12/2004

Inflight Queries Double by 04

Projected workload (average) (concurrent inflight queries)

0% 20% 40% 60%

2 Yr Growth 4 Yr growth

DSS Workload Triples by 04

Projected workload growth rate (average)

SLIDE 6

Database Grow th

1 2 3 4 5 6

Respondents' Projected Database Size (average) (TB)

On Jan 2001 On 12 2002 On 12 2004

Size: OLTP Doubles; DSS Triples by 04

OLTP DSS OLTP N = 43 DSS N = 67

0% 50% 100% 150%

Jan 01 - Dec 02 Jan 01 - Dec 04

Growth: DSS & OLTP Double by 04

OLTP DSS

Respondents' Projected Database Growth Rate (average)

SLIDE 7

VLE Storage Grow th

500 1000 1500 2000 2500 3000 3500 Terabytes 2001 2002 2003 2004 2005 2006 Calendar Year

VLE X Storage Triples 02-06

IP SAN/NAS OS Vertical OS Shared FC SAN System390

SLIDE 8

Data Life Cycle

Life Cycle Actions Life Cycle Actions

Create
Store
Replicate
Protect
Update
Archive
Exchange, exchange, exchange, …

Factors Factors

History: 40+ years of Mergers & Acquisitions
Growth: Automation & Partnering
Protection: security, confidentiality, …
The Grand Challenge: semantics of data

Data Droppings Problem

SLIDE 9

Grand Challenge: $ 1 Trillion/ year

Integration Cost Estimates Integration Cost Estimates

24% of IT budgets: $180 B / year US (InfoWorld, January 2002)
13% of IT spend: $752 B / year US (Giga estimate; May 2002)
25-40% of all IT projects (various)
6% of US IT spending: $610B / year US (IDC, May 2002)
7% of IT spending: $1.3T / year worldwide (IDC, May 2002)
28+% of worldwide consulting: $ 160 B/year (Gartner, March 2002)
43% of e-business worldwide consulting: $53 B / year (Gartner)
1.75% to annual IT budget on EAI and B2Bi (Forrester, Dec 2001)
10-30% of IT budgets (David Sink, IBM, InformationWeek, May 27,

2002)

Data Quality Cost Estimates Data Quality Cost Estimates

$600 B / year US (Data Warehouse Institute, 2002)

SLIDE 10

VLE VLDB Challenges

Data Management Data Management

Global Data Managem ent

– Significant improvement in dealing automatically with semantics

Database Engineering
Automated DBA
Com prehensive Data Managem ent Architecture
Data architecture: Web Services, mid-tier, distributed data

Storage Management Storage Management

Data Protection

– Integrated products: DBMS, replication, …

Data Utilization

– Automated DBA, Storage Virtualization, Hierarchical Storage Management for distributed system