[PPT] - Data Virtualization An Agile Approach To Improving Profitability PowerPoint Presentation

SLIDE 1

Data Virtualization – An Agile Approach To Improving Profitability

Mike Ferguson Managing Director Intelligent Business Strategies Denodo Executive Briefing Brussels, November 2016

SLIDE 2

2

About Mike Ferguson

Mike Ferguson is Managing Director of Intelligent Business Strategies Limited. As an analyst and consultant he specializes in business intelligence, data management and enterprise business

integration. With over 35 years of IT experience,

Mike has consulted for dozens of companies, spoken at events all over the world and written numerous

articles. Formerly he was a principal and co-founder
f Codd and Date Europe Limited – the inventors of

the Relational Model, a Chief Architect at Teradata on the Teradata DBMS and European Managing Director of DataBase Associates.

www.intelligentbusiness.biz mferguson@intelligentbusiness.biz Twitter: @mikeferguson1 Tel/Fax (+44)1625 520700

SLIDE 3

3

Topics

Improving profitability
The increasingly complex data landscape
The impact on the business of distributed data
What is data virtualisation?
Improving business performance and agility using data

virtualisation

Reducing time to value and increasing revenue in the logical

data warehouse

Conclusions

SLIDE 4

4

Improving Profitability Is The Goal Of Every Business

Profit Margin

SLIDE 5

5

Reducing Cost – How?

Reduce business complexity
Reduce location complexity
Reduce business function complexity e,g. across business units
Reduce management complexity – flatten hierarchies
Reduce product complexity
Reduce process complexity
Reduce IT system complexity AND modernise IT systems
Reduce data complexity
Automation and digitalisation across channel and lines of business
Improve performance management
Map cost centres and cost types across the value chain and all entities
Track the business impact of new initiatives
Optimise operations to reduce unplanned down time and costs

SLIDE 6

6

Increasing Revenue – How?

Customer centric organisation and operating model
Allow customers to configure products and services to meet their

needs

Then build and/or ship to order
Complete 360o insight of a customer
Targeted precision marketing for cross-sell / up-sell
Predictive and prescriptive analytics
Highest quality of customer service
Customer self-service via digital channels
Customer centric data integration
Complete view across value chain

SLIDE 7

7

Topics – Where Are We?

Improving profitability
The increasingly complex data landscape
The impact on the business of distributed data
What is data virtualisation?
Improving business performance and agility using data

virtualisation

Reducing time to value and increasing revenue in the logical

data warehouse

Conclusions

SLIDE 8

8

The Data Landscape Is Becoming Increasingly Complex And Lack of Integration Are Working Against Business

Line of business IT initiatives when there is a need for enterprise wide

common infrastructure

Multiple copies of data
Processes not integrated
Different user interfaces
Server platforms complexity
Duplicate application functionality
Point-to-Point “Spaghetti” application integration

Marketing System Customer Service System HR Gen. Ledger Procurement system Billing system Fulfilment System Sales System Gen. Ledger

SLIDE 9

9

Trends – More And More Appliances Appearing On The Market Causing ‘Islands’ of Data

Oracle Exadata IBM PureData System for Analytics

Pivotal Greenplum DCA

Teradata

SLIDE 10

10

Big Data Is Also Now In The Enterprise Introducing More Data Stores e.g. Hadoop, NoSQL, Analytic RDBMS

Graph DBMS MPP Analytical RDBMS

BI tools

SQL

indexes Search based BI tools Custom Analytic apps Spark & MR BI tools

OLTP data

Unstructured / semi-structured content actions users business analysts developers real-time

DW

social graph data RDBMS Files

clickstream

Web logs social data

Graph analytics tools Enterprise Information Management

event streams

Stream processing

IoT, markets

SLIDE 11

11

On-Premise Systems Within the Enterprise

Complexity Is Increasing Further As Companies Adopt and Deploy A Mix of On-Premise, SaaS and Cloud Based Systems

employees partners customers Private cloud Private or public cloud Enterprise Service Bus Enterprise Portal Mashups Office Applications SaaS BI Off-premise OLTP apps OLTP Systems

WWW

corporate firewall Data is now potentially fractured even more than before BI/DW Systems

SLIDE 12

12

Hundreds of New Data Sources Are Emerging

The Internet of Things (IoT)

High velocity, high volume data

SLIDE 13

13

The Challenge Of Fractured Data

Data in different locations
Data in different data storage

technologies

Data in different data structures
Different data definitions for the

same data in different data stores

Some data too big to move
Different APIs and query languages

needed to access data

Excessive use of ETL to copy data
Expensive and not agile
Synchronization nightmare

<XML>Text</XML> Digital media RDBMSs Web content E-mail Flat files Packaged applications Office documents Legacy applications BI systems

Big Data applications Cloud based applications

ECMS

“Where is all the Customer Data?”

Accessing, governing and managing data is becoming increasingly complex as it becomes more distributed

SLIDE 14

14

Topics– Where Are We?

Improving profitability
The increasingly complex data landscape
The impact on the business of distributed data
What is data virtualisation?
Improving business performance and agility using data

virtualisation

Reducing time to value and increasing revenue in the logical

data warehouse

Conclusions

SLIDE 15

15

Core Business Processes Often Now Execute Across A Hybrid Computing Environment

rder

credit check fulfil ship invoice payment package

Process Example - Manufacturing Order to cash

schedule

Order entry system Finance credit control system Production planning & scheduling system CAM system Inventory system Distribution system Billing Gen Ledger

Orders data Customer data Product data This makes data difficult to track, maintain, synchronise and manage

SLIDE 16

16

XYZ Corp.

Many Companies Have Business Units, Processes & Systems Organised Around Products and Services

Customers/ Prospects

Product/service line 1

rder

credit check fulfill ship invoice payment package

Product/service line 2 Product/ service line 3

Channels/ Outlets

rder

credit check fulfill ship invoice payment package

rder

credit check fulfill ship invoice payment package

Order

(product line 1)

Order

(product line 2)

Order

(product line 3)

Enterprise

SLIDE 17

17

Business and Data Complexity Can Spiral Out Of Control if Processes And Systems Are Duplicated Across Geographies

Product line 1 Product line 2 Product line 3 Product line 1 Product line 2 Product line 3 Product line 1 Product line 2 Product line 3 Product line 1 Product line 2 Product line 3 Product line 1 Product line 2 Product line 3

Suppliers Products/ Services Accounts Assets Employees Customers Partners Materials

SLIDE 18

18

Business Implications Of Product Orientation and Fractured Customer Data In A World Where Customer Is Now King

Different marketing campaigns from different divisions aimed at the same

customer

Different sales teams from different divisions selling to the same customer
Customer service is hard
e.g. “What is my order status for all products ordered?”
Cost of operating is much higher due to duplicate processes across

product lines

Can’t see customer / product ownership
Can’t see customer risk and customer profitability
Higher chance of poor data quality
Difficult to maintain customer data fractured across multiple applications

SLIDE 19

19

This Makes It Difficult To Access And Report on Data Across The Process To Manage Business Operations

rder

credit check fulfill ship invoice payment package Order-to-Cash Process Orders

What order changes in the last 10 mins? What shipments are impacted by the changes e.g. lack of inventory or shipping capacity? Which customers are affected?

Operational reporting is not timely Inability to respond quickly to problems Problems not seen until long after they happen e.g. incorrect shipments Operational oversights cause processing errors & unplanned operational cost Inability to see across multiple instances of a system can cause errors & duplication of effort

Business impact

SLIDE 20

20

Planning Also Requires Data From Across A Value Chain

Fore- casting Product, Materials Supplier Master data Planning

ERP ERP

CAD

Manufacturing execution system Shipping system SCADA systems

SCM

CRM system Need to see sales, inventory, shipments, manufacturing capacity, resources and forecasts

Plans are too resource intensive Planning slow and is

ut-dated by the time it

is finalised No flexibility, not dynamic, no scenarios

r simulations

Business impact

SLIDE 21

21

Multiple Data Warehouses Are Also Common

Fore- casting Product, Materials Supplier Master data Planning

ERP ERP

CAD

Manufacturing execution system Shipping system SCADA systems Manufacturing volumes & inventory DW Finance DW Sales & mktng DW

SCM

CRM system

Management reporting? KPIs?

SLIDE 22

22

Data Inconsistency Across DW Systems Is Common

BI tool BI tool DW

mart

BI tool BI tool DW

mart

BI tool BI tool DW

mart

Data Integration Data Integration Data Integration

Common data definitions across all tools for the same data? Common data definitions across all DWs for the same data? Common data transformations across all DWs for the same data?

SLIDE 23

23

Many Organisations Have Created Multiple DWs And A Lot Of Data Marts - E.g. Country Specific Data Marts

Sales DW

UK DE FR NL ESP CH IT BE US ETL ETL ETL ETL ETL ETL ETL ETL ETL

Inventory DW

UK DE FR NL ESP CH IT BE US ETL ETL ETL ETL ETL ETL ETL ETL ETL

Business Impact Very high total cost of ownership No Agility – Takes time and is costly to change Risk of inconsistency across DWs and marts No drill down across marts

SLIDE 24

24

New Data Sources Have Emerged Inside And Outside The Enterprise That Business Now Wants To Analyse

Web data
Clickstream data, e-commerce logs
Social networks data e.g., Twitter
Semi-structured data
e.g. JSON, XML, BSON
Unstructured content
How much is TEXT worth to you
Sensor data
Temperature, light, vibration, location, liquid flow, pressure, RFIDs
Vertical industries structured transaction data
E.g. Telecom call data records, retail

SLIDE 25

25

The Changing Landscape – We Now Have Different Platforms Optimised For Different Analytical Workloads

Streaming data Hadoop data store Data Warehouse RDBMS NoSQL DBMS EDW

DW & marts NoSQL Graph

DB

Advanced Analytic (multi-structured data)

mart

DW Appliance

Advanced Analytics (structured data)

Analytical RDBMS Big Data workloads result in multiple platforms now being needed for analytical processing

C R U D

Prod Asset Cust

MDM

Traditional query, reporting & analysis

Real-time stream processing & decision m’gmt

Data mining, model development Investigative analysis, Data refinery Data mining, model development Graph analysis Graph analysis

SLIDE 26

26

Business Users Need To Combine Data In These Systems To Get Deeper Insights

MDM System C R U D

Prod Asset Cust

Who are our customers? What products do we sell? What is the online behaviour of loyal, low risk, low fee customers so we can offer them higher fee products? DW Who are our most loyal, low risk customers that generate low fees?

How do I get at data in multiple analytical data stores to answer this?

What are the most popular navigational paths through our web site that lead to high fee products

SLIDE 27

27

Topics– Where Are We?

Improving profitability
The increasingly complex data landscape
The impact on the business of distributed data
What is data virtualisation?
Improving business performance and agility using data

virtualisation

Reducing time to value and increasing revenue in the logical

data warehouse

Conclusions

SLIDE 28

28

How Does Data Virtualization Work? Example – Integrated Claims Information

Data sources

Oracle Customer Data DB2 Claims SQL Server Payment JSON Incidents

Query

Claim status is stored in DB2
Claims payment info is stored in

SQL Server

Claims form submitted was

captured and stored in JSON

Data Virtualization Server

Source: IBM

SLIDE 29

29

How Does Data Virtualization Work?

Virtual table

1. Define common

integrated data model for the virtual tables

3. Define mappings

from source systems to common virtual tables mapping mapping mapping mapping mapping SQL, X/Query, Web services

4. Query the virtual table(s)

Application, BI tool or portal Data Virtualization Server Results can come back as a SQL result set, XML, HTML, CSV etc. web content RDBMS XML File

2. Define the data

sources Spread sheets

WWW

Virtual table

SLIDE 30

30

Data Virtualization Servers Can Support Multiple Virtual Views AND Nested Virtual Views

Virtual table

Multiple virtual tables can be created if needed

web content RDBMS XML File SQL, X/Query, Web services Application

r BI tool

Virtual table

Each federated query dynamically creates the virtual table at run time

Portal Data Virtualization Server Virtual table

WWW

Spread sheets mapping mapping mapping mapping mapping

SLIDE 31

31

Topics– Where Are We?

Improving profitability
The increasingly complex data landscape
The impact on the business of distributed data
What is data virtualisation?
Improving business performance and agility using data

virtualisation

Reducing time to value and increasing revenue in the logical

data warehouse

Conclusions

SLIDE 32

32

Data Virtualization Make It Easy To Access And Report on Data Across The Process To Manage Business Operations

rder

credit check fulfill ship invoice payment package Order-to-Cash Process Orders Data virtualization and Virtual Data Services Benefits Simplified access Access to real-time data across the process Agile and responsive Avoid unplanned operational costs See across multiple instances of apps See across on-premises & cloud apps cost Agility

SLIDE 33

33

XYZ Corp.

Data Virtualisation - See Views Of Customer Orders, Shipments And Payments Across Line Of Business Product Lines

Customers/ Prospects

Product/service line 1

rder

credit check fulfill ship invoice payment package

Product/service line 2 Product/ service line 3

Channels/ Outlets

rder

credit check fulfill ship invoice payment package

rder

credit check fulfill ship invoice payment package

Order

(product line 1)

Order

(product line 2)

Order

(product line 3)

Enterprise Data virtualization Data virtualization Data virtualization

SLIDE 34

34

Data Virtualization Helps You Quickly See Across Your Value Chain Making Planning Much Easier

Fore- casting Product, Materials Supplier Master data Manufacturing volumes & inventory DW Planning

ERP ERP

Finance DW Shipping system CRM system Sales & mktng DW SCADA systems

Data virtualization SCM

Manufacturing execution system

CAD Benefits Management Reports easy to produce See real-time data across value chain Easier to do planning Dynamic planning on real-time data cost Agility

SLIDE 35

35

Data Virtualization Allow Simplification Of Data Warehouse Achitecture And Reduced Total Cost Of Ownership

DW

ETL

Operational data Agile BI = Agile Architecture Data virtualization can easily accommodate change and speed up time to value Data visualisation BI tools

Virtual ODS virtual mart personalised virtual views virtual mart personalised virtual views

Data Virtualization

data mart cube ODS

ETL ETL ETL

personal data store cost Agility

Adapted from a slide originally by R/20 Consultancy

SLIDE 36

36

Data Virtualisation Simplifies Architecture, Reduces Total Cost Of Ownership, Improves Agility, Speeds Development

DW

UK DE FR NL ESP CH IT BE US

Country Specific Data Marts

ETL ETL ETL ETL ETL ETL ETL ETL ETL

Cost Agility

SLIDE 37

37

Agile Business Led BI/DW Development – Early Prototyping Using Data Virtualisation

Virtual data model Easy to change Quickly produce insight 100% agile development

Bi Tool Build prototype Data virtualisation OLTP OLTP Business Led Prototype Development

virtual data model

DW

SLIDE 38

38

Agile BI Development Process Often Starts In The Business And Is Handed Over To IT – DV Helps With Smooth Handover

Bi Tool DW

Bi Tool

Build prototype Data virtualisation OLTP Data virtualisation OLTP OLTP OLTP ETL Deploy in production Business Led Prototype Development IT deploy

virtual data model virtual data model physical data model Fast deployment Re-use BI tool reports and dashboards Reuse Data Virtualization metadata in ETL Re-use data

DW

Virtual data model Easy to change Quickly produce insight 100% agile development

SLIDE 39

39

Data Virtualisation

Common Data Definitions in A Data Virtualization Server Removes Inconsistencies Across Multiple BI Tools

Common data names and definitions (shared business vocabulary)

mapping mapping mapping mapping web content RDBMS XML File

Disparate data names and definitions (different data vocabularies)

, BI tool (vendor X)

WWW

MsgQ BI tool (vendor Y) Spread sheets mapping

Simplified data access Quick to implement Non-disruptive Agile

SLIDE 40

40

LDW, Governance And Self-Service DI – Data Virtualisation Reduces Copying, Enforces Security And Increases Agility

C R U prod client asset D

MDM Systems

C R U

risk rates pricing Country codes

D

RDM Systems

Business User Self-Service DI Important to make sure business users re-use rather than re- invent AND have simplified access to data

RDBMS Cloud XML, JSON web services NoSQL Files

Data Virtualisation IT Data Architect

SLIDE 41

41

Topics– Where Are We?

Improving profitability
The increasingly complex data landscape
The impact on the business of distributed data
What is data virtualisation?
Improving business performance and agility using data

virtualisation

Reducing time to value and increasing revenue in the logical

data warehouse

Conclusions

SLIDE 42

42

Using Data Virtualisation To Combine Data In These Systems To Get Deeper Insights

MDM System C R U D

Prod Asset Cust

Who are our customers? What products do we sell? DW Who are our most loyal, low risk customers that generate low fees? What are the most popular navigational paths through our web site that lead to high fee products What is the online behaviour of loyal, low risk, low fee customers so we can offer them higher fee products?

How do I get at data in multiple analytical data stores to answer this?

Data Virtualisation

SLIDE 43

43

New Insights In Hadoop Can Integrated With A DW Using Data Virtualization To Provide Enriched Information To Drive Revenue

DW D I e.g. Deriving insight from social web sites like for sentiment analytics new insights OLTP systems

sandbox

Data Scientists

social Web logs

web cloud Data Vitualisation (Logical DW)

SQL on Hadoop

Virtual marts Virtual views of DW & Big Data

SLIDE 44

44

Using Hadoop As A Data Archive Means Data Can Be Kept On- line, Analysed And Still Integrated With Data In The DW

DW D I OLTP systems Archived data Archive unused

r data > n years

new insights Data Vitualisation (Logical DW)

SQL on Hadoop

Virtual marts Virtual views of DW & Big Data

SLIDE 45

45

Logical DW - Real-time Data From NoSQL DBMSs Can Also Be Joined To DW And Big Data Using Data Virtualization

DW D I Nested data like JSON needs to be handled by the data virtualisation server real-time insights OLTP systems

social Web logs

Column Family DB Document DB

NoSQL DB

sensors Flatten nested data SQL on Hadoop

Data Vitualisation (Logical DW) Virtual marts Virtual views of DW & Big Data

SLIDE 46

46

Data Virtualisation Virtual Views Can Connect To Different SQL

n Hadoop Engines To Support Multiple Query Workloads

Source: Hortonworks

SparkSQL Drill Storage is independent

f any SQL engine

Self-Service BI tool Jethro Analytic Application Data Virtualisation (Logical Dat rehouse) DV connectivity to search is also possible Logical DW

SLIDE 47

47

Data Virtualisation – The Logical Data Warehouse

Reducing Time To Value Using Data Virtualization to Create The Logical Data Warehouse

EDW

DW & marts NoSQL DB

e.g. graph DB mart

DW Appliance

Advanced Analytics (structured data)

Self-Service BI tool

Advanced Analytics

Streaming data

RT Analytics

C R U prod cust asset

master data

SLIDE 48

48

Conclusions

Companies are demanding more agility while the data

landscape becomes increasingly more distributed

In addition, the want to reduce costs while also improving

customer engagement and growth

Data virtualisation
Allows organisations to see across hybrid processes so they can

still see the entire business operation

Helps avoid unplanned operational cost
Reduces complexity, improves agility and reduces total cost of
wnership in data warehousing
Removes inconsistency across multiple BI tools for better decisions
Reduces time to value in better customer engagement and growth
Enables the logical data warehouse
Reducing cost and improving growth improves profitability

SLIDE 49

49