DRIVING INTELLIGENCE TO THE EDGE FEATURING AN END-TO-END OPEN - - PowerPoint PPT Presentation

driving intelligence to the edge featuring an end to end
SMART_READER_LITE
LIVE PREVIEW

DRIVING INTELLIGENCE TO THE EDGE FEATURING AN END-TO-END OPEN - - PowerPoint PPT Presentation

DRIVING INTELLIGENCE TO THE EDGE FEATURING AN END-TO-END OPEN ARCHITECTURE FOR IoT Strata Data Conference, September 12, 2018 Dave Shuman, Cloudera, Industry Lead for Manufacturing and IoT Bryan Dean, Red Hat, Director Business Development


slide-1
SLIDE 1

1

DRIVING INTELLIGENCE TO THE EDGE – FEATURING AN END-TO-END OPEN ARCHITECTURE FOR IoT

Strata Data Conference, September 12, 2018 Dave Shuman, Cloudera, Industry Lead for Manufacturing and IoT Bryan Dean, Red Hat, Director Business Development for IoT Solutions

slide-2
SLIDE 2

2

Securely connect, authenticate and manage disparate connected devices that speak different protocols Apply analytics at the edge with machine learning and business rules to enable local, low-latency decision making Centralize IoT data processing, analytics and machine learning to enable deep business insights and actionable intelligence Create and deliver cloud-native

  • applications. Integrate business

applications and processes. Tools to enable end-to-end data security, compliance, authorization and authentication

KEY FUNCTIONALITY FOR AN END-TO-END IoT ARCHITECTURE

Device Management & Connectivity Intelligent Edge Processing & Analytics Advanced Analytics & Machine Learning Business & Application Integration End-to-End Security & Compliance

slide-3
SLIDE 3

3

WHY OPEN SOURCE FOR IoT

“We believe the best way to support this complex environment is to base

  • ur commercial IoT platform, the

Bosch IoT Suite, on open source components and open standards. These projects establish a horizontal

  • pen technology for IoT and provide

the technical breeding grounds for successful business ecosystems.”

  • Dr. Stefan Ferber, VP of Engineering,

Bosch Software Innovations

2.4 30* 250+ 130K

million lines of code projects developers monthly visitors

slide-4
SLIDE 4

4

ADDRESSING ENTERPRISE IoT NEEDS

Data Management & Analytics

  • Enterprise Data Mgmt.
  • Persistent Data Storage
  • Big Data Processing & Analytics
  • Real-Time Analytics
  • Machine Learning
  • Data Security & Compliance

Operational Technology (OT)

  • Device Management
  • Industrial protocols
  • OT Middleware
  • Intelligent gateways
  • MQTT co-inventors
  • OT security

Information Technology (IT)

  • Messaging & Integration
  • Business Rules & CEP
  • Open Hybrid Platform-as-a-Service
  • Enterprise Linux Platform
  • IT security

Enterprise IoT open source community

Operational Technology (OT) Information Technology (IT)

slide-5
SLIDE 5

5

public, private, hybrid cloud

DATA MANAGEMENT & ANALYTICS PLATFORM IoT EDGE CONNECTED “THINGS” IoT INTEGRATION HUB

OPEN END-TO-END IoT ARCHITECTURE

APPLICATION DEVELOPMENT, DELIVERY, & INTEGRATION

Integrating IoT operating technology, data management, analytics, and applications

Cloud-native apps Traditional apps

  • Modular, secure, end-to-end architecture
  • Streaming analytics and machine learning
  • Open, interoperable on hybrid cloud
  • Modern application development and agile integration
slide-6
SLIDE 6

6

CONNECTED “THINGS”

Sensors, Actuators, Data Sources Edge Processing & Analytics

DATA MANAGEMENT & ANALYTICS PLATFORM IoT INTEGRATION HUB APPLICATION DEVELOPMENT, DELIVERY, & INTEGRATION

Data Integration, Routing, Device Command/Control Application Development, Deployment, Integration Advanced Analytics & Machine Learning

EDGE PROCESSING AND ANALYTICS

IoT EDGE

  • Device connectivity
  • Data transformation
  • Intelligent routing
  • Edge applications
  • Edge analytics & real-time

decisions

slide-7
SLIDE 7

7

Edge Processing & Analytics Data Integration, Routing, Device Command/Control

Telemetry Data

Application Development, Deployment, Integration Advanced Analytics & Machine Learning

DATA INTEGRATION, ROUTING, DEVICE COMMAND/CONTROL

IoT INTEGRATION HUB CONNECTED “THINGS”

Sensors, Actuators, Data Sources

DATA MANAGEMENT & ANALYTICS PLATFORM

  • Device management,

security, access control

  • Data aggregation
  • Event processing
  • Agile integration
  • Container-based

application platform

IoT EDGE APPLICATION DEVELOPMENT, DELIVERY, & INTEGRATION

slide-8
SLIDE 8

8

Edge Processing & Analytics Data Integration, Routing, Device Command/Control Advanced Analytics & Machine Learning Application Development, Deployment, Integration

  • Data ingest
  • Stream / batch processing
  • Secure data storage
  • Machine learning and

real-time analytics

IoT EDGE

ADVANCED ANALYTICS AND MACHINE LEARNING

Machine Learning Model

CONNECTED “THINGS”

Sensors, Actuators, Data Sources

DATA MANAGEMENT & ANALYTICS PLATFORM IoT INTEGRATION HUB APPLICATION DEVELOPMENT, DELIVERY, & INTEGRATION

Telemetry Data

slide-9
SLIDE 9

9

Application Data Advanced Analytics & Machine Learning

APPLICATION DEVELOPMENT, DEPLOYMENT, INTEGRATION

Application Development, Deployment, Integration

Cloud-native apps Traditional apps

APPLICATION DEVELOPMENT, DELIVERY, & INTEGRATION

Edge Processing & Analytics

IoT EDGE CONNECTED “THINGS”

Sensors, Actuators, Data Sources

IoT INTEGRATION HUB

Data Integration, Routing, Device Command/Control

DATA MANAGEMENT & ANALYTICS PLATFORM

  • Container-based application platform

with Kubernetes orchestration

  • DevOps: automated build-test-deploy
  • Agile integration
  • Multi-cloud portability
slide-10
SLIDE 10

10

IoT EDGE CONNECTED “THINGS” IoT INTEGRATION HUB DATA MANAGEMENT & ANALYTICS PLATFORM

Cloud-native apps Traditional apps

OPEN END-TO-END IoT ARCHITECTURE

  • Device connectivity
  • Data transformation
  • Intelligent routing
  • Edge applications
  • Edge analytics & real-time

decisions

  • Data ingest
  • Stream / batch processing
  • Secure data storage
  • Machine learning and

real-time analytics

  • Container-based application platform

with Kubernetes orchestration

  • DevOps: automated build-test-deploy
  • Agile integration
  • Multi-cloud portability

APPLICATION DEVELOPMENT, DELIVERY, & INTEGRATION

  • Device management,

security, access control

  • Data aggregation
  • Event processing
  • Agile integration
  • Container-based platform
slide-11
SLIDE 11

11

IoT EDGE CONNECTED “THINGS” IoT INTEGRATION HUB DATA MANAGEMENT & ANALYTICS PLATFORM

Cloud-native apps Traditional apps

OPEN END-TO-END IoT ARCHITECTURE: PRODUCTS

APPLICATION DEVELOPMENT, DELIVERY, & INTEGRATION

ENTERPRISE DATA HUB

slide-12
SLIDE 12

12

IoT EDGE IoT INTEGRATION HUB CONNECTED “THINGS” P r

  • t
  • c
  • l

T r a n s l a t i

  • n

I n t e l l i g e n t F i l t e r i n g A g g r e g a t i

  • n

R

  • u

t i n g DATA MANAGEMENT & ANALYTICS PLATFORM

Real-Time Analytics Data Ingest Real-Time Processing Data Storage Machine Learning Data Security

Telemetry Data

Application Integration Management

Data flow to derive deep business insights and actionable intelligence

END-TO-END ANALYTICS

Telemetry Data

D e e p d a t a a n a l y s i s & i n s i g h t s

Application Data

Cloud-native apps Traditional apps

APPLICATION DEVELOPMENT, DELIVERY, & INTEGRATION

slide-13
SLIDE 13

13

P r

  • t
  • c
  • l

T r a n s l a t i

  • n

I n t e l l i g e n t F i l t e r i n g A g g r e g a t i

  • n

R

  • u

t i n g D e e p d a t a a n a l y s i s & i n s i g h t s

Real-Time Analytics Data Ingest Real-Time Processing Data Storage Machine Learning Data Security

Telemetry Data

Application Integration Application Data

Data flow to derive deep business insights and actionable intelligence

END-TO-END ANALYTICS

Machine Learning

M L M

  • d

e l Actions P r e d i c t i

  • n

/ A l e r t M L M

  • d

e l IoT INTEGRATION HUB DATA MANAGEMENT & ANALYTICS PLATFORM

Cloud-native apps Traditional apps

CONNECTED “THINGS” IoT EDGE APPLICATION DEVELOPMENT, DELIVERY, & INTEGRATION

slide-14
SLIDE 14

14

Red Hat Partner Confidential

PUTTING IT INTO PRODUCTION

Data Mgt. and Analytics

/home/cdsw/trained-model.h5

IoT Edge IoT Hub Connected “Things”

Predict Locally Capture Images Download Model Report Result

A trained model becomes a “downloadable brain”

slide-15
SLIDE 15

15

REAL WORLD DEPLOYMENTS

slide-16
SLIDE 16

16

END-TO-END IOT ARCHITECTURE - PoC

Application Data Telemetry Data MQTT over WiFi Telemetry Data Command & Control Telemetry Data Application Integration Client XDK

Data Mgt. and Analytics

IoT Edge

Network: Red Zone Network: Green Zone

Telemetry Data

IoT Integration Hub Applications Digital Twin

slide-17
SLIDE 17

17

IoT Integration Hub

Application Data Telemetry Data MQTT over WiFi OT Middleware

OT Middleware Smart Services Machine Learning Business Logic Device Management Device Connectivity Administration Platform-as-a-Service Ingest Machine Learning Store Analyze Process

Telemetry Data Command & Control Telemetry Data Application Integration Client XDK

Digital Twin

Data Mgt. and Analytics

BI

IoT Edge Network: Red Zone Network: Green Zone

Apps CDSW PPM

Telemetry Data

END-TO-END IOT ARCHITECTURE - PoC

slide-18
SLIDE 18

18 OPC- UA

Kafka Spark Kudu

Integrate Store

Spark Impala

Analyze Unified Services Layer

YARN Sentry

Data Science Workbench

TCP/IP Mqtt / OPC- UA IoT Calibrators Edge IoT Site Hub TCP/IP mqtt

  • Device connectivity
  • Data transformation
  • Intelligent routing
  • Business logic
  • Edge analytics & real-time

decisions

  • Device management,

security, and access control

  • Data aggregation
  • Event processing
  • Integration services (API’s)

TCP/IP Kafka

Model Deployment

DATA PIPELINE & ARCHITECTURE

  • Data ingest
  • Stream / batch processing
  • Persistent data storage
  • Machine learning and

real-time analytics

slide-19
SLIDE 19

19

Manifold Pressure

slide-20
SLIDE 20

20

Stacks for Machine Learning at the Core and Edge

CORE EDGE Machine Learning Library Machine Learning Platform Operating System Cloud / On-Prem Infrastructure Model Training Application Machine Learning Library Edge Software Library / Framework Operating System Physical Device Model Serving Application

slide-21
SLIDE 21

21

So what about a Classifier?

CORE EDGE DL4J Cloudera Data Science Workbench RHEL AWS EC2 & S3 TrainClassifier.java DL4J Everyware Software Framework (Java OSGi) Yocto Linux Eurotech ReliaGATE 20-25 ServeClassifier.java Machine Learning Library Machine Learning Platform Operating System Infrastructure Model Training Application

slide-22
SLIDE 22

22

Or…

CORE EDGE Keras (w/ TensorFlow backend) Cloudera Data Science Workbench RHEL Azure & ADLS train_classifier.py Tensorflow Lite (Java API) Everyware Software Framework (Java OSGi) Yocto Linux HPE Moonshot ServeClassifier.java Machine Learning Library Machine Learning Platform Operating System Infrastructure Model Training Application

slide-23
SLIDE 23

23

TECHNICAL OVERVIEW: FUNCTIONALITY

IoT EDGE CONNECTED “THINGS”

Telemetry Management

IoT INTEGRATION HUB

Machine learning model

APPLICATION DEVELOPMENT, DELIVERY, & INTEGRATION

CONTAINER-BASED APPLICATION PLATFORM

Traditional applications Cloud-native applications ERP CRM DBs

DATA MANAGEMENT & ANALYTICS

Data engineering Data science Data warehouse Operational database Security & governance Machine learning Storage Services

ENTERPRISE DATA HUB

Telemetry Management App integration

CONTAINER-BASED APPLICATION PLATFORM IOT EDGE FRAMEWORK

DevOps Integration & API mgmt Orchestration Developer services

Edge analytics Machine learning

Field connectivity Cloud connectivity Edge applications Telemetry

IOT INTEGRATION FRAMEWORK

Device management Device connectivity Data integration

slide-24
SLIDE 24

24

Red Hat Partner Confidential

INDUSTRY 4.0 DEMO

slide-25
SLIDE 25

25

TO LEARN MORE… Visit the Cloudera Booth

Industry 4.0 Demo

Featuring the End-to-end architecture and solution Join us for a Webinar on Oct 3rd to hear about the end-to-end Solution

slide-26
SLIDE 26

26

THANK YOU…

slide-27
SLIDE 27

27

BACKUP SLIDES

slide-28
SLIDE 28

28

Red Hat Partner Confidential

PREDOMINANT END-TO-END IoT ALTERNATIVES

Alternative Strength Weakness

Full-Stack Public Cloud Providers

  • Quick to get started
  • Low initial investment
  • Offload in-house expertise
  • Single vendor source/support
  • Loss of data control
  • Proprietary lock-in, limited portability
  • Rigid architecture
  • Limitations at edge for many use cases
  • Cost model at scale

Proprietary IoT Platforms

  • Single vendor packaging
  • Medium-quick to get started
  • Optional vertical use-case packages
  • Proprietary lock-in, poor portability
  • Black-box architecture
  • Rigid architecture
  • Requires vendor-specific in-house knowledge
  • Limited interoperability

Do It Yourself

  • Perceived low initial software cost
  • Customized to use case
  • No vendor middleman
  • Vast in-house expertise required
  • Slow time to value; high development and ongoing

costs

  • Support and indemnification
  • Complexity, maintainability
slide-29
SLIDE 29

29

TECHNICAL OVERVIEW: FUNCTIONALITY

IoT EDGE CONNECTED “THINGS”

Telemetry Management

IoT INTEGRATION HUB

Machine learning model

APPLICATION DEVELOPMENT, DELIVERY, & INTEGRATION

CONTAINER-BASED APPLICATION PLATFORM

Traditional applications Cloud-native applications ERP CRM DBs

DATA MANAGEMENT & ANALYTICS

Data engineering Data science Data warehouse Operational database Security & governance Machine learning Storage Services

ENTERPRISE DATA HUB

Telemetry Management App integration

CONTAINER-BASED APPLICATION PLATFORM IOT EDGE FRAMEWORK

DevOps Integration & API mgmt Orchestration Developer services

Edge analytics Machine learning

Field connectivity Cloud connectivity Edge applications Telemetry

IOT INTEGRATION FRAMEWORK

Device management Device connectivity Data integration

slide-30
SLIDE 30

30

TECHNICAL OVERVIEW: PRODUCTS

IoT EDGE CONNECTED “THINGS”

Telemetry Management

IoT INTEGRATION HUB

Machine learning model

APPLICATION DEVELOPMENT, DELIVERY, & INTEGRATION

Telemetry Management App integration Telemetry

Edge analytics Machine learning

DATA MANAGEMENT & ANALYTICS

ENTERPRISE DATA HUB

slide-31
SLIDE 31

31

TECHNICAL OVERVIEW: PROJECTS

IoT EDGE CONNECTED “THINGS”

Telemetry Management

IoT INTEGRATION HUB

Machine learning model

APPLICATION DEVELOPMENT, DELIVERY, & INTEGRATION DATA MANAGEMENT & ANALYTICS

Telemetry Management App integration

Edge analytics Machine learning

Telemetry

CLOUDERA’S DISTRIBUTION INCLUDING HADOOP (CDH)

slide-32
SLIDE 32

32

IoT Gateways IoT Integration Hub

REMOTE MAINTENANCE & SUPPORT ON-SITE DATA ANALYTICS MACHINE LEARNING & ADVANCED ANALYTICS

Data Mgmt, Analytics & ML Manufacturing Equipment

  • Data ingest
  • Stream / batch

processing

  • Secure data storage
  • Machine learning and

real-time analytics

  • Device connectivity
  • Data transformation
  • Intelligent routing
  • Business logic
  • Edge analytics &

real-time decisions

  • Device management,

security, and access control

  • Data aggregation
  • Event processing
  • Integration services

INDUSTRY 4.0 DEMO ARCHITECTURE

slide-33
SLIDE 33

33

VALUE PROPOSITION

Open and interoperable

Future-proof open source architecture |

  • pen standards | deployment flexibility

Modular

Avoid lock-in | capitalize on existing investments

End-to-end analytics

Analytics at the edge | advanced analytics and machine learning | ML model execution at the edge

Reduce risk and complexity

Simplify development, deployment, and integration tasks | save costs

End-to-end security

Security across devices, access, authentication, and applications as well as data in motion and at rest

Control your data

Privacy | security | regulatory

slide-34
SLIDE 34

34

Device connectivity

Open standards – MQTT, AMQP, OPC-UA, CoAP, HTTP(s)

Flexible deployment Data management & analytics

Based on Apache open source ecosystem libraries for machine learning and advanced analytics

Open application interfaces

Enterprise visibility | real-time anomaly detection | future-proof

Community innovation

Collaboration driven by some of the leading enterprises in the IoT space Any of the leading cloud providers

  • r your data center or hybrid cloud

No vendor lock-in

No rigid architectures or proprietary formats and components

OPEN SOURCE, OPEN STANDARDS, FLEXIBLE DEPLOYMENT