Efficiencies Analytics Solutions Architect using IoT and Machine - - PowerPoint PPT Presentation

▶
efficiencies
SMART_READER_LITE
LIVE PREVIEW

Efficiencies Analytics Solutions Architect using IoT and Machine - - PowerPoint PPT Presentation

Nov 09, 2022 235 likes •509 views

Strata Data Conference 2018 #StrataData How Komatsu is Improving Mining September 2018 Shawn Terry Efficiencies Analytics Solutions Architect using IoT and Machine @globalcoder1 Learning www.mining.komatsu @KomatsuMining 1 Strata Data

slide-1
SLIDE 1

Strata Data Conference 2018 #StrataData 1

How Komatsu is Improving Mining Efficiencies using IoT and Machine Learning

September 2018 Shawn Terry Analytics Solutions Architect @globalcoder1 www.mining.komatsu @KomatsuMining

slide-2
SLIDE 2

Strata Data Conference 2018 #StrataData 2

AGENDA

  • An Introduction to Komatsu
  • What Motivates Us
  • The Komatsu Mining Data Analytics Journey
  • Architecture
  • Lessons Learned
  • Questions?
slide-3
SLIDE 3

Strata Data Conference 2018 #StrataData 3

Video 视频

Customer Value

slide-4
SLIDE 4

Strata Data Conference 2018 #StrataData 4

COMPANY OVERVIEW

Smart Solution Centers

  • Around the clock monitoring
  • Custom Data Solutions
  • Productivity analysis
  • Training
  • And more…
slide-5
SLIDE 5

Strata Data Conference 2018 #StrataData 5

BIG DATA for BIG MACHINES

4800XPC – Electric Rope Shovel

  • 3.65 million pounds
  • Nearly 75 feet high
  • Move 270,000lbs of every

33s

  • More than 1250 sensors

reporting at frequencies up to 10Hz

  • 50,000 unique

measurement points every minute

slide-6
SLIDE 6

Strata Data Conference 2018 #StrataData 6

CUSTOMER FOCUS

Optimizing mineral extraction for the least impact

  • Safety
  • Autonomy, increased reliability
  • Environment
  • Less waste, improved efficiency
  • Minimal footprint
  • Productivity
  • Continuous improvement
  • Bringing lean practices to mining
  • Maintenance
  • Less wear and tear on the machines
  • Reducing unplanned downtime
  • Reducing maintenance time
slide-7
SLIDE 7

Strata Data Conference 2018 #StrataData 7

MINING MARKET CHALLENGES Mines are driven by cost per ton

Today’s challenges:

  • Increasing social and regulatory issues
  • Difficult mining conditions – deeper ore

deposits

  • Declining commodity prices
  • Inventory management
  • Aging workforce
slide-8
SLIDE 8

Strata Data Conference 2018 #StrataData 8

MOVING TO A HOLISTIC SOLUTION

From machine focus to Systems Focus to help our customers solve their toughest challenges

  • Smart equipment with built-in sensors
  • Automated Monitoring of machine health
  • Proactive Recommendations for Maintenance and Operations
  • Understanding dependencies between machines and optimizing the entire process
  • Delivering the right information to the right people at the right time to make decisions that lower $/ton.

Monitoring

Continuous Improvement(CI) Repair & Maintenance (R&M )

Productivity

(ton/h)

Production Cost ($/ton) TCO

($/h)

slide-9
SLIDE 9

Strata Data Conference 2018 #StrataData 9

MANAGING THE DATA

This machine: 1,250 sensor points 550 alarms & events

All machines: 17 billion time series data points with 3xPeak 6 million Alarms & events 150k analytics executed

Each day

slide-10
SLIDE 10

Strata Data Conference 2018 #StrataData 10

ANALYTICS PLATFORM OVERVIEW

slide-11
SLIDE 11

Strata Data Conference 2018 #StrataData 11

OUR ANALYTICS JOURNEY

Future Opportunities

  • Edge Analytics
  • Optimize Data Handling
  • Improved Dev/Ops
  • Leverage Cloud Services

(separate compute/storage)

  • Scale for Broader Growth

Gen 1 Gen 2 Gen 3 Gen 4

  • Limited analytical

capabilities

  • Data processing and

storage costs

  • Limited machine

learning capabilities

  • Data silos

Increasing value and business impacts Historian Based Out-of-the-Box Proprietary Solution Cloud based Open Architecture & Platform Edge Analytics and Improved Orchestration

  • Limited flexibility/

interoperability

  • Scaling to data growth

was a challenge

  • Data processing and

storage costs

  • Challenges around

custom use case

2016-Now Future State

  • Cloudera on

Microsoft Azure

  • Processing, analytics

and machine learning for IoT data

  • Processing 200,000

data points/second

  • Rapid prototyping

2014-2015 2008-2014

slide-12
SLIDE 12

Strata Data Conference 2018 #StrataData 12

INGEST STORE CALCULATE VISUALIZE REPORT

CORE FUNCTIONS OF PLATFORM

slide-13
SLIDE 13

Strata Data Conference 2018 #StrataData 13

ORIGINAL PLATFORM (2008)

FTP/SFTP CUSTOM MSSQL IP.21 MATLAB CUSTOM WEB SSRS

  • First connected machine
  • Near Real-Time Trending and Alerts in a

browser

  • New visibility into equipment health and
  • perations
  • New insights and actions never before

possible

  • Everything fresh and cool
slide-14
SLIDE 14

Strata Data Conference 2018 #StrataData 14

REACHED THE LIMIT -- (~2013)

FTP/SFTP CUSTOM MSSQL IP.21 MATLAB CUSTOM WEB SSRS MSSQL IP.21 MSSQL IP.21 MSSQL IP.21 FTP/SFTP CUSTOM MSSQL IP.21 MATLAB CUSTOM WEB SSRS MSSQL IP.21 MSSQL IP.21 MSSQL IP.21

Private Cloud 1 Surface Mining Private Cloud 2 Underground Mining

  • Max sustained ingest ~60,000 points per

second per environment

  • ~150 pieces of connected equipment
  • Complex and costly to scale out storage
  • Operationally a nightmare to support
  • Performance Sucked
  • Unable to bring on new workloads / limiting

the business

slide-15
SLIDE 15

Strata Data Conference 2018 #StrataData 15

THE TEAM OBJECTIVES

  • Get out of the old platform / cloud
  • Ensure new platform can grow with business
  • No new budget for first year
  • Fixed deadline (Feb1 - 2017)
  • Same core team (platform team: 5 + 3 offshore)
  • Lift and shift what we can
  • Address the new feature wish list
slide-16
SLIDE 16

Strata Data Conference 2018 #StrataData 16

DATA DEMOCRATIZATION

Goal

  • Let regional teams interact with the data
  • Let regional teams develop analytics that

can run quickly, reliably, and responsibly

  • Let regional teams create reports and

dashboards specific for their customers

  • Share knowledge and best practices

globally Challenges

  • Multi-tenant security
  • Analytics development and testing

processes and change management (BYOA)

  • Isolating analytics development from

backend changes

  • Maintaining consistent SLAs
slide-17
SLIDE 17

Strata Data Conference 2018 #StrataData 17

A NEW BEGINNING -- (2016)

  • One unified cloud in Azure with Cloudera at the core.
  • Microservices protect from over coupling
  • Easily scalable storage but not all bottlenecks

removed

  • From “Go” to Production in 6 months
  • Operations – Still challenging, but sustainable
  • Happy Users = Rapid Growth and Adoption
  • Still need to rethink our Analytics patterns

SFTP/CUSTOM STREAMSETS KAFKA MSSQL* OPENTSDB/HBASE KUDU HDFS MATLAB PYTHON R CUSTOM WEB GRAFANA SSRS POWERBI ARCADIA MICROSERVICES (REST)

slide-18
SLIDE 18

Strata Data Conference 2018 #StrataData 18

ANALYTICS PLATFORM STACK

DATA STORAGE AND MANAGEMENT ANALYTICS DATA INGEST / PIPELINES VISUALIZATION & REPORTING

{REST:API} {REST:API} {REST:API}

slide-19
SLIDE 19

Strata Data Conference 2018 #StrataData 19

Enabling Next Gen Analytics -- (2018+)

  • Further Decoupling and

Removing Bottlenecks

  • Stream Based Analytics
  • Containerization
  • Shifting Analytics to Edge
  • Simplifying Architecture
  • Improving DEV/OPS
  • Improving DR and HA

VARIOUS MSSQL* OPENTSDB/HBASE KUDU HDFS PYTHON R MATLAB CUSTOM WEB GRAFANA POWERBI ARCADIA KAFKA MICROSERVICES (REST) INGEST CONSUMERS STREAMSETS

slide-20
SLIDE 20

Strata Data Conference 2018 #StrataData 20

CHALLENGES AT THE EDGE

  • Getting data out of the mine site (customer friendly/extreme environments)
  • Data Quality / Reliability
  • Monitoring edge communications
  • Is it down or off / planned or unplanned?
  • Managing time and time zones
  • Every machine is unique and lots more to enable
  • Explosive data volumes (Catch Up / Bad Sensors)
  • Accurate and timely responses – Right Data to Right Person at Right Time
  • Maintaining the Edge Footprint
slide-21
SLIDE 21

Strata Data Conference 2018 #StrataData 21

Lessons Learned Do

  • Start with a clear problem to solve
  • Limit scope – get quick wins
  • Expect to experiment, fail, and iterate quickly
  • Engage experts
  • Decouple as much as possible
  • Expect every piece to be replaceable
  • Ensure your sponsors understand iterative/agile

development processes

  • Learn to automate dev/ops
  • Instrument and Measure Performance
slide-22
SLIDE 22

Strata Data Conference 2018 #StrataData 22

Lessons Learned Don’t

  • Expect cloud to solve all of your problems
  • Though it does creates amazing new possibilities
  • Expect Cloud infrastructure to be “Always On”
  • Things will fail in unexpected ways – expect and plan for it.
  • Expect technology selection to be a one and done

decision

  • There will be something better tomorrow.
  • Be afraid or unable to change course if something

doesn’t work

  • Be afraid of open source
  • Underestimate the impacts of parallelization
  • Horizontal scaling creates different bottlenecks – you are only

as strong as your weakest link.

slide-23
SLIDE 23

Strata Data Conference 2018 #StrataData 23

Even Whole Data Centers Can Fail…

slide-24
SLIDE 24

COMPLEXITY

slide-25
SLIDE 25

Strata Data Conference 2018 #StrataData 25

DATA VALUE

slide-26
SLIDE 26

Strata Data Conference 2018 #StrataData 26

Summary

Komatsu is leveraging the power of IOT, big data and the cloud to help our mining customers get the lowest $/ton in safe, sustainable ways. Thanks to our incredible team, our partners in Cloudera, Microsoft, and others, we are transforming the way we do business, creating new opportunities to serve our customers and now have the road open for future growth. Through democratization of data analytics and BI we are enabling our Smart Services teams to partner with

  • ur customers to solve their toughest challenges.
slide-27
SLIDE 27

Strata Data Conference 2018 #StrataData 27

QUESTIONS?