Industrial Internet of Things Chenyang Lu Cyber-Physical Systems - - PowerPoint PPT Presentation

Dependable

Industrial Internet of Things

Chenyang Lu

Cyber-Physical Systems Laboratory Department of Computer Science and Engineering

IoT for Industry 4.0

2

• 11.6+ billion hours
• perating

experience

• 36,800+ wireless

field networks

[Emerson]

• $944.92 million by 2020

[Market and Market]

Courtesy: Emerson Process Management

NOT your best-effort IoT at home!

WirelessHART

Industrial wireless standard for process automation

3

Ø Reliability and predictability

q Multi-channel TDMA MAC q One transmission per channel q Redundant routes q Over IEEE 802.15.4 PHY

Ø Centralized network manager

q Collect topology information q Generate routes and schedule q Change when devices/links break

pr pressur essure e tank tank level level temperatur temperature e vibration vibration motor motor valve valve Contr Controller

• ller

The Control Challenge

4

Dependable control requires

• real-time
• control performance
• resilience to loss

Most of today’s industrial wireless networks are for monitoring.

Sour Source: ce: https://www https://www.automation.com .automation.com

Towards Dependable Wireless Control

1. Real-time wireless networks and analysis 2. Optimizing control performance over wireless 3. Resilient yet efficient wireless control under loss.

5

Cannot be accomplished by wireless or control design alone à Cyber-Physical Co-design of Wireless and Control

Towards Dependable Wireless Control

• 1. Real-time wireless networks and analysis

2. Optimizing control performance over wireless 3. Resilient yet efficient wireless control under loss.

6

Cannot be accomplished by wireless or control design alone à Cyber-Physical Co-design of Wireless and Control

The Real-Time Problem

Ø A feedback control loop incurs a flow Fi

q Route: sensor à … à controller à … à actuator q Generate packet every period Pi q Multiple control loops share a network

Ø Each flow must meet deadline Di (≤ Pi)

q Stability and predictable control performance

Ø Research problems

q Real-time transmission scheduling à meet deadlines q Fast delay analysis à adapt to dynamics

7

Delays in WirelessHART

A transmission is delayed by Ø channel contention when all channels are assigned to other transmissions Ø transmission conflict over shared node

8

2 1

• 1 and 4 conflict
• 4 and 5 conflict

4 5 3

Fast Delay Analysis

Ø Compute upper bound of the delay for each flow

q

Sufficient condition for real-time guarantees

q

Enable fast adaptation to wireless dynamics

Ø Channel contention à multiprocessor task scheduling

q

A channel à a processor

q

Flow Fi à a task with period Pi, deadline Di, execution time Ci

q

Leverage real-time scheduling theory!

q

Response time analysis for multiprocessors

Ø Account for delays due to transmission conflicts

9

• A. Saifullah, Y. Xu, C. Lu and Y. Chen, End-to-End Communication Delay Analysis in Industrial

Wireless Networks, IEEE Transactions on Computers, 64(5): 1361-1374, May 2015.

!"#$%&'"(!"# &!"#$%&'"(!"$

)*$%+*,

Delay due to Conflict

Ø Low-priority flow Fl and high-priority flow Fh conflict à delay Fl Ø Q(I,h): #transmissions of Fh sharing nodes with Fl

q In the worst case, Fh can

delay Fl by Q(l,h) slots

Ø Conflicts contribute significantly to delays

q Delay analysis [TC 2015] q Scheduling [RTSS 2010, 2015] q Routing [IoTDI 2018]

10

Fl delayed by 2 slots Fl delayed by 2 slots Fl delayed by 1 slot

Real-Time Wireless Networking

Ø WirelessHART stack [IoT

• J 2017]

q Implementation on a 69-node testbed q Network manager (scheduler + routing)

Ø Real-time and efficiency for industrial IoT

q Emergency communication [ICCPS 2015] q Channel selection [INFOCOM 2017] q Channel reuse [ICDCS 2018] q Energy-efficient, real-time routing [IoTDI 2016, 2018]

Ø Low-Power Wide-Area Networks

q SNOW: Sensor Network Over TV White Spaces

[SenSys 2016, 2017]

11

Towards Dependable Wireless Control

1. Real-time wireless networks and analysis

• 2. Optimizing control performance over wireless

3. Resilient yet efficient wireless control under loss.

12

Cannot be accomplished by wireless or control design alone à Cyber-Physical Co-design of Wireless and Control

Wireless-Control Co-Design

Observation Ø Wireless resource is scarce and dynamic Ø Cannot afford separating wireless and control designs Cyber-Physical Co-Design Ø Cojoin the design of wireless and control Examples Ø Rate selection for wireless control [TECS 2014] Ø Scheduling-control co-design [ICCPS 2013] Ø Routing-control co-design [ICCPS 2015]

13

Rate Selection for Wireless Control

Ø Optimize the sampling rates of control loops sharing a WirelessHART network. Ø Rate selection must balance control and communication.

q Low sampling rate à poor control performance q High sampling rate à long delay à poor control performance

q Rate selection must balance control and communication.

14

Co-Design: incorporate the impacts of rates

• n both control and communication

Ø Control cost of control loop i under rate fi [Seto RTSS’96]

q

Approximated as with sensitivity coefficients

Cyber-Physical Design Interface

15

Ø Digital implementation of control loop i

q

Periodic sampling at rate fi

q

Performance deviates from continuous counterpart

αi e−β i fi

Ø Overall control cost of n loops:

αi e−β i fi

i=1 n

∑

αi, βi

Interface between cyber and physical designs!

• D. Seto, J.P

. Lehoczky, L. Sha, K.G. Shin, On Task Schedulability in Real-Time Control Systems. RTSS 1996

delayi ≤1/ fi

The Rate Selection Problem

16

f = { f1, f2,, fn}

fi

min ≤ fi ≤ fi max

minimize control cost

αi e−β i fi

i=1 n

∑

subject to Ø Constrained non-linear optimization Ø Determine sampling rates

Communication delay Control performance

A Challenging Optimization Problem!

Ø In terms of decision variables (rates), the delay bounds are

17

q

non-linear

q

non-convex

q

non-differentiable

Lagrange dual of objective R a t e

• f

c

• n

t r

• l

l

• p

6

Control cost

Cyber-Physical Co-Design

Ø Relax delay bound à simplify control optimization

q Derive a convex and smooth, but less precise delay bound. q Rate selection becomes a convex optimization problem.

➠ Optimize control performance efficiently at run time!

18

• A. Saifullah, C. Wu, P

. Tiwari, Y. Xu,

• Y. Fu, C. Lu and
• Y. Chen, Near Optimal Rate Selection for Wireless

Control Systems, ACM Transactions on Embedded Computing Systems, 13(4s), Article 128, April 2014.

Towards Dependable Wireless Control

1. Real-time wireless networks and analysis 2. Optimizing control performance over wireless

• 3. Resilient yet efficient control under data loss.

19

This cannot be accomplished by wireless or control design alone à Cyber-Physical Co-design of Wireless and Control

Resilient Control under Data Loss

Ø Data loss causes instability and degrades control performance. Ø Traditionally addressed in separation

q Control: control design to tolerate data loss. q Wireless: redundancy reduces loss at high resource cost. q But how much redundancy is sufficient?

Ø Cyber-physical co-design

q Incorporate robust control design. q Tailor wireless protocols for control needs. q Resilient and efficient wireless control.

20

Handle Data Loss from Sensors

21

Ø State Observer estimates system states based on a system model even if there is no new data from sensors

Model Predictive Control Extended Kalman Filter Actuators Sensors Reference

[ ( ), ( 1), , ( )] u k u k u k w + + , ( , ( , ( , (

( ) y k ˆ( ) y k

ˆ( ) x k

Buffer Plant

ˆ( ) u k

• B. Sinopoli, L. Schenato, M. Franceschetti, K. Poolla, M.I. Jordan, S.S. Sastry, Kalman filtering with

intermittent observations. IEEE Transactions on Automatic Control, 49(9):1453–1464, 2004.

22

Handle Data Loss from Controller

Ø Model Predictive Control

q Controller computes control inputs in the next w+1 sampling periods:

u(k), u(k+1), ... u(k+w).

q Actuator applies u(k).

Ø Buffered actuation

q Actuator buffers previous control inputs u(k+1), ... u(k+h) (h<=w). q Applies buffered control input if updated input is lost. q Buffer size of h à tolerate h consecutive packet loss.

Model Predictive Control Extended Kalman Filter Actuators Sensors Reference

[ ( ), ( 1), , ( )] u k u k u k w + + , ( , ( , ( , (

( ) y k ˆ( ) y k

ˆ( ) x k

Buffer Plant

ˆ( ) u k

Pump 1 Heater

1

L

2

L

Tank 1 Tank 2 Reagent Tank 2

2

u b Reagent Tank 1

1

u

a Pump 2

23

Case Study: Exothermic Reaction Plant

Plant: nonlinear chemical reaction Control input: u1 and u2 Objective: Maintain temperature in Tank 2

Wireless Cyber-Physical Simulator (WCPS)

• Integrate TOSSIM and Simulink
• Capture dynamics of both wireless

networks and physical plants

• Holistic simulations of wireless control
• Open source: wcps.cse.wustl.edu

24

Impact of Data Loss from Sensor

System is highly resilient to packet loss from sensors

Extended Kalman filter under 60% loss from sensor

25

Impact of Data Loss to Actuator

Actuation is more sensitive to data loss than sensing. à Data losses are not equal!

Actuation buffer (size 8) under 60% loss to actuator

Routing in WirelessHART

Ø Existing approach to routing

q Source routing: single path routing à efficient but unreliable. q Graph routing: every node on the primary path has a backup path à

reliable at cost of capacity and energy.

q Entire network uses a uniform routing strategy.

Ø But sensing and actuation need different levels of reliability!

26

Asymmetric Routing

Ø Differentiated routing for sensing and actuation Ø State observer handles data loss from sensors à Ø Source routing from sensors

q State observer compensates for lower reliability q Save network resource

Ø Actuation is more sensitive to data loss à Ø Graph routing to actuators

q High reliability q High resource cost, but needed for control

Tailor routing to control à Spend wireless resource where control needs it

27

28

Maximum Absolute Error

• 73dBm Noise
• 73dBm Noise
• B. Li,
• Y. Ma, T. Westenbroek, C. Wu, H. Gonzalez and C. Lu, Wireless Routing and Control: a Cyber-Physical Case

Study, ACM/IEEE International Conference on Cyber-Physical Systems (ICCPS’16).

Ø Source/Graph performs close to Graph/Graph at 3Hz sampling rate. Ø Efficiency allows higher sampling rate with Source/Graph à further improve control performance!

Towards Dependable Wireless Control

Ø Real-time wireless networking

q Protocols and delay analysis for latency guarantees

Ø Optimize control performance over wireless

q Incorporate scheduling analysis in rate selection

Ø Resilient wireless control under data loss

q Tailor routing strategies for control needs

29

Cannot be accomplished by wireless or control design alone à Cyber-Physical Co-design of Wireless and Control

Beyond Design: Holistic Cyber-Physical Control

Ø Today: network management and control operate in isolation

q Controller controls physical plants q Network manager configures networks q Ignore interdependencies à vulnerable and inefficient industrial plants.

Ø Holistic control: close the loop between control and network

q Holistic controller controls both physical plants and networks.

30

Holistic Controller Plant

Reconfiguration Signals Performance Measurements Control Inputs Outputs

• f Plant

Wireless Sensor Network

Network Manager Network State Observer States of Network Network Configuration

• Y. Ma, D. Gunatilaka, B. Li, H. Gonzalez

and C. Lu, Holistic Cyber-Physical Management for Dependable Wireless Control Systems, ACM Transactions on Cyber-Physical Systems, Special Issue on Dependability in Cyber Physical Systems and Applications, 3(1), Article 3, 2018..

Beyond Design: Holistic Cyber-Physical Control

Ø Today: network management and control operate in isolation

q Controller controls physical plants q Network manager configures networks q Ignore interdependencies à vulnerable and inefficient industrial plants.

Ø Holistic control: close the loop between control and network

q Holistic controller controls both physical plants and networks.

31

Holistic Controller Plant

Reconfiguration Signals Performance Measurements Control Inputs Outputs

• f Plant

Wireless Sensor Network

Network Manager Network State Observer States of Network Network Configuration

• Y. Ma, D. Gunatilaka, B. Li, H. Gonzalez

and C. Lu, Holistic Cyber-Physical Management for Dependable Wireless Control Systems, ACM Transactions on Cyber-Physical Systems, Special Issue on Dependability in Cyber Physical Systems and Applications, 3(1), Article 3, 2018..

How to coordinate networks and control at run-time for resiliency?

Beyond Wireless: Real-Time Edge and Cloud

Ø Support real-time applications in the cloud.

q Latency guarantees. q Real-time performance isolation. q Resource sharing between real-time and non-real-time workloads.

Ø Real-time cloud stack.

q RT

• Xen à real-time virtual machine scheduling (included in Xen)

q VATC à real-time network I/O on a virtualized host. q RT

• OpenStack à real-time cloud resource management.

32

VATC: RT Network I/O RT

• OpenStack

Latency guarantees

Cyber-Physical Event Processing RT Cilk Plus

Beyond Wireless: Real-Time Edge and Cloud

Ø Support real-time applications in the cloud.

q Latency guarantees. q Real-time performance isolation. q Resource sharing between real-time and non-real-time workloads.

Ø Real-time cloud stack.

q RT

• Xen à real-time virtual machine scheduling (included in Xen)

q VATC à real-time network I/O on a virtualized host. q RT

• OpenStack à real-time cloud resource management.

33

VATC: RT Network I/O RT

• OpenStack

Latency guarantees

Cyber-Physical Event Processing RT Cilk Plus

How to orchestrate edge and cloud for dependable control?

The Dependability Challenges

Ø Industrial IoT have started!

q Industrial drivers: standards, consortia, deployments q System building blocks: from wireless to edge and cloud q Holistic modeling, simulation and design tools

Ø We must address the dependability challenges

q Real-time, resiliency, safely, security… q Cyber-physical co-design is a necessity!

34

Application driver from Industry 4.0

CPS: Solving the Right Problem at the Right Time!

For More Information

Ø C. Lu, A. Saifullah, B. Li, M. Sha, H. Gonzalez, D. Gunatilaka, C. Wu, L. Nie and

• Y. Chen,

Real-Time Wireless Sensor-Actuator Networks for Industrial Cyber-Physical Systems, Special Issue on Industrial Cyber-Physical Systems, Proceedings of the IEEE, 104(5): 1013-1024, May 2016. Ø Wireless Cyber-Physical Simulator: http://wcps.cse.wustl.edu Ø Real-Time Industrial Wireless Control Networks: http://cps.cse.wustl.edu/index.php/Real-Time_Wireless_Control_Networks Ø RT

• Xen: https://sites.google.com/site/realtimexen/

35