From Reflexes to In-Network Processing Enabling Ultra-low Latency - PowerPoint PPT Presentation

From Reflexes to In-Network Processing Enabling Ultra-low Latency and High Reliability for Cyber-physical Networking Klaus Wehrle, joint work by the COMSYS team http://comsys.rwth-aachen.de NIPAA@ICNP, 13.10.2020

Motivation Cyber-physical networking Evolution of „ Remote control of machines, humans not involved Communication „ High Precision required Systems „ Challenges: Ultra-low latency and high reliability Cyber Physical CPS / IIoT Control of Things Internet Web & Cloud Good old Internet WiFi 3G 4G Physical Human Cyber Human-centric communication „ Humans are ‘slow’ and compensate (comm. & system) errors „ Latency (<20ms) was never a big issue http://comsys.rwth-aachen.de 2

Cyber-Physical Networking – Challenge 1: Ultra-low Latency Cloud Challenge: Ultra-low latency with Controller Switch Switch Application Controller Sensors Actuators Application Transport Transport Network IP IP Network MAC MAC MAC MAC MAC MAC PHY PHY PHY PHY PHY PHY http://comsys.rwth-aachen.de 3

Cyber-Physical Networking – Challenge 1: Ultra-low Latency Cloud Cloud Challenge: Ultra-low latency with with Controller Controller Switch Switch „ Problem 1: Physical distance Controller Sensors Actuators Controller „ Solution: Reduce the distance 😐 Transport Transport Transport à Edge Cloud reduces horizontal Network IP IP Network Network MAC MAC MAC MAC MAC MAC MAC distance, e.g. in 5G PHY PHY PHY PHY PHY PHY PHY „ Edge cloud still not nearest to plant http://comsys.rwth-aachen.de 4

Cyber-Physical Networking – Challenge 1: Ultra-low Latency — Problem 2: Vertical distance Cloud with of the layered system approach Controller Switch Switch Controller Actuators Sensors „ Control and Communication layers heavily abstract from each other Transport Transport Network IP IP Network ¾ Control is just an(other) application MAC MAC MAC MAC MAC MAC PHY PHY PHY PHY PHY PHY ¾ Network seen as (stable) black box „ No joint optimization possible SENSE SENSOR TRANS NET MAC à Cyber-Physical Networking Initiative MAC NET MAC „ Co-Designed Networked Control MAC NET MAC Seperated by abstraction MAC NET DECIDE TRANS CONTR TRANS NET MAC Pendulum unstable MAC NET MAC MAC NET MAC MAC NET TRANS ACTUATOR http://comsys.rwth-aachen.de ACT 5

Co-Designing Communication and Control Cloud Controller Switch Switch Simple Control Task Reflexes Approximated Low lat Low atenc ency Co Control path Control Rules data pat dat path Reflex n Reflex n Reflex n Reflexes Software-defined IN R1 „ SDN actions not suitable wrong IN R1 token OUT R2 token Networking (SDN) rcv for control actions IN R2 wait pre- token OUT R1 alert OUT R2 token TO R1 „ Simply pre-computing bloats missing Control SDN rule tables Softw. lost Plane token Controller Token rcvd Simple Reduced à But modern programmable Token passed Communication SDN switches (Tofino, FPGA, Communication Rules Data Task smartNICs) are paving the Plane way towards In-Network- http://comsys.rwth-aachen.de Processing 6

Accuracy-Latency-(Throughput) Trade-Off — Computing Platforms Faster & Less 1) End-host computations more predictable computational restrictions But very 2) In-kernel processing (XDP, TC) But restricted more unpredictable operations 3) SmartNIC and slower execution 4) Switch (e.g. Tofino) Userspace Kernel NIC http://comsys.rwth-aachen.de 10

REFLEXES – A Co-Designed Architecture for In-Network Control Reflexes Challenge: Cloud Reflex n Reflex n with Reflex n „ Make joint decision on control Controller Switch Switch and communication decision Controller Actuators Sensors „ Combine possible reactions to Transport Transport Network IP IP Network many reflex candidates and MAC MAC MAC MAC MAC MAC PHY PHY PHY PHY PHY PHY „ push reflexes nearer to plant SENSE SENSOR Pendulum TRANS NET MAC very stable MAC NET Reflexes MAC MAC NET MAC MAC NET DECIDE TRANS CONTR TRANS NET MAC Pendulum ACT earlier unstable and Com/Con-optimized MAC NET MAC MAC NET MAC MAC NET TRANS ACTUATOR http://comsys.rwth-aachen.de ACT 11

General REFLEXES Framework Sensor Access Point Remote R Cloud Switch Sensor Access Point Switch Actuator Edge Switch Cloud Actuator Low Low lat atenc ency dat data a pat path Co Control path — Task separation: Separating data processing and coordination „ Fast and simple reactions based on INP ¾ Use computation in the network to execute simple tasks ¾ Push simplified control algorithm (reflex) to the switch „ Main control algorithm stays in edge cloud to do delay-insensitive adaptation ¾ Slow path processing, coordination and state management stays in the cloud ¾ Cloud updates reflex if necessary, e.g. latency change, process is mobile, etc. http://comsys.rwth-aachen.de 12

Two Real-world Examples (Cluster Internet of Production) — Arc welding robots — Mobile robot cooperation R S R S HD HD … 6x S S S A HD HD A S S S S S S R A — Control loops — Control loops „ Single-digit millisecond latency „ Positioning coordinated by many inputs „ Multiple sensor sources ¾ e.g. indoor coordinate system, camera, etc. ¾ In-network coordinate transformation ¾ HD and infrared camera „ Human in the loop detection (safety zone) ¾ Current draw of light arc ¾ e.g. logical safety loop among cameras, lasers, Lidar „ Actuators „ Robot interaction via multiple sensors ¾ Robot positioning ¾ Light arc voltage http://comsys.rwth-aachen.de „ Augmented Reality … 13

Networked Control – Real-World Example Laser Tracker Coordinate Transformation C http://comsys.rwth-aachen.de 14

In-Network Coordinate Transformation – Fundamentals 𝒔 sin 𝜾 cos 𝝌 𝑦 Challenge: Coordinate transformation 𝑧 = 𝒔 sin 𝜾 sin 𝝌 (Spherical to Cartesian) 𝑨 𝒔 cos 𝜾 — Restricted Fixed-Point Arithmetic — Approximate trigonometric functions „ ± 0 …2 ! . [0 …2 "#$! ] 1. Chebyshev polynomials „ Choose fixed point to 2. Table Lookup ¾ ensure range is sufficiently large (application range) 𝜾 𝐭𝐣𝐨 ¾ maximize fractional part 0.000000 0 (required accuracy) sin 𝜾 0.000488 0.000488 0.000977 0.000977 … „ Problem: Large table space needed „ Use sum of angle identity sin𝑏 + 𝑐 = sin𝑏 / cos 𝑐 + cos 𝑏 / sin𝑐 http://comsys.rwth-aachen.de 15

In-Network Coordinate Transformation – Fundamentals 𝒔 sin 𝜾 cos 𝝌 𝑦 Challenge: Coordinate transformation 𝑧 = 𝒔 sin 𝜾 sin 𝝌 (Spherical to Cartesian) 𝑨 𝒔 cos 𝜾 — Restricted Fixed-Point Arithmetic — Approximate trigonometric functions „ ± 0 …2 ! . [0 …2 "#$! ] 1. Chebyshev polynomials „ Choose fixed point to 2. Table Lookup ¾ ensure range is sufficiently large (application range) 𝜾 𝒊𝒋𝒉𝒊 𝜾 𝒎𝒑𝒙 (sin 𝜾 𝒊𝒋𝒉𝒊 , cos 𝜾 𝒊𝒋𝒉𝒊 ) (sin 𝜾 𝐦𝐩𝐱 , cos 𝜾 𝐦𝐩𝐱 ) 𝜾 𝒊𝒋𝒉𝒊 𝜾 𝒊𝒋𝒉𝒊 (sin 𝜾 𝒊𝒋𝒉𝒊 , cos 𝜾 𝒊𝒋𝒉𝒊 ) (sin 𝜾 𝒊𝒋𝒉𝒊 , cos 𝜾 𝒊𝒋𝒉𝒊 ) ¾ maximize fractional part 0.000000 0.000000 (0,1) (0,1) 000000 000000 (0,1) (0,1) (required accuracy) 000001 (2.980232e-8, 1) 0.000488 0.000488 (0.000488, 0.999999) (0.000488, 0.999999) 000001 (2.980232e-8, 1) 000002 000002 (5.960464e-8, 1) (5.960464e-8, 1) 0.000977 0.000977 (0.000977, 0.999995) (0.000977, 0.999995) … … … … 000488 000488 (-0.000470, 0.999999) (-0.000470, 0.999999) 6.282714 6.282714 (-0.000470, 0.999999) (-0.000470, 0.999999) sin 6.282714000002 ≈ −0.000471 sin 6.282714000002 ≈ −0.000471 sin 6.282714000002 ≈ −0.000471 ≈ sin 𝜾 𝒊𝒋𝒉𝒊 / 𝑑𝑝𝑡 𝜾 ()* + cos 𝜾 𝒊𝒋𝒉𝒊 / sin 𝜾 +,- ≈ −0.000470 / 1 + 0.999999 / 5.960464 / 10 $. ≈ −0.000470 http://comsys.rwth-aachen.de 16

In-Network Image Processing — Low-latency computer vision often needed • Fast reactions to the environment — Camera images rarely fit into single packet • Use local computation strategies like convolution Middle position between two highest responses Turn left Forward Turn right http://comsys.rwth-aachen.de 17

Data Stream Processing Sensor Access Point Remote Re Cloud Cl Switch Sensor Access Point Switch Sensor Ed Edge Switch Cl Cloud Actuator — Collection and Analysis of Process Data „ Data-driven improvement of production and efficiency ¾ Collect every data item the process and machines are emitting ¾ Derive immediate feedback on process status and product quality ¾ Realtime-feedback for production process „ Problem: Data rate of produced process data http://comsys.rwth-aachen.de 18

Real-world example: Fine Blanking • Infrared camera: 160 Mbps • Sampling: 2.5-5kHz • Data not relevant • 64 signals at 32bit • Press control/sensors: 25 Mbps • Sampling: 5 kHz • Vibr. Sensor: 1 Mhz, 150Mbps • Data rate: 45-90 Mbps • D. Rate: 10 Mbps • ~500 Mbps per 4K camera Decoiler Leveler Lubricator Press http://comsys.rwth-aachen.de 19