CROSS-LAYER CROSS-LAYER LATENCY-AWARE AND -PREDICTABLE - - PowerPoint PPT Presentation

▶
cross layer cross layer latency aware and predictable
SMART_READER_LITE
LIVE PREVIEW

CROSS-LAYER CROSS-LAYER LATENCY-AWARE AND -PREDICTABLE - - PowerPoint PPT Presentation

Jan 22, 2023 98 likes •704 views

CROSS-LAYER CROSS-LAYER LATENCY-AWARE AND -PREDICTABLE LATENCY-AWARE AND -PREDICTABLE DATA COMMUNICATION DATA COMMUNICATION Andreas Schmidt Apr 07th, 2020 Anywhere on Earth Doctoral Colloquium 1

slide-1
SLIDE 1

CROSS-LAYER CROSS-LAYER LATENCY-AWARE AND -PREDICTABLE LATENCY-AWARE AND -PREDICTABLE DATA COMMUNICATION DATA COMMUNICATION

Andreas Schmidt

Apr 07th, 2020 Anywhere on Earth Doctoral Colloquium 復 􏥧  􏞣

1

slide-2
SLIDE 2

 

CYBER-PHYSICAL SYSTEMS CYBER-PHYSICAL SYSTEMS

 􏋜  􏡠  􏂆

2 . 1

slide-3
SLIDE 3

DISTRIBUTED CYBER-PHYSICAL SYSTEMS DISTRIBUTED CYBER-PHYSICAL SYSTEMS

 􏈴  􏂆  􏂆  􏙢  􏂆  􏂆  􏂆  􏇁  􏡠

2 . 2

slide-4
SLIDE 4

A REAL-WORLD A REAL-WORLD DISTRIBUTED CPS DISTRIBUTED CPS

Proprietary Solution

Enhanced Shock Burst (ESB)

Our Open Solution

Wi-Fi, IP, PRRT Publication-in-Progress [BSGPH20]

0:00 / 0:41

 􏡠  􏡠

2 . 3

slide-5
SLIDE 5

2008 2008

Cyber Physical Systems: Design Challenges [Lee08] "Passing of time is inexorable" Most abstractions insufficiently capture timing Edward A. Lee

UC Berkeley

 􏇛

2 . 4

slide-6
SLIDE 6

LATENCY-* LATENCY-*

AWARENESS AWARENESS Know what latency you require Know what latency you cause Share this with others PREDICTABILITY PREDICTABILITY Built upon awareness Improve confidence Share confidence with others  􏁯  􏇦

2 . 5

slide-7
SLIDE 7

STATE OF AFFAIRS IN NETWORKING STATE OF AFFAIRS IN NETWORKING

The Internet most dominant interconnecting communication systems best effort, i.e. no predictable timing (by design) Link: Transmission latencies known, but not exposed IP: No timing at all Transport Layer: TCP:  UDP:  RTP:  Let's go back to proprietary, closed networks? No! Why?  􏃂  􏇡  􏑶 􏃬 

2 . 6

slide-8
SLIDE 8

Internet Architecture [Cla18] provides

INTEROPERABILITY INTEROPERABILITY

2 . 7

slide-9
SLIDE 9

RESEARCH QUESTION RESEARCH QUESTION*

CAN WE MAINTAIN CAN WE MAINTAIN INTEROPERABILITY INTEROPERABILITY AND AND AT THE SAME TIME AT THE SAME TIME ACHIEVE ACHIEVE LATENCY-AWARENESS AND -PREDICTABILITY? LATENCY-AWARENESS AND -PREDICTABILITY?

* as you might assume, this is a question too big for a PhD thesis

2 . 8

slide-10
SLIDE 10

TODAY'S RESEARCH QUESTION TODAY'S RESEARCH QUESTION*

CAN WE BUILD CAN WE BUILD TRANSPORT LAYERS TRANSPORT LAYERS WITH WITH LATENCY-AWARENESS AND -PREDICTABILITY? LATENCY-AWARENESS AND -PREDICTABILITY?

*

2 . 9

slide-11
SLIDE 11

OUTLINE OUTLINE

PRRT PRRT X-LAP X-LAP X-PACE X-PACE TTS TTS OUTLOOK OUTLOOK  􏃭  􏙄  􏋳  􏕜  􏇦

2 . 10

slide-12
SLIDE 12

PRRT PRRT

[SH16a, GGG+19]

 􏃭

3 . 1

slide-13
SLIDE 13

WHAT IS THE PROBLEM? WHAT IS THE PROBLEM?

Full reliability infinite time [Sha48, Fei54] Most applications... ... have (some) time constraint ... can tolerate (some) missing data  (Today's) transport protocols do not consider that

⇒

3 . 2

slide-14
SLIDE 14

FLAVOURS OF RELIABILITY FLAVOURS OF RELIABILITY

 Full  Partial

3 . 3

slide-15
SLIDE 15

PRRT PROTOCOL PRRT PROTOCOL

Predictably Reliable Real-time Transport Fundamental work in [Gor12] Redesign for control applications since 2015

3 . 4

slide-16
SLIDE 16

PRRT'S PLACE IN THE ISO MODEL PRRT'S PLACE IN THE ISO MODEL

Default Control/Multimedia App PRRT UDP IP Any MAC In General Control/Multimedia App PRRT Any Layer with Process-to-Process Datagram Delivery

3 . 5

slide-17
SLIDE 17

PRRT CHANNEL MEASUREMENT PRRT CHANNEL MEASUREMENT

Parameter Approach Round-trip Time NTP Algorithm [MMBK10] Delivery Rate IETF Dra [CCYJ17] Loss Rate & Correlation Statistics over success-failure sequences Selective ACKs with feedback Measurements exposed via API

3 . 6

slide-18
SLIDE 18

PRRT'S RECEIVE MODES PRRT'S RECEIVE MODES

3 . 7

slide-19
SLIDE 19

PRRT'S FUNCTIONS PRRT'S FUNCTIONS

Error Control Adaptive Hybrid Error Coding [Gor12] (not covered in this talk) Congestion Control Based on BBR ideas [CCG+16] Utilization & measurement Latency-avoidance Rate and Flow Control Later: X-Pace  􏋳

3 . 8

slide-20
SLIDE 20

EVALUATION EVALUATION

[GGG+19]

3 . 9

slide-21
SLIDE 21

USABILITY USABILITY

Policy Publicly available at MIT licensed Technology C implementation & API Python API (via Cython) Rust API (via bindgen)  􏃤 prrt.larn.systems  􏄢

3 . 10

slide-22
SLIDE 22

X-LAP X-LAP

[RSH+17, RSH+18]

 􏙄

4 . 1

slide-23
SLIDE 23

WHAT IS THE PROBLEM? WHAT IS THE PROBLEM?

 CPS demand predictable latency and timing Cross-layer, intra-host profiling is required

valgrind, wireshark, packetdrill... insufficient

Microsecond regime is increasingly relevant [BMPR17]

4 . 2

slide-24
SLIDE 24

STAMPS STAMPS

Wall-Time

CLOCK_MONOTONIC

expensive to capture (≈ 70ns) Processor Cycles

rdtsc

cheap to capture (≈ 10ns) non-trivial relation to wall-time  􏀘  􏋜

4 . 3

slide-25
SLIDE 25

WHAT IS X-LAP? WHAT IS X-LAP?

X = Cross, Lap = time to do one round

X-Lap is a cross-layer, intra-host, timing & latency analysis tool

download at xlap.larn.systems

4 . 4

slide-26
SLIDE 26

STAMPS IN X-LAP STAMPS IN X-LAP

e.g.: ChannelTransmit [ms] [1]  􏀘

↑t = 5 ms ↓c = 7 156 294

 􏋜

4 . 5

slide-27
SLIDE 27

X-LAP IN ACTION X-LAP IN ACTION

4 . 6

slide-28
SLIDE 28

TRACES DATA FORMAT TRACES DATA FORMAT

SN ... ChannelTransmit_T ChannelTransmit_C ... 42 ... 5.012ms ... 43 ... 5.029ms ... ... ... ... .... ...

7 156 294 47 961 303

4 . 7

slide-29
SLIDE 29

RUNTIME TRACING CODE RUNTIME TRACING CODE

XlapStamp_Cycle(LinkTransmitStart, pkt->seqno); XlapStamp_TimeValue(ChannelTransmit, timestamp, pkt->seqno); XlapStamp_CycleValue(ChannelTransmit, cyclestamp, pkt->seqno); XlapStamp_Cycle(LinkTransmitEnd, pkt->seqno); void send_packet(PrrtPacket* pkt) { 1 pace(); // delays sending of data using cross-layer pacing 2 wait_for_free_congestion_window_space(); 3 compute_next_send_time(); // for cross-layer pacing 4 char[] bytes = serialize_packet_to_bytes(pkt); 5 6 struct timespec timestamp; 7 uint64_t cyclestamp; 8 send(sock_fd, bytes, & timestamp, &cyclestamp); 9 10 11 track_outstanding_packet(pkt); 12 13 } 14

4 . 8

slide-30
SLIDE 30

CYCLE TO TIME CYCLE TO TIME

4 . 9

slide-31
SLIDE 31

PACKET TRACE PACKET TRACE

4 . 10

slide-32
SLIDE 32

TRACE JITTER TRACE JITTER

4 . 11

slide-33
SLIDE 33

MULTISERIES CORRELATION MULTISERIES CORRELATION

4 . 12

slide-34
SLIDE 34

X-PACE X-PACE

[SRGP+19]

 􏋳

5 . 1

slide-35
SLIDE 35

WHAT IS THE PROBLEM? WHAT IS THE PROBLEM?

 Bufferbloat [GN12] Unpaced Paced

5 . 2

slide-36
SLIDE 36

PACE PACE PACING PACING

intentionally delaying a transmission

P, [P] =

time unit of work

5 . 3

slide-37
SLIDE 37

CROSS-LAYER PACING CROSS-LAYER PACING

every step considers and adapts its pace to the bottleneck pace

PACED SYSTEMS PACED SYSTEMS

i ∈ [0 : n − 1] P (i) P (btl) S is paced iff ∀i, j ∈ [0 : n − 1] : i < j ⇒ ≥ P (i) P (j)

5 . 4

slide-38
SLIDE 38

PROPAGATE PACES PROPAGATE PACES

5 . 5

slide-39
SLIDE 39

REDUCE 無駄 REDUCE 無駄*

* muda or waste, originally described by Toyota Production System [Ohn88]

Inventory: only a delivered message is useful Waiting: a message loses value over time Overprocessing: as-fast-as-possible causes losses  􏑩  􏇛  􏒌

5 . 6

slide-40
SLIDE 40

X-PACE IN PRRT X-PACE IN PRRT

5 . 7

slide-41
SLIDE 41

CROSS-LAYER API CROSS-LAYER API

send_sync()

Passive

socket.btl_pace

Active

5 . 8

slide-42
SLIDE 42

EVALUATION APPROACH EVALUATION APPROACH

PRRT with X-Pace vs. Optimized TCP Variants (CUBIC, BBR)

NODELAY, QUICKACK low_latency SNDBUF & RCVBUF low write() not send()

5 . 9

slide-43
SLIDE 43

INTERNET SCENARIO INTERNET SCENARIO

traceroute to 79.199.28.123 (79.199.28.123), 30 hops max, 60 byte packets 1 vlan-herfet-neu.nt.uni-saarland.de (134.96.86.1) 0.400 ms 0.358 ms 0.420 ms 2 c65eb36-win.net.uni-saarland.de (134.96.6.54) 0.328 ms 0.314 ms 0.384 ms 3 cr-dui1-te0-5-0-2-4.x-win.dfn.de (188.1.241.185) 8.687 ms 8.688 ms 8.680 ms 4 cr-fra2-be16.x-win.dfn.de (188.1.144.178) 9.618 ms 9.732 ms 9.727 ms 5 ffm-b12-link.telia.net (213.248.97.40) 9.226 ms 9.563 ms 9.191 ms 6 ffm-bb3-link.telia.net (62.115.142.46) 10.069 ms 9.761 ms 9.704 ms 7 ffm-b4-link.telia.net (62.115.120.6) 9.802 ms 9.792 ms 10.024 ms 8 dtag-ic-319284-ffm-b4.c.telia.net (213.248.93.187) 10.374 ms 10.175 ms 10.17 9 91.23.246.213 (91.23.246.213) 13.230 ms 13.252 ms 13.234 ms 10 * * * ... 30 * * *

5 . 10

slide-44
SLIDE 44

RESULTS RESULTS

5 . 11

slide-45
SLIDE 45

TRANSPARENT TRANSMISSION TRANSPARENT TRANSMISSION SEGMENTATION SEGMENTATION

[SH16b, SH17b]

 􏕜

6 . 1

slide-46
SLIDE 46

WHAT IS A PROBLEM? WHAT IS A PROBLEM?

Small receiver buffer flow-limited communication 􏁲 

⇒

6 . 2

slide-47
SLIDE 47

TRANSPARENT TRANSMISSION TRANSPARENT TRANSMISSION SEGMENTATION SEGMENTATION

Relay: Transport-Layer Performance-Enhancing-Proxy [ ]  􏃭 RFC3135

6 . 3

slide-48
SLIDE 48

TTS IN ACTION TTS IN ACTION

6 . 4

slide-49
SLIDE 49

IDEAL RECEIVER BUFFER IDEAL RECEIVER BUFFER

derived from a theoretical flow control model

= ⋅ Brelay RTT1 RTT2 Brecv Λ = = + 1 Reff,TTS Reff,E2E RTT1 RTT2

6 . 5

slide-50
SLIDE 50

EVALUATION EVALUATION

= 4 KiB, RT = 20 ms, RT = 5 ms Brecv T1 T2 ⇒ Λ = 5, = 4 ⋅ 4KiB Brelay

6 . 6

slide-51
SLIDE 51

BENEFITS OF TTS BENEFITS OF TTS

Lossy Last Mile High Reordering due to Jitter Insufficient Receiver Buffer

6 . 7

slide-52
SLIDE 52

OUTLOOK OUTLOOK

e.LARN, [SGPH20]

 􏇦

7 . 1

slide-53
SLIDE 53

FUTURE WORK FUTURE WORK

Energy- Awareness Statistical Shaping Multicast Support  􏍷  􏣲  􏇡

7 . 2

slide-54
SLIDE 54

WRAP-UP WRAP-UP

NETWORKED CPS NETWORKED CPS PRRT PRRT

[SH16a, GGG+19]

X-LAP X-LAP

[RSH+17, RSH+18]

X-PACE X-PACE

[SRGP+19]

TTS TTS

[SH16b, SH17b]

OUTLOOK OUTLOOK

e.LARN, [SGPH20]

 􏂆  􏃭  􏙄  􏋳  􏕜  􏇦

8 . 1

slide-55
SLIDE 55

QUESTIONS QUESTIONS  􏋾

8 . 2

slide-56
SLIDE 56

APPENDIX APPENDIX

9 . 1

slide-57
SLIDE 57

PUBLICATIONS PUBLICATIONS

[SH16a] Schmidt, Andreas; Herfet, Thorsten: “Approaches for Resilience- and Latency-Aware Networking“, International Symposium on Networked Cyber-Physical Systems (NetCPS, Poster Session), Munich, Germany, September 2016 [SH16b] Schmidt, Andreas; Lange, Tobias; Herfet, Thorsten: “Low-Latency Multimedia Streaming using Open Networking Environments“, International Conference on Computer and Communications (ICCC), Chengdu, China, October 2016 [SH17b] Schmidt, Andreas; Herfet, Thorsten: “Transparent Transmission Segmentation in Soware-Defined Networks”, IEEE Conference on Network Sowarization (NetSo), Bologna, Italy, July 2017 [RSH+17] Reif, Stefan; Schmidt, Andreas; Hönig, Timo; Herfet, Thorsten; Schröder-Preikschat, Wolfgang: “X-Lap: A Systems Approach for Cross-Layer Profiling and Latency Analysis for Cyber-Physical Networks”, 15th International Workshop on Real-Time Networks (ECRTS RTN), Dubrovnic, Croatia, June 2017 [RSH+18] Reif, Stefan; Schmidt, Andreas; Hönig, Timo; Herfet, Thorsten; Schröder-Preikschat, Wolfgang: “∆elta: Differential Energy-Efficiency, Latency, and Timing Analysis for Real-Time Networks“, 16th International Workshop on Real-Time Networks (ECRTS RTN), Barcelona, Spain, July 2018 [GGG+19] Gallenmüller, Sebastian; Glebke, René; Günther, Stephan; Hauser, Eric; Leclaire, Maurice; Reif, Stefan; Rüth, Jan; Schmidt, Andreas; Carle, Georg; Herfet, Thorsten; Schröder-Preikschat, Wolfgang; Wehrle, Klaus: “Enabling wireless network support for gain scheduled control“. 2nd International Workshop on Edge Systems, Analytics and Networking (EdgeSys), Dresden, Germany, March 2019 [SRGP+19] Schmidt, Andreas; Reif, Stefan; Gil Pereira, Pablo; Hönig, Timo; Herfet, Thorsten; Schröder-Preikschat, Wolfgang: “Cross-layer Pacing for Predictably Low Latency”. 6th International IEEE Workshop on Ultra-Low Latency in Wireless Networks (ULLWN), Paris, France, April 2019 [SGPH20] Schmidt, Andreas; Gil Pereira, Pablo; Herfet, Thorsten: "Predictably Reliable Real-time Transport Services for Wireless Cyber-Physical Systems", IFAC World Congress, Berlin, Germany, July 2020

􏅝 

9 . 2

slide-58
SLIDE 58

REFERENCES REFERENCES

[Sha48] Claude Elwood Shannon. A Mathematical Theory of Communication. 1948. [Fei54] Amiel Feinstein. A new basic theorem of information theory. 1954. [Lee08] Edward A. Lee. Cyber Physical Systems: Design Challenges. In Proceedings of the 11th IEEE International Symposium on Object and Component-Oriented Real-Time Distributed Computing (ISORC), 2008. [MMBK10] D. Mills, J. Martin, J. Burbank, and W. Kasch. Network time protocol version 4: Protocol and algorithms

  • specification. RFC 5905, RFC Editor, June 2010. http://www.rfc-editor.org/rfc/rfc5905.txt.

[GN12] Jim Gettys and Kathleen Nichols. Bufferbloat: Dark Buffers in the Internet. Communications of the ACM, 55(1):57 – 65, 2012. [Gor12] Manuel Gorius. Adaptive Delay-constrained Internet Media Transport. PhD thesis, Saarland University, 2012. [CCG+16] Neal Cardwell, Yuchung Cheng, C Stephen Gunn, Soheil Hassas Yeganeh, and Van Jacobson. BBR: Congestion- based congestion control. ACM Queue, 14(5):50, 2016. [BMPR17] Luiz Barroso, Mike Marty, David Patterson, and Parthasarathy Ranganathan. Attack of the Killer Microseconds Communications of the ACM, 60(4):48–54, March 2017. [CCYJ17] Yuchung Cheng, Neal Cardwell, Soheil Hassas Yeganeh, and Van Jacobson. Delivery rate estimation - IETF DRAFT, 2017. [Cla18] David D Clark. Designing an Internet. MIT Press, 2018.

 􏗜

9 . 3

slide-59
SLIDE 59

IMAGE SOURCES IMAGE SOURCES  􏌃

Edward Lee

9 . 4