Identifying Load-Balanced Backends Ian Rodney 1
Why does it matter? • Targeted DDoS • Service degradation - - - 2
Load Balancers • Terminate & regenerate :( • Pass through :) • Hashing – IP/Port - - - 3
Side Channels • Information leaks around shared state ���� • Well studied • Setup: �������� 4
IPID Mechanism • Unique fragment ID • Counter types: • 16-bit field in IPv4 • Global • IPv6 Extension • Per-Destination • Hybrid (2048 counters) 5
IPID Side Channel • Global Counter • Covered in lecture! • Per-Dest • Hybrid 6
IPID Side Channel • Global Counter • Pretty hard to defeat • Per-Dest • But there is a way • Hybrid 7
IPID Side Channel • Global Counter Source: IPv6 Test; S/A • Per-Dest RST, IPID: n • Hybrid Source: Victim; S/A RST, IPID: ? Source: IPv6 Test ; S/A RST, IPID: n+1 Global Counter with or n+2 found IPv6 Address 10 Xu 2018
Timestamps Mechanisms • Systems have a unique clock drift TCP TCP ICMP ICMP HTTP HTTP Resolution 1Hz – 1 kHz 1Hz – 1 kHz 1kHz 1kHz 1Hz 1Hz Constant Drift Yes Yes NTP adjusted NTP adjusted Yes Yes NTP removes offset • Is 1Hz too low? 12:00:01 12:00:00 Kohno & Et. Al. 2005, Zander 2008, Rye 2019 11
Shared State Mechanisms • Fragment reassembly buffer • TCP SYN Cache • Challenge ACK rate limit 13
Rate-Limit Mechanism • Challenge ACK rate limit SEQ: 900 • SYN or RST variants ACK: 1000 RST: 1201 Challenge; ACK: 1000 99 left RST: 1000 RFC 5961 14
Rate-Limit Mechanism • Challenge ACK rate limit SEQ: 900 • SYN or RST variants ACK: 1000 RST; SEQ: 60,000 RST; SEQ: 1200 Challenge; ACK: 1000 99 left RFC 5961 15
Rate-Limit Side Channel • Infer presence of connection ���� Seq: 1000 Challenge ACK 99 left 0 left Source: User, SYN Seq: 500 Cao & Et. Al 2016 16
Rate-Limit Side Channel • Infer presence of connection ���� Seq: 1000 Challenge ACK 100x 99 left 0 left Source: A, SYN Challenge 99x ACK Cao & Et. Al 2016 17
Buffer Side Channel U, IPID: 10 • Fragment buffer A, IPID: 80 U, IPID: 20 & per-destination Source: A, Frag A, IPID: 90 Source: U, Frag IPID • subtle IPID: 20 ���� Source: V, Full Cao & Et. Al 2016 18
Buffer Side Channel U, IPID: 10 • Fragment buffer A, IPID: 80 Source: A, New Frag A, IPID: 100 & per-destination A, IPID: 90 IPID • subtle ���� Zhang 2018 19
Buffer Side Channel U, IPID: 10 Source: A, Remainders: • Fragment buffer A, IPID: 80 A, IPID: 100 & per-destination A, IPID: 90 80, 90, 100 IPID Reply for 80, 90, 100 • subtle ���� Zhang 2018 20
SYN Cache Side Channel V, SYN • Fill up cache (SYN cookies) V, SYN : V e • Different source ports c r u o S SYN Exists: RST S/A ? �������� Zhang & Et. Al 2015 21
SYN Cache Side Channel V, SYN • Fill up cache (SYN cookies) V, SYN Source: A • Different source ports SYN SYN Cookie �������� Zhang & Et. Al 2015 22
How to leverage? • IPID: • Global --> straight forward • Per-Dest/2048 --> impossible/hard • Timestamps --> straight forward • Shared State --> overwhelm and check 25
My contributions • Check for side-channel presence • Alexa Top 1000 26
My contributions • Check for side-channel presence • Alexa Top 1000 • ICMP/TCP/HTTP timestamps • TCP traceroute (termination location) 27
Tools • Scapy • Raw pcaps • Packet manipulation • Requests • HTTP • Ray • Distributed programming (scanning) 28
(a few) Results • 986 responses • 98% had TCP responses • 60% had TCP timestamps • 85% had HTTP responses • 0 ICMP 29
(a few) Results 30
(a few) Results 31
(a few) Results ICMP … TCP 32
(a few) Results ICMP ICMP … TCP TCP 33
(a few) Results ICMP ICMP … TCP 34
(a few) Results 35
Lessons Learned • Don't underestimate the kernel Kernel Scapy TCP NIC 36
Lessons Learned • Don't underestimate the kernel • ISPs can be annoying
Lessons Learned • Don't underestimate the kernel • ISPs can be annoying • I don't get IPv6 • Google IPv6 DNS + IPv6 ISP support = No connection?! 38
Experiments Next Steps • Existence of Challenge ACKs • IPv6 reachability • HTTP timestamp analysis 39
Validation Next Steps • Simple GCP Load Balanced Web Server • Easy ground-truth • In-the-wild validation server: mw1325.eqiad.wmnet 41
Questions? Thanks for listening! 42
Recommend
More recommend
