WITH VERFPLOETER WOUTER B. DE VRIES , RICARDO DE O. SCHMIDT, WES - PowerPoint PPT Presentation

BROAD AND LOAD-AWARE ANYCAST MAPPING WITH VERFPLOETER WOUTER B. DE VRIES , RICARDO DE O. SCHMIDT, WES HARDAKER, JOHN HEIDEMANN, PIETER-TJERK DE BOER AND AIKO PRAS London - November 3, 2017

ACM Internet Measurement Conference 2017 INTRODUCTION Anycast site User 2

ACM Internet Measurement Conference 2017 INTRODUCTION Anycast site User Users are routed to a (potentially) nearby site 2

ACM Internet Measurement Conference 2017 INTRODUCTION The set of clients that reach a site define that sites catchment Anycast site User Users are routed to a (potentially) nearby site 2

ACM Internet Measurement Conference 2017 INTRODUCTION The set of clients that reach a site define that sites catchment Catchments are often messy and not divided into neat geographic areas Anycast site User Users are routed to a (potentially) nearby site 2

ACM Internet Measurement Conference 2017 INTRODUCTION Peering agreements, Transits and BGP policies influence the routing process. You don’t know where a client from a certain prefix will be routed until he is routed there. 3

ACM Internet Measurement Conference 2017 INTRODUCTION Peering agreements, Transits and BGP policies influence the routing process. Suboptimal routing can lead to severely You don’t know where a client from a certain prefix will degraded performance, in terms of RTT, be routed until he is routed there. load balancing and resilience 3

ACM Internet Measurement Conference 2017 DETERMINING CATCHMENT - TODAY • Analyze log files • Only if you have them • Use active probing (e.g. RIPE Atlas) • Requires VPs at many locations 4

ACM Internet Measurement Conference 2017 DETERMINING CATCHMENT - TODAY • Analyze log files • Only if you have them • Use active probing (e.g. RIPE Atlas) • Requires VPs at many locations NL: 563 VPs 4

ACM Internet Measurement Conference 2017 DETERMINING CATCHMENT - TODAY • Analyze log files • Only if you have them • Use active probing (e.g. RIPE Atlas) • Requires VPs at many locations NL: 563 VPs Brazil: 63 VPs China: 19 VPs 4

ACM Internet Measurement Conference 2017 GOALS • Develop a method to accurately map anycast catchments • Without requiring input from users • “Calibrate” the anycast catchment to the actual client base of the service 5

ACM Internet Measurement Conference 2017 VERFPLOETER: METHODOLOGY ACTIVE MEASUREMENT WITH “PASSIVE” VANTAGE POINTS Vantage points Anycast sites 6

ACM Internet Measurement Conference 2017 VERFPLOETER: METHODOLOGY ACTIVE MEASUREMENT WITH “PASSIVE” VANTAGE POINTS Vantage points Send out ICMP Echo Requests Anycast sites 6

ACM Internet Measurement Conference 2017 VERFPLOETER: METHODOLOGY ACTIVE MEASUREMENT WITH “PASSIVE” VANTAGE POINTS Vantage points ICMP Echo Replies are routed towards “nearest” anycast site Anycast sites 6

ACM Internet Measurement Conference 2017 VERFPLOETER: METHODOLOGY ACTIVE MEASUREMENT WITH “PASSIVE” VANTAGE POINTS Vantage points This defines the catchment that each VP belongs to Anycast sites 6

ACM Internet Measurement Conference 2017 WHAT ARE THE PASSIVE VPS ICMP-responding IPv4 or IPv6 addresses We use a recent IPv4 hitlist from USC/ISI: https://ant.isi.edu/datasets/ip_hitlists/ 1 IPv4 address per /24 block: 14.7 million addresses 6.3 million after removing those in which there was never a responding host 1.0.0.0/24 – 1.0.0.57 1.0.1.0/24 – 1.0.1.85 1.0.2.0/24 – 1.0.2.4 1.0.3.0/24 – 1.0.3.204 7

ACM Internet Measurement Conference 2017 USING VERFPLOETER: TWO CASE STUDIES • The B Root DNS • Anycast since 1 st of May 2017 • 2 anycast sites • Our anycast testbed • 9 anycast sites Measured catchment using both Atlas and Verfploeter 8

ACM Internet Measurement Conference 2017 FROM ATLAS TO VERFPLOETER B-ROOT CASE STUDY Geolocated using MaxMind GeoLite2 Atlas Verfploeter /24 blocks seen 8,677 (of 9083) 3,786,907 (of 6,877,175) Geolocatable 8,677 3,786,229 Unique 2,079 3,606,300 9

ACM Internet Measurement Conference 2017 FROM ATLAS TO VERFPLOETER B-ROOT CASE STUDY Note scale 10

ACM Internet Measurement Conference 2017 FROM ATLAS TO VERFPLOETER B-ROOT CASE STUDY Note scale Each “dot” is a pie chart 10

ACM Internet Measurement Conference 2017 FROM ATLAS TO VERFPLOETER B-ROOT CASE STUDY Note scale Atlas scale 11

ACM Internet Measurement Conference 2017 FROM ATLAS TO VERFPLOETER B-ROOT CASE STUDY Note scale Atlas scale Atlas Verfploeter 11

ACM Internet Measurement Conference 2017 FROM ATLAS TO VERFPLOETER B-ROOT CASE STUDY Note scale Atlas scale Atlas Verfploeter 11

ACM Internet Measurement Conference 2017 FROM ATLAS TO VERFPLOETER B-ROOT CASE STUDY Note scale Atlas scale Atlas Verfploeter 11

ACM Internet Measurement Conference 2017 MEASURING AN ANYCAST TESTBED: TANGLED A more diverse ground for verfploeter measurements: Our 9-site anycast testbed Location Host Upstream AU Sydney Vultr AS20473 FR Paris Vultr AS20473 JP Tokyo WIDE AS2500 NL Enschede Univ. of Twente AS1103 UK London Vultr AS20473 US Washington USC/ISI AS1972 US Miami Florida Int. Univ. AS20080 BR Sao Paulo Florida Int. Univ. AS1251 DK Copenhagen DK-Hostmaster AS39389 12

ACM Internet Measurement Conference 2017 MEASURING AN ANYCAST TESTBED: TANGLED A more diverse ground for verfploeter measurements: Our 9-site anycast testbed Location Location Host Host Upstream Upstream AU AU Sydney Sydney Vultr Vultr AS20473 AS20473 FR FR Paris Paris Vultr Vultr AS20473 AS20473 JP JP Tokyo Tokyo WIDE WIDE AS2500 AS2500 Shared upstream NL NL Enschede Enschede Univ. of Twente Univ. of Twente AS1103 AS1103 UK UK London London Vultr Vultr AS20473 AS20473 US US Washington Washington USC/ISI USC/ISI AS1972 AS1972 US US Miami Miami Florida Int. Univ. Florida Int. Univ. AS20080 AS20080 BR BR Sao Paulo Sao Paulo Florida Int. Univ. Florida Int. Univ. AS1251 AS1251 DK DK Copenhagen Copenhagen DK-Hostmaster DK-Hostmaster AS39389 AS39389 12

ACM Internet Measurement Conference 2017 MEASURING AN ANYCAST TESTBED: TANGLED ATLAS VS VERFPLOETER 13

ACM Internet Measurement Conference 2017 MEASURING AN ANYCAST TESTBED: TANGLED ATLAS VS VERFPLOETER Atlas Verfploeter 13

ACM Internet Measurement Conference 2017 MEASURING AN ANYCAST TESTBED: TANGLED ATLAS VS VERFPLOETER Atlas Verfploeter 13

ACM Internet Measurement Conference 2017 CALIBRATING CATCHMENT Not all measured prefixes are equal Some will send no traffic, others a lot e.g. due to NAT or ISP DNS Forwarders 14

ACM Internet Measurement Conference 2017 CALIBRATING CATCHMENT Obtained from (historic) log files or from a service with a similar Obtained with Verfploeter user base Prefix Load Prefix Site 1.0.0.0/24 10 Q/s 1.0.0.0/24 LAX RIGHT 1.0.1.0/24 0 Q/s 1.0.1.0/24 LAX JOIN 1.0.2.0/24 0 Q/s 1.0.2.0/24 LAX 1.0.3.0/24 15 Q/s 1.0.3.0/24 MIA 15

ACM Internet Measurement Conference 2017 CALIBRATING CATCHMENT Prefix Load Prefix Site 1.0.0.0/24 10 Q/s 1.0.0.0/24 LAX RIGHT 1.0.1.0/24 0 Q/s 1.0.1.0/24 LAX JOIN 1.0.2.0/24 0 Q/s 1.0.2.0/24 LAX 1.0.3.0/24 15 Q/s 1.0.3.0/24 MIA Prefix Site Load 1.0.0.0/24 LAX 10 Q/s 1.0.1.0/24 LAX 0 Q/s 1.0.2.0/24 LAX 0 Q/s 1.0.3.0/24 MIA 15 Q/s 15

ACM Internet Measurement Conference 2017 CALIBRATING CATCHMENT Prefix Load Prefix Site 1.0.0.0/24 10 Q/s 1.0.0.0/24 LAX RIGHT SITE Fraction of traffic 1.0.1.0/24 0 Q/s 1.0.1.0/24 LAX JOIN LAX 0.40 1.0.2.0/24 0 Q/s 1.0.2.0/24 LAX MIA 0.60 1.0.3.0/24 15 Q/s 1.0.3.0/24 MIA Prefix Site Load 1.0.0.0/24 LAX 10 Q/s 1.0.1.0/24 LAX 0 Q/s 1.0.2.0/24 LAX 0 Q/s 1.0.3.0/24 MIA 15 Q/s 15

ACM Internet Measurement Conference 2017 CALIBRATING CATCHMENT AT B-ROOT 1. Record traffic from unicast B-root (prior to anycast deployment) Date Site Queries/day Queries/sec 2017-04-12 LAX (Unicast) 2.34G 27.1k 2017-05-15 Both 2.20G 25.4k LAX 1.78G 20.6k MIA 0.407G 4.71k 2. Run Verfploeter on anycast B-root 16

ACM Internet Measurement Conference 2017 CALIBRATING CATCHMENT AT B-ROOT RECORDED TRAFFIC COMBINED WITH VERFPLOETER Before calibration Date Method Measurement % to LAX May 15 th RIPE Atlas 9682 VPs 82.4% May 15 th Verfploeter 3.923M /24s 87.8% May 15 th Verfploeter x Load 81.6% May 15 th Actual load 2.188G q/day 81.4% After calibration 17

ACM Internet Measurement Conference 2017 CALIBRATING CATCHMENT AT B-ROOT Verfploeter allows you to experiment with various configurations and how that affects the load on the sites of your service 18

ACM Internet Measurement Conference 2017 INSIDE AUTONOMOUS SYSTEMS Anycast site User 19

ACM Internet Measurement Conference 2017 INSIDE AUTONOMOUS SYSTEMS Is one vantage point per AS enough? Anycast site User 19

ACM Internet Measurement Conference 2017 INSIDE AUTONOMOUS SYSTEMS NUMBER OF SITES VERSUS NUMBER OF ANNOUNCED PREFIXES We see more than 1 site from around 12.7% of all ASes 20