Tier-1s break Anycast DNS Zhihao Li, Neil Spring D-Root: - PowerPoint PPT Presentation

Tier-1’s break Anycast DNS Zhihao Li, Neil Spring

D-Root: 199.7.91.13 • 111 Anycast replicas: • 19 global (red): advertised without restriction • 92 local (black): advertised one hop in BGP

Anycast • Mental model: • Packets sent to an anycast address travel to the nearest* replica, subject to global/local constraints. • More replicas should mean lower latency, better distribution, reliability against denial-of-service attacks.

Anycast • Mental model: • Packets sent to an anycast address travel to the nearest* replica, subject to global/local constraints. • More replicas should mean lower latency, better distribution, reliability against denial-of-service attacks.

Reality • 4-5x optimal delay (to a local), 2x expected (nearest global) Actual average distance Distance to nearest global replica Distance to nearest replica 2000 Average miles per query traveled 1500 1000 500 0 Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec 2015 - 2016

Reality • Despite doubling the number of (local) replicas Average miles per query traveled Actual average distance 2000 Distance to nearest global replica Distance to nearest replica 1500 1000 500 0 Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec # Replicas 100 80 All 60 Global 40 20 0 Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec 2015 - 2016

Reality • 80% of queries should take under 1000 miles (16ms RTT) • 50% are traveling farther. 100 80 60 CDF 40 20 Distance to nearest replica Distance to nearest global replica Actual average distance 0 0 2000 4000 6000 8000 10000 Miles per query traveled, Oct 1 2016

Reality • Same data, first week in Oct 2016, log scale x-axis. • Even when there’s a global replica in your city… 100 80 60 CDF 40 20 Distance to nearest replica Distance to nearest global replica Actual average distance 0 1 10 100 1000 10000 Miles per query traveled, Oct 1 2016

How do we fix it? • More sites? • More peerings? • Better policies? • Make local replicas global? • What if ISPs chose cleverly from their providers? • Pathological behavior must be atypical, right? • Is it even broken?

Similar observations • Anycast Latency: How Many Sites Are Enough? Schmidt, Heidemann, Kuipers • Used Atlas probes (not traces) to look at C, F, K, L root. • More sites doesn’t correlate with lower latency • Making local sites global didn’t help K

It’s the tier-1’s (I think)

Source (resolver) location • For addresses originated by Tier 1’s, what is their nearest replica. Intensity by query volume. mcva cpmd sewa paca bbca abva nyny zuch sgsg hkcn atga laca louk ffde tojp mifl viat chil 1 NTT TATA SPRINTLINK ZAYO GTT XO ATT Client portion TELEFONICA KPN 0.5 SEABONE LEVEL3 DTAG OPENTRANSIT COGENT TELIANET QWEST UUNET 0 sewa paca bbca laca chil atga mifl abva mcva cpmd nyny louk zuch ffde viat sgsg hkcn tojp Global replicas

Request destination • For addresses originated by Tier 1’s, what is their chosen replica. Intensity by query volume. mcva cpmd sewa paca bbca abva nyny zuch sgsg hkcn atga laca louk ffde tojp mifl viat chil 1 NTT TATA SPRINTLINK ZAYO GTT XO ATT Query portion TELEFONICA KPN 0.5 SEABONE LEVEL3 DTAG OPENTRANSIT COGENT TELIANET QWEST UUNET 0 sewa paca bbca laca chil atga mifl abva mcva cpmd nyny louk zuch ffde viat sgsg hkcn tojp Global replicas

Would you like to see them again?

Often McLean, VA. • Traffic from tier-1 address space can arrive on other replicas, but generally does not. cpmd cpmd mcva mcva sewa sewa paca bbca abva paca bbca abva nyny zuch sgsg hkcn nyny zuch sgsg hkcn atga atga laca louk laca louk ffde tojp ffde mifl viat mifl viat tojp chil chil 1 NTT NTT TATA TATA SPRINTLINK SPRINTLINK ZAYO GTT GTT XO ATT ATT Query portion TELEFONICA TELEFONICA KPN 0.5 SEABONE SEABONE LEVEL3 LEVEL3 DTAG OPENTRANSIT OPENTRANSIT COGENT COGENT TELIANET TELIANET QWEST QWEST UUNET UUNET 0 sewa paca bbca laca chil atga mifl abva mcva cpmd nyny louk zuch ffde viat sgsg hkcn tojp sewa paca bbca laca chil atga mifl abva mcva cpmd nyny louk zuch ffde viat sgsg hkcn tojp Global replicas Global replicas

Could just be us.

Could just be us. No.

Could just be us. No. This time using RIPE Atlas data, same Oct 1, 2016. Now counting vantage points whose queries transit a tier-1 (since we have traceroutes) instead of queries received.

A-Root • Better. Notably, DTAG sends to London, not Frankfurt. hkg nyc hkg nyc lon lax lon fra lax fra 1 NTT COGENT TELIANET TATA OPENTRANSIT SEABONE TELEFONICA Query portion LEVEL3 GTT 0.5 ZAYO XO LGI LGI ATT UUNET DTAG KPN QWEST SPRINTLINK 0 lax nyc lon fra hkg lax nyc lon fra hkg Global replicas Global replicas

C-Root • The best at matching tier-1-carried queries to a nearby site. mad mad ord par ord par iad bts iad bts lax lax fra fra jfk jfk 1 COGENT COGENT TELIANET TELIANET NTT NTT XO LEVEL3 LEVEL3 UUNET UUNET SPRINTLINK SPRINTLINK Query portion TATA TATA ATT ATT 0.5 GTT GTT QWEST QWEST OPENTRANSIT OPENTRANSIT ZAYO LGI LGI SEABONE SEABONE TELEFONICA TELEFONICA DTAG KPN KPN 0 lax ord iad jfk mad par fra bts lax ord iad jfk mad par fra bts Global replicas Global replicas

E-Root • Similar to D in that northern Virginia is preferred, despite Paris, Frankfurt, London query sources. pao qpg pao qpg mia cdg syd mia cdg syd bur ord bur ord sfo iad lga sfo iad lga fra fra lhr lhr atl atl 1 NTT TATA XO ATT COGENT GTT Query portion ZAYO LGI LGI 0.5 SPRINTLINK OPENTRANSIT TELEFONICA LEVEL3 LEVEL3 SEABONE TELIANET UUNET DTAG KPN 0 pao sfo bur ord atl mia iad lga lhr cdg fra qpg syd pao sfo bur ord atl mia iad lga lhr cdg fra qpg syd Global replicas Global replicas

F-Root • Mostly European RIPE probes served by Chicago despite an Amsterdam replica. ams ams pao pao ord ord lga lga atl atl 1 SEABONE SEABONE LGI LGI LEVEL3 LEVEL3 NTT NTT TELIANET TELIANET TELEFONICA TELEFONICA ZAYO Query portion XO UUNET UUNET 0.5 TATA TATA SPRINTLINK SPRINTLINK ATT ATT GTT GTT DTAG COGENT COGENT KPN KPN OPENTRANSIT OPENTRANSIT QWEST QWEST 0 pao ord atl lga ams pao ord atl lga ams Global replicas Global replicas

OPENTRANSIT TELEFONICA SEABONE TELIANET COGENT • Still picking just one server, not typically the server with the QWEST LEVEL3 UUNET DTAG ZAYO TATA KPN GTT NTT ATT LGI XO most clients. sox sox chi chi mia mia ash ash was was mtv mtv poa poa lon lon par par amx amx lnx lnx gva gva dex dex mln mln osl osl gur gur wie wie sth sth lul lul rig rig Global replicas tll tll fix fix rox rox jnb jnb rwx rwx spb spb ukx ukx ank ank yan yan bah bah qtr qtr dbi dbi khi khi mum mum kat kat thi thi bkx bkx ula ula sin sin hkx hkx prt prt i-Root bnx bnx mix mix tai tai tok tok vux vux LEVEL3 wel wel LGI sox sox chi chi mia mia ash ash was was mtv mtv poa poa lon lon par par amx amx lnx lnx gva gva dex dex mln mln osl osl gur gur wie wie sth sth lul lul rig rig Global replicas tll tll fix fix rox rox jnb jnb rwx rwx spb spb ukx ukx ank ank yan yan bah bah qtr qtr dbi dbi khi khi mum mum kat kat thi thi bkx bkx ula ula sin sin hkx hkx prt prt bnx bnx mix mix tai tai tok tok vux vux wel wel 0 0.5 1

OPENTRANSIT TELEFONICA SPRINTLINK SEABONE TELIANET • Fairly good, although preference for “tpe” despite no clients. COGENT QWEST LEVEL3 UUNET DTAG ZAYO TATA KPN GTT NTT ATT LGI XO yvr yvr sfo sfo sea sea dfw dfw eau eau ord ord btl btl atl atl sjo sjo ilg ilg mia mia iad iad cbb cbb sju sju rao rao aju aju cpv cpv jpa jpa rkv rkv mad mad lgw lgw Global replicas par par ams ams lju lju arn arn cpt cpt waw waw sof sof kun kun rix rix tll tll led led wil wil evn evn kwi kwi bom bom mle mle del del dac dac dmk dmk iph iph sin sin J-Root pek pek tpe tpe OPENTRANSIT tbh tbh TELEFONICA SPRINTLINK sel sel SEABONE TELIANET hnd hnd COGENT gum QWEST LEVEL3 gum UUNET mel TATA mel GTT NTT ATT wlg LGI wlg yvr yvr sfo sfo sea sea dfw dfw eau eau ord ord btl btl atl atl sjo sjo ilg ilg mia mia iad iad cbb cbb sju sju rao rao aju aju cpv cpv jpa jpa rkv rkv mad mad lgw lgw Global replicas par par ams ams lju lju arn arn cpt cpt waw waw sof sof kun kun rix rix tll tll led led wil wil evn evn kwi kwi bom bom mle mle del del dac dac dmk dmk iph iph sin sin pek pek tpe tpe tbh tbh sel sel hnd hnd gum gum mel mel wlg wlg 0 0.5 1