Rollover and Die? George Michaelson, APNIC Geoff Huston, APNIC - - PowerPoint PPT Presentation

▶
rollover and die
SMART_READER_LITE
LIVE PREVIEW

Rollover and Die? George Michaelson, APNIC Geoff Huston, APNIC - - PowerPoint PPT Presentation

May 08, 2023 340 likes •954 views

Rollover and Die? George Michaelson, APNIC Geoff Huston, APNIC Patrik Wallstrm, IIS Roy Arends, Nominet UK 1 Were under attack!!! On the 16th of december, traffic more than doubled 2 DNSKEY amplification attack 3 DNSKEY response size

slide-1
SLIDE 1

Rollover and Die?

George Michaelson, APNIC Geoff Huston, APNIC Patrik Wallström, IIS Roy Arends, Nominet UK

1

slide-2
SLIDE 2

We’re under attack!!!

2

On the 16th of december, traffic more than doubled

slide-3
SLIDE 3

DNSKEY amplification attack

3

slide-4
SLIDE 4

DNSKEY response size

4

Response size: 990 Bytes Query rate: 2000 qps Additional load

15.8 Mbps

slide-5
SLIDE 5

Who does this?

5

slide-6
SLIDE 6

Who does this?

5

slide-7
SLIDE 7

What was special about the 16th?

6

slide-8
SLIDE 8

What was special about the 16th?

7

slide-9
SLIDE 9

Hanlon’s razor

8

Never attribute to malice that which can

be explained by stupidity.

slide-10
SLIDE 10

Why so many clients?

  • Fedora bug report 17th Jan 2010

– (1 month after the roll)

  • operator reports getting 240.000 log entries in 24hr

– “no valid key”

  • dnssec-conf tool contained a hard-configured trust anchor file

– obsolete after the 16th.

9

slide-11
SLIDE 11

What was special about the 16th?

10

slide-12
SLIDE 12

What was special about the 16th?

10

what a great lesson

Randy Bush’s response

slide-13
SLIDE 13

Current load for in-addr.arpa

11

getting better, below 1000 qps right now But decline not fast enough before new roll

slide-14
SLIDE 14

The Load Projection

12

slide-15
SLIDE 15

The Load Projection

12

remove

slide-16
SLIDE 16

The Load Projection

12

remove remove

slide-17
SLIDE 17

The Load Projection

12

remove remove add

slide-18
SLIDE 18

Was this a one off event?

13

Sweden, june 2009

slide-19
SLIDE 19

Was this a one off event?

14

Sweden, june 2008

slide-20
SLIDE 20

Was this a one off event?

14

Sweden, june 2008 1st resolver fix

slide-21
SLIDE 21

Was this a one off event?

14

Sweden, june 2008 1st resolver fix 2nd resolver fix

slide-22
SLIDE 22

Why so many Queries?

  • Resolvers are supposed to cache dnskey
  • Even when those are bad
  • Resolvers should be nice, not aggressive
  • So many resolvers, so few servers

15

slide-23
SLIDE 23

Why so many Queries?

  • Bind bug in all versions
  • Depth First Search (DFS) problem
  • Chain of trust validation:

16

slide-24
SLIDE 24

Why so many Queries?

  • Bind bug in all versions
  • Depth First Search (DFS) problem
  • Chain of trust validation:

16

TA root

slide-25
SLIDE 25

Why so many Queries?

  • Bind bug in all versions
  • Depth First Search (DFS) problem
  • Chain of trust validation:

16

TA root A SIG www.dnssec.se

slide-26
SLIDE 26

Why so many Queries?

  • Bind bug in all versions
  • Depth First Search (DFS) problem
  • Chain of trust validation:

16

KEY dnssec.se TA root A SIG

slide-27
SLIDE 27

Why so many Queries?

  • Bind bug in all versions
  • Depth First Search (DFS) problem
  • Chain of trust validation:

16

KEY dnssec.se DS se TA root A SIG

slide-28
SLIDE 28

Why so many Queries?

  • Bind bug in all versions
  • Depth First Search (DFS) problem
  • Chain of trust validation:

16

KEY KEY dnssec.se DS se TA root A SIG

slide-29
SLIDE 29

Why so many Queries?

  • Bind bug in all versions
  • Depth First Search (DFS) problem
  • Chain of trust validation:

16

KEY KEY dnssec.se DS se DS root TA root A SIG

slide-30
SLIDE 30

Why so many Queries?

  • Bind bug in all versions
  • Depth First Search (DFS) problem
  • Chain of trust validation:

16

KEY KEY KEY dnssec.se DS se DS root TA root A SIG

slide-31
SLIDE 31

Why so many Queries?

  • Bind bug in all versions
  • Depth First Search (DFS) problem
  • Chain of trust validation:

16

KEY KEY KEY dnssec.se DS se DS root TA root A SIG

slide-32
SLIDE 32

Why so many Queries?

  • Bind bug in all versions
  • Depth First Search (DFS) problem
  • Chain of trust validation:

16

KEY 3 3 13 13 20 20 KEY KEY dnssec.se DS se DS root TA root A SIG

slide-33
SLIDE 33

Why so many Queries?

  • Bind bug in all versions
  • Depth First Search (DFS) problem
  • Chain of trust validation:

16

KEY 3 3 13 13 20 20 KEY KEY dnssec.se DS se DS root TA root A SIG

slide-34
SLIDE 34

Why so many Queries?

  • Bind bug in all versions
  • Depth First Search (DFS) problem
  • Chain of trust validation:

16

KEY 3 3 13 13 20 20 KEY KEY dnssec.se DS se DS root TA root A SIG 3 * 3 * 13 * 13 * 20 * 20 = 608400 queries

slide-35
SLIDE 35

ISC

  • Reported the depth first search bug on februari 8th

17

slide-36
SLIDE 36

ISC

  • Reported the depth first search bug on februari 8th
  • Acknowledged the problem

– fundamental fix, needs thorough testing.

17

slide-37
SLIDE 37

ISC

  • Reported the depth first search bug on februari 8th
  • Acknowledged the problem

– fundamental fix, needs thorough testing.

  • released BIND 9.7.0 with bug

– first version that can validate the root – “Exercise caution”, ignores the lame DS issue

17

slide-38
SLIDE 38

ISC

  • Reported the depth first search bug on februari 8th
  • Acknowledged the problem

– fundamental fix, needs thorough testing.

  • released BIND 9.7.0 with bug

– first version that can validate the root – “Exercise caution”, ignores the lame DS issue

  • released BIND 9.6.2 with bug

– root-validation back ported, no 5011

  • tick tock

17

slide-39
SLIDE 39

ISC

  • Reported the depth first search bug on februari 8th
  • Acknowledged the problem

– fundamental fix, needs thorough testing.

  • released BIND 9.7.0 with bug

– first version that can validate the root – “Exercise caution”, ignores the lame DS issue

  • released BIND 9.6.2 with bug

– root-validation back ported, no 5011

  • tick tock
  • Announced a patch as soon as possible.

– still waiting – folks are deploying 9.7.0 and 9.6.2 right now

17

slide-40
SLIDE 40

The Perfect Storm

  • DNSSEC deployment at root (DURZ)

– guess what: lame trust-anchor, don’t configure

18

slide-41
SLIDE 41

The Perfect Storm

  • DNSSEC deployment at root (DURZ)

– guess what: lame trust-anchor, don’t configure

18

slide-42
SLIDE 42

The Perfect Storm

19

slide-43
SLIDE 43

The Perfect Storm

  • No automatic trust anchor roll (5011)

– 9.7.0 implementation is buggy

  • promised fix in 9.7.1

– 9.6.2 not planned

19

slide-44
SLIDE 44

The Perfect Storm

  • No automatic trust anchor roll (5011)

– 9.7.0 implementation is buggy

  • promised fix in 9.7.1

– 9.6.2 not planned

  • DLV mishaps:

– DLV registry promiscuously scrapes TLD keys

  • Just another chain of trust

– .PR rolled its key

  • was unavailable to DLV users for days
  • caused a major packet storm

19

slide-45
SLIDE 45

The Perfect Storm

  • Multiple trust anchor problem

– TLD Trust Anchors trump Root Trust Anchor

  • stale TLD Trust Anchor trumps valid Root Trust Anchor

20

slide-46
SLIDE 46

The Perfect Storm

  • Multiple trust anchor problem

– TLD Trust Anchors trump Root Trust Anchor

  • stale TLD Trust Anchor trumps valid Root Trust Anchor
  • Doom scenario:

– TLD registers DS in root – new policy: don’t announce rolls, depend on root

  • That is the way NS records works as well

– Operators won’t update TLD trust anchor anymore

  • Why would they, they’ve configured root trust-anchor

20

slide-47
SLIDE 47

A Series Of Unfortunate Events

21

slide-48
SLIDE 48

A Series Of Unfortunate Events

  • buggy software

21

slide-49
SLIDE 49

A Series Of Unfortunate Events

  • buggy software
  • DNSSEC @ root

21

slide-50
SLIDE 50

A Series Of Unfortunate Events

  • buggy software
  • DNSSEC @ root
  • multiple trust anchor problem

21

slide-51
SLIDE 51

A Series Of Unfortunate Events

  • buggy software
  • DNSSEC @ root
  • multiple trust anchor problem
  • no 5011 deployment

21

slide-52
SLIDE 52

A Series Of Unfortunate Events

  • buggy software
  • DNSSEC @ root
  • multiple trust anchor problem
  • no 5011 deployment
  • opportunistic DLV scraping

21

slide-53
SLIDE 53

A Series Of Unfortunate Events

  • buggy software
  • DNSSEC @ root
  • multiple trust anchor problem
  • no 5011 deployment
  • opportunistic DLV scraping
  • Frequent Rollover Syndrome

– rolling rolling rolling, keep them DNSKEYs rolling.

21

slide-54
SLIDE 54

Frequent Rollover Syndrome

  • Advice seems to be:

– roll the key as often as you can – Some roll twice a year, some roll monthly

22

slide-55
SLIDE 55

Frequent Rollover Syndrome

  • Advice seems to be:

– roll the key as often as you can – Some roll twice a year, some roll monthly

  • Advice is completely misguided:

– too many sigs do not leak the key. – Intention is to mitigate a compromised key fallout – but there is no perfect forward security

22

slide-56
SLIDE 56

Frequent Rollover Syndrome

  • Advice seems to be:

– roll the key as often as you can – Some roll twice a year, some roll monthly

  • Advice is completely misguided:

– too many sigs do not leak the key. – Intention is to mitigate a compromised key fallout – but there is no perfect forward security

  • If a key can be compromised in 1 year, it can be

compromised in 6 months for twice the cost

22

slide-57
SLIDE 57

Frequent Rollover Syndrome

  • Advice seems to be:

– roll the key as often as you can – Some roll twice a year, some roll monthly

  • Advice is completely misguided:

– too many sigs do not leak the key. – Intention is to mitigate a compromised key fallout – but there is no perfect forward security

  • If a key can be compromised in 1 year, it can be

compromised in 6 months for twice the cost

  • Other reasons: educate operators, exercise procedures

– all irrelevant, never mess with a critical production system

22

slide-58
SLIDE 58

Solution

  • Fix the buggy software already

– stop releasing versions that have problems – (Help fund BIND-10)

  • Don’t roll keys (too often)

– be practical

  • Do not endorse configuration of trust-anchors when parent is

signed. – no 5011, no web-page with listed keys, no DLV, no ITAR – Manage all through a signed parent.

  • When parent is not signed:

– Use proper 5011. Use ISC’s DLV.

slide-59
SLIDE 59

Questions ? Remarks ? Observations ?

http://www.potaroo.net/ispcol/2010-02/rollover.html

Thanks to Anand Buddhdev Patrik Wallström George Michaelson Geoff Huston David Conrad Folks at ISC

slide-60
SLIDE 60

Questions ? Remarks ? Observations ?

http://www.potaroo.net/ispcol/2010-02/rollover.html

Thanks to Anand Buddhdev Patrik Wallström George Michaelson Geoff Huston David Conrad Folks at ISC Question: If you’ve deployed DNSSEC and rolled your (ksk) key, look at the stats around that period, and (pretty) please report them back to us.