[PPT] - Tr I nc: Small Trusted Hardware for Large Distributed Systems Dave PowerPoint Presentation

SLIDE 1

TrInc: Small Trusted Hardware for Large Distributed Systems

John R. Douceur Jacob R. Lorch Thomas Moscibroda Microsoft Research Dave Levin University of Maryland

SLIDE 2

TrInc – NSDI 2009 Dave Levin

Trust in distributed systems

2

Selfish Participants Malicious Participants

SLIDE 3

TrInc – NSDI 2009 Dave Levin

Trust in distributed systems

2

Selfish Participants Malicious Participants

A participant “equivocates” by sending conflicting messages to others

Powerful tool: Equivocation

SLIDE 4

TrInc – NSDI 2009 Dave Levin

Equivocation is common and powerful

3

Byz. Generals

SLIDE 5

TrInc – NSDI 2009 Dave Levin

Equivocation is common and powerful

3

Byz. Generals

“Retreat” “Advance”

SLIDE 6

TrInc – NSDI 2009 Dave Levin

Equivocation is common and powerful

3

Byz. Generals

“Retreat” “Advance”

Voting

SLIDE 7

TrInc – NSDI 2009 Dave Levin

Equivocation is common and powerful

3

Byz. Generals

“Retreat” “Advance” Tally w/o ’s vote “Counted your vote”

Voting

SLIDE 8

TrInc – NSDI 2009 Dave Levin

Equivocation is common and powerful

3

Byz. Generals

“Retreat” “Advance” Tally w/o ’s vote “Counted your vote”

Voting BitTorrent

SLIDE 9

TrInc – NSDI 2009 Dave Levin

Equivocation is common and powerful

3

Byz. Generals

“Retreat” “Advance” Tally w/o ’s vote “Counted your vote” “I don’t have piece 5” “I have piece 5”

Voting BitTorrent

SLIDE 10

TrInc – NSDI 2009 Dave Levin

Equivocation is common and powerful

3

Byz. Generals

“Retreat” “Advance” Tally w/o ’s vote “Counted your vote” “I don’t have piece 5” “I have piece 5”

Voting BitTorrent

Auctions Leader election Trusted logs Digital cash Version control Online games DHTs soBGP

SLIDE 11

TrInc – NSDI 2009 Dave Levin

f malicious users
If completely untrusted,

3f+1 users needed for consensus [Lamport et al, 1982]

Equivocation is common and powerful

3

Byz. Generals

“Retreat” “Advance”

SLIDE 12

TrInc – NSDI 2009 Dave Levin

If users cannot equivocate, only

2f+1 users are needed [Chun et al, 2007]

f malicious users
If completely untrusted,

3f+1 users needed for consensus [Lamport et al, 1982]

Equivocation is common and powerful

3

Byz. Generals

“Retreat” “Advance”

SLIDE 13

TrInc – NSDI 2009 Dave Levin

New design space
All participants have a

trusted component

Enter Trusted Hardware

4

Equivocation can be rendered impossible with trusted hardware

SLIDE 14

TrInc – NSDI 2009 Dave Levin

New design space
All participants have a

trusted component

Enter Trusted Hardware

4

Equivocation can be rendered impossible with trusted hardware

SLIDE 15

TrInc – NSDI 2009 Dave Levin

New design space
All participants have a

trusted component

Enter Trusted Hardware

4

Equivocation can be rendered impossible with trusted hardware

SLIDE 16

TrInc – NSDI 2009 Dave Levin

New design space
All participants have a

trusted component

Enter Trusted Hardware

4

Equivocation can be rendered impossible with trusted hardware

To be practical, the hardware

must be small

Ubiquity via low cost
Tamper-resilient
Easier to verify a small TCB

SLIDE 17

TrInc – NSDI 2009 Dave Levin

3 2 1

Contributions

5

TrInc – A new, practical primitive for eliminating equivocation Applications of TrInc Implementation in currently available hardware

SLIDE 18

TrInc – NSDI 2009 Dave Levin

3 2 1

Contributions

5

1

TrInc – A new, practical primitive for eliminating equivocation Applications of TrInc Implementation in currently available hardware

SLIDE 19

TrInc – NSDI 2009 Dave Levin

Motivating question

6

What is the minimal abstraction needed to make equivocation impossible?

SLIDE 20

TrInc – NSDI 2009 Dave Levin

Motivating question

6

What is the minimal abstraction needed to make equivocation impossible? A counter and a key are enough

SLIDE 21

TrInc – NSDI 2009 Dave Levin

1. Monotonically increasing counter
2. Key for signing attestations

TrInc: Trusted Incrementer

7

34

K

SLIDE 22

TrInc – NSDI 2009 Dave Levin

1. Monotonically increasing counter
2. Key for signing attestations

TrInc: Trusted Incrementer

7

34

Attestations bind data to counters

K

SLIDE 23

TrInc – NSDI 2009 Dave Levin

1. Monotonically increasing counter
2. Key for signing attestations

TrInc: Trusted Incrementer

7

34

Attestations bind data to counters “Bind this data to counter value 36”

K

SLIDE 24

TrInc – NSDI 2009 Dave Levin

1. Monotonically increasing counter
2. Key for signing attestations

TrInc: Trusted Incrementer

7

34

Attestations bind data to counters Attest( 36, data) “Bind this data to counter value 36”

K

SLIDE 25

TrInc – NSDI 2009 Dave Levin

1. Monotonically increasing counter
2. Key for signing attestations

TrInc: Trusted Incrementer

7

34

Attestations bind data to counters Attest( 36, data) “Bind this data to counter value 36”

K

SLIDE 26

TrInc – NSDI 2009 Dave Levin

1. Monotonically increasing counter
2. Key for signing attestations

TrInc: Trusted Incrementer

7

34 36

Attestations bind data to counters Attest( 36, data) “Bind this data to counter value 36”

K

SLIDE 27

TrInc – NSDI 2009 Dave Levin

1. Monotonically increasing counter
2. Key for signing attestations

TrInc: Trusted Incrementer

7

34 36

Attestations bind data to counters Attest( 36, data) “Bind this data to counter value 36”

K

SLIDE 28

TrInc – NSDI 2009 Dave Levin

1. Monotonically increasing counter
2. Key for signing attestations

TrInc: Trusted Incrementer

7

34 36

Attestations bind data to counters Attest( 36, data)

< 34, 36, data >K

“Bind this data to counter value 36”

K

SLIDE 29

TrInc – NSDI 2009 Dave Levin

TrInc Attestations

8

< 34, 36, data >K < 36, 36, nonce >K

SLIDE 30

TrInc – NSDI 2009 Dave Levin

TrInc Attestations

8

Can only move to a state once
“data” is forever bound to 36
There was nothing bound to 35

< 34, 36, data >K < 36, 36, nonce >K Advance attestation

“What is your current counter?”
Nonces assure freshness
There is nothing beyond 36 (yet)

Status attestation

SLIDE 31

TrInc – NSDI 2009 Dave Levin

Multiple counters

Need multiple trusted counters
Systems running concurrently
Some systems benefit from more counters

9

34 17 42

SLIDE 32

TrInc – NSDI 2009 Dave Levin

Hardware that contains ≥1 counter is a Trinket
Allocates and frees counters
Establishes session keys

Trinket

Multiple counters

Need multiple trusted counters
Systems running concurrently
Some systems benefit from more counters

9

34 17 42

SLIDE 33

TrInc – NSDI 2009 Dave Levin

TrInc is practical

Trusted Platform Module (TPM)

is ubiquitous

Has what we need
Tamper-resistance
Counters (currently 4)
Crypto
Small amount of storage
It just lacks the right interface

10

0% 20% 40% 60% 80% 100% 2005 2009 TPM Penetration

Source: IDC 2006

Desktop PCs Mobile PCs x86 Servers

SLIDE 34

TrInc – NSDI 2009 Dave Levin

3 2 1

Contributions

11

1

TrInc – A new, practical primitive for eliminating equivocation Applications of TrInc Implementation in currently available hardware

SLIDE 35

TrInc – NSDI 2009 Dave Levin

3 2 1

Contributions

11

2

TrInc – A new, practical primitive for eliminating equivocation Applications of TrInc Implementation in currently available hardware

SLIDE 36

TrInc – NSDI 2009 Dave Levin

What can TrInc do?

Trusted append-only logs
Prevent under-reporting in BitTorrent
Reduces communication in PeerReview
BFT with fewer nodes and messages
Ensure fresh data in DHTs
Prevent Sybil attacks

12

SLIDE 37

TrInc – NSDI 2009 Dave Levin

What can TrInc do?

Trusted append-only logs
Prevent under-reporting in BitTorrent
Reduces communication in PeerReview
BFT with fewer nodes and messages
Ensure fresh data in DHTs
Prevent Sybil attacks

12

SLIDE 38

TrInc – NSDI 2009 Dave Levin

What can TrInc do?

Trusted append-only logs
Prevent under-reporting in BitTorrent
Reduces communication in PeerReview
BFT with fewer nodes and messages
Ensure fresh data in DHTs
Prevent Sybil attacks

12

SLIDE 39

TrInc – NSDI 2009 Dave Levin

Implementing a trusted log in TrInc

13

Append(data):

Bind new data to the

end of the log

Lookup(sequence num):

No equivocating on what

is or is not stored

SLIDE 40

TrInc – NSDI 2009 Dave Levin

10 Implementing a trusted log in TrInc

13

Append(data):

Bind new data to the

end of the log

Lookup(sequence num):

No equivocating on what

is or is not stored

SLIDE 41

TrInc – NSDI 2009 Dave Levin

10 Implementing a trusted log in TrInc

13

< >

8,9,

< >

3,8,

Untrusted storage

< >

9,10,

Append(data):

Bind new data to the

end of the log

Lookup(sequence num):

No equivocating on what

is or is not stored

< >

9,10,

SLIDE 42

TrInc – NSDI 2009 Dave Levin

10 Implementing a trusted log in TrInc

13

< >

8,9,

< >

3,8,

append

Untrusted storage

< >

9,10,

Append(data):

Bind new data to the

end of the log

Lookup(sequence num):

No equivocating on what

is or is not stored

< >

9,10,

SLIDE 43

TrInc – NSDI 2009 Dave Levin

10 Implementing a trusted log in TrInc

13

< >

8,9,

< >

3,8,

Untrusted storage

< >

9,10,

attest(11,, )

Append(data):

Bind new data to the

end of the log

Lookup(sequence num):

No equivocating on what

is or is not stored

< >

9,10,

SLIDE 44

TrInc – NSDI 2009 Dave Levin

10 Implementing a trusted log in TrInc

13

< >

8,9,

< >

3,8,

Untrusted storage

< >

9,10,

Append(data):

Bind new data to the

end of the log

Lookup(sequence num):

No equivocating on what

is or is not stored

< >

9,10,

SLIDE 45

TrInc – NSDI 2009 Dave Levin

10 Implementing a trusted log in TrInc

13

11

< >

8,9,

< >

3,8,

Untrusted storage

< >

9,10,

Append(data):

Bind new data to the

end of the log

Lookup(sequence num):

No equivocating on what

is or is not stored

< >

9,10,

SLIDE 46

TrInc – NSDI 2009 Dave Levin

10 Implementing a trusted log in TrInc

13

11

< >

8,9,

< >

3,8,

< >

10,11,

Untrusted storage

< >

9,10,

Append(data):

Bind new data to the

end of the log

Lookup(sequence num):

No equivocating on what

is or is not stored

< >

9,10,

SLIDE 47

TrInc – NSDI 2009 Dave Levin

10 Implementing a trusted log in TrInc

13

11

< >

8,9,

< >

3,8,

< >

10,11,

Untrusted storage

< >

9,10,

Append(data):

Bind new data to the

end of the log

Lookup(sequence num):

No equivocating on what

is or is not stored

< >

9,10,

SLIDE 48

TrInc – NSDI 2009 Dave Levin

10 Implementing a trusted log in TrInc

13

11

< >

8,9,

< >

3,8,

< >

10,11,

lookup 10

Untrusted storage

< >

9,10,

Append(data):

Bind new data to the

end of the log

Lookup(sequence num):

No equivocating on what

is or is not stored

< >

9,10,

SLIDE 49

TrInc – NSDI 2009 Dave Levin

10 Implementing a trusted log in TrInc

13

11

< >

8,9,

< >

3,8,

< >

10,11,

lookup 10

Untrusted storage

< >

9,10,

Append(data):

Bind new data to the

end of the log

Lookup(sequence num):

No equivocating on what

is or is not stored

< >

9,10,

SLIDE 50

TrInc – NSDI 2009 Dave Levin

10 Implementing a trusted log in TrInc

13

11

< >

8,9,

< >

3,8,

< >

10,11,

lookup 10

Untrusted storage

< >

9,10,

Append(data):

Bind new data to the

end of the log

Lookup(sequence num):

No equivocating on what

is or is not stored

< >

9,10,

SLIDE 51

TrInc – NSDI 2009 Dave Levin

10 Implementing a trusted log in TrInc

13

11

< >

8,9,

< >

3,8,

< >

10,11,

Untrusted storage

< >

9,10,

Append(data):

Bind new data to the

end of the log

Lookup(sequence num):

No equivocating on what

is or is not stored

< >

9,10,

SLIDE 52

TrInc – NSDI 2009 Dave Levin

10 Implementing a trusted log in TrInc

13

11

< >

8,9,

< >

3,8,

< >

10,11,

Untrusted storage

< >

9,10,

Append(data):

Bind new data to the

end of the log

Lookup(sequence num):

No equivocating on what

is or is not stored

< >

9,10,

SLIDE 53

TrInc – NSDI 2009 Dave Levin

10 Implementing a trusted log in TrInc

13

11

< >

8,9,

< >

3,8,

< >

10,11,

Untrusted storage

< >

9,10,

Few hardware accesses Fast lookups

Append(data):

Bind new data to the

end of the log

Lookup(sequence num):

No equivocating on what

is or is not stored

< >

9,10,

SLIDE 54

TrInc – NSDI 2009 Dave Levin

TrInc-A2M

Attested Append-only Memory (A2M)
Stores logs in trusted storage
Accesses trusted storage for all methods
A2M shown to solve
Byzantine fault tolerance using fewer nodes
SUNDR file system
Quorum/Update protocol
By construction, TrInc solves these systems, too

14

SLIDE 55

TrInc – NSDI 2009 Dave Levin

What can TrInc do?

Trusted append-only logs
Prevent under-reporting in BitTorrent
Reduces communication in PeerReview
BFT with fewer nodes and messages
Ensure fresh data in DHTs
Prevent Sybil attacks

15

SLIDE 56

TrInc – NSDI 2009 Dave Levin

What can TrInc do?

Trusted append-only logs
Prevent under-reporting in BitTorrent
Reduces communication in PeerReview
BFT with fewer nodes and messages
Ensure fresh data in DHTs
Prevent Sybil attacks

15

SLIDE 57

TrInc – NSDI 2009 Dave Levin

BitTorrent primer

16

SLIDE 58

TrInc – NSDI 2009 Dave Levin

BitTorrent primer

16

Fast, users share the work File pieces

SLIDE 59

TrInc – NSDI 2009 Dave Levin

BitTorrent primer

16

Fast, users share the work File pieces

1 1 0 1 1 0

SLIDE 60

TrInc – NSDI 2009 Dave Levin

BitTorrent primer

16

Fast, users share the work File pieces

1 1 0 1 1 0

Does not have piece 2

SLIDE 61

TrInc – NSDI 2009 Dave Levin

BitTorrent primer

16

Fast, users share the work File pieces

1 1 0 1 1 0

SLIDE 62

TrInc – NSDI 2009 Dave Levin

BitTorrent primer

16

Fast, users share the work File pieces

1 0 1 1 1 0 1 0 1 1 1 0

SLIDE 63

TrInc – NSDI 2009 Dave Levin

BitTorrent primer

16

Fast, users share the work File pieces

1 0 1 1 1 0 1 0 1 1 1 0

SLIDE 64

TrInc – NSDI 2009 Dave Levin

BitTorrent primer

16

Fast, users share the work File pieces

1 1 0 1 0 1 1 1 0 1 0 1 1 1 0 1 0 1

Interested

SLIDE 65

TrInc – NSDI 2009 Dave Levin

BitTorrent primer

16

Fast, users share the work File pieces

1 1 0 1 0 1 1 1 0 1 0 1 1 1 0 1 0 1 1 0 1 1 1 0

Interested Interested

SLIDE 66

TrInc – NSDI 2009 Dave Levin

BitTorrent primer

16

Fast, users share the work File pieces

1 1 0 1 0 1 1 1 0 1 0 1 1 1 0 1 0 1 1 0 1 1 1 0

Interested Interested

SLIDE 67

TrInc – NSDI 2009 Dave Levin

Piece under-reporting is equivocation

17

Yields prolonged interest from others and faster download times

[SIGCOMM’08]

SLIDE 68

TrInc – NSDI 2009 Dave Levin

Piece under-reporting is equivocation

17

SLIDE 69

TrInc – NSDI 2009 Dave Levin

Piece under-reporting is equivocation

17

SLIDE 70

TrInc – NSDI 2009 Dave Levin

Piece under-reporting is equivocation

17

Ack

SLIDE 71

TrInc – NSDI 2009 Dave Levin

Piece under-reporting is equivocation

17

Ack

SLIDE 72

TrInc – NSDI 2009 Dave Levin

Piece under-reporting is equivocation

17

I received

SLIDE 73

TrInc – NSDI 2009 Dave Levin

Piece under-reporting is equivocation

17

I never received

I received

SLIDE 74

TrInc – NSDI 2009 Dave Levin

Applying TrInc

What does the counter represent?
The number of pieces received
To what do peers attest?
Their bitfield
The most recent piece received
When do peers attest?
When they receive
When they sync their counters

18

SLIDE 75

TrInc – NSDI 2009 Dave Levin

TrInc-BitTorrent

19

SLIDE 76

TrInc – NSDI 2009 Dave Levin

TrInc-BitTorrent

19

SLIDE 77

TrInc – NSDI 2009 Dave Levin

TrInc-BitTorrent

19

I have and most recently received

1

SLIDE 78

TrInc – NSDI 2009 Dave Levin

TrInc-BitTorrent

19

I have and most recently received

1 I have and most recently received

2 I have and most recently received

3

SLIDE 79

TrInc – NSDI 2009 Dave Levin

TrInc-BitTorrent

19

Counter matches the bitfield size

I have and most recently received

1 I have and most recently received

2 I have and most recently received

3

SLIDE 80

TrInc – NSDI 2009 Dave Levin

TrInc-BitTorrent

19

Counter matches the bitfield size

I have and most recently received

1 I have and most recently received

2 I have and most recently received

3

SLIDE 81

TrInc – NSDI 2009 Dave Levin

TrInc-BitTorrent

19

Counter matches the bitfield size Attests to most recent piece

I have and most recently received

1 I have and most recently received

2 I have and most recently received

3

SLIDE 82

TrInc – NSDI 2009 Dave Levin

TrInc-BitTorrent

19

Counter matches the bitfield size Attests to most recent piece

I have and most recently received

1 I have and most recently received

2 I have and most recently received

3

SLIDE 83

TrInc – NSDI 2009 Dave Levin

Why attest to the latest piece?

20

SLIDE 84

TrInc – NSDI 2009 Dave Levin

Why attest to the latest piece?

20

SLIDE 85

TrInc – NSDI 2009 Dave Levin

Why attest to the latest piece?

20

1 I have

SLIDE 86

TrInc – NSDI 2009 Dave Levin

Why attest to the latest piece?

20

1 I have

SLIDE 87

TrInc – NSDI 2009 Dave Levin

Why attest to the latest piece?

20

1 I have

SLIDE 88

TrInc – NSDI 2009 Dave Levin

Why attest to the latest piece?

20

1 I have

2 I have

SLIDE 89

TrInc – NSDI 2009 Dave Levin

Why attest to the latest piece?

20

1 I have

2 I have

Looks good to me Looks good to me Looks good to me

SLIDE 90

TrInc – NSDI 2009 Dave Levin

Why attest to the latest piece?

20

1 I have

2 I have

Looks good to me Looks good to me Looks good to me

SLIDE 91

TrInc – NSDI 2009 Dave Levin

Why attest to the latest piece?

20

1 I have

2 I have

Looks good to me Looks good to me Looks good to me

Lesson: Without the full log, must ensure proper behavior at each step

SLIDE 92

TrInc – NSDI 2009 Dave Levin

Macrobenchmarks

TrInc-BitTorrent
Solves piece under-reporting
TrInc-A2M
Reduces hardware requirements
Higher throughput
TrInc-PeerReview
Reduces the communication

necessary to achieve fault detection

21

SLIDE 93

TrInc – NSDI 2009 Dave Levin

3 2 1

Contributions

22

2

TrInc – A new, practical primitive for eliminating equivocation Applications of TrInc Implementation in currently available hardware

SLIDE 94

TrInc – NSDI 2009 Dave Levin

3 2 1

Contributions

22

3

TrInc – A new, practical primitive for eliminating equivocation Applications of TrInc Implementation in currently available hardware

SLIDE 95

TrInc – NSDI 2009 Dave Levin

Implementation

Gemalto .NET Smartcard
Crypto unit (RSA & 3-DES)
32-bit micro-controller
80 KB persistent memory
A few dozen lines of C#
Case studies
TrInc-A2M
TrInc-PeerReview
TrInc-BitTorrent

23

SLIDE 96

TrInc – NSDI 2009 Dave Levin

TrInc microbenchmarks

24

50 100 150 200 250 noop Asym Attest Asym Attest Symm attest Symm Attest Verify

(advance) (status)

Operation time (msec)

(advance) (status)

SLIDE 97

TrInc – NSDI 2009 Dave Levin

TrInc microbenchmarks

24

50 100 150 200 250 noop Asym Attest Asym Attest Symm attest Symm Attest Verify

(advance) (status)

Operation time (msec)

(advance) (status)

SLIDE 98

TrInc – NSDI 2009 Dave Levin

TrInc microbenchmarks

24

50 100 150 200 250 noop Asym Attest Asym Attest Symm attest Symm Attest Verify

(advance) (status)

Operation time (msec)

(advance) (status)

32 msec to write a counter

SLIDE 99

TrInc – NSDI 2009 Dave Levin

TrInc microbenchmarks

24

50 100 150 200 250 noop Asym Attest Asym Attest Symm attest Symm Attest Verify

(advance) (status)

Operation time (msec)

(advance) (status)

32 msec to write a counter Only 2x

SLIDE 100

TrInc – NSDI 2009 Dave Levin

Why so slow?

Fundamentally new application of trusted hardware
Typically used for bootstrapping
TrInc makes it intrinsic to the protocol
It can be faster
There just has not been the call for it prior to TrInc

25

SLIDE 101

TrInc – NSDI 2009 Dave Levin

Summary

Equivocation is a versatile and powerful
A small amount of trust can secure a large system
TrInc is
Minimal – A counter and a key
Versatile – Applies to a wide range of systems
Practical – Uses the same components available today

26

SLIDE 102

TrInc – NSDI 2009 Dave Levin

TrInc speeds up A2M

27

150 300 450 600 Append Lookup Lookup Lookup End Truncate Advance

TrInc-A2M A2M

(successful) (too early) (forgotten)

Operation time (msec)

SLIDE 103

TrInc – NSDI 2009 Dave Levin

TrInc speeds up A2M

27

150 300 450 600 Append Lookup Lookup Lookup End Truncate Advance

TrInc-A2M A2M

(successful) (too early) (forgotten)

Operation time (msec)

TrInc does not go to h/w for successful lookups

SLIDE 104

TrInc – NSDI 2009 Dave Levin

TrInc speeds up A2M

27

150 300 450 600 Append Lookup Lookup Lookup End Truncate Advance

TrInc-A2M A2M

(successful) (too early) (forgotten)