Effective and Efficient Compromise Recovery for Weakly Consistent - - PowerPoint PPT Presentation

▶
effective and efficient compromise recovery for weakly
SMART_READER_LITE
LIVE PREVIEW

Effective and Efficient Compromise Recovery for Weakly Consistent - - PowerPoint PPT Presentation

Dec 05, 2022 472 likes •1.34k views

Effective and Efficient Compromise Recovery for Weakly Consistent Replication Prince Mahajan (UT Austin), Ramakrishna Kotla, Cathy Marshall, Venugopalan Rama Ramasubramanian, Tom Rodeheffer, Doug Terry, Ted Wobber (Microsoft Research

slide-1
SLIDE 1

Effective and Efficient Compromise Recovery for Weakly Consistent Replication

Prince Mahajan (UT Austin), Ramakrishna Kotla, Cathy Marshall, Venugopalan “Rama” Ramasubramanian, Tom Rodeheffer, Doug Terry, Ted Wobber (Microsoft Research Silicon Valley)

slide-2
SLIDE 2

Scenario

Sam

slide-3
SLIDE 3

Scenario

Sam

slide-4
SLIDE 4

Scenario

Alex

slide-5
SLIDE 5

Scenario

Alex

slide-6
SLIDE 6

Scenario

Corrupted “contact” Alex

slide-7
SLIDE 7

Scenario

Corrupted “contact” Alex

slide-8
SLIDE 8

Scenario

Corrupted “contact” Workgroup Alex

slide-9
SLIDE 9

Scenario

Corrupted “contact” Sam Workgroup

slide-10
SLIDE 10

Scenario

Corrupted “contact” Workgroup

slide-11
SLIDE 11

Scenario

Corrupted “contact”

In replicated systems, even inappropriate updates propagate automatically.

Workgroup

slide-12
SLIDE 12

Another Scenario

  • L. A. customers

San Francisco customers Chicago customers

slide-13
SLIDE 13

Another Scenario

  • L. A. customers

San Francisco customers Chicago customers

slide-14
SLIDE 14

Another Scenario

  • L. A. customers

San Francisco customers Chicago customers

slide-15
SLIDE 15

Another Scenario

  • L. A. customers

San Francisco customers Chicago customers

slide-16
SLIDE 16

Our Contributions

Polygraph: A framework that

  • Extends weakly consistent replication
  • Removes corrupted updates
  • Recovers uncorrupted updates
  • While being

Effective: Retain most uncorrupted updates Efficient: Incur less bandwidth cost

slide-17
SLIDE 17

Outline

  • Motivation
  • System Model
  • Backup-based approach
  • Polygraph: Effective and Efficient Recovery
  • Results
  • Conclusion
slide-18
SLIDE 18

System Model

slide-19
SLIDE 19

System Model

  • Replicas can independently update items
  • Each update produces a new version of item
  • New versions propagate asynchronously
slide-20
SLIDE 20

System Model

  • Replicas can independently update items
  • Each update produces a new version of item
  • New versions propagate asynchronously
  • Replicas retain the most recent version
  • Archive replica logs all received versions
slide-21
SLIDE 21

Example System

Replica A

updates

Replica B Replica C

stored versions updates updates

Archive Log

slide-22
SLIDE 22

Update Timeline

wall clock time

A C B

1 2 3 4 5

replica

A1 A4 C4 B5 i k j

6 7

B1 C3 C1 l B4

8

B2 C2 A3 B3 A2

influence items i j k l replicas A, B, C versions A1, A2,...,C4

slide-23
SLIDE 23

Update Timeline

wall clock time

A C B

1 2 3 4 5

replica

A1 A4 C4 B5 i k j

6 7

B1 C3 C1 l B4

8

B2 C2 A3 B3 A2

influence items i j k l replicas A, B, C versions A1, A2,...,C4

slide-24
SLIDE 24

Threat Model

  • Compromises can result from malice or misuse
  • Corrupted versions

Versions injected by compromised replicas Versions influenced by corrupted versions

  • An external agent detects and reports

compromises after the fact

  • Archive and replication layer is not compromised
slide-25
SLIDE 25

Update Timeline

wall clock time

A C B

1 2 3 4 5

replica

A1 A4 C4 B5

i k j

6 7

B1 C3 C1

l

B4

8

B2 C2 A3 B3 A2

influence items i j k l replicas A, B, C versions A1, A2,...,C4

slide-26
SLIDE 26

Update Timeline (with compromise)

wall clock time

A C B

1 2 3 4 5

replica

A1 A4

i k j

6 7

B1 C3 C1

l

8

C4 B5 B4 B2 C2 A3 B3 A2

compromise notification influence

innocent version

items i j k l corrupt version replicas A, B, C compromise versions A1, A2,...,C4

slide-27
SLIDE 27

Update Timeline (after recovery)

wall clock time

A C B

1 2 3 4 5

replica

A1 A4

i k j

6 7

B1 C3 C1

l

8

B2 C2 A3 B3 A2

compromise notification influence

innocent version

items i j k l corrupt version replicas A, B, C compromise versions A1, A2,...,C4

slide-28
SLIDE 28

Backup-based Approach

Replica B Backup Replica A Replica C

stored versions updates updates updates

Innocent version Corrupt version

Checkpoints

Checkpoint

slide-29
SLIDE 29

Backup-based Approach

Replica B Backup Replica A Replica C

stored versions updates updates updates

Innocent version Corrupt version

Checkpoints

Checkpoint

slide-30
SLIDE 30

Backup-based Approach

Replica B Backup Replica A Replica C

stored versions updates updates updates

Innocent version Corrupt version

Checkpoints

Checkpoint

slide-31
SLIDE 31

Backup-based Approach

Replica B Backup Replica A Replica C

stored versions updates updates updates

Compromise notification: B compromised at time t

Innocent version Corrupt version

Checkpoints

Checkpoint

slide-32
SLIDE 32

Backup-based Approach

Replica B Backup Replica A Replica C

stored versions updates updates updates

Compromise notification: B compromised at time t

Innocent version Corrupt version

Checkpoints

Checkpoint

slide-33
SLIDE 33

Backup-based Approach

Replica B Backup Replica A Replica C

stored versions updates updates updates

Compromise notification: B compromised at time t

Innocent version Corrupt version

Checkpoints

Checkpoint

slide-34
SLIDE 34

Backup-based Approach

Replica B Backup Replica A Replica C

stored versions updates updates updates

Compromise notification: B compromised at time t

Innocent version Corrupt version

Checkpoints

Checkpoint

slide-35
SLIDE 35

Backup-based Approach

Replica B Backup Replica A Replica C

stored versions updates updates updates

Compromise notification: B compromised at time t

Innocent version Corrupt version

Checkpoints

Checkpoint

slide-36
SLIDE 36

Backup-based Approach

Replica B Backup Replica A Replica C

stored versions updates updates updates

Compromise notification: B compromised at time t

Innocent version Corrupt version

Checkpoints

Checkpoint

slide-37
SLIDE 37
  • Inefficient: Re-propagation from backup to replicas
  • Ineffective: Updates subsequent to checkpoint are lost

Drawbacks of Backup-based Approach

A C B

1 2 3 4 5

replica

A1 A4 C4 B5

i k j

6 7

B1 C3 C1

l

B4

8

B2 C2 A3 B3 A2

compromise notification

slide-38
SLIDE 38

Polygraph: Key Ideas

slide-39
SLIDE 39

Polygraph: Key Ideas

  • Innocent version identification

Effectiveness: innocent versions created post- compromise are recovered

slide-40
SLIDE 40

Polygraph: Key Ideas

  • Innocent version identification

Effectiveness: innocent versions created post- compromise are recovered

  • Replica-local retention
slide-41
SLIDE 41

Polygraph: Key Ideas

  • Innocent version identification

Effectiveness: innocent versions created post- compromise are recovered

  • Replica-local retention

Replicas retain innocent versions

slide-42
SLIDE 42

Polygraph: Key Ideas

  • Innocent version identification

Effectiveness: innocent versions created post- compromise are recovered

  • Replica-local retention

Replicas retain innocent versions Effectiveness: newer versions recovered

slide-43
SLIDE 43

Polygraph: Key Ideas

  • Innocent version identification

Effectiveness: innocent versions created post- compromise are recovered

  • Replica-local retention

Replicas retain innocent versions Effectiveness: newer versions recovered Efficiency: retained versions save bandwidth

slide-44
SLIDE 44

Innocent Versions

Version is innocent if it is Generated before compromise, or Not influenced by any corrupt version from the compromised replica

slide-45
SLIDE 45

Is a version generated before compromise?

Update A1

A2 C1 B3 B1 B4 A4 C3

Archive Log

B2

slide-46
SLIDE 46

Is a version generated before compromise?

Update A1

A2 C1 B3 B1 B4 A4

1 2 3 4 5 6 7 8

wall clock time A3 C3

Archive Log

B2

slide-47
SLIDE 47

Is a version generated before compromise?

Update A1

A2 C1 B3 B1 B4 A4

1 2 3 4 5 6 7 8

wall clock time A3 C3

Archive Log

B2

compromise notification

slide-48
SLIDE 48

Is a version generated before compromise?

Update A1

A2 C1 B3 B1 B4 A4

1 2 3 4 5 6 7 8

wall clock time A3 C3

Archive Log

B2

compromise notification versions generated prior to compromise

slide-49
SLIDE 49

Precompromise cut summarizes versions archived prior to compromise

Is a version generated before compromise?

Update A1

A2 C1 B3 B1 B4 A4

1 2 3 4 5 6 7 8

wall clock time A3 C3

Archive Log

precompromise cut

B2

compromise notification versions generated prior to compromise

slide-50
SLIDE 50

Is a version v influenced by any corrupt version from the compromised replica?

slide-51
SLIDE 51

Is a version v influenced by any corrupt version from the compromised replica?

  • Sufficient to check: is the most recent version from

the compromised replica that influenced v is corrupt?

slide-52
SLIDE 52

Is a version v influenced by any corrupt version from the compromised replica?

  • Sufficient to check: is the most recent version from

the compromised replica that influenced v is corrupt?

  • Each version has a taint vector
  • Taint vector of a version v tracks the most recent

version from each replica that has influenced v

slide-53
SLIDE 53

Is a version v influenced by any corrupt version from the compromised replica?

  • Sufficient to check: is the most recent version from

the compromised replica that influenced v is corrupt?

  • Each version has a taint vector
  • Taint vector of a version v tracks the most recent

version from each replica that has influenced v

A1 B2 C2

A1 A1 B2 C2

Taint vector

A1 B2

C3

A1 B2 C3

slide-54
SLIDE 54

Is a version v influenced by any corrupt version from the compromised replica?

  • Sufficient to check: is the most recent version from

the compromised replica that influenced v is corrupt?

  • Each version has a taint vector
  • Taint vector of a version v tracks the most recent

version from each replica that has influenced v A version v is innocent if the influencing version from the compromised replica in v’s taint vector is innocent

A1 B2 C2

A1 A1 B2 C2

Taint vector

A1 B2

C3

A1 B2 C3

slide-55
SLIDE 55

Innocent Version Identification

Can identify innocent versions received after compromise

slide-56
SLIDE 56

A C B

1 2 3 4 5

replica

A1 A4 C4 B5

i k j

6 7

B1 C3 C1

l

B4

8

B2 C2 A3 B3 A2

Innocent Version Identification

compromise notification

Can identify innocent versions received after compromise

slide-57
SLIDE 57

A C B

1 2 3 4 5

replica

A1 A4 C4 B5

i k j

6 7

B1 C3 C1

l

B4

8

B2 C2 A3 B3 A2

Innocent Version Identification

Innocent because in precompromise cut compromise notification

Can identify innocent versions received after compromise

slide-58
SLIDE 58

A C B

1 2 3 4 5

replica

A1 A4 C4 B5

i k j

6 7

B1 C3 C1

l

B4

8

B2 C2 A3 B3 A2

Innocent Version Identification

Innocent because in precompromise cut Innocent because not influenced by any corrupt version from the compromised replica compromise notification

Can identify innocent versions received after compromise

slide-59
SLIDE 59

A C B

1 2 3 4 5

replica

A1 A4 C4 B5

i k j

6 7

B1 C3 C1

l

B4

8

B2 C2 A3 B3 A2

Innocent Version Identification

Innocent because in precompromise cut Innocent because not influenced by any corrupt version from the compromised replica compromise notification

Can identify innocent versions received after compromise

slide-60
SLIDE 60

Replica-local Retention

  • Replicas retain innocent versions
  • Key mechanism: innocence predicate
slide-61
SLIDE 61

Innocent Version Identification at Archive

v is included in the precompromise cut, or The version from the compromised replica in v’s taint vector is innocent

Version v Innocent? Yes/No precompromise cut compromised replicaID

slide-62
SLIDE 62

Innocence Predicate: Innocent Version Identification at Replica

v is included in the precompromise cut, or The version from the compromised replica in v’s taint vector is innocent

Version v Innocent? Yes/No

precompromise cut replicaID

Innocence Predicate (from archive)

slide-63
SLIDE 63

Replica B Replica A Replica C

stored versions updates updates updates

Compromise notification: B compromised at time t

Innocent version Corrupt version

Archive Log

Replica-local Retention

slide-64
SLIDE 64

Replica B Replica A Replica C

stored versions updates updates updates

Compromise notification: B compromised at time t

Innocent version Corrupt version

Archive Log

Innocence Predicate

Replica-local Retention

slide-65
SLIDE 65

Replica B Replica A Replica C

stored versions updates updates updates

Compromise notification: B compromised at time t

Innocent version Corrupt version

Archive Log

Innocence Predicate

Replica-local Retention

slide-66
SLIDE 66

Replica B Replica A Replica C

stored versions updates updates updates

Compromise notification: B compromised at time t

Innocent version Corrupt version

Archive Log

Innocence Predicate

Replica-local Retention

slide-67
SLIDE 67

Replica B Replica A Replica C

stored versions updates updates updates

Compromise notification: B compromised at time t

Innocent version Corrupt version

Archive Log

Innocence Predicate

Replica-local Retention

Innocence Predicate Filter

slide-68
SLIDE 68

Replica B Replica A Replica C

stored versions updates updates updates

Compromise notification: B compromised at time t

Innocent version Corrupt version

Archive Log

Replica-local Retention

Innocence Predicate Filter Innocence Predicate Filter

slide-69
SLIDE 69

Replica B Replica A Replica C

stored versions updates updates updates

Compromise notification: B compromised at time t

Innocent version Corrupt version

Archive Log

Replica-local Retention

Innocence Predicate Filter Innocence Predicate Filter

slide-70
SLIDE 70

Implementation

  • Implemented on Cimbiosys
  • Multiple compromises

Can recover simultaneously

  • Multiple independent archives

Improves effectiveness, efficiency, and fault-tolerance

slide-71
SLIDE 71

Evaluation

slide-72
SLIDE 72

Evaluation

Setup

  • 10 replicas and 1000 items
  • 3000 updates overall
  • 1000 updates after

compromise

  • Random updates and

synchronizations

  • 5 updates between

synchronizations

slide-73
SLIDE 73

Metric

  • (in)effectiveness:

lost items

  • (in)efficiency:

network overhead

Evaluation

Setup

  • 10 replicas and 1000 items
  • 3000 updates overall
  • 1000 updates after

compromise

  • Random updates and

synchronizations

  • 5 updates between

synchronizations

slide-74
SLIDE 74

Effectiveness and Efficiency

% of items lost Network transfers due to recovery (% items transferred per replica)

20 40 60 80 100 20 40 60 80 100

Network transfers Lost items

slide-75
SLIDE 75

Effectiveness and Efficiency

% of items lost Network transfers due to recovery (% items transferred per replica)

20 40 60 80 100 20 40 60 80 100

B a c k u p Network transfers Lost items

slide-76
SLIDE 76

Effectiveness and Efficiency

% of items lost Network transfers due to recovery (% items transferred per replica)

20 40 60 80 100 20 40 60 80 100

B a c k u p Network transfers Lost items

all items updated after compromise are lost

slide-77
SLIDE 77

Effectiveness and Efficiency

% of items lost Network transfers due to recovery (% items transferred per replica)

20 40 60 80 100 20 40 60 80 100

B a c k u p Network transfers Lost items P

  • l

y g r a p h

slide-78
SLIDE 78

Effectiveness and Efficiency

% of items lost Network transfers due to recovery (% items transferred per replica)

20 40 60 80 100 20 40 60 80 100

B a c k u p Network transfers Lost items B a c k u p T a i n t P

  • l

y g r a p h

slide-79
SLIDE 79

Effectiveness and Efficiency

% of items lost Network transfers due to recovery (% items transferred per replica)

20 40 60 80 100 20 40 60 80 100

B a c k u p Network transfers Lost items B a c k u p T a i n t P

  • l

y g r a p h

taint saves over 40% items

slide-80
SLIDE 80

Effectiveness and Efficiency

% of items lost Network transfers due to recovery (% items transferred per replica)

20 40 60 80 100 20 40 60 80 100

B a c k u p Network transfers Lost items B a c k u p T a i n t P

  • l

y g r a p h

local retention saves over 20% items

slide-81
SLIDE 81

Effectiveness and Efficiency

% of items lost Network transfers due to recovery (% items transferred per replica)

20 40 60 80 100 20 40 60 80 100

B a c k u p Network transfers Lost items B a c k u p T a i n t P

  • l

y g r a p h

local retention reduces b/w by 85%

slide-82
SLIDE 82

Effectiveness and Efficiency

% of items lost Network transfers due to recovery (% items transferred per replica)

20 40 60 80 100 20 40 60 80 100

B a c k u p Network transfers Lost items B a c k u p T a i n t P

  • l

y g r a p h P

  • l

y g r a p h P r e c

  • m

p r

  • m

i s e C u t

slide-83
SLIDE 83

Effectiveness and Efficiency

% of items lost Network transfers due to recovery (% items transferred per replica)

20 40 60 80 100 20 40 60 80 100

B a c k u p Network transfers Lost items B a c k u p T a i n t P

  • l

y g r a p h P

  • l

y g r a p h

T a i n t

P

  • l

y g r a p h P r e c

  • m

p r

  • m

i s e C u t

slide-84
SLIDE 84

Effectiveness and Efficiency

% of items lost Network transfers due to recovery (% items transferred per replica)

20 40 60 80 100 20 40 60 80 100

B a c k u p Network transfers Lost items B a c k u p T a i n t P

  • l

y g r a p h P

  • l

y g r a p h

T a i n t

P

  • l

y g r a p h P r e c

  • m

p r

  • m

i s e C u t

polygraph saves versions not saved by precompromise cut & taint

slide-85
SLIDE 85

Conclusion

  • In a weakly consistent system, Polygraph

reverses the effect of corrupt updates Effective: retains most uncorrupted updates Efficient: recovery uses less bandwidth

  • Implemented on Cimbiosys replication system