Intrusion Detection and Response in LOCKSS Rachel Greenstadt - - PowerPoint PPT Presentation

Intrusion Detection and Response in LOCKSS

Rachel Greenstadt greenie@eecs.harvard.edu Harvard University Advisors: HT Kung and Mike Smith September 30, 2004

Intrusion Detection and Response in LOCKSS – p.1/43

Overview

Problem: Electronic Archiving Approach: LOCKSS project Security Threats and Countermeasures Intrusion Detection and Response: Alarms Do we need alarms? Can alarms catch instrusions? Can we take local steps to correct the network after an intrusion? Can we catch intrusions faster?

Intrusion Detection and Response in LOCKSS – p.2/43

A Crisis in Archiving

Libraries know how to cooperate to preserve paper data With move to electronic data, publishers offer subscriptions Offer “perpetual access,” but no business model for this Libraries want to preserve their own data BUT Digital preservation is hard (and most people don’t believe it)

Intrusion Detection and Response in LOCKSS – p.3/43

Why is Digital Preservation Hard?

Storage media are unreliable in the long term MTBF of components Human error Many anecdotes of backup failures Suggest bit rot/failures happen in Byzantine ways Almost everyone has a story Companies don’t like to talk about it

Intrusion Detection and Response in LOCKSS – p.4/43

Overview

Problem: Electronic Archiving Approach: LOCKSS project Security Threats and Countermeasures Intrusion Detection and Response: Alarms Do we need alarms? Can alarms catch instrusions? Can we take local steps to correct the network after an intrusion? Can we catch intrusions faster?

Intrusion Detection and Response in LOCKSS – p.5/43

The LOCKSS Approach

Build P2P community of libraries Each library maintains a replica of the Archival Unit (AU) Goal: Maintain consensus on content of AUs Libraries help each other repair replicas Audit and repair detected damage with voting Publishers provide content for archiving Without responsibility for preservation

Intrusion Detection and Response in LOCKSS – p.6/43

Why LOCKSS?

LOCKSS is a real project Deployed version Consortia of libraries Engineering team Good vehicle to study intrusion detection for

• ther P2P/distributed systems

Intrusion Detection and Response in LOCKSS – p.7/43

Peer Relationships

LOCKSS has an internal PKI. If I’m a LOCKSS peer, other peers are: Friends—Peers with which I have out-of-band trust relationships. Friends sign certificates for each other. Discovered Peers—These peers form my trust web. Reference List—A subset of discovered peers that can be polled. Undiscovered Peers—Peers unknown (or untrusted) by me.

Intrusion Detection and Response in LOCKSS – p.8/43

Opinion Polls

Periodically, peers poll a subset of the reference list and compare the votes to their local AU

Landslide agreement Do nothing. Landslide disagreement Repair Inconclusive Alarm!

Intrusion Detection and Response in LOCKSS – p.9/43

Updating the Reference List

Before voting, peers nominate other peers for inclusion in reference list/trust web. At the end of poll, voting peers are purged from reference list Add some friends Add some nominees

Intrusion Detection and Response in LOCKSS – p.10/43

Overview

Problem: Electronic Archiving Approach: LOCKSS project Security Threats and Countermeasures Intrusion Detection and Response: Alarms Do we need alarms? Can alarms catch instrusions? Can we take local steps to correct the network after an intrusion? Can we catch intrusions faster?

Intrusion Detection and Response in LOCKSS – p.11/43

Security Concerns

Basically giving away write permission on your archive!!!! For LOCKSS to be useful, benefits must

• utweigh costs

Intrusion Detection and Response in LOCKSS – p.12/43

Alarms

Idea Simultaneous bit rot is rare If voters disagree, adversary activity likely LOCKSS didn’t specify response This is my contribution

Intrusion Detection and Response in LOCKSS – p.13/43

My Contributions

Ran experiments to verify alarms were needed and could detect intrusions Devised a localized protocol to respond to alarms by Healing compromised peers Ran simulations to evaluate and tune this protocol Devised and tested an augmented protocol to trigger alarms earlier in an attack

Intrusion Detection and Response in LOCKSS – p.14/43

Evaluation Measures

Iterative process of simulation and reasoning about the system design and simulation results Proofs would be nice, but system complexity would render them inaccurate or intractable Problem with many P2P systems Initially, goal was to keep adversary from damaging > 50% of the AUs, reached that, see how close we can get to 0.

Intrusion Detection and Response in LOCKSS – p.15/43

Simulations

Simulate 1000 peers participating in the LOCKSS system Each peer has one AU that can be good or bad. Some variable fraction of these peers are adversarial Adversary follows the strategy of lurk and try to get a presence on good peers’ reference lists, then attack when they’ll win decisively.

Intrusion Detection and Response in LOCKSS – p.16/43

Static Variables (Assumptions)

Total Peers 1000 Sybils 200 Topology Cluster Poll Size 10-20 Supermajority 70% Reference List Goal 60 Churn Ratio 10% Lurktime 3600 of 7200 ticks MTBF Doc

1 200 yrs

Intrusion Detection and Response in LOCKSS – p.17/43

Dynamic Variables

# of Adversarial Peers (10-400) Random seeds Alarm Response

Intrusion Detection and Response in LOCKSS – p.18/43

Overview

Problem: Electronic Archiving Approach: LOCKSS project Security Threats and Countermeasures Intrusion Detection and Response: Alarms Do we need alarms? Can alarms catch instrusions? Can we take local steps to correct the network after an intrusion? Can we catch intrusions faster?

Intrusion Detection and Response in LOCKSS – p.19/43

Do we need alarms?

0.2 0.4 0.6 0.8 1 0.1 0.15 0.2 0.25 0.3 0.35 0.4 Proportion of replicas damaged Proportion of peers initially subverted initial damage irrecoverable damage

Figure 1: MUTE alarm: when an alarm is called, do noth-

ing (proceed as if we won the poll)

Intrusion Detection and Response in LOCKSS – p.20/43

Can alarms detect intrusions?

Theory—Why do alarms occur? Alarm might happen if an adversary tries to win a poll without enough votes (can’t count

• n this)

Peers with corrupted copies + peers with good copies => alarms Adversary can’t corrupt enough copies at

• nce to win without giving alarms a chance to

fix things Arrange reference list updating and rate limiting to make this so

Intrusion Detection and Response in LOCKSS – p.21/43

Can alarms detect intrusions?

0.2 0.4 0.6 0.8 1 0.1 0.15 0.2 0.25 0.3 0.35 0.4 Proportion of replicas damaged Proportion of peers initially subverted 0.148 0.185 0.189 0.208 0.219 0.239 0.238 0.264 0.276 0.299 0.298 0.31 0.326 0.34 0.347 0.346 0.36 0.371 0.395 0.419 0.431 0.44 0.433 0.471 0.47 initial damage irrecoverable damage

Figure 2: Foothold ratio for a very patient adversary and results after the first alarm.

Intrusion Detection and Response in LOCKSS – p.22/43

Alarms

Do we need alarms? Can alarms catch instrusions? Can we take local steps to correct the network after an intrusion? Can we catch intrusions faster?

Intrusion Detection and Response in LOCKSS – p.23/43

How can we respond?

Change our state (not enough) Ask our friends to change their state Ask our friends to ask their friends · · ·

Intrusion Detection and Response in LOCKSS – p.24/43

Peer states

Admin Computer AU Healthy Good Good Good Damaged Good Good Bad Subverted Good Bad Good/Bad Evil Bad Bad Good/Bad Figure 3: Classification of Peers We can heal subverted peers and revoke certificates of nominated evil peers

Intrusion Detection and Response in LOCKSS – p.25/43

Healing Alarm Procedure

Contact friends in trust web, ask them to check for compromise and patch. Treat all unhealed peers as undiscovered. What if we want to do more than just our friends? We can heal nodes at depth 2, by asking friends to ask them to heal themselves Healed peers revoke certificates they signed when subverted

Intrusion Detection and Response in LOCKSS – p.26/43

Types of Alarms

Intrusion Detection and Response in LOCKSS – p.27/43

Healing Results 1

0.2 0.4 0.6 0.8 1 0.1 0.15 0.2 0.25 0.3 0.35 0.4 Proportion of replicas damaged Proportion of peers initially subverted initial damage irrecoverable damage 0.2 0.4 0.6 0.8 1 0.1 0.15 0.2 0.25 0.3 0.35 0.4 Proportion of replicas damaged Proportion of peers initially subverted initial damage irrecoverable damage

Figure 4: Doing nothing, compared to healing d=1 (end of

simulation)

Intrusion Detection and Response in LOCKSS – p.28/43

Healing Results 2

0.2 0.4 0.6 0.8 1 0.1 0.15 0.2 0.25 0.3 0.35 0.4 Proportion of replicas damaged Proportion of peers initially subverted initial damage irrecoverable damage 0.2 0.4 0.6 0.8 1 0.1 0.15 0.2 0.25 0.3 0.35 0.4 Proportion of replicas damaged Proportion of peers initially subverted initial damage irrecoverable damage

Figure 5: Doing nothing, compared to healing d=1 (worst

pt of simulation)

Intrusion Detection and Response in LOCKSS – p.29/43

Healing Results 3

0.2 0.4 0.6 0.8 1 0.1 0.15 0.2 0.25 0.3 0.35 0.4 Adversary foothold ratio Proportion of peers initially subverted Lurk End Max 0.2 0.4 0.6 0.8 1 0.1 0.15 0.2 0.25 0.3 0.35 0.4 Adversary foothold ratio Proportion of peers initially subverted Lurk End Max

Figure 6: Doing nothing, compared to healing d=1

Intrusion Detection and Response in LOCKSS – p.30/43

Results with Evil Peers

Intrusion Detection and Response in LOCKSS – p.31/43

Imperfect Healing p = 0.5

0.2 0.4 0.6 0.8 1 0.1 0.15 0.2 0.25 0.3 0.35 0.4 Proportion of replicas damaged Proportion of peers initially subverted initial damage irrecoverable damage 0.2 0.4 0.6 0.8 1 0.1 0.15 0.2 0.25 0.3 0.35 0.4 Proportion of replicas damaged Proportion of peers initially subverted initial damage irrecoverable damage

Figure 8: d=1,p=0.5, compared to healing d=1, p=1 What if the sys admins are unable to heal all sub- verted nodes?

Intrusion Detection and Response in LOCKSS – p.32/43

Healing With Depth 2

0.2 0.4 0.6 0.8 1 0.1 0.15 0.2 0.25 0.3 0.35 0.4 Proportion of replicas damaged Proportion of peers initially subverted initial damage irrecoverable damage 0.2 0.4 0.6 0.8 1 0.1 0.15 0.2 0.25 0.3 0.35 0.4 Proportion of replicas damaged Proportion of peers initially subverted initial damage irrecoverable damage

Figure 9: d=2,p=1, compared to healing d=1, p=1

Intrusion Detection and Response in LOCKSS – p.33/43

Depth 2, Prob 0.5

0.2 0.4 0.6 0.8 1 0.1 0.15 0.2 0.25 0.3 0.35 0.4 Proportion of replicas damaged Proportion of peers initially subverted initial damage irrecoverable damage 0.2 0.4 0.6 0.8 1 0.1 0.15 0.2 0.25 0.3 0.35 0.4 Proportion of replicas damaged Proportion of peers initially subverted initial damage irrecoverable damage

Figure 10: d=2,p=0.5, compared to healing d=1, p=0.5

Intrusion Detection and Response in LOCKSS – p.34/43

Benefits of Localized Alarms

The system does not have to halt for human intervention for every alarm A Nuisance Adversary might try to DOS the system by calling many alarms In the local alarm case these adversaries’ victims will be their friends. These friends can revoke their certs if they are too obnoxious, so self-policing.

Intrusion Detection and Response in LOCKSS – p.35/43

Alarms

Do we need alarms? Can alarms catch instrusions? Can we take local steps to correct the network after an intrusion? Can we catch intrusions faster?

Intrusion Detection and Response in LOCKSS – p.36/43

Adding Hashing to LOCKSS

Old idea to use local hashing to augment LOCKSS Try a simple idea to see how much it can help Each peer keeps a local hash of its AU If poll result is repair, call alarm if hash matches the document Current research on more sophisticated use

• f hashing

Intrusion Detection and Response in LOCKSS – p.37/43

Hashing Results

0.2 0.4 0.6 0.8 1 0.1 0.15 0.2 0.25 0.3 0.35 0.4 Proportion of replicas damaged Proportion of peers initially subverted initial damage irrecoverable damage 0.2 0.4 0.6 0.8 1 0.1 0.15 0.2 0.25 0.3 0.35 0.4 Adversary foothold ratio Proportion of peers initially subverted Lurk End Max

Figure 11: Depth 1, prob 1, keeping a hash.

Intrusion Detection and Response in LOCKSS – p.38/43

More Hashing Thoughts

What happens when the hash goes bad? If you have a good doc and a bad hash (or a bad document) then you are still vulnerable. Current research to mitigate this issue...

Intrusion Detection and Response in LOCKSS – p.39/43

Conclusions

Alarms are necessary and can be effective Local healing alarms can recover from the compromise of up to 40% of LOCKSS peers Adding hashing techniques to LOCKSS is effective A combination of hashing and alarms can mitigate the risks involved in LOCKSS

Intrusion Detection and Response in LOCKSS – p.40/43

Future Directions

Hashing Periodic re-infections Adversaries that go back into lurk mode

Intrusion Detection and Response in LOCKSS – p.41/43

Reset Results

0.2 0.4 0.6 0.8 1 0.1 0.15 0.2 0.25 0.3 0.35 0.4 Proportion of replicas damaged Proportion of peers initially subverted initial damage irrecoverable damage 0.2 0.4 0.6 0.8 1 0.1 0.15 0.2 0.25 0.3 0.35 0.4 Proportion of replicas damaged Proportion of peers initially subverted initial damage irrecoverable damage

Figure 12: Compare resetting the ref list to doing nothing

Intrusion Detection and Response in LOCKSS – p.42/43

Some Other Approaches

Other approaches can be complementary RAID/backups Use hashing (handwave) Have peers store signed hashes for each

• ther

Reasonable, but won’t achieve concensus Incentive issues LOCKSS models current library interactions

But LOCKSS is happening! End of talk: look at ways to add redundancy to LOCKSS with hashing

Intrusion Detection and Response in LOCKSS – p.43/43