Mechanical Support for Efficient Dissemination on the CAN Overlay - - PowerPoint PPT Presentation

Mechanical Support for Efficient Dissemination on the CAN Overlay Network

• Francesco Bongiovanni -

INRIA Sophia Antipolis OASIS team

Work done in collaboration with Dr. Ludovic Henrio

12 October 2011

Introduction Mechanizing formal proofs Contributions Goals Future Work Motivation

↓ correct-by-construction efficient

broadcast P2P protocol ∗

QED

∗ conditions apply

• Francesco Bongiovanni -

Mechanical Support for Efficient Dissemination on CAN (2/26)

Introduction Mechanizing formal proofs Contributions Goals Future Work Motivation

IP multicast

Broadcasting a message M in a network IP multicast Network-level Pros

bandwidth efficiency no redundant packets

• Francesco Bongiovanni -

Mechanical Support for Efficient Dissemination on CAN (3/26)

Introduction Mechanizing formal proofs Contributions Goals Future Work Motivation

IP multicast Issues

Broadcasting a message M in a network IP multicast Network-level Pros

bandwidth efficiency no redundant packets

Broadcasting a message M in a network IP multicast Network-level Pros

bandwidth efficiency no redundant packets

Issues

Scalability Best-effort Deployment

• Francesco Bongiovanni -

Mechanical Support for Efficient Dissemination on CAN (3/26)

Introduction Mechanizing formal proofs Contributions Goals Future Work Motivation

Can we achieve efficient multi-point delivery without support from the IP layer ?

• Francesco Bongiovanni -

Mechanical Support for Efficient Dissemination on CAN (4/26)

Introduction Mechanizing formal proofs Contributions Goals Future Work Motivation

Can we achieve efficient multi-point delivery without support from the IP layer ?

Broadcasting a message M in a network Overlay Multicast Application-level Pros

easier deployment reliability guarantees

IP multicast Network-level Pros

bandwidth efficiency no redundant packets

Issues

Scalability Best-effort Deployment

• Francesco Bongiovanni -

Mechanical Support for Efficient Dissemination on CAN (4/26)

Introduction Mechanizing formal proofs Contributions Goals Future Work Motivation

Can we achieve efficient multi-point delivery without support from the IP layer ?

Broadcasting a message M in a network Overlay Multicast Application-level Pros

easier deployment reliability guarantees

IP multicast Network-level Pros

bandwidth efficiency no redundant packets

Issues

Scalability Best-effort Deployment

Issues

hard to design hard to program hard to verify

• Francesco Bongiovanni -

Mechanical Support for Efficient Dissemination on CAN (4/26)

Introduction Mechanizing formal proofs Contributions Goals Future Work Motivation

Can we build such delivery mechanism correctly and formally prove its properties ?

• Francesco Bongiovanni -

Mechanical Support for Efficient Dissemination on CAN (5/26)

Introduction Mechanizing formal proofs Contributions Goals Future Work Motivation

Can we build such delivery mechanism correctly and formally prove its properties ? − → Using an interactive proof assistant

• Francesco Bongiovanni -

Mechanical Support for Efficient Dissemination on CAN (5/26)

Introduction Mechanizing formal proofs Contributions Goals Future Work Motivation

Can we build such delivery mechanism correctly and formally prove its properties ? *** − → Using an interactive proof assistant

• Francesco Bongiovanni -

Mechanical Support for Efficient Dissemination on CAN (5/26)

Introduction Mechanizing formal proofs Contributions Goals Future Work Motivation

Can we build such delivery mechanism correctly and formally prove its properties ? *** − → Using an interactive proof assistant ***

Specification Implementation

PROOF

• Francesco Bongiovanni -

Mechanical Support for Efficient Dissemination on CAN (5/26)

Introduction Mechanizing formal proofs Contributions Goals Future Work Motivation

Can we build such delivery mechanism correctly and formally prove its properties ? *** − → Using an interactive proof assistant ***

Specification Implementation

PROOF

Specification Implementation

PROOF

• Francesco Bongiovanni -

Mechanical Support for Efficient Dissemination on CAN (5/26)

Introduction Mechanizing formal proofs Contributions Goals Future Work Motivation

Context - FP7 STREP PLAY

Event Cloud : Publish/Subscribe system for large scale RDF data ∗ processing and storage (based a modified version of CAN).

Chord Layer CAN Layer c1 c2 c3 ci

p

• s

p

• s

p

• s

p

• s

p

• s

* RDF quadruple =”{subject, predicate, object, context}”

• Francesco Bongiovanni -

Mechanical Support for Efficient Dissemination on CAN (6/26)

Introduction Mechanizing formal proofs Contributions Goals Future Work Motivation

Context - FP7 STREP PLAY

Event Cloud : Publish/Subscribe system for large scale RDF data ∗ processing and storage (based a modified version of CAN).

Chord Layer CAN Layer c1 c2 c3 ci

p

• s

p

• s

p

• s

p

• s

p

• s

* RDF quadruple =”{subject, predicate, object, context}”

Need for dissemination algorithms for retrieving RDF data efficiently

• Francesco Bongiovanni -

Mechanical Support for Efficient Dissemination on CAN (6/26)

Introduction Mechanizing formal proofs Contributions Goals Future Work Motivation

Motivation

Dissemination algorithms on top of large-scale P2P systems are hard to :

• Francesco Bongiovanni -

Mechanical Support for Efficient Dissemination on CAN (7/26)

Introduction Mechanizing formal proofs Contributions Goals Future Work Motivation

Motivation

Dissemination algorithms on top of large-scale P2P systems are hard to :

Design Verify

Program Simulate Experiment Analyze

• Francesco Bongiovanni -

Mechanical Support for Efficient Dissemination on CAN (7/26)

Introduction Mechanizing formal proofs Contributions Goals Future Work Motivation

Motivation

Dissemination algorithms on top of large-scale P2P systems are hard to :

Design Verify

Program Simulate Experiment Analyze

Distributed Algorithms are subtle & error-prone...yet few have been formally verified Formal methods to the rescue

• Francesco Bongiovanni -

Mechanical Support for Efficient Dissemination on CAN (7/26)

Introduction Mechanizing formal proofs Contributions Goals Future Work

Mechanizing formal proofs

What’s in it for you ?

Papers with “just” a description of the algorithm

• Francesco Bongiovanni -

Mechanical Support for Efficient Dissemination on CAN (8/26)

Introduction Mechanizing formal proofs Contributions Goals Future Work

Mechanizing formal proofs

What’s in it for you ?

Papers with “just” a description of the algorithm

• [Chord, CAN, Pastry,...]
• Francesco Bongiovanni -

Mechanical Support for Efficient Dissemination on CAN (8/26)

Introduction Mechanizing formal proofs Contributions Goals Future Work

Mechanizing formal proofs

What’s in it for you ?

Papers with “just” a description of the algorithm

• [Chord, CAN, Pastry,...]

Papers with a more precise description of the algorithm and rough hand proofs of correctness

• Francesco Bongiovanni -

Mechanical Support for Efficient Dissemination on CAN (8/26)

Introduction Mechanizing formal proofs Contributions Goals Future Work

Mechanizing formal proofs

What’s in it for you ?

Papers with “just” a description of the algorithm

• [Chord, CAN, Pastry,...]

Papers with a more precise description of the algorithm and rough hand proofs of correctness papers with formal hand proofs

• Francesco Bongiovanni -

Mechanical Support for Efficient Dissemination on CAN (8/26)

Introduction Mechanizing formal proofs Contributions Goals Future Work

Mechanizing formal proofs

What’s in it for you ?

Papers with “just” a description of the algorithm

• [Chord, CAN, Pastry,...]

Papers with a more precise description of the algorithm and rough hand proofs of correctness papers with formal hand proofs Papers with machine-checkable proofs ([Charron-Bost & Merz 2009])

• Francesco Bongiovanni -

Mechanical Support for Efficient Dissemination on CAN (8/26)

Introduction Mechanizing formal proofs Contributions Goals Future Work

Mechanizing formal proofs

It’s all about trust...

Nothing is ever certain, but we can achieve high levels of reliability... ...and theorem provers are more reliable than most human hand proofs.

• Francesco Bongiovanni -

Mechanical Support for Efficient Dissemination on CAN (9/26)

Introduction Mechanizing formal proofs Contributions Goals Future Work

Mechanizing formal proofs

It’s all about trust...

Nothing is ever certain, but we can achieve high levels of reliability... ...and theorem provers are more reliable than most human hand proofs. Working in an interactive theorem prover gives you :

Confidence in correctness

(*assuming the theorem prover is sound)

Automatic assistance in tedious parts of the proof

• Francesco Bongiovanni -

Mechanical Support for Efficient Dissemination on CAN (9/26)

Introduction Mechanizing formal proofs Contributions Goals Future Work Informal description of CAN Sketch of the algorithm A glimpse of the formalization process Summary

Contributions

Correct construction of an efficient broadcast algorithm using Isabelle/HOL interactive proof assistant.

• Francesco Bongiovanni -

Mechanical Support for Efficient Dissemination on CAN (10/26)

Introduction Mechanizing formal proofs Contributions Goals Future Work Informal description of CAN Sketch of the algorithm A glimpse of the formalization process Summary

Contributions

kind of properties to prove

Correct construction of an efficient broadcast algorithm using Isabelle/HOL interactive proof assistant. The type of props we would like to prove:

• Efficiency: a node receives the message only once
• Coverage: all the nodes within a zone must be covered
• Termination: all the nodes have received the message (only
• nce)
• Francesco Bongiovanni -

Mechanical Support for Efficient Dissemination on CAN (10/26)

Introduction Mechanizing formal proofs Contributions Goals Future Work Informal description of CAN Sketch of the algorithm A glimpse of the formalization process Summary

Background

Content Addressable Network

• Francesco Bongiovanni -

Mechanical Support for Efficient Dissemination on CAN (11/26)

Introduction Mechanizing formal proofs Contributions Goals Future Work Informal description of CAN Sketch of the algorithm A glimpse of the formalization process Summary

Background

Content Addressable Network d − dimensional Cartesian coordinate space

• Francesco Bongiovanni -

Mechanical Support for Efficient Dissemination on CAN (11/26)

Introduction Mechanizing formal proofs Contributions Goals Future Work Informal description of CAN Sketch of the algorithm A glimpse of the formalization process Summary

Background

Content Addressable Network d − dimensional Cartesian coordinate space each peer manages a portion of the space

A B C D E H I J K L M F G N O P Q R S T U V W X Y Z A1 B1 C1 D1 F1 E1 G1 H1 I1

• Francesco Bongiovanni -

Mechanical Support for Efficient Dissemination on CAN (11/26)

Introduction Mechanizing formal proofs Contributions Goals Future Work Informal description of CAN Sketch of the algorithm A glimpse of the formalization process Summary

Background

Content Addressable Network d − dimensional Cartesian coordinate space each peer manages a portion of the space a peer only knows its adjacent neighbors

A B C D E F G

• Francesco Bongiovanni -

Mechanical Support for Efficient Dissemination on CAN (11/26)

Introduction Mechanizing formal proofs Contributions Goals Future Work Informal description of CAN Sketch of the algorithm A glimpse of the formalization process Summary

Background

Routing in CAN

X

insert (k,v) retrieve (k)

A B C D E H I J K L M F G N O P

• Francesco Bongiovanni -

Mechanical Support for Efficient Dissemination on CAN (12/26)

Introduction Mechanizing formal proofs Contributions Goals Future Work Informal description of CAN Sketch of the algorithm A glimpse of the formalization process Summary

Background

Routing in CAN

X

insert (k,v) retrieve (k)

A B C D E H I J K L M F G N O P

State overhead: O(d) Lookup complexity: O(dN

1 d )

• Francesco Bongiovanni -

Mechanical Support for Efficient Dissemination on CAN (12/26)

Introduction Mechanizing formal proofs Contributions Goals Future Work Informal description of CAN Sketch of the algorithm A glimpse of the formalization process Summary

Contributions

Main idea

From To

• Francesco Bongiovanni -

Mechanical Support for Efficient Dissemination on CAN (13/26)

Introduction Mechanizing formal proofs Contributions Goals Future Work Informal description of CAN Sketch of the algorithm A glimpse of the formalization process Summary

Contributions

Main idea

From To formally

• Francesco Bongiovanni -

Mechanical Support for Efficient Dissemination on CAN (13/26)

Introduction Mechanizing formal proofs Contributions Goals Future Work Informal description of CAN Sketch of the algorithm A glimpse of the formalization process Summary

Contributions - intuition

We split the CAN into Zones

• Francesco Bongiovanni -

Mechanical Support for Efficient Dissemination on CAN (14/26)

Introduction Mechanizing formal proofs Contributions Goals Future Work Informal description of CAN Sketch of the algorithm A glimpse of the formalization process Summary

Contributions - intuition

We split the CAN into Zones Zones do not intersect Zone 3 Zone 2 Zone 1

* Assuming a method for Zone division E.g.: Knowing the Space coordinates and its neighbors coordinates, initiator computes the geometrical difference and assigns non overlapping zones to each of its neighbors

• Francesco Bongiovanni -

Mechanical Support for Efficient Dissemination on CAN (14/26)

Introduction Mechanizing formal proofs Contributions Goals Future Work Informal description of CAN Sketch of the algorithm A glimpse of the formalization process Summary

Contributions - intuition

We split the CAN into Zones Zones do not intersect There are valid paths within zones (finite) Neighbors are connected . . . Zone 3 Zone 2 Zone 1

* Assuming a method for Zone division E.g.: Knowing the Space coordinates and its neighbors coordinates, initiator computes the geometrical difference and assigns non overlapping zones to each of its neighbors

• Francesco Bongiovanni -

Mechanical Support for Efficient Dissemination on CAN (14/26)

Introduction Mechanizing formal proofs Contributions Goals Future Work Informal description of CAN Sketch of the algorithm A glimpse of the formalization process Summary

Contributions

Algorithm

• Francesco Bongiovanni -

Mechanical Support for Efficient Dissemination on CAN (15/26)

Introduction Mechanizing formal proofs Contributions Goals Future Work Informal description of CAN Sketch of the algorithm A glimpse of the formalization process Summary

Contributions

Algorithm a message M to bcast is received by a peer (initiator) Zone 3 Zone 2 Zone 1

Message M to bcast 1

• Francesco Bongiovanni -

Mechanical Support for Efficient Dissemination on CAN (15/26)

Introduction Mechanizing formal proofs Contributions Goals Future Work Informal description of CAN Sketch of the algorithm A glimpse of the formalization process Summary

Contributions

Algorithm a message M to bcast is received by a peer (initiator) the initiator sends M to all its neighbors Zone 3 Zone 2 Zone 1

Message M to bcast 1

Zone 3 Zone 2 Zone 1

M M 2 M

• Francesco Bongiovanni -

Mechanical Support for Efficient Dissemination on CAN (15/26)

Introduction Mechanizing formal proofs Contributions Goals Future Work Informal description of CAN Sketch of the algorithm A glimpse of the formalization process Summary

Contributions

Algorithm a message M to bcast is received by a peer (initiator) the initiator sends M to all its neighbors within a zone, M is propagated and stays within the zone Zone 3 Zone 2 Zone 1

Message M to bcast 1

Zone 3 Zone 2 Zone 1

M M 2 M

Zone 3 Zone 2 Zone 1

3 M M M M M

• Francesco Bongiovanni -

Mechanical Support for Efficient Dissemination on CAN (15/26)

Introduction Mechanizing formal proofs Contributions Goals Future Work Informal description of CAN Sketch of the algorithm A glimpse of the formalization process Summary

Contributions

the formalization process P2P Protocol CAN

definition

(reusable) abstractions Messages Zones Nodes ...

• Francesco Bongiovanni -

Mechanical Support for Efficient Dissemination on CAN (16/26)

Introduction Mechanizing formal proofs Contributions Goals Future Work Informal description of CAN Sketch of the algorithm A glimpse of the formalization process Summary

Contributions

Definitions

Definitional approach

• Francesco Bongiovanni -

Mechanical Support for Efficient Dissemination on CAN (17/26)

Introduction Mechanizing formal proofs Contributions Goals Future Work Informal description of CAN Sketch of the algorithm A glimpse of the formalization process Summary

Contributions

Definitions

Definitional approach definition of a Node, Space

• Francesco Bongiovanni -

Mechanical Support for Efficient Dissemination on CAN (17/26)

Introduction Mechanizing formal proofs Contributions Goals Future Work Informal description of CAN Sketch of the algorithm A glimpse of the formalization process Summary

Contributions

Definitions

Definitional approach definition of a Node, Space definition of a Message

• Francesco Bongiovanni -

Mechanical Support for Efficient Dissemination on CAN (17/26)

Introduction Mechanizing formal proofs Contributions Goals Future Work Informal description of CAN Sketch of the algorithm A glimpse of the formalization process Summary

Contributions

Definitions

Definitional approach definition of a Node, Space definition of a Message

• Francesco Bongiovanni -

Mechanical Support for Efficient Dissemination on CAN (17/26)

Introduction Mechanizing formal proofs Contributions Goals Future Work Informal description of CAN Sketch of the algorithm A glimpse of the formalization process Summary

Contributions

Definitions

Definitional approach definition of a Node, Space definition of a Message definition of a CAN . . .

• Francesco Bongiovanni -

Mechanical Support for Efficient Dissemination on CAN (17/26)

Introduction Mechanizing formal proofs Contributions Goals Future Work Informal description of CAN Sketch of the algorithm A glimpse of the formalization process Summary

Contributions

Definitions

Definitional approach definition of a Node, Space definition of a Message definition of a CAN . . .

• Francesco Bongiovanni -

Mechanical Support for Efficient Dissemination on CAN (17/26)

Introduction Mechanizing formal proofs Contributions Goals Future Work Informal description of CAN Sketch of the algorithm A glimpse of the formalization process Summary

Contributions

Definitions

Definitional∗ approach definition of a Node, Space definition of a Message definition of a CAN . . .

* Basically we define the needed abstractions of the protocol

• Francesco Bongiovanni -

Mechanical Support for Efficient Dissemination on CAN (17/26)

Introduction Mechanizing formal proofs Contributions Goals Future Work Informal description of CAN Sketch of the algorithm A glimpse of the formalization process Summary

Contributions

the formalization process P2P Protocol CAN (reusable) abstractions Messages Zones Nodes ...

Finer grain properties + Proofs

Finite Zones ... Neighbors Connected exists neighbor ... Finite Msgs Finite Paths inside Zone

• Francesco Bongiovanni -

Mechanical Support for Efficient Dissemination on CAN (18/26)

Introduction Mechanizing formal proofs Contributions Goals Future Work Informal description of CAN Sketch of the algorithm A glimpse of the formalization process Summary

Contributions

On a day-to-day basis

User ITP Write a theorem to prove Write few lemmas necessary to prove Add new lemmas + defs Prove lemmas (set of goals) Prove the main theorem

All lemmas proven Subgoal is too difficult

1 1

• Francesco Bongiovanni -

Mechanical Support for Efficient Dissemination on CAN (19/26)

Introduction Mechanizing formal proofs Contributions Goals Future Work Informal description of CAN Sketch of the algorithm A glimpse of the formalization process Summary

Contributions

formalization process pictured

P2P Protocol CAN (reusable) abstractions Messages Zones Nodes ...

Finer grain properties + Proofs

Finite Zones ... Neighbors Connected exists neighbor ... Finite Msgs Finite Paths inside Zone

Combining Proofs

Coverage Efficiency ...

• Francesco Bongiovanni -

Mechanical Support for Efficient Dissemination on CAN (20/26)

Introduction Mechanizing formal proofs Contributions Goals Future Work Informal description of CAN Sketch of the algorithm A glimpse of the formalization process Summary

Contributions

Summary

What we have done so far:

• Francesco Bongiovanni -

Mechanical Support for Efficient Dissemination on CAN (21/26)

Introduction Mechanizing formal proofs Contributions Goals Future Work Informal description of CAN Sketch of the algorithm A glimpse of the formalization process Summary

Contributions

Summary

What we have done so far:

A formalization of an abstraction of CAN overlay network + theorems and correctness proofs.

• Francesco Bongiovanni -

Mechanical Support for Efficient Dissemination on CAN (21/26)

Introduction Mechanizing formal proofs Contributions Goals Future Work Informal description of CAN Sketch of the algorithm A glimpse of the formalization process Summary

Contributions

Summary

What we have done so far:

A formalization of an abstraction of CAN overlay network + theorems and correctness proofs. A formalization of abstract geometric notions related to CAN, neighboring and communication aspects + correctness proofs

• Francesco Bongiovanni -

Mechanical Support for Efficient Dissemination on CAN (21/26)

Introduction Mechanizing formal proofs Contributions Goals Future Work Informal description of CAN Sketch of the algorithm A glimpse of the formalization process Summary

Contributions

Summary

What we have done so far:

A formalization of an abstraction of CAN overlay network + theorems and correctness proofs. A formalization of abstract geometric notions related to CAN, neighboring and communication aspects + correctness proofs An example explaining how to define formally a broadcast algorithm for a static CAN. Current spec + proofs : around 2000 lines of Isabelle code

• Francesco Bongiovanni -

Mechanical Support for Efficient Dissemination on CAN (21/26)

Introduction Mechanizing formal proofs Contributions Goals Future Work

Goals...

Initial goal: to develop the algorithm correctly and prove its correctness properties.

• Francesco Bongiovanni -

Mechanical Support for Efficient Dissemination on CAN (22/26)

Introduction Mechanizing formal proofs Contributions Goals Future Work

Goals...

Initial goal: to develop the algorithm correctly and prove its correctness properties. Additional goal: to build a generic reasoning framework which will ease the promotion of formal correctness proofs of existing multicast algorithms and also facilitate the design of new ones (which are efficient and fault-tolerant,. . . ).

• Francesco Bongiovanni -

Mechanical Support for Efficient Dissemination on CAN (22/26)

Introduction Mechanizing formal proofs Contributions Goals Future Work

Future work

Implementation Consider a dynamic CAN (churn) Test different (possibly existing) dissemination schemes

multiple initiators, ...

Fault-tolerant broadcast Structured proofs

• Francesco Bongiovanni -

Mechanical Support for Efficient Dissemination on CAN (23/26)

Introduction Mechanizing formal proofs Contributions Goals Future Work

Take away message

“ Programs are not released without being tested, why should algorithms be published without being model checked * ? ”

• Leslie Lamport

* proved correct

• Francesco Bongiovanni -

Mechanical Support for Efficient Dissemination on CAN (24/26)

Introduction Mechanizing formal proofs Contributions Goals Future Work

Questions...?

Thank you

• Francesco Bongiovanni -

Mechanical Support for Efficient Dissemination on CAN (25/26)

Introduction Mechanizing formal proofs Contributions Goals Future Work

Backup slides

Model checking VS theorem proving

lightweight but limited heavy but really rigorous model checking theorem proving Formal verification State space Counter-example Verification procedure Obtaining insight of the system Finite Automatic Automatic Tell how the system is incorrect Infinite Limited automatic Not automatic Tell how the system is correct

• Francesco Bongiovanni -

Mechanical Support for Efficient Dissemination on CAN (26/26)