Advanced Publish/Subscribe Communication Services Antonio Carzaniga - - PowerPoint PPT Presentation

Advanced Publish/Subscribe Communication Services

Antonio Carzaniga and Alexander L. Wolf

University of Colorado at Boulder Boulder, Colorado USA {carzanig alw}@cs colorado edu {carzanig,alw}@cs.colorado.edu

About the Tutors

Antonio Carzaniga

– PhD: Politecnico di Milano, Italy – Previously at CEFRIEL

Alexander L. Wolf

– PhD: University of Massachusetts, Amherst USA PhD: University of Massachusetts, Amherst USA – Previously at AT&T Bell Laboratories

General research interests General research interests

– software engineering, distributed systems, security and networking security, and networking – Developers of the Siena pub/sub system

Outline

What is publish/subscribe communication? Service models Implementation models Implementation models Survey of representative pub/sub systems

U i bli h/ b ib i ti

Using publish/subscribe communication Advanced topics

Outline

What is publish/subscribe communication? Service models Implementation models Implementation models Survey of representative pub/sub systems

U i bli h/ b ib i ti

Using publish/subscribe communication Advanced topics

Origins of Publish/Subscribe

Database triggers

– Ingres

li t 2 client

Message-oriented middleware

– IBM MQSeries

mailbox mailbox

client 2

Message-based software bus architecture

client 1

• g

– Field – Sun ToolTalk

bus

client 1 client 2

Sun ToolTalk – DEC Fuse

client 3

Common Characteristics

Asynchrony

– send and forget

Reactive

– computation driven by receipt of message

Location/identity abstraction Location/identity abstraction

– destination determined by receiver, not sender

Loose coupling Loose coupling

– senders/receivers added without reconfiguration – one-to-many, many-to-one, many-to-many

Levels of Deployment

From machines to whole networks

– Single machine

» call back, signal

– LAN

» RPC

– WAN

» client/server, overlay router network Single machine/LAN solutions are mature WAN (e.g., the Internet) solutions still require

( g , ) q significant new research advances

Basic Approaches to Publish/Subscribe

Listener model

– receivers register with senders – sender/receiver communication is point-to-point

Database trigger model

– “senders” update tables senders update tables – “receivers” (or internal code) notified of changes

Broker model Broker model

– receivers register with common message broker d / i i ti i di t d b – sender/receiver communication is mediated by message broker through a middleware service

Publish/Subscribe Middleware Service

want low air fares

subscribe

United offers DEN MXP low air fares to Europe want

notify

DEN-MXP October

publish

notification

want special offers by United

notify

publication

publish/subscribe service publish/subscribe service

Alitalia offers DEN-MXP

subscription

want to fly December DEN MXP Nov-Feb

subscriber subscription

DEN-MXP

publisher

Sampling of Applications

Gaming File sharing Workflow Auctioning Sensor grids News distribution Intrusion detection Network management Group communication Resource discovery Application

management

Software component

integration

General Goals and Objectives

Dynamic integration of heterogeneous

computing and information services

Timely information dissemination Support for loosely-coupled, distributed

li ti applications

Scalable deployment

Outline

What is publish/subscribe communication? Service models Implementation models Implementation models Survey of representative pub/sub systems

U i bli h/ b ib i ti

Using publish/subscribe communication Advanced topics

Basic Elements of a Service Model

Data model

– structure – types

Subscription model

– filter language filter language – scope – subscription modifications subscription modifications

General challenge

i l bilit – expressiveness versus scalability

Data Model

Fare Offer United DEN MXP 850 USD United DEN MXP 850 USD

getAirline setAirline tO i i string getOrigin setOrigin getDestination

Airline United

list of values

<fare-offer>

setDestination getPrice setPrice

Origin DEN Destination MXP fare offer <al>United</al> <orig>DEN</orig> <dest>MXP</dest>

setPrice getCurrency setCurrency

Price 850 Currency USD <dest>MXP</dest> <price>850</price> <cur>USD</cur>

set of named values

</fare-offer>

tree of named values full type system

Subscription Model: Filter Language

Operators are implied by the data model

string prefix, suffix, substring, regular g g expression list value operators + indexing named values value operators + selection tree value operators + structure walk p full type user-defined operators

Example: Siena Data Model

A publication is a set of typed attributes

attribute ≡ (type,name,value)

Example: Siena Filter Language

A filter is a list of attribute constraints

attribute constraint ≡ (type,name,operator,value)

Example: Siena Matching Rules

Matching is defined by covering relations

p matches f (or f covers p)

publication p filter f airline = United price < 900 refundable = no price = 850 price < 900 price > 300 destination any price = 850 destination = MXP airline = United y

Scope/Expressiveness of Subscriptions

Subject Fare Offer Ai li U it d

Channel

joinChannel(37)

Airline United Origin DEN Destination MXP Price 850

– channel identifier – equality over channel identifiers

channel 37 Price 850 Currency USD

identifiers

Subject (or topic)

– designated subject field

Subject Fare Offer Airline United Origin DEN

subscribe(“Fare∗”)

designated subject field – general filter applied to designated field

Destination MXP Price 850 Currency USD Subject Fare Offer

Content

– all message fields

Airline United Origin DEN Destination MXP

subscribe(“Origin=DEN,Price<900”)

– general filter applied to full message content

Price 850 Currency USD

Multicast vs. Content-Based Pub/Sub

Multicast Service Content-Based Service joinGroup(IF,group) leaveGroup(IF,group) ifconfig(IF,predicate) ifconfig(IF,false) ip_send(datagram)

dest=224.11.22.33 dest=224.11.22.33

send(message)

resource=printer resource=printer 224.11.22.35 224.11.22.33 dest=224.11.22.35 dest=224.11.22.35 dest=224.11.22.33 dest=224.11.22.33 resource=printer language=PS resource=printer language=PS resource=printer language=plc5 resource=printer language=plc5 224.11.22.34 dest=224.11.22.34 dest=224.11.22.34 resource=printer status=ready resource=printer status=ready maker= HP age > 3 maker= HP age > 3

partition

age 3 age 3

arbitrary subsets

Multicast vs. Content-Based Pub/Sub

Special case of content-based pub/sub

– joining multicast group g is equivalent to defining a receiver predicate that selects messages dd d addressed to g

Multicast designed for streaming media

– relatively stable set of groups and membership – simple channel-based interface

Content-based pub/sub is intended to

support unpredictable interaction patterns pp p p

– sophisticated content-based interface

Some Filter Language Issues

Negation operator

increased expressiveness unclear utility

set-of(¬f) ≡ Χpub \ set-of(f)

User-defined operator

increased expressiveness increased expressiveness unpredictable execution

Patterns

md5-hash(signature) = “7dc81ef”

Patterns

increased expressiveness

• l

h i f ti unclear choice of semantics unpredictable execution

(f1•f2)* • (f3|f4)

Example: Siena Filter Pattern

A pattern correlates messages of interest

f = f1 followed by f2

airline = United price < 900 filter f2 airline = United price < 1200 filter f1 price < 900 price > 300 destination any price < 1200 price ≥ 900 destination any y y

Subscription Modifications

Incremental style

– effect of each subscription is “additive” – new filter represents logical relation among subscriptions – service must provide an explicit unsubscribe command – client must “remember” subscription history

Subscribe-and-forget style

– effect of each subscription is complete p p – no need for unsubscribe nor history

Incremental Subscriptions

airline = United i 400 airline = United i 400

=

price < 400 price < 400 airline = United airline = United

∨

airline = United

= ⊕

destination = MXP price < 400

∨

destination = MXP

=

airline = United airline = United airline = United

⊕ ⊕

price < 900 price < 900

∨

destination = MXP

=

airline = United airline = United airline = United

⊕

airline = United price < 650 airline = United price < 900

∨

airline = United destination = MXP

=

i li U it d i li U it d airline = United price < 900 airline = United destination = MXP

=

Subscribe and Forget

airline = United i 400 airline = United i 400

=

price < 400 price < 400 airline = United airline = United

= ⊕

destination = MXP destination = MXP

=

airline = United airline = United

⊕

airline = United airline = United

⊕

price < 900 price < 900

=

airline = United price < 650 airline = United price < 650

= ⊕ = ∨ f1 f1 f2 f2 ∨ f1 f1 f2 f2

Outline

What is publish/subscribe communication? Service models Implementation models Implementation models Survey of representative pub/sub systems

U i bli h/ b ib i ti

Using publish/subscribe communication Advanced topics

Basic Elements of Implementation Model

Architecture

– centralized – static routing, federation, replication – store-and-forward routing – peer-to-peer versus client/server p p

Platform

– multicast TCP overlay CORBA active multicast, TCP overlay, CORBA, active distributed database, web services

Gateways Gateways

General Service Implementation

subscribe publish notify

publish/subscribe service p

Publications must meet subscriptions somewhere Service must decide whether publications match

subscriptions

Service must notify subscribers

Centralized Implementation

subscribe publish notify server/dispatcher/broker

publish/subscribe service

Evaluating Centralized Implementation

Simple

– every (client) application sends subscriptions and publications to a single server – the server maintains and evaluates all subscriptions locally

But not very “scalable”

– one server handles every subscription and every message of every client – the server becomes a network bottleneck – all clients must trust the same server (privacy)

Naïve Extension: Broadcast Replication

replicated servers replicated servers

subscribe publish notify

Publications are relayed to every server

y y

Subscriptions remain local to their servers

Evaluating Broadcast Replication

Simple

– hopefully making use of network-level broadcast

Reduces computation p

– only local subscriptions – same number of publications same number of publications

Still not very scalable

little flexibility – little flexibility – little privacy i d t k t ffi – increased network traffic

Another Simple Extension: Federation

conditional relay conditional relay

subscribe publish notify

1 3 2 … 4 1 3

… …

Import/export statements

4

x,y,a,b →1 x,a,~b →3 x,y,z →4

…

Import/export statements Static, user-defined routes

Evaluating Federation

Reduces traffic

– only imported/exported traffic goes through

Improves privacy p p y Reduces computation

– local subscriptions plus import/export – local subscriptions plus import/export

Still not very scalable

littl fl ibilit – little flexibility – suboptimal network usage – potential errors due to misconfiguration

Advanced: Fully Distributed Architecture

dynamic routing dynamic routing

subscribe … … … … … … … … …

Advanced: Fully Distributed Architecture

dynamic routing dynamic routing

publish notify subscribe … … … … … … … … …

Dynamic Routing

Servers are interconnected through statically

configured links

Servers function as store-and-forward

publication dispatchers

Servers exchange subscription information Servers exchange subscription information

according to a routing protocol

Servers forward publications according to Servers forward publications according to

– subscription information f di t l – forwarding protocol

Evaluating Fully Distributed Architecture

Increases reliability

– exploiting multi-connected topologies

Adapts automatically

– to changes in applications (subscriptions) – to changes in network (topology)

Reduces computation and communication

– using optimized routing and forwarding

Requires complex protocols

– to handle topological routing information – to handle content-based routing information

Example: Hierarchical Routing

simple extension of centralized server simple extension of centralized server

“master” server connection

B

…→A connection …→1

A 3

…→1

1 3 1 C 1 3 2 1 2 C 5 4 5

Example: Hierarchical Routing

simple extension of centralized server simple extension of centralized server

“master” server connection

B

…→A connection …→1

A 3

…→1

1 3 1 C 1 3 2 1 2 C 5 4 5

Evaluating Hierarchical Routing

Reduces traffic and computation

– “leaf” nodes process only local traffic and traffic that matches local subscriptions

» no unnecessary traffic goes down the hierarchy

Still not very scalable

– root server processes every subscription and publication

Hierarchical Routing Improved

combine and simplify subscriptions combine and simplify subscriptions

S1 →A

B

S1 →1 S1

A

S1

1 3 2

Hierarchical Routing Improved

combine and simplify subscriptions combine and simplify subscriptions

S1 covers S2 S =[airline=“UA”] S1 →A

B

S1=[airline= UA ] S2=[airline=“UA”, dest=“MXP”] ⇒ do not propagate S2 S1 →1 S1 →1 S

2

A

S2 →2

1 3 2

S2

2

Hierarchical Routing Improved

combine and simplify subscriptions combine and simplify subscriptions

S1 covers S2 S =[airline=“UA”] S1 →A S* →A

B

S1=[airline= UA ] S2=[airline=“UA”, dest=“MXP”] ⇒ do not propagate S2 S1 →1 S

2

S1 →1 S

2

S*

A

S3 S2 →2 S2 →2 S3 →3 S1 ,S2, and S3 are similar

1 3 2

S1=[airline=“UA”] S2=[airline=“UA”, dest=“MXP”] S3=[airline=“AZ”]

2

3

⇒ propagate S*=[airline=any]

Evaluating Improved Hierarchical Routing

Reduce traffic and computation

– only “new” subscriptions are propagated – servers store and process fewer, more generic subscriptions

Open to a variety of heuristic optimizations p y p

– e.g., based on actual traffic profiles

Applies to other architectures Applies to other architectures

– concept of “content-based subnet address” (see demonstration) (see demonstration)

Other Considerations

Reliability

– reliable transfer and storage of subscriptions – reliable transfer of publications

d d li » assured delivery » causal ordering Value added services Value-added services

– support for temporarily disconnected clients – support for mobile clients – support for mobile clients – publication history service – authentication service authentication service

Outline

What is publish/subscribe communication? Service models Implementation models Implementation models Survey of representative pub/sub systems

U i bli h/ b ib i ti

Using publish/subscribe communication Advanced topics

Survey

Standards Research systems

– CORBA Notification Service – Java Message Service – Le Subscribe – Elvin Hermes – Java Message Service

Commercial systems

– FioranoMQ – Hermes – Steam – JEDI FioranoMQ – IBM WebSphere MQ EB – TIBCO Rendezvous JEDI – Siena – IBM Gryphon

Caveat

– not all features of the systems are presented

OMG’s CORBA Notification Service

Data model: predefined message object

– header contains standard attribute/value pairs and user-defined attribute/value pairs – body contains user-defined attribute/value pairs and user-defined “any” field

Scope/expressiveness

– publishers publish to a topic channel – content-based filters applied to standard attribute/value pairs and body’s user-defined attribute/value pairs

CORBA Notification Service (cont.)

Filter language

– filters modeled as objects containing constraints – constraints correspond to a standard grammar – operators are SQL-like

Sun’s Java Message Service

Data model: predefined message object

– header contains standard attribute/value pairs – properties are user-defined attribute/value pairs – body comes in five standard types

» stream, map, text, object, uninterpreted bytes Scope/expressiveness

– publishers publish to a topic channel pub s e s pub s to a top c c a e – content-based filters applied to header attributes and properties p p

Filter language: SQL-like

Fiorano’s FioranoMQ

Data model

– compliant with JMS 1.1 – also supports XML documents as messages

Scope/expressiveness

– compliant with JMS 1.1 compliant with JMS 1.1

Filter language

compliant with JMS 1 1 – compliant with JMS 1.1 – also supports XPath queries over XML messages messages

Fiorano’s FioranoMQ (cont.)

Implementation model

– “hub and spoke” (i.e., centralized broker) – centralized brokers can be federated using “repeaters” – a repeater is a JMS client (statically) configured to bridge between brokers

repeater p

IBM’s WebSphere MQ Event Broker

Data model

– headers and attribute/value pairs – also supports XML documents in message body

Scope/expressiveness

– topic-based publication and subscription topic based publication and subscription – “subscription point” property of a publisher

» allows subscribers to identify desired publisher set allows subscribers to identify desired publisher set

– filters applied to topics and content

Filter language Filter language

– SQL and XPath

WebSphere MQ Event Broker (cont.)

Implementation model

– broker and clients use message queues – brokers can be federated, with subscription information shared among federation members

» tree of brokers » collective of fully connected brokers (i.e., broadcast replication) » tree of collectives » tree of collectives

WebSphere MQ Event Broker (cont.)

Value-added services

– compliance with JMS 1.0.2 – publications can be retained for later retrieval

» implies database support

– brokers can be configured to apply application processing logic to publications

» e.g., converting currency values

– publication ordering

» sequence number or time stamp

TIBCO’s Rendezvous

Data model

– subject field consisting of a list of strings – opaque body

Scope/expressiveness

– filters applied to subject field filters applied to subject field

Filter language

indexed – indexed – string equality and limited regular expression

TIBCO’s Rendezvous

Implementation model

– federated brokers – statically configured sharing among brokers

IBM’s Gryphon

Data model

– attribute/value pairs

Scope/expressiveness

p p

– both subject- and content-based filtering

Filter language Filter language

– SQL-like

Implementation model Implementation model

– fully connected, replicated brokers

Sampling of Research Systems

Le Subscribe (INRIA/NYU/Univ. of Toronto)

– high-performance matching algorithm

Elvin (DSTC, Australia)

( , )

– publisher-side filtering (“quenching”)

Hermes (Cambridge Univ

UK)

Hermes (Cambridge Univ., UK)

– typed data model routing based on per type “rendezvous” points – routing based on per-type rendezvous points

» message unicast to rendezvous point, disseminated to tree of subscribers (c.f., IP multicast routing) ( , g)

Sampling of Research Systems (cont.)

Steam (Trinity College, Dublin)

– publish/subscribe in ad hoc networks – “proximity” filters, in addition to subject and content filters

JEDI (Politecnico di Milano/CEFRIEL)

( )

– publish/subscribe for mobile clients – uses explicit “moveOut” and “moveIn” functions p

Siena (Univ. of Colorado, USA)

scalability through fully distributed architecture – scalability through fully distributed architecture – filter optimization

Outline

What is publish/subscribe communication? Service models Implementation models Implementation models Survey of representative pub/sub systems

U i bli h/ b ib i ti

Using publish/subscribe communication Advanced topics

Using Publish/Subscribe Systems

Siena

– research prototype – mature implementation f di t ib t d hit t – focus on distributed architecture – advanced implementation features

JMS

JMS

– industry “standard” l i l t ti – several implementations – focus on Java integration and smaller-scale applications applications – rich interface

Siena

Fully functional publish/subscribe service

– first release: 1998 – available for anonymous download

htt // l d d / l/ i / http://www.cs.colorado.edu/serl/siena/ Content-based subscription language

SQL lik d t d l d t – SQL-like data model and operators

Java and C++ implementations

B i i h h SSL

Basic security through SSL Integrated with XML Architected as a distributed service

Sampling of Registered Users

.EDU

– CMU, Columbia, Dartmouth, Indiana Univ., MIT, Oregon, UCI, UMass, Imperial College, TU- Vi FU B li U i f T U i Vienna, FU-Berlin, Univ. of Toronto, Univ. College London, Univ. of Bologna, Univ. of Oslo, Univ of Manitoba Univ de Valladolid

• Univ. of Manitoba, Univ. de Valladolid

.COM

Obj T k l d L kh d M ti – Objsys, Tecknowledge, Lockheed Martin, MITRE Corp., Boeing, Siemens AG Austria, Air France Borland Organix Consulting France, Borland, Organix Consulting, Diverminds, DataVision Consulting Inc.

Hello World!

import siena.*; HelloWorld.java public class HelloWorld { public static void main(String [] args) public static void main(String [] args) throws SienaException { Siena siena = new ThinClient("tcp:host.domain.net:1234"); N tifi ti N tifi ti () Notification n = new Notification(); n.putAttribute("greeting", "Hello World!"); siena.publish(n); } }

Compiling and Running

> javac -classpath siena.jar HelloWorld.java > java -classpath siena.jar j p j Exception in thread "main" siena.comm.PacketSenderException: Connection refused at siena.comm.TCPPacketSender.send(TCPPacketSender.java:80) siena.comm.TCPPacketSender.send(TCPPacketSender.java:80) at siena.ThinClient.publish(ThinClient.java:384) at HelloWorld.main(HelloWorld.java:9)

We need to set up a server to connect to…

Setting up a Siena Network

First we start up a server to…

• n host1

server address: “tcp:host1:1234” > java -classpath siena.jar \ siena.StartServer -port 1234

• n host1

1

tcp:host1:1234

Then we connect other servers…

> java -classpath siena.jar \ siena StartServer port 2222 \

• n host2

2 3

siena.StartServer -port 2222 \

• master tcp:host1:1234

> java -classpath siena jar \

• n host3

> java classpath siena.jar \ siena.StartServer -port 3333 \

• master tcp:host1:1234
• n host4

4

> java -classpath siena.jar \ siena.StartServer -port 4444 \

• master tcp:host3:3333

Siena Connectors and Addresses

Siena communications

– one-way, message oriented – receiver creates a Receiver object sender creates a Sender object – sender creates a Sender object

» passing the address of the Receiver object to a SenderFactory

Address ::= connector-type:type-specific-part

– available connector types

h t t » tcp:host:port » udp:host:port » ka:host:port » ssl:host:port

Back to Our Example…

> java -classpath siena.jar:. HelloWorld

Nothing happens: there are no subscribers so the notification Nothing happens: there are no subscribers, so the notification goes nowhere We need a subscriber to complete the HelloWorld example p p

Hello World Subscriber

import siena.*; Subscriber.java public class Subscriber implements Notifiable { public void notify(Notification n) { System.out.println(e); System.out.println(e); } public static void main(String [] args) th Si E ti { throws SienaException { Siena siena = new ThinClient(args[0]); Filter f = new Filter(); f.addConstraint("greeting", Op.PF, "Hello"); siena.subscribe(f, new Subscriber()); } }

Hello World!

The Siena network is up and running

> java -classpath siena.jar:. Subscriber tcp:localhost:1234 > java -classpath siena.jar:. Subscriber tcp:localhost:1234

• > java -classpath siena.jar:. Subscriber tcp:localhost:1234

event{ greeting="Hello World!"}

So, we first run the subscriber…

• event{ greeting= Hello World! }
• And then run the publisher…

> java -classpath siena.jar:. HelloWorld > java -classpath siena.jar:. HelloWorld >

Synchronous Subscriber

import siena.*; SynchronousSubscriber.java public class SynchronousSubscriber { public static void main(String [] args) throws Exception { Siena siena = new ThinClient(args[0]); Siena siena new ThinClient(args[0]); Filter f = new Filter(); f.addConstraint("greeting", Op.PF, "Hello"); N tifi ti B ff b f N tifi ti B ff () NotificationBuffer buf = new NotificationBuffer(); siena.subscribe(f, buf); Notification n = buf.getNotification(-1); // wait forever System.out.println(n); siena.shutdown(); } }

HelloWorld in JMS [JMS Documentation]

Get a ConnectionFactory through JNDI Get a Destination (i.e., Topic) through JNDI Get a Connection from the ConnectionFactory Get a Session from the Connection Create a MessageProducer from Session and

Destination

Create a Message from the Session Fill in message data Send the Message through the MessageProducer

JMS HelloWorld Publisher

import javax.naming.*;

JMSHelloWorld.java

import javax.jms.*; public class JMSHelloWorld { public static void main(String [] args) throws Exception { Context context = new InitialContext(); ConnectionFactory cf; cf = (ConnectionFactory)context.lookup("ConnectionFactory"); Topic greetings = (Topic)context.lookup("Greetings"); Connection connection = cf.createConnection(); Session session = connection.createSession(false,Session.AUTO_ACKNOWLEDGE); MessageProducer sender = session.createProducer(greetings); TextMessage m = session.createTextMessage(); m.setText("Yo!"); m.setStringProperty("greeting", "Hello World!"); connection.start(); sender.send(m); } }

JMS (Synchronous) Subscriber

import javax.naming.*;

JMSSubscriber.java

import javax.jms.*; public class JMSSubscriber { public static void main(String [] args) throws Exception { Context context = new InitialContext(); Context context = new InitialContext(); ConnectionFactory cf; cf = (ConnectionFactory)context.lookup("ConnectionFactory"); Topic greetings = (Topic)context.lookup("Greetings"); C ti ti f t C ti () Connection connection = cf.createConnection(); Session session = connection.createSession(false,Session.AUTO_ACKNOWLEDGE); MessageConsumer receiver = session.createConsumer(greetings); connection.start(); TextMessage m = (TextMessage)receiver.receive(5000); // 5 sec System.out.println(m.getText()); } }

JMS (Asynchronous) Subscriber

import javax.naming.*;

JMSAsyncSubscriber.java

import javax.jms.*; class HelloListener implements MessageListener { public void onMessage(Message message) { System out println(message); System.out.println(message); } } public class JMSAsyncSubscriber { public static void main(String [] args) { // // ...Initialization... // receiver.setMessageListener(new HelloListener()); connection.start(); } }

Content-Based Subscriptions in JMS

import javax.naming.*;

JMSSubscriber.java

import javax.jms.*; public class JMSSubscriber { public static void main(String [] args) throws Exception { // // // ...Initialization... // MessageConsumer receiver; ( receiver = session.createConsumer(greetings, "(greeting = ‘Hello’)"); connection.start(); TextMessage m = (TextMessage)receiver.receive(5000); // 5 sec System.out.println(m.getText()); } }

Outline

What is publish/subscribe communication? Service models Implementation models Implementation models Survey of representative pub/sub systems

U i bli h/ b ib i ti

Using publish/subscribe communication Advanced topics

Advanced Topics

Security Client mobility Client mobility

B h ki

Benchmarking

Security of Publish/Subscribe Systems

Reliability Access control (i e

policies)

Access control (i.e., policies)

A th ti ti

Authentication Privacy

Reliability

Fault tolerance

– heart-beat propagation of routing information – dynamic discovery of (optimal) paths within n- connected network – generic, robust content-based routing

Fair resource allocation Fair resource allocation

– flow control accounting – accounting

Access Control Policies

Publish/subscribe firewalls

– filters on outgoing publications

» confidentiality of intra-domain events » blocking and/or rewriting (anonymization)

– filters on incoming publications

» mandatory restrictions on inter-domain traffic » additional filtering

filt t i b i ti – filters on outgoing subscriptions

» confidentiality of subscriptions » blocking and/or rewriting (over generalization) » blocking and/or rewriting (over-generalization)

Authentication

Value-added service

– clients apply and verify MACs

Native implementation

– “MAC” operator – MAC operator

[airline=“UA”, price<900, has_valid_mac(K)]

Privacy: “Strong” Security

Confidentiality of subscriptions with an untrusted

i f t t infrastructure

– clients want the service to filter publications on their behalf without “knowing” what it is filtering behalf without knowing what it is filtering

» privacy of publications » privacy of subscriptions

Approximate solutions

– encrypted computations – filter obfuscation – anonymization (hiding many subscribers behind a proxy)

• ver generality in subscriptions

– over-generality in subscriptions

Mobility Goal

Combine the benefits of mobile applications

with the communication services offered by advanced publish/subscribe services

Kinds of Mobility

Host mobility Code mobility

– movement of mobile hosts

code code code

Code mobility

– movement of code among hosts

» weak mobility: code fragment is relocated by creating » weak mobility: code fragment is relocated by creating a fresh copy at the destination point » strong mobility: movement of code while maintaining th t t f ti the state of execution

d d

code code

code code

code

Mobility Introduces Problems

Client may lose some messages while

disconnected

Part of client’s state is maintained by the

service service

– subscriptions – routing information – routing information – messages in transit

Enabler: Message Persistence

m m m m

S S S

m m m m C C C m m m m

Persistence and Mobility

code code code code

Message Download Protocol

2) Disconnect(c,H) 1) Save(c,H) T H 3) Move(c) 4) Connect(c,T) 6) Subscribe(f,c) 7) D l d( H) 5) Save(c,T)

C

7) Download(c,H) Merge(c) 8) Download(T)

C

8) Download(T) Problem: race conditions Solutions: scout message, delay

Evaluating Publish/Subscribe Systems

Very large parameter space

– network topology – distribution of clients – rate of publication – rate of subscription p – publication content

» distribution of attribute names, types, and values y

– subscription content

» distribution of constraint names, types, and values We need a benchmark

A Publish/Subscribe Benchmark

Requirements

– evaluate support for existing applications – evaluate performance with existing applications – evaluate performance for future applications

Benchmark sections

– interface suitability: capability model interface suitability: capability model – traces from existing applications synthetic benchmark – synthetic benchmark

Outline

What is publish/subscribe communication? Service models Implementation models Implementation models Survey of representative pub/sub systems

U i bli h/ b ib i ti

Using publish/subscribe communication Advanced topics

Conclusion

Prospects for publish/subscribe

– could it one day dominate network traffic?

» multiparty versus point-to-point communication Open Problems and Future Directions

– seamless integration with other services g

» file transfer, remote procedure call, naming, …

– finding an appropriate network foundation g

» multicast, content-based networking

– communication ontologies