Distributed Applications Software Engineering 2017 Alessio Gambi - - PowerPoint PPT Presentation

Distributed Applications Software Engineering 2017 Alessio Gambi - Saarland University Based on the work of Cesare Pautasso, Christoph Dorn, and other sources

ReCap

Software Architecture A software system’s architecture is the set of principal design decisions made about the system. N. Taylor et al. Abstraction Communication Visualization and Quality Attributes Representation

Every system a software architecture has What designers want

Design • Architectural Styles • Architectural Patterns • Building Blocks Software Components - Component API/Interfaces - Software Connectors -

3-Tier Architecture Presentation Business logic Web Browser Data source Front End App Server Back End

UI Page Static Submit Logic ArticleView Resources Skinning Localization ArticleEdit Parser Writes Reads Backend

UI Page Cached Cached Loader Submit Logic HTML File Skinning Localization Cache Static Resources ArticleView Cached Parser ArticleEdit Cache Job Queue Parser Writes Job Runner Reads Backend Writes Precompile/Recompile Regenerate/Invalidate

Distributed Applications

Client/Server • Many clients, active, close to users • One server, passive, close to data • Single point of failure, scalability • Security, scalability

Distribution and Lifecycle In distributed applications the lifecycle of remote objects is disjoint from the local ones. We must explicitly design the lifecycle of those remote entities

Distribution and Lifecycle In distributed applications the lifecycle of remote objects is disjoint from the local ones. We must explicitly design the lifecycle of those remote entities Static and Lazy instances Leasing Per-request instances Pooling Client-dependent instances Passivation

Static Instances Process A Server Process 2) invoke Client Object 1) create 3) invoke n) destroy Process B Server Boundary Client Application Machine n-1) shutdown Remote object instances exist independently of any clients They last as long as their container (server)

Lazy Instances Process A Server Process 4) create 3) Invoke Client Servant on Obj X Invoker for Obj X 5) invoke Boundary 1) create 2) register Object “X” Machine Server Application Instantiate object upon first request Save computational resources

Per-request Instances Process A Server Process 2a) create 2) Invoke Client 2b) invoke Servant on Obj X Invoker 2c) destroy for Obj X 1) create Servant 3) Invoke Process B for Obj X on Obj X 3a) create Server Client 3b) invoke Boundary Application Machine 3c) destroy Each request processed by a fresh instance Provide max logical isolation (but high cost)

Client-dependent Instances 3) invoke 4) invoke Process A Server Process 2b) create 2a) new Client Remote Instance Object Factory 1) create 5a) new Process B Object Instance 5b) create Server Client Application 6) invoke Requests from the same client processed by the same instance (but there might be a one-to-many mapping) Remote objects extend client logic and share its state

Leasing Server Process 5) invoke Object “X” Process A 4) Invoke on Obj X Client Invoker 2) create 3) create lease 4) renew lease 1) Create X Lifecycle Manager After lease expired: 6) destroy Avoid removal of per-client objects when not used by periodically renew the lease

Pooling Process A Server Process Servant 6) invoke Servant for X Servant 4) Invoke Client for X Servant for X on Obj X Invoker for X 7) put servant back 3) register pooled instance “X” 5) get idle servant 2) create Boundary Machine Server Object Application Pool “X” 1) create Maintain a (possibly dynamic) set of generic objects to serve clients requests Clean up state before returning to the pool

Passivation Process A Servant Server Process 1b,8) invoke for “X” 1a,4) Invoke on Client Invoker Obj X 7a) create 7b) activate 5) activate (objId) 3) storeState (objId) After timeout Lifecycle 2a) passivate Manager 2b) destroy 6) getState (objId) Save resources by freezing “per-client” objects Objects are reactivated upon first request

(A)Synchronization Remote invocations can be either synchronous or asynchronous. For asynchronous invocations we must handle the evolution of the distributed state across the nodes. One-way Patterns Two-way Patterns Fire and Forget Poll Object Sync with Server Callback

Fire and Forget Process B Process A Requestor Invoker 2b) send 1) invoke Boundary 2a) return Machine Client Best effort (or nobody cares) semantics

Sync with Server Process B Process A Requestor 2) send Invoker 1) invoke 3c) invoke 3b) return Client Object 3a) reply Requestor ensures that the request correctly arrived 
 to server (but not processed) Delivery confirmation semantics

Poll Object (or Future) Process B Process A Requestor 2) invoke Invoker 1) invoke 3) isAvailable = false 4) storeResult Client Poll Object 5) isAvailable = true 6) getResult Local stub on client’s machine checks if results are ready

Callback Process B Process A Requestor 3) invoke Invoker 2) invoke 4) finished(result) Callback Client Object 1) create 5) execute Execute code whenever the remote request returns

Publish/Subscribe • Subscription to queues or topics • Loose coupling

Pub/Sub vs Event-Driven

Pub/Sub vs Event-Driven no specific roles local/distributed

Pub/Sub vs Event-Driven opposite roles no specific roles mostly distributed local/distributed

Message Bus • Publish • Subscribe • Notify

MOM Message-Oriented Middleware Messaging Middleware Receive Message Receiver Send Message Queue Sender Receiver Queue Receive Reply Receiver Send Reply

MOM Message-Oriented Middleware Messaging Middleware • Processing always on Receive Message consumer Receiver Send Message Queue • Queues provide Sender Receiver persistence and Queue Receive Reply decoupling (async) Receiver Send Reply

Reply or don’t reply? MOMs enable both request-only and request-reply interactions despite sender/receiver do not know each other addresses

Reply or don’t reply? MOMs enable both request-only and request-reply interactions despite sender/receiver do not know each other addresses Uniquely identify a request message (ID) + MessageType=REQUEST|REPLY & MessageID = ID = Correlation between the requests and replies

Handling Messages • Routing 
 Content-based, Dynamic • Filtering 
 Message filter • Transforming messages 
 Splitter, Aggregator • Transforming messages content 
 Normalizer, Content Enricher, Content Filter • Transforming message envelope 
 Envelope wrapper

Content-based Routing Destination decided using the payload

Dynamic Routing Destination not fixed but chosen using rules

Message Filter Remove un-needed messages

Splitter Decompose a composite message in parts

Aggregator Use the parts to create a composite message

Content Filter Filter from a composite message unneeded payload

Content Enricher Use additional data to augment messages

Normalizer Route messages to translators which transform the to a common format

Enveloper Wrapper Bridged delivery via wrapping messages into other messages

Messaging Bridge link multiple messaging systems to make messages exchanged on one also available on the others

Pipe & Filter • Clean separation: filter process, pipe transport • Heterogeneity and distribution • Only batch processing, serializable data • Composability, Reuse

Stream • Send • Receive

Streaming • Infinite sequence of messages 
 simple/primitive, complex • Discrete - Messages 
 stock markets, twitter • Continuous - Data 
 video, audio

Streaming and Data Analytics Streaming Sink Streaming Streaming Sink MW Source Streaming Sink Unicast or multicast communication channels No discrete unit of interaction (request/response) Low overhead, but mostly transport/communication

Sync/Async Streams • Synchronous 
 Time matters (e.g., minimum transfer rate) • Asynchronous 
 Sequence matters (e.g., no specific transfer rate)

Sync/Async Streams • Synchronous 
 Time matters (e.g., minimum transfer rate) • Asynchronous 
 Sequence matters (e.g., no specific transfer rate) • Isochronous 
 Time is essence (e.g., both min and max transfer rate)

Processing Model Complex Event Processor Application-specific data processing Incoming streams Output complex messages Streaming m3 m2 m1 data m m3 m2 m1 … … … … … … s3 s2 s1 Streaming s3 s2 s1 data s Complex stream/multiple streams data processing

Processing Model Complex Event Processor • Event representation 
 POJO, Maps, Object-Arrays, XML, etc.. • Continuous processing 
 events processes as they arrive and sent to output • Listeners and notifications 
 both incoming and outgoing events • Domain specific languages (DSL) 
 describe conditions, transformations, etc.