Communication Middleware Software Layers 1.1 A distributed system - - PowerPoint PPT Presentation

Communication Middleware

Software Layers

A distributed system organized as middleware. Note that the middleware layer extends over multiple machines. 1.1

2

3

Communication Alternatives

฀ Raw message passing

฀ TCP/IP: reliable byte stream ฀ UDP/IP: unreliable, packet oriented, connectionless

Problem: low level --> not easy to use

฀ OS Abstractions: Distributed Shared Memory

Approach: on page fault, get the remote page Pros: easy to program. Cons: high overhead, hard to handle failures

4

RPC (Xerox RPC)

n Remote Procedure Call (RPC)

฀ Main idea: Call remote procedures as if they are local ones. ฀ Key adv.: general, easy to use, and efficient. l

Uses the well-known procedure call semantics.

l

The caller makes a procedure call and then waits. If it is a local procedure call, then it is handled normally; if it is a remote procedure, then it is handled as a remote procedure call.

l

Caller semantics is blocked send; callee semantics is blocked receive to get the parameters and a nonblocked send at the end to transmit results.

5

RPC Architecture

6

Example

Interface{ // has enough information for compile time // checking, and generating proper calling sequence int func1(int arg) int func2(char * buffer, int size) }

7

Example: Client stub

int func1(int arg)

Client stub for func1 int func1(int arg) {

Create req Pack table_index, fid, arg, …etc Send req Wait for reply Unpack results Return result

}

8

Example: Server stub

int func1(int arg)

Server stub fun1{ Unpack request Find the requested function Call the function Pack results Send results }

9

Example: How about fun2

Interface{ // has enough information for compile time // checking, and generate proper calling sequence int func1(int arg) int func2(char * buffer, int size) } Problems:

฀ *buffer points into different address space.

(solution: copy/restore if needed)

฀ Can be input or output

(solution: add an argument type (in|out|both)

฀ Complex data structures

10

Binding

Example RPC-based file system Server Load function in memory Populate export table (fid, fname, call_back function) Export interface Registry Store s_ip, fname

function id (fid), unique id that is generated every time the function is exported

11

Binding

Example: RPC-based file system Client Query registry Query server Store info in Memory (fname, server_ip, fid, index) Registry Server

Index Proc. Name fid call back pointer Export table

12

Runtime

Client call fun1() Server Check export table Run function Return result

Runtime choices: TCP/IP?

• High overhead on server, many connections and state
• High latency (setting/closing connections)

UDP/IP

• Unreliable. (used by Xerox RPC)

Index Proc. Name fid call back pointer Export table

13

Fault tolerance

Lost request Client call fun1() Server Run function

time out

14

Fault tolerance

Lost reply Client call fun1() Problem? function executed twice!! Server Run function Run function

time out

15

Call semantics

฀ At least once: the function is executed at least once. ฀ At most once: the function is executed once or not at all

Neither one is perfect: exactly-once is desired but not possible with a single server RPC.

Xerox RPC implements, at most once semantic:

฀ If success is returned  exactly once: it is guaranteed that it

was executed exactly once

฀ On failure: at most once.

16

Xerox at most once semantics

To avoid duplicates Client call fun1() Server Check export table Check prev. calls table Run function? Return result

(client ip, process ID, seq #, result) Strictly increasing sequence number

Unique processes id

client_ip

• Proc. Id

Seq# Index Proc. Name fid call back pointer Export table

17

Fault tolerance

Lost reply Client call fun1() Server Update the prev. call table Run function Check prev. calls tbl Duplicate, return without execution

time out client_ip

• Proc. Id

Seq# IP1 22 10

18

Fault tolerance

Client failure with repeated seq#? Client call fun1() Later call of fun1() Server Run function Check prev. calls tbl

• >Duplicate, return

reboot

Fun1() was acked a second time without execution.

Solution: add clock based conversation id, to every call. <Conversation_id, seq#> is strictly increasing client_ip

• Proc. Id

Seq# IP1 22 10

19

Fault tolerance: Server crash

Client call fun1() Server Run function update prev. calls tbl Run function again !!

time out reboot Solution: Rebind on server restart with new fid client_ip

• Proc. Id

Seq# IP1 22 10

20

Fault tolerance: Server crash

Client call fun1() Server Run function Rebind functions with a different fid fid does not match

time out reboot What should the application do with this error?

CS454/654 3-21

Message-Based Communication

฀ Lower-level interface to provide more flexibility ฀ Popular systems (RabbitMQ, ActiveMQ, Kafka) ฀ Two (abstract) primitives are used to implement these

฀

send

฀

receive

฀ Issues:

฀ Are primitives blocking or nonblocking (synchronous or asynchronous)? ฀ Are primitives reliable or unreliable (persistent or transient)?

CS454/654 3-22

Message-Queuing System: P2P

฀

The relationship between queue-level addressing and network-level addressing.

From Tanenbaum and van Steen, Distributed Systems: Principles and Paradigms

CS454/654 3-23

Message Brokers

฀

The general organization of a message broker in a message-queuing system.

2-30

From Tanenbaum and van Steen, Distributed Systems: Principles and Paradigms

CS454/654 3-24

Message-Queuing System: flexible MQ network

2-29

From Tanenbaum and van Steen, Distributed Systems: Principles and Paradigms

CS454/654 3-25

Synchronous/Asynchronous Messaging

฀ Synchronous

฀ The sender is blocked until its message is stored in the local buffer at the receiving host or

delivered to the receiver.

฀ Asynchronous

฀ The sender continues immediately after executing a send ฀ The message is stored in the local buffer at the sending host or at the first communication server.

CS454/654 3-26

Transient/Persistent Messaging

฀ Transient

฀ The sender puts the message on the net and if it cannot be delivered to the sender or to the next

communication host, it is lost.

฀ There can be different types depending on whether it is asynchronous or synchronous

฀ Persistent

฀ The message is stored in the communication system as long as it takes to deliver the message to

the receiver