Distributed Systems Principles and Paradigms Maarten van Steen VU - - PowerPoint PPT Presentation

Distributed Systems Principles and Paradigms

Maarten van Steen

VU Amsterdam, Dept. Computer Science Room R4.20, steen@cs.vu.nl

Chapter 10: Distributed Object-Based Systems

Version: December 2, 2009

Contents

Chapter 01: Introduction 02: Architectures 03: Processes 04: Communication 05: Naming 06: Synchronization 07: Consistency & Replication 08: Fault Tolerance 09: Security 10: Distributed Object-Based Systems 11: Distributed File Systems 12: Distributed Web-Based Systems 13: Distributed Coordination-Based Systems

2 / 40

Distributed Object-Based Systems 10.1 Architecture

Remote distributed objects

Data and operations encapsulated in an object Operations implemented as methods grouped into interfaces Object offers only its interface to clients Object server is responsible for a collection of objects Client stub (proxy) implements interface Server skeleton handles (un)marshaling and object invocation

Server machine Object Client machine Proxy Same interface as object Interface State Method Client invokes a method Network Skeleton invokes same method at object Marshalled invocation is passed across network Client OS Server OS Server Skeleton Client 3 / 40

Distributed Object-Based Systems 10.1 Architecture

Remote distributed objects

Types of objects I Compile-time objects: Language-level objects, from which proxy and skeletons are automatically generated. Runtime objects: Can be implemented in any language, but require use of an object adapter that makes the implementation appear as an object. Types of objects II Transient objects: live only by virtue of a server: if the server exits, so will the object. Persistent objects: live independently from a server: if a server exits, the object’s state and code remain (passively) on disk.

4 / 40

Distributed Object-Based Systems 10.1 Architecture

Example: Enterprise Java Beans (EJB)

What is it Java object hosted by special server that allows for different means of calling the object by remote clients.

Local OS Network Server kernel JMS JNDI JDBC RMI EJB EJB EJB Container Server Services

5 / 40

Distributed Object-Based Systems 10.1 Architecture

Types of EJBs

Four different types Stateless session bean: Transient object, called once, does its work and is done. Example: execute an SQL query and return result to caller. Stateful session bean: Transient object, but maintains client-related state until the end of a session. Example: shopping cart. Entity bean: Persistent, stateful object, can be invoked during different sessions. Example: object maintaining client info on last number of sessions. Message-driven bean: Reactive objects, often triggered by message types. Used to implement publish/subscribe forms of communication.

6 / 40

Distributed Object-Based Systems 10.1 Architecture

Globe distributed objects

Observation Most distributed objects are not distributed at all: state is kept at a single node. Alternative: Globe objects, which are physically distributed across multiple machines.

Local object Distributed shared object Process Interface Network

7 / 40

Distributed Object-Based Systems 10.1 Architecture

Globe distributed objects

Note To make DSOs generic, we need to separate function from distribution support

Control subobject Replication subobject Semantics subobject Communication subobject Communication with other local

• bjects

Same interface as implemented by semantics subobject

8 / 40

Distributed Object-Based Systems 10.1 Architecture

Globe distributed objects

Control subobject Replication subobject Semantics subobject Communication subobject Communication with other local

• bjects

Same interface as implemented by semantics subobject

Note Replication subobject essentially decides how and when the local semantics subobject will be invoked.

9 / 40

Distributed Object-Based Systems 10.2 Processes

Processes: Object servers

Servant The actual implementation of an object, sometimes containing only method implementations: Collection of C or COBOL functions, that act on structs, records, database tables, etc. Java or C++ classes Skeleton Server-side stub for handling network I/O: Unmarshalls incoming requests, and calls the appropriate servant code Marshalls results and sends reply message Generated from interface specifications

10 / 40

Distributed Object-Based Systems 10.2 Processes

Processes: Object servers

Object adapter The “manager” of a set of objects: Inspects (as first) incoming requests Ensures referenced object is activated (requires identification of servant) Passes request to appropriate skeleton, following specific activation policy Responsible for generating object references

11 / 40

Distributed Object-Based Systems 10.2 Processes

Processes: Object servers

Local OS Request demultiplexer Object adapter Object's stub (skeleton) Server with three objects Server machine Object adapter

Observation Object servers determine how their objects are constructed

12 / 40

Distributed Object-Based Systems 10.2 Processes

Example: Ice

main(int argc, char* argv[]) { Ice::Communicator ic; Ice::ObjectAdapter adapter; Ice::Object

• bject;

ic = Ice::initialize(argc, argv); adapter = ic->createObjectAdapterWithEndPoints ( "MyAdapter","tcp -p 10000");

• bject

= new MyObject; adapter->add(object, objectID); adapter->activate(); ic->waitForShutdown(); }

Note Activation policies can be changed by modifying the properties attribute of an adapter. Ice aims at simplicity, and achieves this partly by putting policies into the middleware.

13 / 40

Distributed Object-Based Systems 10.3 Communication

Client-to-object binding

Object reference Having an object reference allows a client to bind to an object: Reference denotes server, object, and communication protocol Client loads associated stub code Stub is instantiated and initialized for specific object Two ways of binding Implicit: Invoke methods directly on the referenced object Explicit: Client must first explicitly bind to object before invoking it

14 / 40

Distributed Object-Based Systems 10.3 Communication

Client-to-object binding: implicit/explicit

Distr object* obj ref;

• bj ref = ...;
• bj ref->do something();

Distr object* obj ref; Local object* obj ptr;

• bj ref = ...;
• bj ptr = bind(obj ref);
• bj ptr->do something();

Some remarks Reference may contain a URL pointing to an implementation file (Server,object) pair is enough to locate target object We need only a standard protocol for loading and instantiating code Observation Remote-object references allow us to pass references as parameters. This was difficult with ordinary RPCs.

15 / 40

Distributed Object-Based Systems 10.3 Communication

Remote Method Invocation (RMI)

Basics (Assume client stub and server skeleton are in place) Client invokes method at stub Stub marshals request and sends it to server Server ensures referenced object is active: Create separate process to hold object Load the object into server process ... Request is unmarshaled by object’s skeleton, and referenced method is invoked If request contained an object reference, invocation is applied recursively (i.e., server acts as client) Result is marshaled and passed back to client Client stub unmarshals reply and passes result to client application

16 / 40

Distributed Object-Based Systems 10.3 Communication

RMI: Parameter passing

Object reference Much easier than in the case of RPC: Server can simply bind to referenced object, and invoke methods Unbind when referenced object is no longer needed

17 / 40

Distributed Object-Based Systems 10.3 Communication

RMI: Parameter passing

Object-by-value A client may also pass a complete object as parameter value: An object has to be marshaled:

Marshall its state Marshall its methods, or give a reference to where an implementation can be found

Server unmarshals object. Note that we have now created a copy

• f the original object.

Object-by-value passing tends to introduce nasty problems

18 / 40

Distributed Object-Based Systems 10.3 Communication

RMI: Parameter passing

Local object O1 Copy of O1 Remote object O2 Local reference L1 New local reference Remote reference R1 Remote invocation with L1 and R1 as parameters Copy of R1 to O2 Machine A Machine B Machine C Client code with RMI to server at C (proxy) Server code (method implementation)

Note Systemwide object reference generally contains server address, port to which adapter listens, and local object ID. Extra: Information on protocol between client and server (TCP , UDP , SOAP , etc.)

19 / 40

Distributed Object-Based Systems 10.3 Communication

RMI: Parameter passing

Local object O1 Copy of O1 Remote object O2 Local reference L1 New local reference Remote reference R1 Remote invocation with L1 and R1 as parameters Copy of R1 to O2 Machine A Machine B Machine C Client code with RMI to server at C (proxy) Server code (method implementation)

Question What’s an alternative implementation for a remote-object reference?

20 / 40

Distributed Object-Based Systems 10.3 Communication

Object-based messaging

Client proxy Callback interface Client RTS Client application

• 2. Request to server
• 4. Call by the RTS
• 1. Call by the

application

• 3. Response from server

Client proxy Polling interface Client RTS Client application

• 2. Request to server
• 1. Call by the

application

• 3. Response from server
• 4. Call by the

application

21 / 40

Distributed Object-Based Systems 10.4 Naming

Object references

Observation In order to invoke remote objects, we need a means to uniquely refer to them. Example: CORBA object references.

Repository identifier IIOP version Host Port Object key Components Profile ID Tagged Profile Object identifier Adapter identifier Other server- specific information Profile Interoperable Object Reference (IOR)

22 / 40

Distributed Object-Based Systems 10.4 Naming

Object references

Observation It is not important how object references are implemented per object-based system, as long as there is a standard to exchange them between systems.

Object system A Object system B Object server Interoperable references (Half) gateway

Solution Object references passed from one RTS to another are transformed by the bridge through which they pass (different transformation schemes can be implemented)

23 / 40

Distributed Object-Based Systems 10.4 Naming

Object references

Object system A Object system B Object server Interoperable references (Half) gateway

Observation Passing an object reference refA from RTS A to RTS B circumventing the A-to-B bridge may be useless if RTS B doesn’t understand refA

24 / 40

Distributed Object-Based Systems 10.4 Naming

Globe object references: location independent

Stacked address Stack of addresses representing the protocol to speak: Field Description Protocol ID Constant representing a (known) protocol Protocol addr. Protocol-specific address

• Impl. handle

Reference to a file in a repository Instance address Contains all that is needed to talk in a propritary way to an object: Field Description

• Impl. handle

Reference to a file in a repository Initialization string Used to initialize an implementation

25 / 40

Distributed Object-Based Systems 10.5 Synchronization

Synchronization

Issue Client stubs hide many of the interactions with objects, which may not always be a good idea

Object Object Process Process Lock

• Unlock

(a) (b)

26 / 40

Distributed Object-Based Systems 10.5 Synchronization

Synchronization

Other problem Should we have server-side or client-side synchronization: Server: Problems when client crashes Clients: Problems when clients at different machines need to synchronize Java Clients at same machine are automatically synchronized; those from different machines are not.

27 / 40

Distributed Object-Based Systems 10.6 Consistency and Replication

Consistency and replication

Observation Objects form a natural means for realizing entry consistency: Data are grouped into units, and protected by a synchronization variable (i.e., lock) Synchronization variables adhere to sequential consistency (i.e., values are set atomically) Operations of grouped data can be nicely grouped: object Problem What happens when objects are replicated? One way or the other we need to ensure that operations on replicated objects are properly

• rdered.

28 / 40

Distributed Object-Based Systems 10.6 Consistency and Replication

Replicated objects

Problem We need to make sure that requests are ordered correctly at the servers and that threads are deterministically sheduled

Middleware Local OS Threads

• Unordered requests

Totally ordered requests Middleware Local OS Threads Thread scheduler Deterministic thread scheduling Unordered requests Object Computer 1 Computer 2 T1

1

T1

2

T2

1

T2

2

29 / 40

Distributed Object-Based Systems 10.6 Consistency and Replication

Replicated objects

Observation We are dealing with nasty issues here. Simplicity may dictate completely serialized (i.e., single-threaded) executions at the server.

30 / 40

Distributed Object-Based Systems 10.6 Consistency and Replication

Replicated invocations

Active replication Updates are forwarded to multiple replicas, where they are carried out. There are some problems to deal with in the face of replicated invocations

Client replicates invocation request All replicas see the same invocation Object receives the same invocation three times Replicated object A B1 B2 B3 C

31 / 40

Distributed Object-Based Systems 10.6 Consistency and Replication

Replicated invocations

Solution Assign a coordinator on each side (client and server), which ensures that only one invocation, and one reply is sent

Coordinator

• f object B

Result Coordinator

• f object C

(a) (b) Client replicates invocation request B1 B1 B2 B2 B3 B3 C1 C1 C2 C2 A A Result

32 / 40

Distributed Object-Based Systems 10.7 Fault Tolerance

Fault tolerance

Observation Nothing really new to report – object systems handle fault tolerance as you would expect them to do. Example CORBA uses special interoperable object group references to allow a client to access replicas:

Repository identifier IIOP ver. IIOP ver. Host-1 Host-N Port-1 Port-N Object key-1 Object key-N Components Components Profile ID Profile ID TAG PRIMARY TAG BACKUP Other group- specific information Other group- specific information Profile-1 Profile-N Interoperable Object Group Reference (IOGR)

33 / 40

Distributed Object-Based Systems 10.7 Fault Tolerance

CORBA FT: Architecture

Replication manager Object group manager Property manager Middleware/RTS Object Interceptor Replication Reliable multicasting Logging & Recovery Log To other replicas Client or

• bject server

34 / 40

Distributed Object-Based Systems 10.8 Security

Security

When objects can (also) be moved/copied around We need to consisder three different security aspects: Secure binding: is client ↔ object binding authorized? Secure invocation: is client authorized to invoke a method? Platform security: is object protected against underlying platform and vice versa? Example For Globe we have the following public/private key pairs: Object key: Owner of the private key can set access rights for users and replica servers. Replica key: Used to make sure that a replica server is hosting an object. User key: Associated with every user.

35 / 40

Distributed Object-Based Systems 10.8 Security

Globe security

sig(O, {R,U,D, }) User certificate U: K+

Alice

K+

Alice

sig(O, {U, }) Replica certificate K+

Repl

K+

Adm

K+

Repl

sig(O, {R, }) K+

Adm

00 00 1 111 0 U:01 00 1 1 111 1 D:1

• 11 00

111 0 R: 11 11 111 0 R: Administrative certificate (c) (b) (a)

User certificate: U[i] = 1 iff user is allowed to invoke method Mi. Replica certificate: R[i] = 1 iff replica is allowed to execute Mi. Administrative certificate: U[i] = 1 iff a user certificate for Mi may be issued. Same for R[i].

36 / 40

Distributed Object-Based Systems 10.8 Security

Secure method invocation

Check user permissions Request secure channel Find suitable replica Establish secure channel Encrypt/sign network packet Decrypt & authenticate request Authorize request Execute

• peration

3 2 4 8 9 11 12 13 10 7 1

Request invocation

5 6

Marshall request Pass request to channel Client-side stub Server-side replica Security

• bject

Security

• bject

Control

• bject

Control

• bject

Repl.

• bject

Repl.

• bject

Comm.

• bject

Comm.

• bject

Semant.

• bject

Unmarshall Pass request

1: Application issues an invocation request. 2: Control subobject checks user permissions; security object should have valid user certificate. 3: Request is marshaled and passed on.

37 / 40

Distributed Object-Based Systems 10.8 Security

Secure method invocation

Check user permissions Request secure channel Find suitable replica Establish secure channel Encrypt/sign network packet Decrypt & authenticate request Authorize request Execute

• peration

3 2 4 8 9 11 12 13 10 7 1

Request invocation

5 6

Marshall request Pass request to channel Client-side stub Server-side replica Security

• bject

Security

• bject

Control

• bject

Control

• bject

Repl.

• bject

Repl.

• bject

Comm.

• bject

Comm.

• bject

Semant.

• bject

Unmarshall Pass request

4: Replication subobject requests middleware to set up secure channel to suitable replica. 5: Security object initiates a replica lookup. 6: Security subobject sets up secure channel. The replica must prove it is allowed to carry out the requested invocation.

38 / 40

Distributed Object-Based Systems 10.8 Security

Secure method invocation

Check user permissions Request secure channel Find suitable replica Establish secure channel Encrypt/sign network packet Decrypt & authenticate request Authorize request Execute

• peration

3 2 4 8 9 11 12 13 10 7 1

Request invocation

5 6

Marshall request Pass request to channel Client-side stub Server-side replica Security

• bject

Security

• bject

Control

• bject

Control

• bject

Repl.

• bject

Repl.

• bject

Comm.

• bject

Comm.

• bject

Semant.

• bject

Unmarshall Pass request

7: Request is passed on to comm. subobject. 8: Comm. subobject encrypts and signs the request. 9: Request is decrypted and authenticated. 10: Request is passed to server-side repl. subobject.

39 / 40

Distributed Object-Based Systems 10.8 Security

Secure method invocation

Check user permissions Request secure channel Find suitable replica Establish secure channel Encrypt/sign network packet Decrypt & authenticate request Authorize request Execute

• peration

3 2 4 8 9 11 12 13 10 7 1

Request invocation

5 6

Marshall request Pass request to channel Client-side stub Server-side replica Security

• bject

Security

• bject

Control

• bject

Control

• bject

Repl.

• bject

Repl.

• bject

Comm.

• bject

Comm.

• bject

Semant.

• bject

Unmarshall Pass request

11: Authorization takes place; user certificate from client-side stub ia passed to replica to verify that request can be carried out. 12: Request is unmarshaled. 13: Operation can be executed.

40 / 40