Course introduction Defining distributed system s A distributed - - PowerPoint PPT Presentation

Distributed System s Fall 2 0 0 8

Course introduction

Defining distributed system s

”A distributed system is one in which components located at networked computers communicate and coordinate their actions by passing messages.” (Coulouris, Dollimore, Kindberg, 2005) ”A distributed system is one in which the failure of a computer you didn't even know existed can render your own computer unusable.” (Leslie Lamport, 1987)

Outline

• Staff presentation
• Course presentation
• Lessons from last year
• This year’s course
• General information on middlewares
• Different kinds of communication
• External data representation
• Interaction, Failure handling, Security
• Design Considerations

Staff presentation

• Daniel Henriksson danielh@cs (teacher and assistant)
• Lars Larsson larsson@cs (teacher and assistant)
• P-O Östberg p-o@cs (teacher)

Questions about the practical assignment should be sent to both Daniel and Lars. Questions about the material covered in lectures should be sent to the appropriate teacher.

Course presentation

Students should obtain:

• Knowledge of theoretical models for distributed

systems

• Knowledge of problems and solutions in designing

and implementation of distributed systems The course covers:

• Architectural models of distributed systems
• Client-Server, peer-to-peer, transactions,

transparency, naming, error handling, resource management, and synchronisation

• Computer security in a broad perspective
• Distributed programming and middlewares

Course presentation

• Theoretical part (4,5 Swedish hp)

– Theory, methods, algorithms, and principles

• Practical part (3 Swedish hp)

– Practical obligatory assignment

Lessons from last year ( positive)

• Practical assignment was praised as being interesting

and hard, but rewarding – gave opportunity to apply the theory to practise

• The material was interesting, useful, and challenging
• Both the material and the assignment gave insight

into several problems and how they can be solved

Lessons from last year ( negative)

• The lectures were not satisfactory, and ”not of any

use” (too vague and not enough detail)

• The practical assignment was very time-consuming,

and did not cover all material of the course

• The book was hard to read, and even if one started

to study it ”on time”, it was hard to study all of it

• The practical assignment was unstructured
• Mixed languages – some material was only presented

in Swedish, and students were allowed to hand in reports and exams in either Swedish or English

This year’s course

• New staff working with the course
• Lectures based on the course as it was given two

years ago

• Only English answers and reports are allowed
• Practical assignment requires more structure (project

plan with milestones) and provides more structure (common interface)

• Same book (it is second to none)

The practical assignm ent

• Group communication middleware

– Group handling – Message ordering – Multicast communication

• Presentation of working implementation at the end of

the course

• Solved in pairs (2 students per group)
• Deals with theory from the first couple of lectures

http: / / www.cs.umu.se/ kurser/ 5DV020/ HT-08/ assignment.html

Teaching style

• Pedagogical approach
• Choice of teaching aids
• Transparencies (overhead slides)

– Most come directly from the book’s website

• What should one learn, and at what level of detail?

Architectural m odels

“The architecture of a system is its structure in terms of separately specified components.” (Section 2.2) Placement of components across the network

• Relationships between components

– Servers, clients, peers

• Dynamic systems (downloadable code, seamless

modifications of network) Definitions:

• Platform: lowest-level hardware and software (up to

OS level)

• Middleware: software layer designed to mask

heterogeneity and provide convenient programming model

System architectures

“Pure” models:

• Client-server

– HTTP, FTP, … countless others

• Peer-to-peer

– BitTorrent, Freenet, Direct Connect, … Variations:

• Multiple servers (load balancing, more reliable)
• Mobile code (download code from server and run it)
• Mobile agents
• Thin clients

Design of distributed system s

• Performance issues

– Responsiveness – Throughput – Load balancing

• Quality of service (QoS)

– Reliability, security, performance (time-critical) – Adaptability

• Caching and replication

– Web caching and Content Distribution Networks

• Dependability

– Correctness, security, and fault tolerance

Middlew are

• Distributed systems often utilise middleware

technologies to aid development

• Offers layers of abstraction
• Extends upon traditional programming models:

– Local procedure call -> Remote Procedure Call – OOP -> Remote Method Invocation – Event-based programming model

Applications, services Middleware layers request-reply protocol marshalling and external data representation UDP and TCP RMI and RPC

Middlew are challenges

• Heterogeneous systems

– Networks, hardware, OS, languages, protocols

• Openness
• Security
• Scalability

– Control physical resource costs and performance loss, resource conservation, bottleneck avoidance

• Failure handling

– Detection, masking, tolerance, recovery, redundancy

• Concurrency

– No global time, simultaneously running processes

• Transparency

– Access, location, concurrency, replication, failure, mobility, performance, scaling

Detailed m iddlew are issues

• How are parameters passed, and how is data

converted?

• How are distributed resources (functions, methods,
• bjects) published and discovered?
• How are errors handled in a distributed system?
• Security concerns
• Memory handling – distributed garbage collection?

I nter-process com m unication

• Communication in several layers

– Network protocol – Transport protocol – Message passing – Request/ reply – Transactions

Message passing

send(destination, message); receive(source, message); Five levels of synchronisation:

• 1. Non-blocking send: call to send() returns

immediately as kernel has loaded the parameters

• 2. Blocking send: send() returns when the receiver’s

kernel has received the message

• 3. (Explicit blocking send: send() returns as the

receiver’s application process has received the message – application level)

• 4. Request and reply: send() returns when the receiver

has created a response and sent it back.

Reliability

• Reliable: messages are guaranteed to be delivered

despite a ”reasonable” number of packets being dropped or lost

• Unreliable: messages are not guaranteed to be

delivered in the face of even a single packed dropped

• r lost
• UDP is totally unreliable
• TCP is (almost) reliable

Request / Reply com m unication

• Chosing the underlying transport protocol

– UDP vs TCP

• Message identifier

– Sender and requestID

• Dealing with lost packets

– How many retransmits?

Request / Reply – Retransm it

• When retransmitting, the receiver may end up with

duplicates

• Filter based on message ID
• Carry out the operation again (or resend cached

results) – Idempotent operation – can be repeated with the same results each time – Some kind of history is required when resending cached results

Request / Reply - Protocols

• Request protocol – R
• Request-reply protocol – RR
• Request-reply-acknowledge protocol - RRA

Exam ple RR protocol: HTTP

• Content negotiation
• Authentication
• Persistent Connections (HTTP 1.1)
• ASCII-based protocol supporting binary data
• Methods

– GET, POST, HEAD, PUT, DELETE, OPTION, TRACE

• Request:

– Method, URL, Protoversion, Headers, (Data)

• Reply

– Protoversion, Status, Reason, Headers, (Data)

• RFC1945 (HTTP 1.0), RFC2616 (HTTP 1.1)

Data com m unication

• Problem when sending and recieving data

– Data structures must be ”flattened” – The representation of data types can differ – The representation of text can differ – Big endian (English digits) vs Little endian (German digits)

• Solution

– Use senders representation and include information about translation – Use a stanardized external format

External data representation

• Standard(s) for external represenation of data

structures and primitives

• Could be binary or text

Marshalling – Unm arshalling

• Marshalling

– Convert internal representation (e.g, object) to an external data representation (e.g, text)

• Unmarshalling

– Convert external data to an internal representation

• Marshalling and Unmarshalling is typically taken care
• f by a middleware, without any involvement with

the application programmer.

External data representation

• Examples

– Sun XDR (eXternal Data Representation) – Java Object Serialization (Java RMI) – Corba CDR (Common Data Representation) – XML (eXtensible Markup Language)

• Used in Web services

– ASN (Abstract Syntax Notation)

• Supports different encodings (such as XML)

I nterface

• Specifies procedures (and variables) that can be used

by other modules

• Parameters: input, output, or both
• No pointers
• Service interface

– Provided services

• Remote interface

– Methods and functions accessible from objects and functions in other processes

• Interface Definition Language (IDL)

– Enables communication between objects implemented in different languages

• Web Service Description Language (WSDL)

– Interface definition for Web services

I nteraction

• Algorithm
• Distributed algorithm

– The steps to be taken by each of the processes, including the messages between them

I nteraction ( continued)

• Communication performance characteristics

– Latency (delay), Bandwidth (throughput), Jitter (variation in time)

• Clocks and timing

– Clock drift

• Interaction models

– Synchronous distributed systems – Asynchronous distributed systems

• Event Ordering

– Delays can cause replies to arrive to a third party before the initial message has arrived – Lamports logical time

Failure handling

• Distributed systems are more likely to experience

errors or failures

• Many different kinds

– Local failures, bit-errors, lost packets, no response, method does not exist, etc...

• What should happend when a failure occours?

Security

• Distributed system = increase exposure
• Client- and Server-side authentication
• Client authorization

– Is the client allowed to do this?

• Proof of execution

– The server must be able to prove that something has been executed

Security ( continued)

• Non-repudiation

– It should not be possible to deny doing something that has been done

Design considerations

• Cannot guarantee that things are executed exacly
• nce.
• Transparency approach

Different sem antics

• (Local call – exacly once)
• Maybe once

– Omission failures (dropped packets, crashes)

• At-least-once

– Crash failure, arbitrary failure (multiple executions) – Used by Sun RPC

• At-most-once

– Executed exacly once or not at all – Used by Java RMI, Corba

Transparency

• Two main approaches
• Make it similar to local calls
• But: Slower and more prone to crashes
• Make it clear that it is a remote call
• Programmer should be able to cancel a call
• Programmer is aware that this is a remote call
• Current trend;
• Let the calls be entirely transparent, but use

different interfaces.

Sum m ary

• Introduction to the course
• What is a distributed system
• Consequenses

– Concurrency – No global time – Unrelated errors

Sum m ary ( continued)

• Challenges

– Heterogenity – Openness – Security – Scalability – Error handling – Concurrency – Transparency

• Architectural models
• Fundamental models

Sum m ary ( continued)

• Middlewares

– Request / Reply communication – External data representation

Next tim e

• P-O gives a lecture on middleware technologies,

including the different external data representations – Remote Procedure Call

• Sun RPC

– Remote Method Invocation

• CORBA
• Java RMI