SLIDE 1
61A Lecture 31
November 14th 2011
Monday, November 14, 2011
Parallel and Distributed Computing
Coordinating groups of computers
Monday, November 14, 2011
61A Lecture 31 November 14th 2011 Monday, November 14, 2011 - - PowerPoint PPT Presentation
61A Lecture 31 November 14th 2011 Monday, November 14, 2011 - - PowerPoint PPT Presentation
61A Lecture 31 November 14th 2011 Monday, November 14, 2011 Parallel and Distributed Computing Coordinating groups of computers Monday, November 14, 2011 So far functions data structures objects abstraction interpretation evaluation 3
SLIDE 2
SLIDE 3
So far
3
functions data structures
- bjects
abstraction interpretation evaluation
Monday, November 14, 2011
SLIDE 4
So far
4
functions data structures
- bjects
abstraction interpretation evaluation
One program One machine One computer
Monday, November 14, 2011
SLIDE 5
Parallel and Distributed Computing
Distributed Computing
Groups of computers communicating and exchanging data with a shared goal.
- Communication networks
- Data storage
- Large scale computing
5
Parallel Computing
One computer with many processes collaborating to execute the same program faster.
- Speeding up computation
Monday, November 14, 2011
SLIDE 6
Lecture plan Today
Distributed computing
Wednesday
Parallel computing: problems
Friday Parallel computing: solutions
6 Monday, November 14, 2011
SLIDE 7
Distributed Computer Systems
7
Interconnected groups of independent computers that collaborate to get work done.
Monday, November 14, 2011
SLIDE 8
Characteristics of distributed systems
- 1. Independent computers
- 2. (Often) In different
locations
- 3. Connected by a network
- 4. Communicate by passing
messages to each other
- 5. A shared computational
goal.
8 Monday, November 14, 2011
SLIDE 9
Examples of distributed systems
Information sharing & communication
Telephone networks, cellular networks The world wide web Skype, IM, Xbox/PlayStation and other online multiplayer systems
Large scale computation
“Cloud computing” - Amazon and Microsoft MapReduce - later in this course
9 Monday, November 14, 2011
SLIDE 10
Topics in Distributed Systems
- Architectures
- Client-server
- Peer-to-peer
- Message passing
- Design principles
- Modularity
- Interfaces
10 Monday, November 14, 2011
SLIDE 11
Architecture
Computers in a distributed system can have different roles depending on the goal of the system. The network of computers can be structured in different ways.
11
Client-server Peer-to-peer
Monday, November 14, 2011
SLIDE 12
Client-Server Architecture
Good for dispensing a service 2 roles Clients: make requests from server Server: listens for requests and responds to them. Many clients Only 1 server.
12
server
client client client client client client client client
request request request request request request request
response
request
response
Monday, November 14, 2011
SLIDE 13
Example: world wide web
13
www.nytimes.com client client client request response The New York Times
Monday, November 14, 2011
SLIDE 14
Example: world wide web Server’s job
Listen for requests Calculate front page
- ads
- personalized content
Send web page back to correct browser
14
Client’s job
Send correct request to server based on user input Display received web page
- fonts & colors
- images
- interactivity
Send further requests
Monday, November 14, 2011
SLIDE 15
Division of labor
15
Client Server
One source Many consumers Provide information
- r service
Use service, or make it usable to humans
Monday, November 14, 2011
SLIDE 16
Single point of failure
16
Server is a single point of failure. System stops working if server goes down. If client goes down,
- nly that client is
affected.
server
client client client client client client client client
request request request request request request request
response
request
response
Monday, November 14, 2011
SLIDE 17
Disadvantage: performance degrades under load
The more clients that want to use a server, the worse the server performs
- Connection speed becomes slow -- limited bandwidth
- Server becomes slow to respond -- limited processing power
Cannot shrink and grow with changing demand
17 Monday, November 14, 2011
SLIDE 18
Peer to peer architecture
Division of labor among all computers All computers send and receive data All computer contribute resources
- Disk space
- Memory
- Processing power
Applications
- Data storage
- Communication
- Large-scale computation
18 Monday, November 14, 2011
SLIDE 19
The importance of an organized network structure
19
A
B
Monday, November 14, 2011
SLIDE 20
The importance of an organized network structure
20
A
B
The shortest path for A to send a message to B.
Monday, November 14, 2011
SLIDE 21
The importance of an organized network structure
21
A
B
A roundabout path Inefficient.
Monday, November 14, 2011
SLIDE 22
The importance of an organized network structure
22
A
B
Everyone sends A’s message to their neighbors, until B is reached. Huge load on network. Inefficient use of resources
Monday, November 14, 2011
SLIDE 23
Supernodes: keep track of network structure
Computers with a special function No longer “pure” peer-to- peer Knows locations of other supernodes Knows which computers are “under” it Keeps track of newcomers and computers that leave
23 Monday, November 14, 2011
SLIDE 24
Example: Skype
Peers: all computers running skype Not a pure peer-to-peer network Supernodes coordinate users and manage sign-ins and sign-outs.
24
A B
Monday, November 14, 2011
SLIDE 25
Disadvantages
Complex network structure Inefficiency in communication
- Can take up a lot of traffic trying to route messages.
Advantages
No single point of failure Can grow and shrink with demand
25 Monday, November 14, 2011
SLIDE 26
Messages
26
A B
1101000110 Used to coordinate behavior Send or receive data Request that a function be executed Signal that a particular event has occurred
Monday, November 14, 2011
SLIDE 27
Message Structure
27
A B
1101000110
SENDER RECIPIENT ... CONTENT ...
Monday, November 14, 2011
SLIDE 28
Message Structure
28
A B
1101000110
SENDER RECIPIENT ... CONTENT ...
So recipient knows where to send response
So message can be routed properly.
Data, remote procedure call, signal, encoded video, text, etc.
Monday, November 14, 2011
SLIDE 29
Within a program
29
A B
(string, ..., ...) Dispatch procedure Sender and recipient implicit Data sent using underlying data structures.
A B
1101000110 ???? Over a network, there are no “underlying data structures”!
Monday, November 14, 2011
SLIDE 30
Message protocol
A set of rules for encoding and decoding messages. All computers in the system must obey the protocol when sending and receiving messages.
Example
The first 3 bytes are the sender The next 3 bytes are the recipient After that is the content, which is video, encoded according to... etc. etc.
30
A B
1101000110
Monday, November 14, 2011
SLIDE 31
Example: messages on the world wide web
31 Monday, November 14, 2011
SLIDE 32
Example: messages on the world wide web
32
Protocol name Server Requested page
Your IP address en.wikipedia.org GET wiki/UC_Berkeley HTTP 1.1
Monday, November 14, 2011
SLIDE 33
Modularity
The components of a system should be black boxes with respect to each other. The black boxes are required to hold up interfaces.
33
Dispatch procedures
Interface = List of messages that can be taken in Responses that should be given to each message
General Systems
Interface = List of inputs that can be taken in Outputs that should be given in response to inputs.
Monday, November 14, 2011
SLIDE 34
Advantages of modularity
Easy to understand => Easy to change and expand If something goes wrong, only defective component needs to be replaced Easy to debug
- Compare real outputs to the supposed interface
- Defective component is the one that doesn’t hold up the
interface any longer.
34 Monday, November 14, 2011