Server Software Design Prof. Chuan-Ming Liu Computer Science and - - PowerPoint PPT Presentation

NTUT, TAIWAN 1

Mobile Computing & Software Engineering Lab Mobile Computing & Software Engineering Lab

Server Software Design

• Prof. Chuan-Ming Liu

Computer Science and Information Engineering National Taipei University of Technology Taipei, TAIWAN

NTUT, TAIWAN 2

Mobile Computing & Software Engineering Lab Mobile Computing & Software Engineering Lab

Conceptual Server Algorithm

Simple Server Algorithm

• 1. Create a socket
• 2. Bind the socket to the port
• 3. Enter an infinite loop

Accept request Process request Reply request

Not enough in practice generally

NTUT, TAIWAN 3

Mobile Computing & Software Engineering Lab Mobile Computing & Software Engineering Lab

Conceptual Server Algorithm

Consider file transfer

First client needs a 200 MB file Second client needs a 20 byte file If the server processes them in FIFO fashion, the second client will wait an unreasonable amount of time for a small transfer

Server usually handles more than one request at a time

NTUT, TAIWAN 4

Mobile Computing & Software Engineering Lab Mobile Computing & Software Engineering Lab

Concurrent vs Iterative

Iterative server: one request at a time Concurrent server: multiple requests at

• ne time

Note: the concurrency is conceptual and generally will use one thread for one request

NTUT, TAIWAN 5

Mobile Computing & Software Engineering Lab Mobile Computing & Software Engineering Lab

Concurrent vs Iterative

Concurrent server refers to whether the server permits multiple requests to proceed concurrently, not whether the underlying implementation uses multiple and concurrent threads

NTUT, TAIWAN 6

Mobile Computing & Software Engineering Lab Mobile Computing & Software Engineering Lab

Concurrent vs Iterative

Concurrent servers are difficult to design and build Why concurrency?

Long delay caused by iterative servers Performance bottleneck resulted from iterative servers effects many clients

NTUT, TAIWAN 7

Mobile Computing & Software Engineering Lab Mobile Computing & Software Engineering Lab

Connection Access or Not

Two major transport protocols:

TCP (Transmission Control Protocol) UDP (User Datagram Protocol)

TCP is connection-oriented and UDP is connectionless The corresponding servers are

Connection-oriented servers Connectionless serves

NTUT, TAIWAN 8

Mobile Computing & Software Engineering Lab Mobile Computing & Software Engineering Lab

TCP Semantics

Point-to-point communication Reliable connection establishment Reliable delivery Flow controlled transfer Full-duplex transfer Stream paradigm

NTUT, TAIWAN 9

Mobile Computing & Software Engineering Lab Mobile Computing & Software Engineering Lab

UPD Semantics

Many-to-many communication Unreliable Service Lack of flow control Message paradigm

NTUT, TAIWAN 10

Mobile Computing & Software Engineering Lab Mobile Computing & Software Engineering Lab

TCP or UDP?

Because the semantics of TCP and UDP differ sharply, a designer cannot choose between connection-oriented and connectionless transport protocols without considering the semantics required by the application protocol Depends on the applications, not the types

NTUT, TAIWAN 11

Mobile Computing & Software Engineering Lab Mobile Computing & Software Engineering Lab

Connection-Oriented Servers

Easier to program since TCP provides all the reliability Drawbacks:

Need a separate socket for each connection Care about the socket allocation and resource consuming Resource limitation problem – due to the clients can crash often

NTUT, TAIWAN 12

Mobile Computing & Software Engineering Lab Mobile Computing & Software Engineering Lab

Connectionless Servers

Access multiple hosts from a single socket No problem of resource depletion No reliable delivery

Need to take care of the reliability Usually, clients take care of retransmitting request Server may retransmit as well

NTUT, TAIWAN 13

Mobile Computing & Software Engineering Lab Mobile Computing & Software Engineering Lab

Connectionless Servers

Achieving reliability can be difficult

Timeout Retransmission

TCP does not allow multicast or broadcast Multicast and broadcast services require UDP Any server that accepts or responds to multicast and broadcast communication must be connectionless

NTUT, TAIWAN 14

Mobile Computing & Software Engineering Lab Mobile Computing & Software Engineering Lab

Stateful or Stateless

The issue of stateless arises from a need to ensure reliability For a service using connectionless transport, like UDP, the application protocols need to ensure the reliability

NTUT, TAIWAN 15

Mobile Computing & Software Engineering Lab Mobile Computing & Software Engineering Lab

Stateless Servers

Consider a connectionless server that allows clients to read information from files

Clients need to specify

Filename Position in the file Amount to read

Straightforward method is to handle each request independently

NTUT, TAIWAN 16

Mobile Computing & Software Engineering Lab Mobile Computing & Software Engineering Lab

Optimizing Stateless Servers

Observe that

The overhead to open and close files is high Data read may be few Clients may sequentially read files

Using buffers to assist reading files

This may be similar to the stateful servers Hashing table is used The IP address and the port number form the index

NTUT, TAIWAN 17

Mobile Computing & Software Engineering Lab Mobile Computing & Software Engineering Lab

Optimizing Stateless Servers

Filename: Y Offset 1024 Buffer pointer: Filename: X Offset: 512 Buffer pointer: Buffer for file X Starting at byte 512 Buffer for file Y Starting at byte 1024 Hash(ip, port)

NTUT, TAIWAN 18

Mobile Computing & Software Engineering Lab Mobile Computing & Software Engineering Lab

Optimizing Stateless Servers

Improvement can be achieved when all the assumptions meet Problem arises when the client programs fail

Consider the situation that the client crashes Buffer will be full

Adjustment can be done by least recently used (LRU) rule for maintaining the hash table Thrashing occurs

NTUT, TAIWAN 19

Mobile Computing & Software Engineering Lab Mobile Computing & Software Engineering Lab

Optimizing Stateless Servers

A programmer must be extremely careful when optimizing a stateless server because managing small amounts

• f

state information can consume resources if clients crash and reboot frequently or if the underlying network duplicates or delay messages

NTUT, TAIWAN 20

Mobile Computing & Software Engineering Lab Mobile Computing & Software Engineering Lab

Types of Servers

Concurrent Connection-oriented Concurrent Connectionless Iterative Connection-oriented Iterative Connectionless

NTUT, TAIWAN 21

Mobile Computing & Software Engineering Lab Mobile Computing & Software Engineering Lab

Iterative or Concurrent

Request processing time (Server) total time the server takes to handle a request Observed response time (Client) total delay from sending a request to receiving the response Server has a queue of requests to handle

NTUT, TAIWAN 22

Mobile Computing & Software Engineering Lab Mobile Computing & Software Engineering Lab

Iterative or Concurrent

If N is the average length of the request queue, then the observed response time is approximately (N/2+1) × the server’s request processing time The observed response time is in proportion to N, N should be reasonable small, say N=5 When N is small, an iterative server is ok;

• therwise, a concurrent serve is preferred

NTUT, TAIWAN 23

Mobile Computing & Software Engineering Lab Mobile Computing & Software Engineering Lab

Iterative or Concurrent

Alternatively, the overall load can be considered

Suppose a server is designed to handle K clients Each client sends R request per second Total number of requests is RK The server’s processing time should be less than 1/KR second per request

If the server can handle at the required rate, iterative way is ok; otherwise, concurrent method is selected

NTUT, TAIWAN 24

Mobile Computing & Software Engineering Lab Mobile Computing & Software Engineering Lab

Iterative Server Algorithms

Easy to design, program, debug, and modify. Working best with simple services accessed by a connectionless access protocol

NTUT, TAIWAN 25

Mobile Computing & Software Engineering Lab Mobile Computing & Software Engineering Lab

Iterative Connection- Oriented Server Algorithm

• 1. Create a socket and bind to the well-known

address for the service being offered

• 2. Place the socket in passive mode and make it

ready for use by a server

• 3. Accept the next connection request from the

socket, and obtain a new socket for the connection

• 4. Repeatedly read, process, and reply a request
• 5. When a particular connection is done, close the

connection and go back to step 3

NTUT, TAIWAN 26

Mobile Computing & Software Engineering Lab Mobile Computing & Software Engineering Lab

Binding a Well-known Address

Like clients, servers use procedure getportbyname to map a service name into a well-known port number Using function bind to specify a connection IP address is necessary for bind, but it is difficult to select the IP address

NTUT, TAIWAN 27

Mobile Computing & Software Engineering Lab Mobile Computing & Software Engineering Lab

INADDR_ANY

Constant, INADDR_ANY, provided by the socket interface helps binding

specifies a wildcard address that matches any

• f the host’s IP address

Has a multi-homes host accept incoming communication addressed to any of the host’s IP address

NTUT, TAIWAN 28

Mobile Computing & Software Engineering Lab Mobile Computing & Software Engineering Lab

Iterative Connection- Oriented Server Algorithm

• 1. Create a socket and bind to the well-known

address for the service being offered

• 2. Place the socket in passive mode and make it

ready for use by a server

• 3. Accept the next connection request from the

socket, and obtain a new socket for the connection

• 4. Repeatedly read, process, and reply a request
• 5. When a particular connection is done, close the

connection and go back to step 3

NTUT, TAIWAN 29

Mobile Computing & Software Engineering Lab Mobile Computing & Software Engineering Lab

Passive socket

Using function listen to place a socket in passive mode Listen also takes an argument that specifies the length of an internal request queue for the socket

NTUT, TAIWAN 30

Mobile Computing & Software Engineering Lab Mobile Computing & Software Engineering Lab

Iterative Connection- Oriented Server Algorithm

• 1. Create a socket and bind to the well-known

address for the service being offered

• 2. Place the socket in passive mode and make it

ready for use by a server

• 3. Accept the next connection request from the

socket, and obtain a new socket for the connection

• 4. Repeatedly read, process, and reply a request
• 5. When a particular connection is done, close the

connection and go back to step 3

NTUT, TAIWAN 31

Mobile Computing & Software Engineering Lab Mobile Computing & Software Engineering Lab

Accepting Connections

Using function accept to obtain the next incoming connection request After connection, a server uses recv to receive requests and send to send reply When the server finishes with the connection, it calls close to release the socket

NTUT, TAIWAN 32

Mobile Computing & Software Engineering Lab Mobile Computing & Software Engineering Lab

Iterative Connectionless Server Algorithm

• 1. Create a socket and bind to the well-

known address for the service being

• ffered
• 2. Repeatedly receive the next request from

a client, formulate a response, and send a reply back to the client according to the application protocol

NTUT, TAIWAN 33

Mobile Computing & Software Engineering Lab Mobile Computing & Software Engineering Lab

Reply Address

Socket interface provides two ways to specify a remote endpoint

Client uses connect Connectionless server generate reply address explicity and use sendto socket call

NTUT, TAIWAN 34

Mobile Computing & Software Engineering Lab Mobile Computing & Software Engineering Lab

Sendto Socket Call

Sendto

specifies a datagram and an address sendto (s, msg, len, flags, toaddr, toaddrlen); s: socket descriptor; msg: address of the data to be sent len: number of bytes in the buffer flags: debugging or control options toaddr: address to which a message should be sent toaddrlen: length of the address structure

How to get this?

NTUT, TAIWAN 35

Mobile Computing & Software Engineering Lab Mobile Computing & Software Engineering Lab

Recvfrom Socket Call

Two arguments specify two buffers

One buffer for arriving datagram The other buffer for the sender’s address recvfrom(s, buf, len, flags, from, fromlen); buf: address of the data to be placed len: space available in the buffer from: address from which the message is sent

NTUT, TAIWAN 36

Mobile Computing & Software Engineering Lab Mobile Computing & Software Engineering Lab

Concurrent Server Algorithms

Improve response time when

Response needs significant I/O Processing time varies dramatically among requests Server executes on multiple processors

NTUT, TAIWAN 37

Mobile Computing & Software Engineering Lab Mobile Computing & Software Engineering Lab

Master and Slave Threads

Most concurrent servers use multiple threads Two types:

Master server thread Slave server thread

It is possible to achieve concurrency by a single thread

NTUT, TAIWAN 38

Mobile Computing & Software Engineering Lab Mobile Computing & Software Engineering Lab

Master and Slave Threads

A master thread executes initially and

• pens a socket at some port and waits for

the next request A slave thread is created by the master for each request The master thread never communicates with client

NTUT, TAIWAN 39

Mobile Computing & Software Engineering Lab Mobile Computing & Software Engineering Lab

Concurrent Connectionless Server Algorithm

Master 1. Create a socket, bind the address, and leave the socket unconnected Master 2. Repeatedly call recvfrom to receive the next request and create a new slave thread to handle the response

NTUT, TAIWAN 40

Mobile Computing & Software Engineering Lab Mobile Computing & Software Engineering Lab

Concurrent Connectionless Server Algorithm

Slave 1. Receive a specific request upon creation as well as access to the socket Slave 2. Form a reply and send it back to the client using sendto Slave 3. Exit

NTUT, TAIWAN 41

Mobile Computing & Software Engineering Lab Mobile Computing & Software Engineering Lab

Concurrent Connectionless Server Algorithm

Because process or thread creation is expensive, few connectionless servers have concurrent implementations.

NTUT, TAIWAN 42

Mobile Computing & Software Engineering Lab Mobile Computing & Software Engineering Lab

Concurrent Connection- Oriented Server Algorithm

Use a connection as the basic paradigm for communication Connection-oriented servers implement concurrency among connections rather than among individual requests

NTUT, TAIWAN 43

Mobile Computing & Software Engineering Lab Mobile Computing & Software Engineering Lab

Concurrent Connection- Oriented Server Algorithm

Master 1. Create a socket, bind the address, and leave the socket unconnected Master 2. Place the socket in passive mode and make it ready for use by a server Master 3. Repeatedly call accept to receive the next request and create a new slave thread or process to handle the request

NTUT, TAIWAN 44

Mobile Computing & Software Engineering Lab Mobile Computing & Software Engineering Lab

Concurrent Connection- Oriented Server Algorithm

Slave 1. Receive a connection request (i.e. socket for the connection) upon creation Slave 2. Interact with the client using the connection Slave 3. Close the connection and exit

NTUT, TAIWAN 45

Mobile Computing & Software Engineering Lab Mobile Computing & Software Engineering Lab

Concurrency Implementation

Master Process Slave Processes Slave Threads Master Thread

NTUT, TAIWAN 46

Mobile Computing & Software Engineering Lab Mobile Computing & Software Engineering Lab

Concurrency by One Thread

The creation of thread and process is expensive Many applications need to share information among connections Apparent concurrency with a single thread

X window

NTUT, TAIWAN 47

Mobile Computing & Software Engineering Lab Mobile Computing & Software Engineering Lab

Concurrency by One Thread

It is possible to share memory between threads, but if the request load does not exceed the server’s capacity to handle them,

• ne single thread is preferred

To achieve this, use select call for asynchronous I/O One key point is that a socket is treated as an I/O device

NTUT, TAIWAN 48

Mobile Computing & Software Engineering Lab Mobile Computing & Software Engineering Lab

Concurrency by One Thread

• 1. Create a socket and bind the address. Add

socket to the list of I/O’s

• 2. Use select to wait for I/O on existing

sockets

• 3. If the original socket is ready, use accept

to obtain the next connection and add the new socket to the list of I/O’s

• 4. If some socket other than the original is

ready, use recv (read) and send (write) to communicate

• 5. Continue with step 2

NTUT, TAIWAN 49

Mobile Computing & Software Engineering Lab Mobile Computing & Software Engineering Lab

When to Use Each Server Type

Iterative vs. Concurrent

Using an iterative implementation if

Short request processing time Tolerated response time

Connection-oriented vs. Connectionless

Depending on

if the application protocol handle reliability the area of the network

NTUT, TAIWAN 50

Mobile Computing & Software Engineering Lab Mobile Computing & Software Engineering Lab

When to Use Each Server Type

Real vs. Apparent concurrency

Using single thread when

Cost of creating and switching among threads is high Server must share or exchanging data among connections

Using multi-threaded solution if

Server must share or exchanging data among connections cost of using threads is low

Using a multi-process solution if each slave can

• perate in isolation it to achieve maximal

concurrency

NTUT, TAIWAN 51

Mobile Computing & Software Engineering Lab Mobile Computing & Software Engineering Lab

Server Types

Iterative, connectionless server

Most common for connectionless servers

Iterative, connection-oriented server

Less common servers

Concurrent, connectionless server

Uncommon type

Concurrent, connection-oriented server

Most general type Multi-process, multi-thread, or single thread

NTUT, TAIWAN 52

Mobile Computing & Software Engineering Lab Mobile Computing & Software Engineering Lab

Server Deadlock

What is deadlock?

A condition in which a program or set of program can not proceed because they are blocked waiting for an event that will never happen In case of server, deadlock means that the server ceases to answer requests

NTUT, TAIWAN 53

Mobile Computing & Software Engineering Lab Mobile Computing & Software Engineering Lab

Server Deadlock

client server An iterative connection-oriented server

Connected but no send Call recv to receive Server is blocked

NTUT, TAIWAN 54

Mobile Computing & Software Engineering Lab Mobile Computing & Software Engineering Lab

Server Deadlock

client server An iterative connection-oriented server

Connected and send Never reads response Flow control interfere Buffer full Server blocks

NTUT, TAIWAN 55

Mobile Computing & Software Engineering Lab Mobile Computing & Software Engineering Lab

Server Deadlock

Deadlock arises because a calling program blocks when the OS can not satisfy a system call Call to send will block the caller if no buffer space for sending data Call to recv will block the caller until TCP receives data Occurs in single slave thread