Server Software Design Prof. Chuan-Ming Liu Computer Science and - - PowerPoint PPT Presentation

server software design
SMART_READER_LITE
LIVE PREVIEW

Server Software Design Prof. Chuan-Ming Liu Computer Science and - - PowerPoint PPT Presentation

Mobile Computing & Software Engineering Lab Mobile Computing & Software Engineering Lab Server Software Design Prof. Chuan-Ming Liu Computer Science and Information Engineering National Taipei University of Technology Taipei, TAIWAN


slide-1
SLIDE 1

NTUT, TAIWAN 1

Mobile Computing & Software Engineering Lab Mobile Computing & Software Engineering Lab

Server Software Design

  • Prof. Chuan-Ming Liu

Computer Science and Information Engineering National Taipei University of Technology Taipei, TAIWAN

slide-2
SLIDE 2

NTUT, TAIWAN 2

Mobile Computing & Software Engineering Lab Mobile Computing & Software Engineering Lab

Conceptual Server Algorithm

Simple Server Algorithm

  • 1. Create a socket
  • 2. Bind the socket to the port
  • 3. Enter an infinite loop

Accept request Process request Reply request

Not enough in practice generally

slide-3
SLIDE 3

NTUT, TAIWAN 3

Mobile Computing & Software Engineering Lab Mobile Computing & Software Engineering Lab

Conceptual Server Algorithm

Consider file transfer

First client needs a 200 MB file Second client needs a 20 byte file If the server processes them in FIFO fashion, the second client will wait an unreasonable amount of time for a small transfer

Server usually handles more than one request at a time

slide-4
SLIDE 4

NTUT, TAIWAN 4

Mobile Computing & Software Engineering Lab Mobile Computing & Software Engineering Lab

Concurrent vs Iterative

Iterative server: one request at a time Concurrent server: multiple requests at

  • ne time

Note: the concurrency is conceptual and generally will use one thread for one request

slide-5
SLIDE 5

NTUT, TAIWAN 5

Mobile Computing & Software Engineering Lab Mobile Computing & Software Engineering Lab

Concurrent vs Iterative

Concurrent server refers to whether the server permits multiple requests to proceed concurrently, not whether the underlying implementation uses multiple and concurrent threads

slide-6
SLIDE 6

NTUT, TAIWAN 6

Mobile Computing & Software Engineering Lab Mobile Computing & Software Engineering Lab

Concurrent vs Iterative

Concurrent servers are difficult to design and build Why concurrency?

Long delay caused by iterative servers Performance bottleneck resulted from iterative servers effects many clients

slide-7
SLIDE 7

NTUT, TAIWAN 7

Mobile Computing & Software Engineering Lab Mobile Computing & Software Engineering Lab

Connection Access or Not

Two major transport protocols:

TCP (Transmission Control Protocol) UDP (User Datagram Protocol)

TCP is connection-oriented and UDP is connectionless The corresponding servers are

Connection-oriented servers Connectionless serves

slide-8
SLIDE 8

NTUT, TAIWAN 8

Mobile Computing & Software Engineering Lab Mobile Computing & Software Engineering Lab

TCP Semantics

Point-to-point communication Reliable connection establishment Reliable delivery Flow controlled transfer Full-duplex transfer Stream paradigm

slide-9
SLIDE 9

NTUT, TAIWAN 9

Mobile Computing & Software Engineering Lab Mobile Computing & Software Engineering Lab

UPD Semantics

Many-to-many communication Unreliable Service Lack of flow control Message paradigm

slide-10
SLIDE 10

NTUT, TAIWAN 10

Mobile Computing & Software Engineering Lab Mobile Computing & Software Engineering Lab

TCP or UDP?

Because the semantics of TCP and UDP differ sharply, a designer cannot choose between connection-oriented and connectionless transport protocols without considering the semantics required by the application protocol Depends on the applications, not the types

slide-11
SLIDE 11

NTUT, TAIWAN 11

Mobile Computing & Software Engineering Lab Mobile Computing & Software Engineering Lab

Connection-Oriented Servers

Easier to program since TCP provides all the reliability Drawbacks:

Need a separate socket for each connection Care about the socket allocation and resource consuming Resource limitation problem – due to the clients can crash often

slide-12
SLIDE 12

NTUT, TAIWAN 12

Mobile Computing & Software Engineering Lab Mobile Computing & Software Engineering Lab

Connectionless Servers

Access multiple hosts from a single socket No problem of resource depletion No reliable delivery

Need to take care of the reliability Usually, clients take care of retransmitting request Server may retransmit as well

slide-13
SLIDE 13

NTUT, TAIWAN 13

Mobile Computing & Software Engineering Lab Mobile Computing & Software Engineering Lab

Connectionless Servers

Achieving reliability can be difficult

Timeout Retransmission

TCP does not allow multicast or broadcast Multicast and broadcast services require UDP Any server that accepts or responds to multicast and broadcast communication must be connectionless

slide-14
SLIDE 14

NTUT, TAIWAN 14

Mobile Computing & Software Engineering Lab Mobile Computing & Software Engineering Lab

Stateful or Stateless

The issue of stateless arises from a need to ensure reliability For a service using connectionless transport, like UDP, the application protocols need to ensure the reliability

slide-15
SLIDE 15

NTUT, TAIWAN 15

Mobile Computing & Software Engineering Lab Mobile Computing & Software Engineering Lab

Stateless Servers

Consider a connectionless server that allows clients to read information from files

Clients need to specify

Filename Position in the file Amount to read

Straightforward method is to handle each request independently

slide-16
SLIDE 16

NTUT, TAIWAN 16

Mobile Computing & Software Engineering Lab Mobile Computing & Software Engineering Lab

Optimizing Stateless Servers

Observe that

The overhead to open and close files is high Data read may be few Clients may sequentially read files

Using buffers to assist reading files

This may be similar to the stateful servers Hashing table is used The IP address and the port number form the index

slide-17
SLIDE 17

NTUT, TAIWAN 17

Mobile Computing & Software Engineering Lab Mobile Computing & Software Engineering Lab

Optimizing Stateless Servers

Filename: Y Offset 1024 Buffer pointer: Filename: X Offset: 512 Buffer pointer: Buffer for file X Starting at byte 512 Buffer for file Y Starting at byte 1024 Hash(ip, port)

slide-18
SLIDE 18

NTUT, TAIWAN 18

Mobile Computing & Software Engineering Lab Mobile Computing & Software Engineering Lab

Optimizing Stateless Servers

Improvement can be achieved when all the assumptions meet Problem arises when the client programs fail

Consider the situation that the client crashes Buffer will be full

Adjustment can be done by least recently used (LRU) rule for maintaining the hash table Thrashing occurs

slide-19
SLIDE 19

NTUT, TAIWAN 19

Mobile Computing & Software Engineering Lab Mobile Computing & Software Engineering Lab

Optimizing Stateless Servers

A programmer must be extremely careful when optimizing a stateless server because managing small amounts

  • f

state information can consume resources if clients crash and reboot frequently or if the underlying network duplicates or delay messages

slide-20
SLIDE 20

NTUT, TAIWAN 20

Mobile Computing & Software Engineering Lab Mobile Computing & Software Engineering Lab

Types of Servers

Concurrent Connection-oriented Concurrent Connectionless Iterative Connection-oriented Iterative Connectionless

slide-21
SLIDE 21

NTUT, TAIWAN 21

Mobile Computing & Software Engineering Lab Mobile Computing & Software Engineering Lab

Iterative or Concurrent

Request processing time (Server) total time the server takes to handle a request Observed response time (Client) total delay from sending a request to receiving the response Server has a queue of requests to handle

slide-22
SLIDE 22

NTUT, TAIWAN 22

Mobile Computing & Software Engineering Lab Mobile Computing & Software Engineering Lab

Iterative or Concurrent

If N is the average length of the request queue, then the observed response time is approximately (N/2+1) × the server’s request processing time The observed response time is in proportion to N, N should be reasonable small, say N=5 When N is small, an iterative server is ok;

  • therwise, a concurrent serve is preferred
slide-23
SLIDE 23

NTUT, TAIWAN 23

Mobile Computing & Software Engineering Lab Mobile Computing & Software Engineering Lab

Iterative or Concurrent

Alternatively, the overall load can be considered

Suppose a server is designed to handle K clients Each client sends R request per second Total number of requests is RK The server’s processing time should be less than 1/KR second per request

If the server can handle at the required rate, iterative way is ok; otherwise, concurrent method is selected

slide-24
SLIDE 24

NTUT, TAIWAN 24

Mobile Computing & Software Engineering Lab Mobile Computing & Software Engineering Lab

Iterative Server Algorithms

Easy to design, program, debug, and modify. Working best with simple services accessed by a connectionless access protocol

slide-25
SLIDE 25

NTUT, TAIWAN 25

Mobile Computing & Software Engineering Lab Mobile Computing & Software Engineering Lab

Iterative Connection- Oriented Server Algorithm

  • 1. Create a socket and bind to the well-known

address for the service being offered

  • 2. Place the socket in passive mode and make it

ready for use by a server

  • 3. Accept the next connection request from the

socket, and obtain a new socket for the connection

  • 4. Repeatedly read, process, and reply a request
  • 5. When a particular connection is done, close the

connection and go back to step 3

slide-26
SLIDE 26

NTUT, TAIWAN 26

Mobile Computing & Software Engineering Lab Mobile Computing & Software Engineering Lab

Binding a Well-known Address

Like clients, servers use procedure getportbyname to map a service name into a well-known port number Using function bind to specify a connection IP address is necessary for bind, but it is difficult to select the IP address

slide-27
SLIDE 27

NTUT, TAIWAN 27

Mobile Computing & Software Engineering Lab Mobile Computing & Software Engineering Lab

INADDR_ANY

Constant, INADDR_ANY, provided by the socket interface helps binding

specifies a wildcard address that matches any

  • f the host’s IP address

Has a multi-homes host accept incoming communication addressed to any of the host’s IP address

slide-28
SLIDE 28

NTUT, TAIWAN 28

Mobile Computing & Software Engineering Lab Mobile Computing & Software Engineering Lab

Iterative Connection- Oriented Server Algorithm

  • 1. Create a socket and bind to the well-known

address for the service being offered

  • 2. Place the socket in passive mode and make it

ready for use by a server

  • 3. Accept the next connection request from the

socket, and obtain a new socket for the connection

  • 4. Repeatedly read, process, and reply a request
  • 5. When a particular connection is done, close the

connection and go back to step 3

slide-29
SLIDE 29

NTUT, TAIWAN 29

Mobile Computing & Software Engineering Lab Mobile Computing & Software Engineering Lab

Passive socket

Using function listen to place a socket in passive mode Listen also takes an argument that specifies the length of an internal request queue for the socket

slide-30
SLIDE 30

NTUT, TAIWAN 30

Mobile Computing & Software Engineering Lab Mobile Computing & Software Engineering Lab

Iterative Connection- Oriented Server Algorithm

  • 1. Create a socket and bind to the well-known

address for the service being offered

  • 2. Place the socket in passive mode and make it

ready for use by a server

  • 3. Accept the next connection request from the

socket, and obtain a new socket for the connection

  • 4. Repeatedly read, process, and reply a request
  • 5. When a particular connection is done, close the

connection and go back to step 3

slide-31
SLIDE 31

NTUT, TAIWAN 31

Mobile Computing & Software Engineering Lab Mobile Computing & Software Engineering Lab

Accepting Connections

Using function accept to obtain the next incoming connection request After connection, a server uses recv to receive requests and send to send reply When the server finishes with the connection, it calls close to release the socket

slide-32
SLIDE 32

NTUT, TAIWAN 32

Mobile Computing & Software Engineering Lab Mobile Computing & Software Engineering Lab

Iterative Connectionless Server Algorithm

  • 1. Create a socket and bind to the well-

known address for the service being

  • ffered
  • 2. Repeatedly receive the next request from

a client, formulate a response, and send a reply back to the client according to the application protocol

slide-33
SLIDE 33

NTUT, TAIWAN 33

Mobile Computing & Software Engineering Lab Mobile Computing & Software Engineering Lab

Reply Address

Socket interface provides two ways to specify a remote endpoint

Client uses connect Connectionless server generate reply address explicity and use sendto socket call

slide-34
SLIDE 34

NTUT, TAIWAN 34

Mobile Computing & Software Engineering Lab Mobile Computing & Software Engineering Lab

Sendto Socket Call

Sendto

specifies a datagram and an address sendto (s, msg, len, flags, toaddr, toaddrlen); s: socket descriptor; msg: address of the data to be sent len: number of bytes in the buffer flags: debugging or control options toaddr: address to which a message should be sent toaddrlen: length of the address structure

How to get this?

slide-35
SLIDE 35

NTUT, TAIWAN 35

Mobile Computing & Software Engineering Lab Mobile Computing & Software Engineering Lab

Recvfrom Socket Call

Two arguments specify two buffers

One buffer for arriving datagram The other buffer for the sender’s address recvfrom(s, buf, len, flags, from, fromlen); buf: address of the data to be placed len: space available in the buffer from: address from which the message is sent

slide-36
SLIDE 36

NTUT, TAIWAN 36

Mobile Computing & Software Engineering Lab Mobile Computing & Software Engineering Lab

Concurrent Server Algorithms

Improve response time when

Response needs significant I/O Processing time varies dramatically among requests Server executes on multiple processors

slide-37
SLIDE 37

NTUT, TAIWAN 37

Mobile Computing & Software Engineering Lab Mobile Computing & Software Engineering Lab

Master and Slave Threads

Most concurrent servers use multiple threads Two types:

Master server thread Slave server thread

It is possible to achieve concurrency by a single thread

slide-38
SLIDE 38

NTUT, TAIWAN 38

Mobile Computing & Software Engineering Lab Mobile Computing & Software Engineering Lab

Master and Slave Threads

A master thread executes initially and

  • pens a socket at some port and waits for

the next request A slave thread is created by the master for each request The master thread never communicates with client

slide-39
SLIDE 39

NTUT, TAIWAN 39

Mobile Computing & Software Engineering Lab Mobile Computing & Software Engineering Lab

Concurrent Connectionless Server Algorithm

Master 1. Create a socket, bind the address, and leave the socket unconnected Master 2. Repeatedly call recvfrom to receive the next request and create a new slave thread to handle the response

slide-40
SLIDE 40

NTUT, TAIWAN 40

Mobile Computing & Software Engineering Lab Mobile Computing & Software Engineering Lab

Concurrent Connectionless Server Algorithm

Slave 1. Receive a specific request upon creation as well as access to the socket Slave 2. Form a reply and send it back to the client using sendto Slave 3. Exit

slide-41
SLIDE 41

NTUT, TAIWAN 41

Mobile Computing & Software Engineering Lab Mobile Computing & Software Engineering Lab

Concurrent Connectionless Server Algorithm

Because process or thread creation is expensive, few connectionless servers have concurrent implementations.

slide-42
SLIDE 42

NTUT, TAIWAN 42

Mobile Computing & Software Engineering Lab Mobile Computing & Software Engineering Lab

Concurrent Connection- Oriented Server Algorithm

Use a connection as the basic paradigm for communication Connection-oriented servers implement concurrency among connections rather than among individual requests

slide-43
SLIDE 43

NTUT, TAIWAN 43

Mobile Computing & Software Engineering Lab Mobile Computing & Software Engineering Lab

Concurrent Connection- Oriented Server Algorithm

Master 1. Create a socket, bind the address, and leave the socket unconnected Master 2. Place the socket in passive mode and make it ready for use by a server Master 3. Repeatedly call accept to receive the next request and create a new slave thread or process to handle the request

slide-44
SLIDE 44

NTUT, TAIWAN 44

Mobile Computing & Software Engineering Lab Mobile Computing & Software Engineering Lab

Concurrent Connection- Oriented Server Algorithm

Slave 1. Receive a connection request (i.e. socket for the connection) upon creation Slave 2. Interact with the client using the connection Slave 3. Close the connection and exit

slide-45
SLIDE 45

NTUT, TAIWAN 45

Mobile Computing & Software Engineering Lab Mobile Computing & Software Engineering Lab

Concurrency Implementation

Master Process Slave Processes Slave Threads Master Thread

slide-46
SLIDE 46

NTUT, TAIWAN 46

Mobile Computing & Software Engineering Lab Mobile Computing & Software Engineering Lab

Concurrency by One Thread

The creation of thread and process is expensive Many applications need to share information among connections Apparent concurrency with a single thread

X window

slide-47
SLIDE 47

NTUT, TAIWAN 47

Mobile Computing & Software Engineering Lab Mobile Computing & Software Engineering Lab

Concurrency by One Thread

It is possible to share memory between threads, but if the request load does not exceed the server’s capacity to handle them,

  • ne single thread is preferred

To achieve this, use select call for asynchronous I/O One key point is that a socket is treated as an I/O device

slide-48
SLIDE 48

NTUT, TAIWAN 48

Mobile Computing & Software Engineering Lab Mobile Computing & Software Engineering Lab

Concurrency by One Thread

  • 1. Create a socket and bind the address. Add

socket to the list of I/O’s

  • 2. Use select to wait for I/O on existing

sockets

  • 3. If the original socket is ready, use accept

to obtain the next connection and add the new socket to the list of I/O’s

  • 4. If some socket other than the original is

ready, use recv (read) and send (write) to communicate

  • 5. Continue with step 2
slide-49
SLIDE 49

NTUT, TAIWAN 49

Mobile Computing & Software Engineering Lab Mobile Computing & Software Engineering Lab

When to Use Each Server Type

Iterative vs. Concurrent

Using an iterative implementation if

Short request processing time Tolerated response time

Connection-oriented vs. Connectionless

Depending on

if the application protocol handle reliability the area of the network

slide-50
SLIDE 50

NTUT, TAIWAN 50

Mobile Computing & Software Engineering Lab Mobile Computing & Software Engineering Lab

When to Use Each Server Type

Real vs. Apparent concurrency

Using single thread when

Cost of creating and switching among threads is high Server must share or exchanging data among connections

Using multi-threaded solution if

Server must share or exchanging data among connections cost of using threads is low

Using a multi-process solution if each slave can

  • perate in isolation it to achieve maximal

concurrency

slide-51
SLIDE 51

NTUT, TAIWAN 51

Mobile Computing & Software Engineering Lab Mobile Computing & Software Engineering Lab

Server Types

Iterative, connectionless server

Most common for connectionless servers

Iterative, connection-oriented server

Less common servers

Concurrent, connectionless server

Uncommon type

Concurrent, connection-oriented server

Most general type Multi-process, multi-thread, or single thread

slide-52
SLIDE 52

NTUT, TAIWAN 52

Mobile Computing & Software Engineering Lab Mobile Computing & Software Engineering Lab

Server Deadlock

What is deadlock?

A condition in which a program or set of program can not proceed because they are blocked waiting for an event that will never happen In case of server, deadlock means that the server ceases to answer requests

slide-53
SLIDE 53

NTUT, TAIWAN 53

Mobile Computing & Software Engineering Lab Mobile Computing & Software Engineering Lab

Server Deadlock

client server An iterative connection-oriented server

Connected but no send Call recv to receive Server is blocked

slide-54
SLIDE 54

NTUT, TAIWAN 54

Mobile Computing & Software Engineering Lab Mobile Computing & Software Engineering Lab

Server Deadlock

client server An iterative connection-oriented server

Connected and send Never reads response Flow control interfere Buffer full Server blocks

slide-55
SLIDE 55

NTUT, TAIWAN 55

Mobile Computing & Software Engineering Lab Mobile Computing & Software Engineering Lab

Server Deadlock

Deadlock arises because a calling program blocks when the OS can not satisfy a system call Call to send will block the caller if no buffer space for sending data Call to recv will block the caller until TCP receives data Occurs in single slave thread