8.2 Logical Platform 8.2 Logical Platform Communication - - PDF document

SLIDE 1

1

§8.2 Logical Platform §8.2 Logical Platform

• communication architecture

communication architecture

• peer

peer-

• to

to-

• peer

peer

• client

client-

• server

server

• server

server-

• network

network

• data and control architecture

data and control architecture

• centralized

centralized

• replicated

replicated

• distributed

distributed

Communication Architecture Communication Architecture

Single node Single node Peer Peer-

• to

to-

• peer

peer Client Client-

• server

server Server Server-

• network

network

Communication Communication Architecture (cont’d) Architecture (cont’d)

• Logical connections

Logical connections

• how the messages flow

how the messages flow

• Physical connections

Physical connections

• the wires

the wires between the between the computers computers

• the limiting factor in

the limiting factor in communication architecture communication architecture design design p1 p2 LAN Two players on a LAN

Example: How Many Players Can We Example: How Many Players Can We Put into a Two Put into a Two-

• Player LAN?

Player LAN?

• Distributed Interactive Simulation

Distributed Interactive Simulation (DIS) protocol data unit (PDU): (DIS) protocol data unit (PDU): 144 bytes (1,152 bits) 144 bytes (1,152 bits)

• Graphics: 30 frames/second

Graphics: 30 frames/second

• PDU rates

PDU rates

• aircraft 12 PDU/second

aircraft 12 PDU/second

• ground vehicle 5 PDU/second

ground vehicle 5 PDU/second

• weapon firing 3 PDU/second

weapon firing 3 PDU/second

• fully articulated human 30

fully articulated human 30 PDU/second PDU/second

• Bandwidth

Bandwidth

• Ethernet LAN 10 Mbps

Ethernet LAN 10 Mbps

• modems 56 Kbps

modems 56 Kbps

• Assumptions:

Assumptions:

• sufficient processor power

sufficient processor power

• no other network usage

no other network usage

• a mix of player types

a mix of player types

⇒ ⇒ LAN: 8,680 packets/second LAN: 8,680 packets/second

fully articulated humans + firing = fully articulated humans + firing = 263 humans 263 humans aircrafts + firing = 578 aircrafts aircrafts + firing = 578 aircrafts ground vehicles + firing = 1,085 ground vehicles + firing = 1,085 vehicles vehicles

• Typical NPSNET

Typical NPSNET-

• IV DIS battle

IV DIS battle

• limits to 300 players on a LAN

limits to 300 players on a LAN

• processor and network

processor and network limitations limitations

Example (cont’d) Example (cont’d)

⇒ ⇒ Modem: 48 packets/second Modem: 48 packets/second

fully articulated humans + firing = 1 fully articulated humans + firing = 1 human human aircrafts + firing = 3 aircrafts aircrafts + firing = 3 aircrafts ground vehicles + firing = 6 vehicles ground vehicles + firing = 6 vehicles

• Redesign packets

Redesign packets

• size 22%, 32 bytes

size 22%, 32 bytes

⇒ ⇒ Modem: 218 packets/second Modem: 218 packets/second

fully articulated humans + firing = 7 fully articulated humans + firing = 7 human human aircrafts + firing = 14 aircrafts aircrafts + firing = 14 aircrafts ground vehicles + firing = 27 vehicles ground vehicles + firing = 27 vehicles

• In a two

In a two-

• player game on a LAN,

player game on a LAN, the protocol selection (TCP, UDP, the protocol selection (TCP, UDP, broadcast,...) hardly matters broadcast,...) hardly matters

• As the number of live or

As the number of live or autonomous players increase an autonomous players increase an efficient architecture becomes efficient architecture becomes more important more important

Multiplayer Client Multiplayer Client-

• Server

Server Systems: Systems: Logical Logical Architecture Architecture

• Client

Client-

• server system

server system

• each player sends packets to

each player sends packets to

• ther
• ther players via a server

players via a server

• Server slows down the message

Server slows down the message delivery delivery

• Benefits of

Benefits of having a having a server server

• no need to

no need to send all packets to all send all packets to all players players

• compress multiple packets to a

compress multiple packets to a single packet single packet

• smooth out

smooth out the packet flow the packet flow

• reliable communication without

reliable communication without the overhead of a fully connected the overhead of a fully connected game game

• administration

administration

p1 p2 pn Communication paths Multiplayer client-server - logical architecture

SLIDE 2

2

Multiplayer Client Multiplayer Client-

• Server Systems:

Server Systems: Physical Architecture (on a LAN) Physical Architecture (on a LAN)

• All messages in the same wire

All messages in the same wire

• Server has to provide some added

Server has to provide some added-

• value function

value function

• collecting data

collecting data

• compressing and redistributing information

compressing and redistributing information

• additional computation

additional computation

p1 p2 LAN Multiplayer client-server - physical architecture on a LAN pn Server

Physical Physical Architecture Can Architecture Can Match Match the the Logical Logical Architecture Architecture

p1 p2 pn Server Phone lines Multiplayer client-server - physical architecture with phone lines

Traditional Client Traditional Client-

• Server

Server

• Server may act as

Server may act as

• broadcast reflector

broadcast reflector

• filtering reflector

filtering reflector

• packet aggregation server

packet aggregation server

• Scalability problems

Scalability problems

• all traffic

all traffic goes through goes through the the server server

⇒ ⇒Server Server-

• network architecture

network architecture

S S C C C C C C C C C C C C C C C C C C C C C C C C C C C C

Multiplayer Multiplayer Server Server-

• Network Architecture

Network Architecture

• Players

Players can locate in the same can locate in the same place in the place in the game world, game world, but but reside on reside on different different servers servers

• real world ≠ game world

real world ≠ game world

• Server

Server-

• to

to-

• server connections

server connections transmit transmit the world the world state state information information

• WAN, LAN

WAN, LAN

• Each server serves a number of

Each server serves a number of client players client players

• LAN, modem, cable modem

LAN, modem, cable modem

• Scalability

Scalability

p1,1 p1,2 p1,n p2,1 p2,2 p2,n p3,1 p3,2 p3,n Server 2 Server 3 Server 1

Partitioning Clients Partitioning Clients across across Multiple Servers Multiple Servers

• The servers exchange control

The servers exchange control messages among themselves messages among themselves

• inform the interests

inform the interests of their

• f their

clients clients

• Reduces the workload on each

Reduces the workload on each server server

• Incurs a greater

Incurs a greater latency latency

• The total

The total processing and processing and bandwidth requirements are bandwidth requirements are greater greater

S S S S S S S S C C C C C C C C C C C C C C C C C C C C C C C C C C C C

Partitioning the Partitioning the Game World Game World across across Multiple Multiple Servers Servers

• Each server manages clients

Each server manages clients located within a certain region located within a certain region

• Client communicates with

Client communicates with different serves as it different serves as it moves moves

• Possibility to aggregate messages

Possibility to aggregate messages

• Eliminates a lot of network traffic

Eliminates a lot of network traffic

• Requires

Requires advanced advanced configuration configuration

• Is

Is a region a region visible from another visible from another region? region?

C C C C C C C C C C C C C C S S S S S S S S

SLIDE 3

3

Server Hierarchies Server Hierarchies

• Servers themselves

Servers themselves act as act as clients clients

• Packet from

Packet from an upstream an upstream server: server:

• deliver to

deliver to the interested the interested downstream clients downstream clients

• Packet from

Packet from a downstream a downstream client: client:

• deliver to

deliver to the interested the interested downstream clients downstream clients

• if other regions are interested in

if other regions are interested in the the packet packet then then deliver it deliver it to to the the upstream upstream server server

C C C C C C C C C C C C C C S S S S S S S S S S S S S S

Peer Peer-

• to

to-

• Peer Architectures

Peer Architectures

• In the

In the ideal ideal large large-

• scale

scale networked networked game game design, design, avoid having avoid having servers servers at all at all

• eventually we cannot scale out

eventually we cannot scale out

• a finite

a finite number of players number of players

• Design

Design goal goal

• peer

peer-

• to

to-

• peer communication

peer communication

• scalable within resources

scalable within resources

• Peer

Peer-

• to

to-

• peer: communication

peer: communication goes goes directly from the sending player to directly from the sending player to the receiving player (or the receiving player (or a set a set of

• f

them) them)

p1 p2 LAN Peer-to-peer on a LAN pn

p1,1 p1,2 p1,n p2,1 p2,2 p2,n p3,1 p3,2 p3,n Server 2 Server 3 Server 1

p3 p4 p2 p1

Peer Peer-

• to

to-

• Peer with Multicast

Peer with Multicast

• For a scalable multiplayer game

For a scalable multiplayer game

• n a LAN, use multicast
• n a LAN, use multicast
• To utilize multicast, assign packets

To utilize multicast, assign packets to proper multicast groups to proper multicast groups

• Area

Area-

• of
• f-
• interest management

interest management

• assign outgoing packets to the

assign outgoing packets to the right groups right groups

• receive incoming packets to the

receive incoming packets to the appropriate multicast groups appropriate multicast groups

• keep track of available groups

keep track of available groups

• even out stream information

even out stream information

p1 p2 Network AOIM software layer pn AOIM 1 AOIM 1 AOIM 1

Peer Peer-

• Server

Server Systems Systems

• Peer

Peer-

• to

to-

• peer: minimizes

peer: minimizes latency, latency, consumes consumes bandwidth bandwidth

• Client

Client-

• server: effective

server: effective aggregation and filtering, aggregation and filtering, increases increases latency latency

• Hybrid

Hybrid peer peer-

• server:

server:

• ver
• ver short

short-

• haul, high

haul, high-

• bandwidth links: peer

bandwidth links: peer-

• to

to-

• peer

peer

• ver
• ver long

long-

• haul, low

haul, low-

• bandwidth links:

bandwidth links: client client-

• server

server

• Each entity has own

Each entity has own multicast group multicast group

• Well

Well-

• connected hosts

connected hosts subscribe directly subscribe directly to a to a multicast group (peer multicast group (peer-

• to

to-

• peer)

peer)

• Poorly

Poorly-

• connected hosts

connected hosts subscribe subscribe to a to a forwarding forwarding server server

• Forwarding server

Forwarding server subscribes to subscribes to the entities the entities’ ’ multicast groups multicast groups

• aggregation,

aggregation, filtering filtering

Data and Control Architectures Data and Control Architectures

• Where does the data reside and how it can be updated?

Where does the data reside and how it can be updated?

• Centralized

Centralized

• ne node holds a full copy of the data
• ne node holds a full copy of the data
• Replicated

Replicated

• all nodes hold a full copy of the data

all nodes hold a full copy of the data

• Distributed

Distributed

• ne node holds a partial copy of the data
• ne node holds a partial copy of the data
• all nodes combined hold a full copy of the data

all nodes combined hold a full copy of the data

• Consistency vs. responsiveness

Consistency vs. responsiveness

Requirements for Data and Control Architectures Requirements for Data and Control Architectures

• Consistency: nodes should have the same view on the data

Consistency: nodes should have the same view on the data

• centralized: simple

centralized: simple— —one node binds them all!

• ne node binds them all!
• replicated: hard

replicated: hard— —how to make sure that every replica gets updated? how to make sure that every replica gets updated?

• distributed: quite simple

distributed: quite simple— —only one copy of the piece of data exists

• nly one copy of the piece of data exists

(but where?) (but where?)

• Responsiveness: nodes should have a quick access to the data

Responsiveness: nodes should have a quick access to the data

• centralized: hard

centralized: hard— —all updates must go through the centre node all updates must go through the centre node

• replicated: simple

replicated: simple— —just do it! just do it!

• distributed: quite simple

distributed: quite simple— —just do it (if data is in the local node) or send just do it (if data is in the local node) or send an update message (but to whom?) an update message (but to whom?)

SLIDE 4

4

Centralized Centralized Architecture Architecture

• Ensure

Ensure that all that all nodes nodes have identical information have identical information

Centralized Centralized Data Store Data Store

State State State Read Update Update Read User User User User User User User User User User User User Synchronization Synchronization Locks Locks

Problem: Who’s Got the Ball Now? Problem: Who’s Got the Ball Now?

x, y, z x, y, z

A A B B

‘ ‘Eventual’ Eventual’ Consistency Consistency

Centralized Centralized Data Store Data Store

State State State Read Update Update Read User User User User User User User User User User User User Per Per-

• client

client FIFO Event FIFO Event Queues Queues Per Per-

• client

client FIFO Event FIFO Event Queues Queues Synchronization Synchronization Locks Locks

Pull and Push Pull and Push

• The clients

The clients ‘pull’ ‘pull’ information when they need it information when they need it

• make

make a request whenever data access is needed a request whenever data access is needed

• problem: unnecessary

problem: unnecessary delays, delays, if the state data has not changed if the state data has not changed

• The server can ‘push’ the information to the clients whenever

The server can ‘push’ the information to the clients whenever the state is updated the state is updated

• clients

clients can maintain a local cache can maintain a local cache

• problem: excessive traffic, if the clients are interested only a

problem: excessive traffic, if the clients are interested only a small small subset of the overall data subset of the overall data

Replicated Architecture Replicated Architecture

• Nodes

Nodes exchange messages directly exchange messages directly

• ensure

ensure that all that all nodes nodes receive updates receive updates

• determine a common global

determine a common global

• rdering for updates
• rdering for updates
• No central host

No central host

• Every

Every node node has an identical view has an identical view

• All state information is accessed from

All state information is accessed from local node local node

Distributed Architecture Distributed Architecture

• State information is distributed among the participating players

State information is distributed among the participating players

• who gets what?

who gets what?

• what to do when a new player joins the game?

what to do when a new player joins the game?

• what to do when an existing player leaves the game?

what to do when an existing player leaves the game?

• ⇒

⇒ Entity ownership Entity ownership

SLIDE 5

5

Problem: Who’s Got the Ball Now? (Part II) Problem: Who’s Got the Ball Now? (Part II)

A A B B

Entity Ownership Entity Ownership

• Ensure that

Ensure that a shared state a shared state can only be updated by one can only be updated by one node node at a at a time time

• exactly

exactly one

• ne node

node has has the ownership the ownership of the state

• f the state
• the owner

the owner periodically periodically broadcasts broadcasts the value of the state the value of the state

• Typically

Typically player’s player’s own

• wn representation (avatar)

representation (avatar) is owned by is owned by that that player player

• Locks on other entities are managed by

Locks on other entities are managed by a lock a lock manager server manager server

• clients

clients query to obtain ownership and contact to release it query to obtain ownership and contact to release it

• the

the server ensures that each entity has only one owner server ensures that each entity has only one owner

• the

the server owns the entity if no one else does server owns the entity if no one else does

• failure

failure recovery recovery

Lock Lock Manager: Example Manager: Example

A A B B

Lock Manager Lock Manager Grant Grant Lock Lock Request Request Lock Lock Request Request Lock Lock Reject Reject Lock Lock Update State Update State

Proxy Update Proxy Update

A A B B

Update Position (A) Update Position (A) Request Update Position Request Update Position Update Position (B) Update Position (B)

• Non

Non-

• owner
• wner sends

sends an update request to the an update request to the owner of the state

• wner of the state
• The owner

The owner decides whether it accepts decides whether it accepts the the update update

• The owner serves as a proxy

The owner serves as a proxy

• Generates an

Generates an extra message on each extra message on each non non-

• owner
• wner update

update

• Suitable when

Suitable when non non-

• owner
• wner updates are rare or many

updates are rare or many nodes nodes want want to update to update the state the state

Ownership Transfer Ownership Transfer

A A B B

Lock Manager Lock Manager Update Position (A) Update Position (A) Request Ownership Request Ownership Notify Lock Notify Lock Transfer Transfer Acknowledge Acknowledge Lock Transfer Lock Transfer Grant Ownership Grant Ownership Update Position (B) Update Position (B)

Ownership Transfer Ownership Transfer (cont’d) (cont’d)

• The lock manager has the lock information at all times

The lock manager has the lock information at all times

• If the

If the node node fails, the lock manager defines the current lock fails, the lock manager defines the current lock

• wnership state
• wnership state
• Lock ownership transfer incurs extra message overhead

Lock ownership transfer incurs extra message overhead

• Suitable when a single

Suitable when a single node node is going to make a series of is going to make a series of updates and there is little contention among updates and there is little contention among nodes nodes wishing to wishing to make updates make updates