Networked Virtual Environments Networked Virtual Environments - - PDF document

SLIDE 1

Special Course on Networked Virtual Environments January 22, 2004 Jouni Smed 1

Special Course on Special Course on

Networked Virtual Environments Networked Virtual Environments

Jouni Smed Jouni Smed

Turku Centre for Computer Science (TUCS) Turku Centre for Computer Science (TUCS) Department of Information Technology, Department of Information Technology, University of Turku University of Turku 2004 2004

Course Syllabus Course Syllabus

• credits: 2 cu

credits: 2 cu

• prerequisites: ‘knowledge on the basic concepts of computer

prerequisites: ‘knowledge on the basic concepts of computer networks’ networks’

• teaching methods: lectures (24 h)

teaching methods: lectures (24 h)

• Thursdays 8

Thursdays 8– –10 and Fridays 8 10 and Fridays 8– –10, Auditorium 10, Auditorium

• from January 22 to February 27

from January 22 to February 27

• assessment: examination

assessment: examination

• course web page:

course web page: ht t p: / / s ht t p: / / st af f . cs. ut u. f t af f . cs. ut u. f i / st af f / i / st af f / ht t p: / / s ht t p: / / st af f . cs. ut u. f t af f . cs. ut u. f i / st af f / i / st af f / j ouni . sm j ouni . sm ed/ scnve/ ed/ scnve/ j ouni . sm j ouni . sm ed/ scnve/ ed/ scnve/

Examinations 1 (2) Examinations 1 (2)

• examination dates

examination dates

1. 1.

March 15, 2004 March 15, 2004

2. 2.

April 5, 2004 April 5, 2004

3. 3.

May 10, 2004 May 10, 2004

• check the exact times and places at

check the exact times and places at ht t p: / / ht t p: / /www . i t . ut u. f i / . ut u. f i / opet us/ t ent i t

• pet us/ t ent i t /

ht t p: / / ht t p: / /www . i t . ut u. f i / . ut u. f i / opet us/ t ent i t

• pet us/ t ent i t /
• if you are

if you are not not a student of University of Turku, you must a student of University of Turku, you must register to receive the credits register to receive the credits

• further instructions are available at

further instructions are available at ht t ht t p: p: / / / / ww www.

• w. t u

t ucs

• cs. f i

. f i / E / Edu duca cat i t i on

• n/ I

/ I nf o nf or m r m at at i o i on/ n/ ht t ht t p: p: / / / / ww www.

• w. t u

t ucs

• cs. f i

. f i / E / Edu duca cat i t i on

• n/ I

/ I nf o nf or m r m at at i o i on/ n/ r eg r egcr cr ed edi t i t s.

• s. ph

php r eg r egcr cr ed edi t i t s.

• s. ph

php

Examinations 2 (2) Examinations 2 (2)

• questions

questions

• based on the lectures and additional literature (3 articles)

based on the lectures and additional literature (3 articles)

• four questions à 8 points

four questions à 8 points

• to pass the examination, at least 16 points (50%) are required

to pass the examination, at least 16 points (50%) are required

• questions are in English, but you can answer in English or in Fi

questions are in English, but you can answer in English or in Finnish nnish

• remember to enrol in time!

remember to enrol in time!

Course Book Course Book

• S. Singhal and M. Zyda,
• S. Singhal and M. Zyda,

Networked Virtual Networked Virtual Environments: Design and Environments: Design and Implementation Implementation, Addison , Addison-

• Wesley, Reading, MA,

Wesley, Reading, MA, 1999. 1999.

• Chapters 1

Chapters 1– –7 (pp. 1 7 (pp. 1– –249) 249)

Additional Literature Additional Literature

• S. Singhal,
• S. Singhal, Effective Remote Modeling in Large

Effective Remote Modeling in Large-

• Scale

Scale Distributed Simulation and Visualization Environments Distributed Simulation and Visualization Environments, PhD , PhD thesis, Stanford University, Stanford, CA, 1996. Chapter 2, pp. thesis, Stanford University, Stanford, CA, 1996. Chapter 2, pp. 13 13– –33. 33.

• S. Benford, C. Greenhalgh, T. Rodden, and J. Pycock,
• S. Benford, C. Greenhalgh, T. Rodden, and J. Pycock,

Collaborative virtual environments, Collaborative virtual environments, Communications of the Communications of the ACM ACM, 44(7):79 , 44(7):79– –85, 2001. 85, 2001.

• J. Smed, T. Kaukoranta, and H. Hakonen, Aspects of
• J. Smed, T. Kaukoranta, and H. Hakonen, Aspects of

networking in multiplayer computer games, networking in multiplayer computer games, The Electronic The Electronic Library Library, 20(2):87 , 20(2):87– –97, 2002. 97, 2002.

SLIDE 2

Special Course on Networked Virtual Environments January 22, 2004 Jouni Smed 2

Outline of the Course Outline of the Course

1. 1.

Introduction Introduction

2. 2.

Background Background

• history

history

• past projects and applications

past projects and applications

3. 3.

Networking Networking

• data transfer and protocols

data transfer and protocols

• communication architectures

communication architectures

4. 4.

Managing dynamic shared state Managing dynamic shared state

• consistency

consistency-

• throughput trade

throughput trade-

• ff
• ff
• centralized information

centralized information repositories repositories

• frequent state regeneration

frequent state regeneration

• dead reckoning

dead reckoning

5. 5.

System design System design

• threads

threads

• polygon culling and level

polygon culling and level-

• of
• f-
• detail

detail

6. 6.

Resource management Resource management

• packet compression and

packet compression and aggregation aggregation

• area

area-

• of
• f-
• interest filtering

interest filtering

• exploiting perceptual limitations

exploiting perceptual limitations

7. 7.

Other issues Other issues

• security

security

• case examples

case examples

§1 Introduction §1 Introduction

• Networked Virtual Environment

Networked Virtual Environment (NVE) ‘is (NVE) ‘is a software system in a software system in which multiple users interact with which multiple users interact with each other in real each other in real-

• time, even

time, even though those users may be located though those users may be located around the around the world.’ world.’ — —Singhal Singhal & Zyda, 1999 & Zyda, 1999

• Keywords:

Keywords:

• global

global

• real

real-

• time

time

• multiple

multiple

• user

user

• interaction

interaction

Winframe Server Network Cable Network Server Mainframe Modem Home PC Modem Laptop Modem Modem Modem

Application Application Areas for NVEs Areas for NVEs

• Military and industrial team training

Military and industrial team training

• Collaborative design and engineering

Collaborative design and engineering

• Multiplayer games

Multiplayer games

• Mobile entertainment

Mobile entertainment

• Virtual shopping malls

Virtual shopping malls

• Online tradeshows and conferences

Online tradeshows and conferences

• Remote customer support

Remote customer support

• Distance learning

Distance learning

Synonyms, Synonyms, Keywords and Abbreviations Keywords and Abbreviations

• Collaborative Virtual Environment (CVE)

Collaborative Virtual Environment (CVE)

• Computer

Computer-

• Supported

Supported Co Co-

• operative
• perative Work (CSCW)

Work (CSCW)

• Media

Media-

• spaces, shared spaces

spaces, shared spaces

• Distributed Interactive Simulation (DIS)

Distributed Interactive Simulation (DIS)

• Distributed Virtual Environment (DVE)

Distributed Virtual Environment (DVE)

• Virtual Reality (

Virtual Reality (VR), Virtual Environment (VE), VR), Virtual Environment (VE), Virtual Virtual Worlds Worlds

• Augmented

Augmented Reality (AR) Reality (AR)

• ...

...

Classification Classification of

• f

Shared Shared-

• Space

Space Technologies 1 (2) Technologies 1 (2)

• Physical reality

Physical reality

• resides in the local, physical

resides in the local, physical world world

• here and now

here and now

• Telepresence

Telepresence

• a real world location remote from

a real world location remote from the participant’s physical the participant’s physical location location

• a

a remote remote-

• controlled

controlled robot robot

Benford et al., 1998 Benford et al., 1998

Augmented Augmented Reality Reality Virtual Virtual Reality Reality Physical Physical Reality Reality Tele Tele-

• presence

presence

Transportation Transportation Artificiality Artificiality local local remote remote synthetic synthetic physical physical

Classification of Classification of Shared Shared-

• Space Technologies 2 (2)

Space Technologies 2 (2)

• Augmented reality

Augmented reality

• synthetic objects are overlaid on

synthetic objects are overlaid on the local environment the local environment

• a head

a head-

• up display (HUD)

up display (HUD)

• Virtual reality

Virtual reality

• the participants are immersed in

the participants are immersed in a remote, synthetic world a remote, synthetic world

• a networked virtual environment

a networked virtual environment (NVE) (NVE)

Benford et al., 1998 Benford et al., 1998

Augmented Augmented Reality Reality Virtual Virtual Reality Reality Physical Physical Reality Reality Tele Tele-

• presence

presence

Transportation Transportation Artificiality Artificiality local local remote remote synthetic synthetic physical physical

SLIDE 3

Special Course on Networked Virtual Environments January 22, 2004 Jouni Smed 3

Features of NVEs 1 (2) Features of NVEs 1 (2)

• A shared sense of space

A shared sense of space

• illusion of being located in the same place

illusion of being located in the same place

• same characteristics for all participants

same characteristics for all participants

• time of day, weather,

time of day, weather, acoustics, haptics... acoustics, haptics...

• A shared sense of presence

A shared sense of presence

• a participant has a virtual persona, an

a participant has a virtual persona, an avatar avatar

• graphical presentation, body structure model, motion model, phys

graphical presentation, body structure model, motion model, physical ical model, etc model, etc. .

• entering and leaving is visible for other participants

entering and leaving is visible for other participants

• all participants

all participants do not do not have to be human have to be human-

• controlled

controlled

Features of NVEs 2 (2) Features of NVEs 2 (2)

• A shared sense of time

A shared sense of time

• see

see other participants’

• ther participants’ actions when they

actions when they occur

• ccur
• enables real

enables real-

• time interaction

time interaction

• A way to communicate

A way to communicate

• by gesture, by typed text, by voice...

by gesture, by typed text, by voice...

• A way to share

A way to share

• interact realistically not only with each other but also with th

interact realistically not only with each other but also with the virtual e virtual environment itself environment itself

Basic Components of (N)VE System 1 (2) Basic Components of (N)VE System 1 (2)

• Graphic engines and displays

Graphic engines and displays

• the cornerstone of the NVE

the cornerstone of the NVE user user interface interface

• head

head-

• mounted displays (HMD)

mounted displays (HMD)

• Cave Automatic Virtual

Cave Automatic Virtual Environment Environment (CAVE) (CAVE)

• Control and communication devices

Control and communication devices

• keyboard, mouse

keyboard, mouse

• joystick

joystick

• dataglove

dataglove

• HMD

HMD

• motion detectors in full

motion detectors in full-

• body immersive environments

body immersive environments

• microphone

microphone

Basic Components of (N)VE System 2 (2) Basic Components of (N)VE System 2 (2)

• Processing systems

Processing systems

• NVEs demand a considerable amount of processing capacity

NVEs demand a considerable amount of processing capacity

• computes the effects of the user’s actions

computes the effects of the user’s actions

• determines when to notify other users

determines when to notify other users

• receives information from other users

receives information from other users

• controls autonomous objects

controls autonomous objects

• computes a visualization of the virtual environment

computes a visualization of the virtual environment

• Data network

Data network

• exchange information

exchange information

• notify about environment changes

notify about environment changes

• synchronize the shared state

synchronize the shared state

• communication among users

communication among users

Challenges in Challenges in Design and Design and Development 1 (3) Development 1 (3)

• Difficult to implement correctly and effectively

Difficult to implement correctly and effectively

• Include multiple traditional software types

Include multiple traditional software types

• NVEs are

NVEs are

• distributed systems

distributed systems

• contend with managing network resources, data loss, network fail

contend with managing network resources, data loss, network failure, ure, concurrency concurrency

• graphical applications

graphical applications

• maintain real

maintain real-

• time display frame rate

time display frame rate

• allocate the CPU among several tasks

allocate the CPU among several tasks

• interactive applications

interactive applications

• process real

process real-

• time input

time input

• users should see the virtual environment as if it exists locally

users should see the virtual environment as if it exists locally

Challenges in Challenges in Design and Design and Development 2 (3) Development 2 (3)

• NVEs

NVEs must work with other applications must work with other applications

• typically

typically integrate with database systems integrate with database systems

• need

need to support user authentication and may interact with commerce to support user authentication and may interact with commerce and other transaction systems and other transaction systems

• to

to support reproducible systems, support reproducible systems, must must be able to log events in real be able to log events in real-

• time

time to a persistent to a persistent storage storage

• the

the complete state of the NVE may not be known at any single host complete state of the NVE may not be known at any single host

• Optimizing one element of

Optimizing one element of the the NVE is NVE is hazardous hazardous

• Consider as a unified system

Consider as a unified system

• NVE development is a difficult

NVE development is a difficult balancing act balancing act of

• f trade

trade-

• offs
• ffs

SLIDE 4

Special Course on Networked Virtual Environments January 22, 2004 Jouni Smed 4

Challenges in Design Challenges in Design and and Development 3 (3) Development 3 (3)

Balancing of Balancing of

i. i.

Network bandwidth Network bandwidth

ii. ii.

Heterogeneity Heterogeneity

iii. iii.

Distributed interaction Distributed interaction

iv. iv.

Real Real-

• time system design and resource management

time system design and resource management

v. v.

Failure management Failure management

vi. vi.

Scalability Scalability

vii.

• vii. Deployment and configuration

Deployment and configuration

i.

• i. Network Bandwidth

Network Bandwidth

• Amount of desired information varies

Amount of desired information varies

• Amount of users varies

Amount of users varies

• How to allocate a limited network

How to allocate a limited network capacity? capacity?

ii.

• ii. Heterogeneity

Heterogeneity

• Users

Users do not have equipment with do not have equipment with the same the same quality quality

• Whether to expose or hide the differences between participants

Whether to expose or hide the differences between participants

• connection speed, processing capacity,…

connection speed, processing capacity,…

• Hide by reducing the system to

Hide by reducing the system to the the lowest common lowest common denominator denominator

• a single

a single ‘bad’ ‘bad’ participant causes problems for participant causes problems for everybody else everybody else

• Take

Take a full a full advance of advance of the available the available resources resources

• user receive different levels of information

user receive different levels of information

• fair play?

fair play?

• Graphical display, computational, and audio capabilities

Graphical display, computational, and audio capabilities

iii.

• iii. Distributed Interaction

Distributed Interaction

• One of the defining qualities of

One of the defining qualities of an an NVE system NVE system

• NVE system must

NVE system must provide provide each user with the illusion that each user with the illusion that

• the entire environment is located on the local

the entire environment is located on the local machine machine

• the actions of the users have a direct and immediate impact on t

the actions of the users have a direct and immediate impact on the he environment environment

• Difficult because of the

Difficult because of the messaging required messaging required

• Each host attempts to

Each host attempts to

• present a consistent real

present a consistent real-

• time

time view view

• cope

cope with out with out-

• of
• f-
• date information

date information

• Problems when multiple users or components interact

Problems when multiple users or components interact

• collision

collision detection, detection, agreement, and agreement, and resolution among participants resolution among participants

iv.

• iv. Real

Real-

• time System

time System Design and Design and Resource Resource Management Management

• Real

Real-

• time interaction defines the process and thread

time interaction defines the process and thread architecture architecture

• many tasks

many tasks have have hard real hard real-

• time

time constraints constraints

• Support quick detection and processing of user action

Support quick detection and processing of user action

• graphical image generation at fixed rate

graphical image generation at fixed rate

• network packets arrive asynchronously, process them soon

network packets arrive asynchronously, process them soon

• perform physics

perform physics modelling modelling and collision detection and collision detection

• Everything in a single thread, use round

Everything in a single thread, use round-

• robin

robin

• Segment into multiple threads, balance them

Segment into multiple threads, balance them

• shared data structures on each host

shared data structures on each host

• shared locks

shared locks

v.

• v. Failure Management

Failure Management

• One or more of the connected hosts

One or more of the connected hosts can can crash at any time crash at any time

• Network connections

Network connections can can fail fail Categories of failure handling: Categories of failure handling:

1. 1.

System stop System stop

• entire NVE terminates due to

entire NVE terminates due to a missing a missing resource resource

2. 2.

System closure System closure

• no impact

no impact on

• n the existing

the existing users but new users but new ones

• nes are unable to login

are unable to login

3. 3.

System hindrance System hindrance

• a required service becomes unvaible; degrades

a required service becomes unvaible; degrades the experience the experience

4. 4.

System continuance System continuance

• a non

a non-

• critical service becomes unavailable; no

critical service becomes unavailable; no noticeable noticeable effect effect

SLIDE 5

Special Course on Networked Virtual Environments January 22, 2004 Jouni Smed 5

vi.

• vi. Scalability

Scalability

• Can be measured

Can be measured with with the number of entities that may the number of entities that may simultaneously participate in the system simultaneously participate in the system

• may include human

may include human-

• and computer

and computer-

• controlled vehicles,

controlled vehicles, a terrain a terrain, and , and even logical objects even logical objects

• Also,

Also, the number the number of hosts,

• f hosts, and physical

and physical distance between distance between the the hosts hosts

• Depends on

Depends on a variety a variety of factors

• f factors
• network capacity, processor capabilities, rendering speeds,…

network capacity, processor capabilities, rendering speeds,…

• The complexity of an NVE increases exponentially with the

The complexity of an NVE increases exponentially with the number of entities because of the number possible interactions number of entities because of the number possible interactions between them between them

• Expensive

Expensive to achieve because it requires enhancements to to achieve because it requires enhancements to virtually all aspects virtually all aspects of

• f the NVE

the NVE system system

vii.

• vii. Deployment

Deployment and Configuration and Configuration

• Deploying the software to participants

Deploying the software to participants

• if

if the software is large, it is inappropriate for downloading the software is large, it is inappropriate for downloading

• a

a small core library with dynamically downloaded components small core library with dynamically downloaded components

• Implications to

Implications to the software the software design, implementation language, design, implementation language, and supported platforms and supported platforms

• In the case of web

In the case of web browsers or light browsers or light-

• weight platforms,

weight platforms, ensure ensure that the enviroment that the enviroment

• can be

can be easily downloaded easily downloaded

• conforms

conforms the security bounds the security bounds

• executes and displays correctly across different platforms

executes and displays correctly across different platforms

• Participants need an access to the configuration information

Participants need an access to the configuration information

• network addresses, encryption keys, access codes, images,

network addresses, encryption keys, access codes, images, computational modes,... computational modes,...

§2 Background §2 Background

• Department of Defense (

Department of Defense (DoD DoD) )

• SIMNET

SIMNET

• Distributed Interactive Simulation (

Distributed Interactive Simulation (DIS) DIS)

• High

High-

• Level Architecture (HLA)

Level Architecture (HLA)

• Academic

Academic NVEs NVEs

• NPSNET

NPSNET

• PARADISE

PARADISE

• DIVE

DIVE

• BrickNet

BrickNet

• ther academic projects
• ther academic projects
• Networked games and demos

Networked games and demos

• SGI

SGI Flight Flight and and Dogfight Dogfight

• Doom

Doom

• ther multiplayer games
• ther multiplayer games

History and Evolution History and Evolution

1980 1980 1990 1990 2000 2000

SIMNET SIMNET

DIS DIS HLA HLA DVE DVE CVE CVE

DIVE, Spline, MASSIVE, Coven DIVE, Spline, MASSIVE, Coven NPSNET, STOW NPSNET, STOW

Military Military Academic Academic Entertainment Entertainment

Amaze Amaze RB2 RB2 Doom Doom

MUD MUD

Battle.net Battle.net Ultima Online Ultima Online Air Warrior Air Warrior

Network Software Architecture (NSA) Network Software Architecture (NSA)

• NSA includes the inseparable

NSA includes the inseparable issues of issues of

• what

what network protocol is used network protocol is used for the system and for the system and

• what

what software architecture supports that protocol, software architecture supports that protocol,

within within the confines of the available bandwidth and processor the confines of the available bandwidth and processor capacity. capacity.

• Important to solve

Important to solve both both problems at problems at once!

• nce!

U.S. Department U.S. Department of

• f Defense (DoD)

Defense (DoD)

• The largest

The largest developer of NVEs for use as simulation systems developer of NVEs for use as simulation systems

• ne of the first to develop NVEs with its SIMNET system
• ne of the first to develop NVEs with its SIMNET system
• the first to do work on large

the first to do work on large-

• scale NVEs

scale NVEs

• SIMNET (simulator networking)

SIMNET (simulator networking)

• begun 1983, delivered 1990

begun 1983, delivered 1990

• a distributed military virtual environment developed for DARPA

a distributed military virtual environment developed for DARPA (Defense Advanced Research Projects Agency) (Defense Advanced Research Projects Agency)

• develop a

develop a ‘low ‘low-

• cost’

cost’ NVE for training small units (tanks, NVE for training small units (tanks, helicopters,…) to fight as a team helicopters,…) to fight as a team