Immersive Telepresence in in Immersive Telepresence Immersive - - PowerPoint PPT Presentation

1

DFN Jahrestagung, Berlin 10.3.99

Immersive Telepresence in Responsive Virtual Environments Immersive Telepresence Immersive Telepresence in in Responsive Virtual Responsive Virtual Environments Environments

Martin Göbel GMD - German National Research Center for Information Technolgy IMK - Institute for MediaCommunication goebel@gmd.de http://imk.gmd.de/docs/ww/ve/

2

DFN Jahrestagung, Berlin 10.3.99

GMD is .... VMSD is ... GMD GMD is is .... VMSD .... VMSD is is ... ...

❏ Geman National Research Center for Information Technology ❏ 1200 employees ❏ ~100M$ budget ❏ 1/3 project funding ❏ 8 institutes: (near) BONN, BERLIN, DARMSTADT ❏ GMD´s high end graphics divison ❏ ~25 scientist & ~15 students ❏ runnig Onyx´s, cave-like CyberStage 2-sided Workbenches wall projections BOOM, virtual studio audio lab ❏ Together with TAN

• HoloBench
• cavelike CUBE
• Tanorama

3

DFN Jahrestagung, Berlin 10.3.99

brief VR (device) History brief brief VR ( VR (device device) ) History History

❏ 88-90 NASA AMES ❏ 90-92 pioneers in Europe: UK, Germany ❏ 92/93 Responsive Workbench (GMD), CAVE (EVL) ❏ 93-97 demo centers in Germany ❏ 94- EUROPEAN project funding ❏ 95- industrial interest (automotive) ❏ 96- many more CAVEs & Benches ❏ 97- industry caves ❏ 98 2-sided Responsive Workbench

4

DFN Jahrestagung, Berlin 10.3.99

Outline Outline Outline

❏ Projective Virtual Environments ❏ TeleConferencing - TelePresence ❏ Immersive TelePresence Trials ❏ Conclusions

5

DFN Jahrestagung, Berlin 10.3.99

The Responsive Workbench

TM 1993

The Responsive Workbench The Responsive Workbench

TM TM

1993

1993

❏ by Wolfgang Krüger ❏ table top display: desk - table - workbench ❏ horizontal plane: 1.8*1.3m ❏ GMD meanwhile version 4: flat, 10°, dual sided ❏ NOW: product by many companies

mirror screen eyes projector

6

DFN Jahrestagung, Berlin 10.3.99

❏ CAVE UIC: Cruz Neira et al 92/93 ❏ cave-like room size display ❏ 8 channel spatial audio ❏ acoustic floor

The CyberStage (1996) The CyberStage The CyberStage (1996) (1996)

side view side view

screens mirrors

acoustic floor

Tracker VW Sharan Interior Design data VW MDC Operation Theatre Prototyping

7

DFN Jahrestagung, Berlin 10.3.99

The 2-sided Resp. Workbench (1998) The The 2- 2-sided sided Resp Resp. . Workbench Workbench (1998) (1998)

❏ THE VE display for interactive applications ❏ extended viewing frustrum ❏ head tracked (as others) ❏ 2-8 channel audio

eyes

screens

mirror mirror

large interaction space (even for 2 viewers) collision free 3D interaction within frustrum dual hand interaction

8

DFN Jahrestagung, Berlin 10.3.99

Compatibility by AVOCADO System Compatibility by Compatibility by AVOCADO System AVOCADO System

❏ since 4/96; beta 1/99 ❏ C++ VE framework ❏ OGL, Performer ❏ scripting in Scheme ❏ modular, extendible ❏ scene graph API ❏ distribution by active object replication and IP multi casting ❏ configurable to sensores and services (IO)

Application Programming

AVOCADO

IRIX 6.5 Performer Scheme

9

DFN Jahrestagung, Berlin 10.3.99

Application Structure Application Structure Application Structure

Service Service Service Service Sensor

Field Field Field

Sensor

Field Field

Node

Field Field Field

Node

Field Field Field

Node

Field Field

Node Reference Field Connection

10

DFN Jahrestagung, Berlin 10.3.99

Applications under Investigation Applications under Investigation Applications under Investigation

❏ Engineering Visualization ❏ Medical Visualization ❏ GeoScience Visualization ❏ Architecture, interior Design ❏ Mathematics ❏ Molecular Design ❏ Education, Entertainment, Story Telling ❏ ...

11

DFN Jahrestagung, Berlin 10.3.99

VR Application development VR VR Application development Application development

❏ Presentation

visual/ audible / haptic rendering

❏ Navigation

point&fly / head tracking / physical probs

❏ Interaction

gloves / pointing devices / task specific tools

❏ Networking

single site / two users / teams & communities

❏ Communication

shared images / audio / video / guestures

❏ Towards Collaboration in virtual teams

12

DFN Jahrestagung, Berlin 10.3.99

TeleConferencing ... TeleConferencing TeleConferencing ... ...

❏ Most real-world applications require communi- cation and - if possible - collaboration between geographically distributed teams ❏ good practice:

• -> pictureTel, shared X applications, …

❏ what is urgently needed: large screen interactive stereo visualization + video-conferencing for synchronous, symmetric and asymmetric collaboration

13

DFN Jahrestagung, Berlin 10.3.99

Teleport Teleport (1995) (1995)

Display rooms with wall size Video wall Merging of real and virtual environments Chroma keying or image segmentation Head tracking Stereo video textures and surround audio Video texture via ATM (D1: 270mbs to 32mbs)

14

DFN Jahrestagung, Berlin 10.3.99

Teleteaching Teleteaching Teleteaching

❏ operating between GMD and University Geneva ❏ bidirectional video + audio ❏ documents as video but also via internet available

U Geneva Session GMD Session

15

DFN Jahrestagung, Berlin 10.3.99

connecting CAVE to RWB (1997) connecting connecting CAVE to RWB (1997) CAVE to RWB (1997)

❏ EUROPEAN ACTS Project for distributed Video Production (DVP) ❏ task with NCSA Urbana, Caterpillar ❏ 97 - 98; on 34mbs ATM (also SC98, Orlando) ❏ aim:

• connect NCSA CAVE to GMD RWB via transatlantic

ATM

• heterogenous VE systems; usability study
• shared industrial application by Caterpillar
• audiovisual communication

16

DFN Jahrestagung, Berlin 10.3.99

NCSA-GMD Caterpillar scenario NCSA-GMD NCSA-GMD Caterpillar scenario Caterpillar scenario

Driver at NCSA

Visible image at Workbench Videotextures

Cat Driver Supervisor Truck Driver

Supervisor at GMD

Responsive Workbench CAVE

17

DFN Jahrestagung, Berlin 10.3.99

Blue Room CyberStage

Audio &Video

simulate the communication and collaboration between two projective VR systems

Immersive TelePresence trial (´97) Immersive TelePresence trial Immersive TelePresence trial (´97) (´97)

remote virtually present

18

DFN Jahrestagung, Berlin 10.3.99

Immersive Telepresence Installation Immersive Telepresence Immersive Telepresence Installation Installation

Infra-red Camera Monitor Stereo Camera

Blue Room Cavelike CyberStage

SP SP SP SP SP Audio Mixer

Onyx 2-pipe RE Onyx 2-pipe IR

Sirius 1 2 Sirius 1 2 Framestore Ultimatte L R

Audio Mixer

19

DFN Jahrestagung, Berlin 10.3.99

Immersive Telepresence Immersive Telepresence Immersive Telepresence

❏ Expert/student roles ❏ Stereo-video integration into virtual world ❏ Interaction and communication in a Virtual Environment ❏ Asymmetric blue room studio - cavelike scenario

Teaching and training within an immersive virtual environment:

20

DFN Jahrestagung, Berlin 10.3.99

Immersive Telepresence Immersive Telepresence Immersive Telepresence

❏ symmetric solutions in caves???

Many open problems … camera position, light situation, unichrome for keying, ...

❏ start with 2-sided RWB

• fixed camera ... field of cameras
• IR stereo camera
• blue wall

❏ at least 155mbs ATM for bidirectional av- stereo; additional video sources cost extra!

21

DFN Jahrestagung, Berlin 10.3.99

Cooperative Responsive Workbench Cooperative Responsive Workbench Cooperative Responsive Workbench

ATM OC12 : ISDN 8 So

2-sided RWB at Site A 2-sided RWB at Site B

• Evtl. More Sites

22

DFN Jahrestagung, Berlin 10.3.99

Immersive Telepresence Immersive Immersive Telepresence Telepresence

❏ Equal roles ❏ Stereo-video integration into virtual world ❏ Interaction and communication in a Virtual Environment ❏ Symmetric Scenario

Collaboration in distributed immersive virtual environment

23

DFN Jahrestagung, Berlin 10.3.99

Video Video Video

24

DFN Jahrestagung, Berlin 10.3.99

Conclusions & further Research Conclusions Conclusions & & further Research further Research

❏ 1:1 cRWB with reasonable depth recognition ❏ moving team members (out of camera field)?? ❏ preservation of parallax and reduction of distortion at any? Time?? ❏ Collaborative interaction ❏ N:m extension is application dependent ❏ Re-extension to CAVE … we would like, BUT less industrial interest and much more difficult.

25

DFN Jahrestagung, Berlin 10.3.99

Acknowledging …. Acknowledging Acknowledging …. ….

❏ Dr Stephen Barras ❏ Jeremy Eccles ❏ Dr Gerhard Eckel ❏ Dr Bernd Fröhlich ❏ Dr Simon Gibbs ❏ Gernot Goebbels ❏ Frank Hasenbrink ❏ Berthold Kirsch ❏ Dr Vali Lalioti ❏ Uli Lechner

http://imk.gmd.de/docs/ww/ve/

❏ Jonny Plate ❏ Michael Tirtasana ❏ Henrik Tramberend ❏ Gerold Wesche ❏ Jürgen Wind ❏ Björn Zehner