This is 4003-590-09 / 4005-769-09 (Applications in Virtual Reality) - - PDF document

9/7/09
 1


4003-590-04 4005-769-04  This is 4003-590-09 / 4005-769-09

• (Applications in Virtual Reality)

 I am Joe Geigel…your host!

9/7/09
 2


 Logistics  Answer the questions

• What is this course about?
• How will the course will run
• What exactly is Virtual Theatre…and why should I

care?

• How do I fit in

 More logistics  But first…

• attendance

 mycourses

• Announcements
• Dropboxes
• Grades
• UGRADS to be ADDED To GRAD SECTION

 Common mycourses shell.

 E-mail

• Be sure that your e-mail is being forwarded

correct.

9/7/09
 3


 Course Web Site:

• http://www.cs.rit.edu/~jmg/vr

 Contact:

• office hours: MW 10-noon or by appt.
• Office: 70 (GCCIS) Rm 3527
• e-mail: jmg@cs.rit.edu
• phone: 475-2051

 Slides:

• Will be available (in PDF) on Web site.

 Official Prerequisite – one of the following:

• Computer Graphics 1(CS 570/761)
• Foundations of 3D Graphics Programming (IT 502)
• 3D Graphics Programming (IT 735)

 However…

• Need expertise in

 Graphics Progamming  Basic programming  Audio  Networking  Logistics  …

9/7/09
 4


 Computing Environment

• ICL6

 Windows Machines  LUSTER

• Graphics Lab (70-3405)

 MoCap development (measurand)  Need access

• Digital Studio Lab

 Building 7A  Organic Motion moCap system

 No required textbooks.  Articles and docs as required

• Will be posted on Web site / mycourses.

9/7/09
 5


 Introduce students to virtual reality

hardware, software, and toolkits

 Apply to a given domain  Apply to a large scale problem.  Teamwork, teamwork, teamwork!

9/7/09
 6


 Virtual Theatre

• A distributed computer system whereby

performers, stage crew, and audience can be in physically separate places yet share in the same live theatrical performance.

 This is a project based course  Teams will be assembled

• Each team will work on a specific component / problem
• Components will integral part of larger system

 Collective goal:

• To build on an existing framework for a virtual theatre system
• Test the framework out on short virtual performance.
• Opening night: Final Exam

9/7/09
 7


 Collaboration

• Within teams
• Between teams
• With artists in School of Design

 Questions so far?

 Schedule will be aggressive  Weeks 1

• Intro material
• Team formation

 Week 2

• Define team goals

 Weeks 3 – 10

• Group work
• Lectures: status meetings / updates / working sessions
• 3 checkpoints

 Finals Week

• Showtime!

Will work towards production schedule of final exam

9/7/09
 8


 Checkpoints

• Periodic Status checks

 Weeks 4,7,10  At least 1 intermediate deliverable  Demo

 Checkpoint deliverables vary by team

• Will be discussed at start of CP cycle.

 Status meetings

• Announcements / new developments
• News from the artistic side

 Updates  Mini-deliverables

• TBD

 Guaranteed team working time.

9/7/09
 9


 Checkpoint 1 -- Sept 28th  Checkpoint 2 – Oct 19th  Checkpoint 3 – Nov 9th  Showtime!

• Finals week

 Team grade

• Meeting checkpoint
• Deliverable + demos
• Working with final demo

 Individual Grades

• Peer review

 Evaluation of teams from those not in teams

• Teammate Evaluation

 Evaluation of students from teammates

• Leader Evaluation

 Evaluation of grad leaders from teammates

9/7/09
 10


 In addition, Grad Students will be

assessed:

• Leadership Role -- Leader evals
• Documentation -- Team documentation is

responsibility of leader.

• Other deliverables -- as agreed upon with

instructor. Graduate Undergrad Checkpoints (Team) 25% 35% Final Demo (Team) 35% 40% Peer Evals (Individual) 20% 25% Leadership / Docs (GRAD) 20%

9/7/09
 11


 Answer the questions

• What is this course about?
• How will the course will run
• What exactly is Virtual Theatre…why should I care?
• How do I fit in

 Any questions?

9/7/09
 12


 Art and Technology Collaboration

 Visuals  Lighting / Staging  Technology

 Our experiments in theatrical storytelling

• Live…action takes place in a distributed virtual space
• Performers / crew control from a physical space
• Active Audience Participation

Integration of distributed virtual reality / gaming technologies into the theatrical storytelling process.

9/7/09
 13


 Machinima

a new form of filmmaking that uses computer games technology to shoot films in the virtual reality of a game engine.

Anna (2003) -- Fountainhead Entertainment

9/7/09
 14


 Machinimation (Fountainhead Entertainment)

http://www.fountainheadent.com

 Cinema

• Recorded
• Editted
• Passive Audience
• POV of filmmaker

 Theatre

• Live
• Real time
• Active Audience
• POV of audience

9/7/09
 15


 Gaming Engines

• Provide object oriented graphical abstraction
• Performs real time rendering
• Provides networking infrastructure

 Requirements for theatrical use

• Adapt to use theatrical metaphor / language
• Physical control of virtual characters and staging
• Theatrical components designed by artisans.

 API

• Set of programming libraries used to create 3D games
• Implies use of low level language (C, C++, Java)

 Scripting Systems

• Extending of existing games (Mods)
• Game specific language
• Constrained by capabilities of game / script
• Examples

 Quake II  Unreal Tournament  Second Life

9/7/09
 16


Graphics Hardware

Graphics API

Gaming Engine

VIRTUAL THEATRE LAYER participant

Networking API

Network Hardware To network To display

Sound API

Sound Hardware To speaker Virtual stage

9/7/09
 17


 Components of a

theatrical production

• Action
• Staging
• Audience
• Sound / Music

 Objects to be defined on

virtual stage

• Actors
• Stage Manager
• Audience Member
• Orchestra

What’s the Buzz (Spring 2004) Getting By (Spring 2005) Critters (Spring 2006 – Fall 2008)

9/7/09
 18


 Controls characters on stage  Actors have associated avatar(s) on

virtual stage.

 Specific control of avatar behavior defined

by subclasses of Actor

 Example 1

• Puppets

9/7/09
 19


 Example 2

• Gesture (What’s the Buzz? Spring 2004)

 Example 3

• Full Body Motion Capture

9/7/09
 20


 Set and Lighting Design

• Created by artistic team
• Imported into Virtual Stage
• Controllable Components

 Triggering of Cues  Continuous Events

 What’s the Buzz?

9/7/09
 21


 Getting By  Controls dynamic staging, lighting, and

sound effects

 Triggering of predefined cues.  Stage manager object manages cues and

allows for manual (or automatic) triggering of these cues.

 Object in virtual space

9/7/09
 22


Stage Mgr object Lights Props Sound

 Each audience member is also an object in

the virtual space

• May (or may not) have a physical avatar
• Human audience member connected via

audience device control

• Interactivity

 Control of what the audience sees

9/7/09
 23


 Interactivity (What’s the Buzz?)  Perspective (What’s the Buzz?)

9/7/09
 24


 Special Effects  Critters

• Rose colored glasses
• Audience decides to see visions or not.

9/7/09
 25


 Let’s take a break…  After break: Our latest show + tasks for

this quarter.

 Spring 2009 (ImagineRIT 09)  Two full body mocap systems  Many characters  Combine with animation and video  Proprietary software replaces game engine

http://www.youtube.com/watch? v=e8rNfSP8zd0

9/7/09
 26


 Let’s go to the video tape.  Development platform / game engine

• LUSTER

 Actor Control

• 2 motion capture systems
• Puppets

 Lighting / staging

• Movable spotlight
• Sound effects
• Confetti

9/7/09
 27


 Emcee (announces theacts)  Crossovers (quick jokes between acts)

• E.g. Taking my case to court

 Acts:

• Marching band (just instruments)
• Two Tens for a Five (video)
• Two Tens for a Five (virtual)
• Singer
• Stars and Stripes forever

 Focus on TOOLS!  Repeat performance of VTVS

• Creation of TOOLS to automate processes
• Documentation of processes

 USERS GUIDE to Virtual Theatre

• Add features left out due to time constraints

in the Spring

 Final exam will be this repeat

performance to a live audience

• LIVE THEATRICAL EVENT

9/7/09
 28


 Production Schedule

• Development
• Checkpoints
• REHEARSAL

 We are tech!  Teams

• Actor Control Interface
• Asset Importing
• Audience Experience
• Audio Systems
• MoCap Integration
• Staging Tools

9/7/09
 29


 Two motion capture systems

• ShapeWrap III (measurand)
• Stage (OrganicMotion – markerless)

ShapeWrap III OM Stage

 Autodesk MotionBuilder

• Interface between moCap system and game

engine.

• Determines character motion from moCap

data.

9/7/09
 30


 Responsible for placement and assigning

control to actors.

 Tasks:

• Tools for positioning actors on virtual stage
• Attaching moCap streams to characters.

 Tasks

• Coordination of actors between 2 mocap

environments

• Exchange of Objects between actors

 Wiimote Interface

• Basic MoCap logistics (setup, calibration, etc).

9/7/09
 31


 Tasks:

 “Packaging” / Distribution of Audience Client  Placement/control of audience avatars  Audience controls  Testing of audience clients in various venues on campus.

 Tasks:

• Define and document pipeline from Maya to

VT stage.

• Easy to use tools for artists to visualize how

their models will look on the stage.

• Documenting modeling limitations and

constraints.

 Experience with Maya preferred.

9/7/09
 32


 Tasks:

• Incorporation of real time streamed audio.

 Either within or external to LUSTER

 Tasks:

• Creation of interface to create, manage, and

trigger cues.

• Create interface for lighting design.
• Explore the use of QLAB as interface to

staging components (could lead to Grad Project).

9/7/09
 33


 Choosing teams

• First Quiz!

 Student survey on mycourses  Please fill out ASAP (by EOD Sept 11)  Will try my best to accommodate  PLEASE Be as specific as possible!

• Leaders

 6 teams / 10 grad students / 5 undergrad students  We’ll see once assignments have been made  One team / several tasks?

 Questions?  VTVS – Fall 2009  Internet’s Got Talent – Spring 2010

9/7/09
 34


 Fill out student questionaire / team

request on mycourses.

 To prepare for next class:

• Paper on Virtual Theatre on Web site.

 Wednesday’s class will be a crash course

in LUSTER.

 Questions?