Outline Gaze-Based Interaction in Cinematic 360 VR Cinematic 360 - - PowerPoint PPT Presentation

Gaze-Based Interaction in Cinematic 360 VR

Luiz Velho

VISGRAF Lab - IMPA

Outline

• Cinematic 360 VR
• Gaze-Based Interaction
• Non-Linear Storytelling
• Media Authoring
• Case Study

Cinematic 360 VR Omnidirectional Video

• Equirectangular Format
• Latitude / Longitude Parametrization
• Stereoscopic Layout

360 Video Player

• Texture Map to Sphere
• Skybox / Panoramic Shader

Spatial Sound

• 3D Sound
• Ambisonics Format

Gaze-Based Interaction Interaction Mechanism

• Flow of Control

Gaze Detection VR Interactive Item VR Camera Gaze Action

Gaze Detection

• VR Eye Raycaster

VR Camera VR Interactive Item

ROI

The Hot-Spot

• Creates Interaction Event

ROI Rectangle

Gaze Action

• Action Trigger

Box

Non-Linear Storytelling

Timeline

• Narrative Structure

Clips Tracks

Time-Flow Control

• Playable Director
• Gaze Interaction Track
• Non-Linear Narratives
• Viewer’s Gaze Direction

Clips

• Clip Element (Gaze Track)
• Clip Types (Gaze Interact)
• Marker
• Player
• Jumper
• Spotter
• Trigger
• Timer
• Choicer

Clip Object Clip Parameters

Marker

• Identifies a Time Instant in the Timeline

Player

• Activates Playback of Audio or Effect

Jumper

• Changes Current Time to Marker Location

Spotter

• Implements Hot-Spot for Audio

Trigger

• Triggers an Action of Interaction Game Object

Timer

• Start Time-Dependent Jump to a Mark

Choicer

• Controls Jump to Element of a Markers Set (based on interaction behaviour)

Media Authoring Interaction Objects

• Perform Action Based on Narrative Interactivity
• Connect with Clips of the Gaze Interaction Track
• Incorporates a VR Interactive Item Component
• Basic Interaction Objects
• Audio Source
• Gaze Rotation
• Gaze Marker Selector

Audio Source

• Controls Audio Playback
• Used for
• Voice-Over
• Music
• Ambient Noise
• Sound Effects

Gaze Rotation

• Changes the VR Camera Direction
• Cut Viewer’s Gaze
• Location in Panoramic Image

Gaze Marker Selector

• Selects a Marker Label
• Based on Procedural Criteria
• Used in Combination with Choicer Clip

Interaction Idioms

• Language of Non-Linear Interactive Narrative
• Defined by Combination of Gaze Interaction Clips
• Some Useful Idioms
• Jumper + Marker
• Trigger + Player
• Spotter + Player + Marker
• Timer + Jumper + Marker
• Choicer + Markers

Jumper + Marker

• Skip to a Different Time

Trigger + Player

• Gaze-Based Play of Game Object

Spotter + Player + Marker

• Selective Temporal Play of Alternative Audio Clips

Timer + Jumper + Marker

• Loop in a Sequence until Time Expires

Choicer + Markers

• Jump to Selective Time Locations

Case Study Context

• Oculus / Facebook

Searching for Aloyo

• 2017 Creators Lab 2.0 Program

Filmmaker Nonprofit Research

• Partnership
• Synopsis

Children Do Not Play War

– Luiz Velho

“Thanks.”