Outline Gaze-Based Interaction in Cinematic 360 VR Cinematic 360 - PowerPoint PPT Presentation

Outline Gaze-Based Interaction in • Cinematic 360 VR Cinematic 360 VR • Gaze-Based Interaction • Non-Linear Storytelling • Media Authoring Luiz Velho VISGRAF Lab - IMPA • Case Study Omnidirectional Video • Equirectangular Format - Latitude / Longitude Parametrization Cinematic 360 VR • Stereoscopic Layout

360 Video Player Spatial Sound • 3D Sound • Texture Map to Sphere • Skybox / Panoramic Shader • Ambisonics Format Interaction Mechanism • Flow of Control Gaze-Based Interaction VR Camera Gaze Action VR Interactive Gaze Detection Item

Gaze Detection The Hot-Spot • Creates Interaction Event • VR Eye Raycaster VR Camera ROI ROI Rectangle VR Interactive Item Gaze Action • Action Trigger Non-Linear Storytelling Box

Timeline Time-Flow Control • Playable Director • Narrative Structure • Gaze Interaction Track - Non-Linear Narratives - Viewer’s Gaze Direction Tracks Clips Clips Marker • Identifies a Time Instant in the Timeline • Clip Element (Gaze Track) Clip Object • Clip Types (Gaze Interact) - Marker - Player - Jumper - Spotter Clip Parameters - Trigger - Timer - Choicer

Player Jumper • Changes Current Time to Marker Location • Activates Playback of Audio or E ff ect Spotter Trigger • Implements Hot-Spot for Audio • Triggers an Action of Interaction Game Object

Timer Choicer • Controls Jump to Element of a Markers Set ( based on interaction behaviour ) • Start Time-Dependent Jump to a Mark Interaction Objects • Perform Action Based on Narrative Interactivity • Connect with Clips of the Gaze Interaction Track Media Authoring • Incorporates a VR Interactive Item Component • Basic Interaction Objects - Audio Source - Gaze Rotation - Gaze Marker Selector

Audio Source Gaze Rotation • Controls Audio Playback • Changes the VR Camera Direction • Used for - Voice-Over • Cut Viewer’s Gaze - Music - Location in Panoramic Image - Ambient Noise - Sound E ff ects Gaze Marker Selector Interaction Idioms • Language of Non-Linear Interactive Narrative • Defined by Combination of Gaze Interaction Clips • Selects a Marker Label • Some Useful Idioms - Based on Procedural Criteria - Jumper + Marker • Used in Combination with Choicer Clip - Trigger + Player - Spotter + Player + Marker - Timer + Jumper + Marker - Choicer + Markers

Jumper + Marker Trigger + Player • Gaze-Based Play of Game Object • Skip to a Di ff erent Time Spotter + Player + Marker Timer + Jumper + Marker • Selective Temporal Play of Alternative Audio Clips • Loop in a Sequence until Time Expires

Choicer + Markers • Jump to Selective Time Locations Case Study Context Searching for Aloyo • 2017 Creators Lab 2.0 Program • Partnership • Oculus / Facebook Filmmaker Nonprofit Research • Synopsis

Children Do Not Play War “Thanks.” – Luiz Velho