Raheel Khalid - @rkhalid890 A game industry and operating systems - - PowerPoint PPT Presentation

Raheel Khalid - @rkhalid890

A game industry and operating systems veteran that serves as the Chief Engineer for Verizon Labs. Using years of experience in building large scale game engines and graphics platforms I work to architect and set the technical vision for Verizon's future in VR & AR streaming services

Confidential and proprietary materials for authorized Verizon personnel and outside agencies only. Use, disclosure or distribution of this material is not permitted to any unauthorized persons or third parties except by written agreement.

2

William Dildine

William brings over eight years of front end web design, systems and network design and integration, and Content Management Systems deployment. He applies project management best practices and agile software development methodology for rapid prototyping, server/client side website development, website administration, cloud migration, online marketing, and e-commerce management. His diverse web experience includes database management, UI/UX design, layout development, 3D web applications, and Search Engine Optimization /Search Engine Marketing management.

Confidential and proprietary materials for authorized Verizon personnel and outside agencies only. Use, disclosure or distribution of this material is not permitted to any unauthorized persons or third parties except by written agreement.

3

• Accommodate next generation content delivery
• Allow for projection or augmentation
• Help users create 3d and 2d content using easy to

use tools

• Build worlds and connect your audience
• Bring traditional advertising into the real world
• View your content anywhere
• Deliver it fast and seamlessly

What did Verizon Labs want to do?

Confidential and proprietary materials for authorized Verizon personnel and outside agencies only. Use, disclosure or distribution of this material is not permitted to any unauthorized persons or third parties except by written agreement.

4

COSMOPOLITAN

Envrmnt turned traditional print media into a more engaging content by overlaying an article with a video and purchasing options for Cosmopolitan. For product purchase, Cosmopolitan’s ecommerce site and a check-

• ut process were integrated into the app.

This implementation showcases a new way of engaging your target audience and increasing sales for brands.

Deeper Customer Engagement Integrated Purchasing Options Customer Data Delivery to Any Device Robust Solution on a Budget

Augmented Reality

6

ALPINE VILLAGE

The Alpine Village gaming experience is a fully immersive, snow-filled world. Players can teleport around the world, fire snowballs at snowmen and virtually interact with

• ther players in real time.

These virtual worlds give your target audience the

• pportunity to experience your brand from anywhere. 


For example, players can encounter branded advertising while teleporting around the world via branded object placement, 3D billboards, etc.

Virtual Reality

Streaming in Real Time VR Monetization Navigable and Social Experiences Customer Data

So how do we get there?

Augmented and Virtual Reality has a base requirement of low latency render, compute and network transmission

8

So how do we get there?

We now have a technological dependency

• n high resolution rendering, high refresh

rate and loads of computer vision. Low resolution and lag are no longer just annoying, they are a deal breaker

When I was your age, this was a dragon … …and we were grateful to have it.

So how do we get there?

It is generally accepted that the greatest acceptable time it takes to update the screen from a user’s movement is roughly

• 20ms. This is referred to as motion-to-

photon latency. Typical displays on VR/AR devices have a refresh rate of between 60Hz and 90Hz

• r 16.66ms to 11.11ms. This is the time to

update pixels on a screen. The fastest networks in the US provide 40-200ms latency round trip. 



 On 5G, however, there is a theoretical 1ms latency.

So how do we get there?

There is a fundamental need on the network and the GPU to handle rending and compute for the foreseeable future

Confidential and proprietary materials for authorized Verizon personnel and outside agencies only. Use, disclosure or distribution of this material is not permitted to any unauthorized persons or third parties except by written agreement.

10

How can we solve these problems

Confidential and proprietary materials for authorized Verizon personnel and outside agencies only. Use, disclosure or distribution of this material is not permitted to any unauthorized persons or third parties except by written agreement.

11

PROJECT DRIVE

Envrmnt created a virtual drive-in theater where multiple players are able to collectively watch movies (2D videos) and interact with each other in real time, while also being exposed to in-scene advertising. In-scene advertising can be incorporated via object placement, 3D billboards, streaming video, animated ads and/or full branding of the virtual world.

Navigable and Social Experiences Customized Worlds Any Platform Streaming in Real Time VR Monetization Customer Data

Virtual Reality

From VR Works SDK we wanted to highlight the following the functionality:

• Lens Matched Shading
• Single Pass Stereo

Rendering

Confidential and proprietary materials for authorized Verizon personnel and outside agencies only. Use, disclosure or distribution of this material is not permitted to any unauthorized persons or third parties except by written agreement.

13

Single Pass Stereo

• Optimizes drawing of left and right eye
• Uses Simultaneous Multi-Projection

architecture to draw geometry once and then project left and right eyes.

Rendering

Confidential and proprietary materials for authorized Verizon personnel and outside agencies only. Use, disclosure or distribution of this material is not permitted to any unauthorized persons or third parties except by written agreement.

14

Rendering

Our scene consists of

• 64 Animated Pumpkins casting point lights
• 8x MSAA 1512x1680 per eye

Confidential and proprietary materials for authorized Verizon personnel and outside agencies only. Use, disclosure or distribution of this material is not permitted to any unauthorized persons or third parties except by written agreement.

15

Without Single Pass Stereo

• Without single pass stereo - CPU 4-5ms

GPU 7-8ms (occasionally dropping a frame)

• With single pass stereo - CPU 3ms GPU

5-6ms

• Net 40% saving on CPU and 25% on GPU

Rendering

Without Single Pass Shading With Single Pass Shading

Confidential and proprietary materials for authorized Verizon personnel and outside agencies only. Use, disclosure or distribution of this material is not permitted to any unauthorized persons or third parties except by written agreement.

16

Rendering

Lens Matched Shading

• Optimizes multi-res pixel shading by

rendering to a surface that approximates the lens correct output of the image.

Up to 2 x performance improvement for VR

Confidential and proprietary materials for authorized Verizon personnel and outside agencies only. Use, disclosure or distribution of this material is not permitted to any unauthorized persons or third parties except by written agreement.

17

Our scene consists of

• 2268x2520 per eye - 1.5x supersampled
• 4x MSAA
• 48 Animated pumpkins casting point

lights

Rendering

Confidential and proprietary materials for authorized Verizon personnel and outside agencies only. Use, disclosure or distribution of this material is not permitted to any unauthorized persons or third parties except by written agreement.

18

Rendering

Lens Matched Shading

• O Original - CPU 5 ms - GPU 9 ms

(occasionally dropping a frame)

• With Lens-Matched-Shading - CPU

3ms - GPU 6 ms

• 15% saving on GPU is worth having
• Next step would be to use a geometry

shader to bin triangles into viewports and cull all the side triangles that are no longer on screen.

No Optimization Single Pass Rendering and Lens Matched Shading

Confidential and proprietary materials for authorized Verizon personnel and outside agencies only. Use, disclosure or distribution of this material is not permitted to any unauthorized persons or third parties except by written agreement.

19

20

NEON

Project Neon is a fully immersive, rich gaming experience that showcases extreme interactivity, flexibility and real- time customization. While multiple players battle monsters and communicate amongst themselves, a “dungeon master” can simultaneously modify the virtual world, introducing new monsters and adding/changing puzzles in real time.

Navigable and Social Experiences Customized Worlds Any Platform Streaming in Real Time VR Monetization Customer Data

Virtual Reality

Prove that real time rendering with no download/install

• works. (No streaming bitmaps)

Build a game using a streaming protocol that is scalable and synchronized 4 weeks to build upon our current streaming infrastructure and build a connected, scalable environment

Case Study Mission Neon - Goals

Merge traditional rendering techniques with a path to use streaming network infrastructure. (Adaptive Bitrate vs. LOD)

Add assets and interpret them using a unified set of rules Dynamically change the scene on the fly Real time communication services (voice chat, video)

22

Case Study Mission Neon - Features

Introduce a set of game master tools that let’s non-developers easily manipulate a world Streaming spacial audio with audio intensity mapping to pulse the scene to the music

Photography

Hampden Sydney College in Virginia looked up one of the more. looked up

• ne of the more

Email Marketing

Hampden Sydney College in Virginia looked up one of the more. looked up

• ne of the more

Cloud Service

Hampden Sydney College in Virginia looked up one of the more. looked up

• ne of the more

App Development

Hampden Sydney College in Virginia looked up one of the more. looked up

• ne of the more

Demo! - Mission Neon

Case Study:

Mission Neon - World Maker

• Rendering our existing WebGL 1.0 solution involved using a Differed Rendering path — expensive lighting

calculations calculated once per light per covered pixel.

• Rendering our WebGL 2.0 solution implemented Tile Forward — Using a two pass technique generating a grid of

light heat map tiles to partition tiles into lights per-tile.

• Noticeably improved render speeds on small light heavy scenes.
• WebGL 2.0 Particles solution saved us from copying the result from CPU memory to GPU memory
• New implementation of transform feedback allowed simulated elements to be rendered simultaneously.
• In some cases a minimum of 64 components can be written to memory per-element vs the earlier max of 4.
• Writing into the same type of buffer as the buffer you are reading from simplifying the whole system.
• Other uses for Transform Feedback - generating a set of unique VBOs for rendering many complex geometries

(skinned meshes).

Confidential and proprietary materials for authorized Verizon personnel and outside agencies only. Use, disclosure or distribution of this material is not permitted to any unauthorized persons or third parties except by written agreement.

24

25

Accept assets on the web. Process and convert on massively parallel GPU’s on the cloud Package assets using a FlatBuffer like format
 Send a message down to clients to pick up a 
 new asset/environment stream (cache on CDN)

Mission Neon

What did we learn?

• FlatAsset Pipeline

User’s world updates and players enjoy
 Saves file size and speed and adds CDN caching

Confidential and proprietary materials for authorized Verizon personnel and outside agencies only. Use, disclosure or distribution of this material is not permitted to any unauthorized persons or third parties except by written agreement.

25

Packaging assets into streaming formats and separating out LOD works really well
 Controlling a world full of players feels good!
 Delivering seamless VR / AR is tough but not impossible

Case Study Mission Neon What did we learn?

The web is getting closer to solving many of these problems in a unified way, but needs implementation from all browsers

WORLD BUILDER

WorldBuilder is a powerful drag and drop tool that lets anyone create and modify immersive streaming virtual spaces, impacting players in real time and providing rich communication services. This tool can be used to promote brands and services

• r monetize via subscriptions, on-demand content,

micro-transactions or dynamic advertising.

Navigable and Social Experiences Customized Worlds Any Platform Streaming in Real Time VR Monetization Customer Data

Virtual Reality

Where do we think there is still work to be done?

As network technology develops on it becomes more

• f a possibility to offload GPU compute tasks to the

cloud. Future enhancements of low latency high bandwidth connections should consider optimizations for graphics processing

• nly. Use, disclosure or distribution of this material is not permitted to any unauthorized

Thank You.