Building a Life-Size Automultiscopic Display Using Consumer Hardware - - PowerPoint PPT Presentation

▶
building a life size automultiscopic
SMART_READER_LITE
LIVE PREVIEW

Building a Life-Size Automultiscopic Display Using Consumer Hardware - - PowerPoint PPT Presentation

Dec 01, 2023 178 likes •803 views

Building a Life-Size Automultiscopic Display Using Consumer Hardware Andrew Jones, Jonas Unger*, Koki Nagano, Jay Busch, Xueming Yu, Hsuan-Yueh Peng, Oleg Alexander, Paul Debevec USC Institute for Creative Technologies *Linkping University

slide-1
SLIDE 1

Building a Life-Size Automultiscopic Display Using Consumer Hardware

Andrew Jones, Jonas Unger*, Koki Nagano, Jay Busch, Xueming Yu, Hsuan-Yueh Peng, Oleg Alexander, Paul Debevec USC Institute for Creative Technologies *Linköping University

slide-2
SLIDE 2

Stereoscopic Automutiscopic

slide-3
SLIDE 3

Automutiscopic

How do we capture, render, display automultiscopic content?

slide-4
SLIDE 4

Projectors Anisotropic screen Audience

slide-5
SLIDE 5

1st prototype Focus on face 2nd prototype Full-size bodies

slide-6
SLIDE 6

3D Geometry custom vertex shader Image-based Light Fields custom pixel shader

slide-7
SLIDE 7

Bandwidth

1920 x 1080 x 60 fps x 360⁰ x 24 bit = 134GB / sec Large number of output streams Data transfer to GPU

slide-8
SLIDE 8

Our Approach

  • Distribute rendering across multiple GPUs

and computers

  • Scalable, additional projectors increases

field of view

slide-9
SLIDE 9

Kara video

slide-10
SLIDE 10

Takanori Okoshi, Three-Dimensional Imaging Techniques, Academic Press 1976

  • Fig. 5.5(b), “projection-type three-dimensional display”, p. 131

projectors screen

Screen materials:

  • Horizontal lenticular
  • Light shaping diffusers
  • Brushed metal
  • Privacy filters
slide-11
SLIDE 11

Anisotropic Projector Arrays

[Agocs et al. 2007]

Holographika

[Kawatika et al. 2012] [Yoshida et al. 2011]

slide-12
SLIDE 12

Projector Array

  • 72 TI DLP Pico

– 480 x 320 Resolution – Mini HDMI input

  • 1.66° Angular Resolution
  • 110° Field of View
slide-13
SLIDE 13
  • 40 lines per inch Lenticular screen from Microlens Inc.
  • 1° horizontal x 60° vertical diffuser from Luminit Co.

Anisotropic Screen

slide-14
SLIDE 14

Graphics Cards

AMD Radeon 7870 graphics cards, 4 x 6 Mini DisplayPort outputs = total 24 outputs DisplayFusion (nView, Ultramon)

slide-15
SLIDE 15

Video Splitters

24 Matrox TripleHeadToGo video splitters

– 1 DisplayPort input, 3 DisplayPort outputs each

slide-16
SLIDE 16

DisplayPort 1.2

  • Multi-Stream Transport (MST)
  • Appear as separate displays
  • Each display can have different

resolution/refresh rate etc

  • Each graphics card still has upper bound

for total number of streams

slide-17
SLIDE 17
slide-18
SLIDE 18

Multiple-center of projection

Every pixel rendered from different viewpoint

slide-19
SLIDE 19
slide-20
SLIDE 20
  • For each vertex, find

corresponding viewer

  • Project back onto

screen from view point

projectors screen current viewer vertex possible viewers

Vertex projection

slide-21
SLIDE 21

current viewer current projector screen vertex possible viewers S

Vertex projection

slide-22
SLIDE 22

Runing in Activision pipeline

  • Picture of Activision

demo

slide-23
SLIDE 23
  • Sum of weighted

Gaussians

  • Can revert back to

default height and distance

  • Falloff distance ≈ width
  • f shoulders

Multiple viewers

slide-24
SLIDE 24
  • 72 TI PICO

projectors

Anisotropic Projector Arrays

Jones et al. “Interpolating Vertical Parallax for an Autostereoscopic 3D Projector Array”. SPIE Stereoscopic Displays and Applications 2014

slide-25
SLIDE 25

Projectors

slide-26
SLIDE 26
slide-27
SLIDE 27

Vivitek Qumi projectors

  • 1280 x 800 pixels
  • LED light source
  • 300 Lumens
  • Low power, small

size

  • ~$300 each
slide-28
SLIDE 28

The Anisotropic Screen

1° horizontal x 60° vertical diffuser from Luminit Co

slide-29
SLIDE 29

The Anisotropic Screen

Light from each projector is scattered as a vertical stripe

slide-30
SLIDE 30

The Anisotropic Screen

Light from each projector is scattered as a vertical stripe

slide-31
SLIDE 31

The Anisotropic Screen

Each view is composed

  • f multiple projector

stripes

slide-32
SLIDE 32

The Anisotropic Screen

Each view is composed

  • f multiple projector

stripes

slide-33
SLIDE 33

Light Stage 6 8 meter geodesic dome LED illumination

30 cameras

slide-34
SLIDE 34

Capture

slide-35
SLIDE 35
slide-36
SLIDE 36

Video showing input clips

slide-37
SLIDE 37

Light Field Sampling

0.625 degrees between projectors

slide-38
SLIDE 38

Light Field Sampling

1.75 degrees between eyes at 2 meters

slide-39
SLIDE 39

Light Field Sampling

6 degrees between cameras

slide-40
SLIDE 40

View Interpolation

Camera 1 Camera 2 Virtual View Proxy geometry Actual geometry

slide-41
SLIDE 41

View Interpolation

Camera 1 Camera 2 Virtual View Proxy geometry Actual geometry

slide-42
SLIDE 42
slide-43
SLIDE 43

Geometry Reconstruction

  • Visual hulls, stereo

reconstruction

  • Relatively slow
  • AGIsoft - 40 minutes per

frame with 30 cameras

Image-Based Visual Hulls Matusik et al., SIGGRAPH ‘00 Free-viewpoint Video of Humans Carranza et al., SIGGRAPH ‘03

slide-44
SLIDE 44
slide-45
SLIDE 45

Einarsson et al. “Relighting Human Locomotion with Flowed Reflectance Fields”, EGSR 2006

slide-46
SLIDE 46
  • M. Werlberger, T. Pock, and H. Bischof: Motion Estimation with Non-Local Total

Variation Regularization, IEEE Conference on Computer Vision and Pattern Recognition (CVPR), San Francisco, CA, USA, June 2010.

slide-47
SLIDE 47

View Interpolation

Camera 1 Camera 2 Virtual View Proxy geometry Actual geometry

slide-48
SLIDE 48

View Interpolation

Camera 1 Camera 2 Virtual View Proxy geometry Actual geometry

slide-49
SLIDE 49
slide-50
SLIDE 50

Modular

  • 4 carts, 72 projectors
slide-51
SLIDE 51
slide-52
SLIDE 52
slide-53
SLIDE 53

The Interview

slide-54
SLIDE 54
slide-55
SLIDE 55

Video Decoding

  • 11 source videos, 20 optical flow videos

per GPU

  • CPU decoding FFMPEG (multi-core)
  • GPU MPEG video decoding (NVCUVID)
slide-56
SLIDE 56

Distributed rendering

Windows 7 Default: commands sent to most single GPU and blitted across Current solution: New instance of application per GPU Next step: OS/Vendor specfic extensions to assign resources to GPUs (ie WGL_NV_gpu_affinity)

Shalini Venkataraman, “Programming Multi-GPUs for Scalable Rendering” GTC 2012

slide-57
SLIDE 57

Ongoing Work

  • Incorporate natural

language processing / artificial intelligence

  • Extend up to 30+ hours of

interview

Arstein et al. “Time-Offset Interaction with a Holocaust Survivor”, Proceedings of International Conference On Intelligent User Interfaces (IUI), 2014

slide-58
SLIDE 58
slide-59
SLIDE 59

Conclusions

  • Simple techniques for rendering geometry

and light fields for automultiscopic displays

  • Limited by GPU bandwidth
  • Need new tools to exploit redundancy, and

distribute resources across views

slide-60
SLIDE 60

Questions

Thanks to CNN, Morgan Spurlock, Inside Man Productions, Shoah Foundation, Pinchas Gutter, Julia Campbell, Bill Swartout, Randall Hill, Randolph Hall, U.S. Air Force DURIP, and U.S. Army RDECOM

http://gl.ict.usc.edu/

slide-61
SLIDE 61