Aug me nte d Re a lity Sung -e ui Yo o n Project Guidelines: - - PDF document

▶
aug me nte d re a lity
SMART_READER_LITE
LIVE PREVIEW

Aug me nte d Re a lity Sung -e ui Yo o n Project Guidelines: - - PDF document

Nov 04, 2022 410 likes •764 views

1 Aug me nte d Re a lity Sung -e ui Yo o n Project Guidelines: Project Topics Any topics related to the course theme are okay You can find topics by browsing recent papers 2 Expectations Mid-term project presentation Introduce

slide-1
SLIDE 1

1

Aug me nte d Re a lity

Sung -e ui Yo o n

slide-2
SLIDE 2

2

Project Guidelines: Project Topics

  • Any topics related to the course theme are okay
  • You can find topics by browsing recent papers
slide-3
SLIDE 3

3

Expectations

  • Mid-term project presentation
  • Introduce problems and explain why it is important
  • Give an overall idea on the related work
  • Explain what problems those existing techniques have
  • (Optional) explain how you can address those problems
  • Explain roles of each member
  • Ack. any external slides/codes, etc.
slide-4
SLIDE 4

4

Expectations

  • Final-term project presentation
  • Review materials that you talked for your mid-term project
  • Present your ideas that can address problems of those state-of-

the-art techniques

  • Give your qualitatively (or intuitive) reasons how your ideas

address them

  • Also, explain expected benefits and drawbacks of your approach
  • (Optional) backup your claims with quantitative results collected

by some implementations

  • Explain roles of each members
  • Ack. any external slides/codes, etc.
slide-5
SLIDE 5

5

A few more comments

  • Start to implement a paper, if you don’t have any clear

ideas

  • While you implement it, you may get ideas about improving it
slide-6
SLIDE 6

6

Project evaluation sheet

Speaker Novelty of the project and idea (1 ~ 5) Practical benefits

  • f the method

(1 ~ 5) Completeness level of the project (1 ~ 5) Total score (3 ~ 15) Role of each student is clear and well balanced? (Yes or No) XXX YYY

You name: ID: Score table: higher score is better.

slide-7
SLIDE 7

7

Go a ls fo r T

  • da y

Goals

  • Ge tting a c q ua inte d with AR
  • Ma king yo ur o wn Aug me nte d Re a lity
  • So me o f re c e nt te c hniq ue s
slide-8
SLIDE 8

8

Wha t is Aug me nte d Re a lity?

Re a l Wo rld E nviro nme nt Virtua l Re a lity Aug me nte d Re a lity

Milgram, Paul, et al. "Augmented reality: A class of displays on the reality-virtuality continuum." Photonics for industrial applications. International Society for Optics and Photonics, 1995. Photo Source: http://smartideasblog.trekk.com/augmented-or-virtual-how-do-you-like-your-reality

Re a l Wo rld E nviro nme nt Virtua l Re a lity Aug me nte d Re a lity

slide-9
SLIDE 9

9

Brie f Histo ry

  • 1968 I

va n Suthe rla nd inve nts first he a d-mo unte d displa y “Swo rd-o f-Da mo c le s” a t Unive rsity o f Uta h.

  • 2010 Vufo ria fo r AR Mo b ile Apps wa s re le a se d b y Qua lc o mm.
  • 2013 Go o g le a nno unc e s Go o g le Gla ss.
  • 2015 Mic ro so ft a nno unc e s the Ho lo L

e ns.

  • 2016 Nia ntic re le a se d Po ké mo n Go .

Original slides are from Frend, Chauncey. ”Augmented Reality & the UITS Advanced Visualization Lab."

slide-10
SLIDE 10

10

AR to da y – I ndustry E xa mple s

AR Color ing Book

(0:00-0:30)

Augme nte d Climbing Wall

(0:00-0:33)

HoloL e ns

Live Texturing of Augmented Reality Characters from Colored Drawings IEEE International Symposium on Mixed and Augmented Reality (ISMAR) 2015

slide-11
SLIDE 11

11

Asse ts + Displa y + I

nte rfa c e

Sour c ing or Building Data

Building E xpe rie nc e s

dedicated workshops on Mar. 24th & 31st

  • Yo u c a n a lso a ug me nt with me dia

– Audio – 2D me dia (I

ma g e & vide o )

– T

e xt

  • 3D Authoring

Autodesk Maya or Max Sketchup TinkerCAD

  • 3D Scanning

3D Scanning Photogrammetry

  • 3D Repositories

Sketchup 3D Warehouse Smithsonian X 3D https://3d.si.edu/browser Sketchfab* Thingiverse*

slide-12
SLIDE 12

12

Mobile De vic e s PC or Mac with We bc am HoloL e ns He adse t

Building E xpe rie nc e s

Asse ts + Display + I nte rfa c e

slide-13
SLIDE 13

13

Onboar ding AR – Pr

  • totyping T
  • ols

Building E xpe rie nc e s

Asse ts + Displa y + Inte r

fac e

No pro g ra mming re q uire d

slide-14
SLIDE 14

14

Base Pac kage AR Plugins AR T e c hnology

  • Mo b ile De vic e Apps
  • I

ma g e o r Ob je c t Ma rke rs

  • De skto p o r L

a pto p Apps

  • F

iduc ia l o r I ma g e Ma rke rs

  • Ho lo L

e ns de vic e Apps

  • Are a L

e a rning o r I ma g e Ma rke rs

De ve lopme nt T

  • ols

Building E xpe rie nc e s

Asse ts + Displa y + Inte r

fac e

slide-15
SLIDE 15

15

Wha t the future ma y ho ld

We b Hoste d AR Applic ations Consume r

  • gr

ade he adse ts

Cybe r Physic al AR Applic ations

(2:16-3:04)

slide-16
SLIDE 16

16

Re so urc e s fo r yo u!

T uto ria ls https:/ / unity3d.c o m/ le a rn

Base Pac kage AR Plugins Vuforia Basics Tutorial By: Chauncey https://www.youtube.com/watch?v=qbI9PrSUo5w AR YouTube Playlist

slide-17
SLIDE 17

17

Technical Components

  • Robotics parts
  • Simultaneous localization and mapping (SLAM)
  • Computer vision parts
  • Geometry, light, and materials estimation
  • Graphics parts
  • Rendering virtual objects
  • Machine learning
  • Understand various things
  • AR spans various fields
slide-18
SLIDE 18

18

Modeling Surface Appearance from a Single Photography using Self-Augmentation [Li et al.]

  • Assume the anisotropic Ward BRDF model
  • Diffuse albedo and normal map per each pixel
  • Specular parameters for each image

Ground truth Reconstructed

  • nes

Original images with relighted images

slide-19
SLIDE 19

19

Network Architectures

  • Two separate architectures
  • Self-augmentation
  • Use unlabeled image

and reconstruct parameters

  • Generate its corresponding image
  • Use them as training pairs with labeled ones
slide-20
SLIDE 20

20

LIME: Live Intrinsic Material Estimation [Meka et al. CVPR 18]

  • Estimate specular information of an object

in the RGB image

  • Starts with the rendering equation, but ends up

with assuming the Phone illumination: diffuse and specular terms

slide-21
SLIDE 21

21

Real-Time Geometry, Albedo, and Motion Reconstruction Using a Single RGB-D Camera

Kaiwen Guo1, Feng Xu1, Tao Yu1,2, Xiaoyang Liu1, Qionghai Dai1, Yebin Liu1

Tsinghua University1 Beihang University2

slide-22
SLIDE 22

22

Offline Volumetric Performance Capture

[Starck and Hilton, 2007], [Liu et al., 2009] [Vlasic et al. 2009], [Debevec, Light Stage], [Collet et al. 2015]

slide-23
SLIDE 23

23

Real-time Volumetric Performance Capture

[Fusion4D, Dou et al. 2016], [Holoportation, Orts‐Escolano et al. 2016]

slide-24
SLIDE 24

24

Real-time Single-view Volumetric Capture

slide-25
SLIDE 25

25

Key Idea: Joint Optimization considering Shading

Geometry & Motion Appearance (diffuse objects)

slide-26
SLIDE 26

26

Geometry Motion Albedo (color)

Overview

Color Depth Input Output

Assume diffuse objects

slide-27
SLIDE 27

27

slide-28
SLIDE 28

28

slide-29
SLIDE 29

29

slide-30
SLIDE 30

30

slide-31
SLIDE 31

31

slide-32
SLIDE 32

32

slide-33
SLIDE 33

33

Near-Eye Light Field Display [ToG 13]

  • Use a microlens array for supporting the light field
slide-34
SLIDE 34

34

Goals for Today were:

  • Getting acquainted with AR
  • Making your own Augmented

Reality

  • Some of recent techniques