SeeTh Throu ough: Findin inding Cha hair irs in Heavily ily - - PowerPoint PPT Presentation

▶

seeth throu ough findin inding cha hair irs in heavily

SeeTh Throu ough: Findin inding Cha hair irs in Heavily ily - - PowerPoint PPT Presentation

Nov 28, 2022 241 likes •645 views

SeeTh Throu ough: Findin inding Cha hair irs in Heavily ily Occlud luded d Ind ndoor or Scene ne Images Moos Hueting Pradyumna Ersin Yumer Vladimir G.Kim Nathan Carr Niloy J.Mitra University College Adobe Research Adobe Research

slide-1

SLIDE 1

SeeTh Throu

ugh: Findin

inding Cha hair irs in Heavily ily Occlud luded d Ind ndoor

r Scene

ne Images

Moos Hueting

University College London

Pradyumna Reddy

University College London

Ersin Yumer

Adobe Research

Vladimir G.Kim

Adobe Research

Nathan Carr

Adobe Research

Niloy J.Mitra

University College London

slide-2

SLIDE 2

slide-3

SLIDE 3

slide-4

SLIDE 4

slide-5

SLIDE 5

Goal: extract 3D scene mock up from single image (focused on chairs and other highly occluded

bjects)

slide-6

SLIDE 6

Context is Important

slide-7

SLIDE 7

Context is Important

slide-8

SLIDE 8

Pipeline

slide-9

SLIDE 9

Pipeline

slide-10

SLIDE 10

Pipeline

slide-11

SLIDE 11

Pipeline

slide-12

SLIDE 12

Pipeline

slide-13

SLIDE 13

Pipeline

slide-14

SLIDE 14

Pipeline

slide-15

SLIDE 15

Keypoint estimation

slide-16

SLIDE 16

Keypoint Dataset

cvgl.stanford.edu/projects/objectnet3d/ Selecting Vertices of the overlaid CAD model Objectnet3D Ground truth annotation Input image

slide-17

SLIDE 17

Keypoint thresholding

slide-18

SLIDE 18

Keypoint thresholding

slide-19

SLIDE 19

Pipeline

slide-20

SLIDE 20

Vanishing point estimation

slide-21

SLIDE 21

Pipeline

slide-22

SLIDE 22

PCA template

slide-23

SLIDE 23

Fit parameters

slide-24

SLIDE 24

Candidate Set

slide-25

SLIDE 25

Pipeline

slide-26

SLIDE 26

Candidate selection

Unary Costs: measure how well the key points explain the object Pairwise Costs: Capture relationship between objects

slide-27

SLIDE 27

Relative transform

Relative Rotation Relative Translation

slide-28

SLIDE 28

Candidate selection

slide-29

SLIDE 29

Pipeline

slide-30

SLIDE 30

Results

http://geometry.cs.ucl.ac.uk/projects/2018/seethrough/

slide-31

SLIDE 31

Results and Dataset

http://geometry.cs.ucl.ac.uk/projects/2018/seethrough/

slide-32

SLIDE 32

Results and Dataset

http://geometry.cs.ucl.ac.uk/projects/2018/seethrough/

slide-33

SLIDE 33

Results

Real World Images SeeingChairs Im2CAD Ours

http://geometry.cs.ucl.ac.uk/projects/2018/seethrough/

slide-34

SLIDE 34

Results

Real World Images SeeingChairs Im2CAD Ours

http://geometry.cs.ucl.ac.uk/projects/2018/seethrough/

slide-35

SLIDE 35

Results

Real World Images SeeingChairs Im2CAD Ours

http://geometry.cs.ucl.ac.uk/projects/2018/seethrough/

slide-36

SLIDE 36

Performance comparison

slide-37

SLIDE 37

Goal: extract 3D scene mock up from single image (focused on chairs and other highly occluded objects) Main insight: cases with significant occlusion can be improved by using high-level contextual knowledge about how scenes “work” Main result: resulting scene mock ups significantly better than combinations of state-of-the-art methods which are reliant on object detection algorithms.

slide-38

SLIDE 38

Limitations

First, we plan to extend the evaluation to more classes of objects beyond

those considered.

Second, one can explore higher fidelity models to better recover fine scale

features in the recovered models.

Finally, we would like to explore templates that can express a broader

understanding of the multi-object spatial relationships including symmetry and regularity.

slide-39

SLIDE 39

This work is in part supported by the Microsoft PhD fellowship program, and ERC Starting Grant SmartGeometry (StG-2013-335373). Also, special thanks to Aron Monszpart, James Hennessey, Carlo Innamorati, Paul Guerrero, and other group members for invaluable help at various stages of the project.

Acknowledgement

slide-40

SLIDE 40

Thank You

Code available: geometry.cs.ucl.ac.uk/projects/2018/seethrough/paper_docs/Code_Data.zip