Unfolding an Indoor Origami World David Fouhey, Abhinav Gupta, - PowerPoint PPT Presentation

Unfolding an Indoor Origami World David Fouhey, Abhinav Gupta, Martial Hebert 1

2

3

Local Evidence Hoiem et al. 2005, Saxena et al. 2005, Fouhey et al. 2013, etc. 4

Constraints 5

Constraints for Single Image 3D Local Smoothness Low Level, Generic Hoiem et al. 2005, Saxena et al. 2005, 2008, Munoz et al., 2009, etc. 6

Constraints for Single Image 3D Local Smoothness Low Level, Generic Hoiem et al. 2005, Saxena et al. 2005, 2008, Munoz et al., 2009, etc. 7

Constraints for Single Image 3D Low Level, High Level, Generic Physical 8

Constraints for Single Image 3D Local Smoothness Low Level, High Level, Generic Physical Coughlan and Yuille 2000, etc. 9

Constraints for Single Image 3D Local Smoothness Low Level, High Level, Generic Physical Hedau et al. 2009, Del Pero et al., 2011, Wang et al., 2012, Schwing et al. 2012, etc. 10

Constraints for Single Image 3D Local Smoothness Low Level, High Level, Generic Physical Lee et al. 2010, Xiao et al. 2012, Zhao et al. 2013, Schwing et al., 2013, etc. 11

Constraints for Single Image 3D Low Level, High Level, Generic Physical 12

Mid-level in the Past Huffman 71, Clowes 71, Kanade 80, 81 Sugihara 86, Malik 87, etc. 13

Our Mid-Level Constraints 14

This Work Input: Output: Single Image Discrete Scene Parse 15

Overview Parameterization Formulation Experimental Results 16

Overview Parameterization Formulation Experimental Results 17

Parameterization 18

Parameterization vp 2 vp 3 vp 1 VP Estimator from Hedau et al., 2009 19

Parameterization Two VPs give grid cell 20

Encoding Surface Normals 21

Encoding Surface Normals 22

Encoding Surface Normals 23

Encoding Surface Normals x 1 ,…, x 400 x 401 ,…, x 800 x 801 ,…, x 1200 24

Related Parameterizations vp 2 x 4 vp 3 vp 1 x 2 x 3 x 1 Hedau et al., 2009; Wang et al. 2010, Schwing et al., 2012, 2013 25

Overview Parameterization Formulation Experimental Results 26

Parameterization 27

Formulation 28

Unaries 29

Unaries Any High c Low c 3D Evidence 30

Unaries Local: Data-Driven 3D Primitives Input 3D Primitive Bank … Fouhey, Gupta, Hebert, 2013 31

Unaries Global: Cuboid Fit + Clutter Mask Input Predicted Walls Clutter Mask Hedau, Hoiem, Forsyth, 2009 32

Binaries 33

Convex/Concave Constraints Convex (+) Concave (-) 34

Convex/Concave Constraints Detected Concave (-) 35

Convex/Concave Constraints Detected Concave (-) 36

Convex/Concave Constraints Detected Concave (-) 37

Convex/Concave Constraints Detected Concave (-) 38

Convex/Concave Constraints Detected Concave (-) 39

Detecting Convex/Concave Use 3DP to Transfer Discontinuities Input 3D Primitive Bank … Ground-Truth Discontinuities similar to Gupta, Arbelaez, Malik, 2013 3DP from Fouhey, Gupta, Hebert, 2013 40

Smoothness 41

Constraints 42

Solving the Model 43

Overview Parameterization Formulation Experimental Results 44

Dataset NYU Depth v2: 795 Train, 654 Test 45

Qualitative Results 46

Qualitative Results 47

Qualitative Results 48

Qualitative Results 49

Surface Connection Graphs Convex Concave 50

Baseline Primary Baseline: 3D Primitives Input Output Fouhey, Gupta, Hebert, 2013 51

Quantitative Results % Good Pixels Summary Stats ( ⁰) (Lower Better) (Higher Better) Mean Median RMSE 11.25⁰ 22.5⁰ 30⁰ Proposed 35.1 19.2 48.7 37.6 53.3 58.9 3DP 36.0 20.5 49.4 35.9 52.0 57.8 Hedau et al. 40.0 23.5 54.1 34.2 49.3 54.4 Lee et al. 43.3 36.3 54.6 18.6 38.6 49.9 52

Quantitative Results 53

Failure Modes Mistaken but Confident Evidence 54

Failure Modes Missing High-Level Modeling 55

Conclusions Parameterization Discrete Parse Single Image Formulation 56

Thank You 57