Occluded Bilateral EPI Regularization 2nd Workshop on Light Fields - - PowerPoint PPT Presentation

▶

Apr 19, 2023 176 likes •340 views

Occluded Bilateral EPI Regularization 2nd Workshop on Light Fields for Computer Vision July 26, 2017 Overview Overview standard minimization approach data term, smoothness term, solver some twists for crisper and smoother results some

SLIDE 1

Occluded Bilateral EPI Regularization

2nd Workshop on Light Fields for Computer Vision

July 26, 2017

SLIDE 2

Overview

OBER - Hendrik Schilling - July 26, 2017 1/15

Overview

standard minimization approach → data term, smoothness term, solver some twists for crisper and smoother results some (over) simplifications

SLIDE 3

Key Insight

OBER - Hendrik Schilling - July 26, 2017 2/15

3D Reconstruction as Minimization Problem

inverse problem underconstrained non-convex

SLIDE 4

Key Insight

OBER - Hendrik Schilling - July 26, 2017 3/15

Key Insights

The Problem is inherently non-differentiable due to

cclusions

→ enforcing differentiabiliy will result in sub-optimal results

Secondary

Intermediate steps impair occlusion border & fine detail performance → formulate error metrics in input domain

SLIDE 5

Method

OBER - Hendrik Schilling - July 26, 2017 4/15

Data T erm

SLIDE 6

Method

OBER - Hendrik Schilling - July 26, 2017 5/15

Smoothness T erm

0.7 0.8 0.8 0.5 0.8 0.5 0.2 0.3 0.8 0.8 0.2 0.3 0.4 0.3 0.2 0.2 0.5 1.0 0.9 0.2 0.1 1.0 0.7 0.6 0.8 0.5 0.6 0.7 candidate d = candidate d =

modified bilateral filter evaluate both color and disparity hard thresholds → crisp occlusion boundaries

SLIDE 7

Method

OBER - Hendrik Schilling - July 26, 2017 6/15

Solver

Algorithm 1 Randomized Solver

1: for 20 iterations do 2:

for Every disp map pixel do

Calc error for current depth

Calc error for depth candidates

Keep best result

end for

7: end for

SLIDE 8

Method

OBER - Hendrik Schilling - July 26, 2017 7/15

Depth Candidates

random change random guess random (large range) neighbour direct neighbour

SLIDE 9

Method

OBER - Hendrik Schilling - July 26, 2017 8/15

Propagation

even pixel processing order neighbour candidates

iteration

SLIDE 10

Results

OBER - Hendrik Schilling - July 26, 2017 9/15

ground truth disparity center view disparity BadPix(0.07) OBER-cross+ANP SPO-MO 2nd best BadPix(0.07) mesh of OBER result (as viewed from above)

SLIDE 11

Results

OBER - Hendrik Schilling - July 26, 2017 10/15 0.00 18.51 3.01 1.76 0.27 0.42 11.17 4.80 10.12 37.01 11.78 6.02 3.52 0.53 0.94 0.85 16.86 22.35 9.59 20.23 12.10 55.52 17.67 9.03 5.27 0.80 1.41 1.27 25.28 33.52 14.39 30.35 18.15 74.03 23.56 12.04 7.03 1.07 1.88 1.69 33.71 44.70 19.19 40.47 24.20

OBER OBER-cross+ANP OFSY_330/DNR PS_RF RM3DE SPO-MO BadPix(0.01) BadPix(0.03) BadPix(0.07) MSE Q25 Bumpiness Contin. Surfaces Bumpiness Planes Discontinuities Fine Fattening Fine Thinning MAE Contin. Surfaces MAE Planes

SLIDE 12

Results

OBER - Hendrik Schilling - July 26, 2017 11/15

SLIDE 13

Results

OBER - Hendrik Schilling - July 26, 2017 12/15 re

reference result (OFSY_330/DNR) OBER-cross+ANP (half-size imgs)

SLIDE 14

Outlook

OBER - Hendrik Schilling - July 26, 2017 13/15

Outlook

A lot of things to improve:

scale space, un-occlusions, heuristics → NN, disp map → mesh

Main strength:

very flexible, per-pixel adaptable solver

SLIDE 15

Outlook

OBER - Hendrik Schilling - July 26, 2017 14/15

SLIDE 16

The End

OBER - Hendrik Schilling - July 26, 2017 15/15