X Disparity Determination in Stereo Vision Lu Sang, Michael Haberl, Raphael Ullmann 22.07.2017 Lu Sang, Michael Haberl, Raphael Ullmann
Overview Problem Description Algorithms Results Lu Sang, Michael Haberl, Raphael Ullmann
Overview Problem Description Algorithms Results Lu Sang, Michael Haberl, Raphael Ullmann
Problem Description Lu Sang, Michael Haberl, Raphael Ullmann
Problem Description Lu Sang, Michael Haberl, Raphael Ullmann
Problem Description • What is the distance to an object? • How to determine the distance by 2D images? 1https://upload.wikimedia.org/wikipedia/commons/4/49/Roboterhand.mit.Gluehbirne.png Lu Sang, Michael Haberl, Raphael Ullmann
Problem Description • What is the distance to an object? • How to determine the distance by 2D images? Applications • Autonomous driving • Robotics • Object recognition 1 1https://upload.wikimedia.org/wikipedia/commons/4/49/Roboterhand.mit.Gluehbirne.png Lu Sang, Michael Haberl, Raphael Ullmann
Stereo Cameras Lu Sang, Michael Haberl, Raphael Ullmann
Corresponding Pixels Figure: Left Picture [1] Figure: Right Picture Lu Sang, Michael Haberl, Raphael Ullmann
Corresponding Pixels Figure: Left Picture [1] Figure: Right Picture Lu Sang, Michael Haberl, Raphael Ullmann
Corresponding Pixels Figure: Left Picture [1] Figure: Right Picture 1 Calculate for the pixels in the left image costs in the right image Lu Sang, Michael Haberl, Raphael Ullmann
Corresponding Pixels Figure: Left Picture [1] Figure: Right Picture 1 Calculate for the pixels in the left image costs in the right image 2 Pixel with minimal cost is the corresponding pixel Lu Sang, Michael Haberl, Raphael Ullmann
Disparity Lu Sang, Michael Haberl, Raphael Ullmann
Disparity Disparity Pixel distance of related pixels. Lu Sang, Michael Haberl, Raphael Ullmann
Getting distance from Disparity Lu Sang, Michael Haberl, Raphael Ullmann
Getting distance from Disparity Lu Sang, Michael Haberl, Raphael Ullmann
Getting distance from Disparity Lu Sang, Michael Haberl, Raphael Ullmann
Getting distance from Disparity Lu Sang, Michael Haberl, Raphael Ullmann
Distance z ✏ f ☎ b d • Distance z • Focal length of the camera f • Disparity d Lu Sang, Michael Haberl, Raphael Ullmann
Overview Problem Description Algorithms Results Lu Sang, Michael Haberl, Raphael Ullmann
Overview Pre-processing Cost Calculation Pyramid Scheme Force Local Consistency Penalty Terms Median Filter Lu Sang, Michael Haberl, Raphael Ullmann
Overview Pre-processing Cost Calculation Pyramid Scheme Force Local Consistency Penalty Terms Median Filter Lu Sang, Michael Haberl, Raphael Ullmann
Pre-processing: Undistortion Original Picture Undistorted Picture Lu Sang, Michael Haberl, Raphael Ullmann
Pre-processing: Undistortion Original Picture Undistorted Picture Lu Sang, Michael Haberl, Raphael Ullmann
Pre-processing: Rectification Lu Sang, Michael Haberl, Raphael Ullmann
Pre-processing: Rectification Lu Sang, Michael Haberl, Raphael Ullmann
Pre-processing: Rectification Lu Sang, Michael Haberl, Raphael Ullmann
Pre-processing: Rectification Left Picture Right Picture Lu Sang, Michael Haberl, Raphael Ullmann
Pre-processing: Rectification Left Picture Right Picture Lu Sang, Michael Haberl, Raphael Ullmann
Modelling: Energy Function Energy Function argmin d C ♣ p , d q � S ♣ p , d q • Cost C ♣ p , d q for every pixel p and disparities d ✏ 1 , ..., D • Regularization S ♣ p , d q • E.g. penalty for deviation of neighbouring pixels Lu Sang, Michael Haberl, Raphael Ullmann
Overview Pre-processing Cost Calculation Pyramid Scheme Force Local Consistency Penalty Terms Median Filter Lu Sang, Michael Haberl, Raphael Ullmann
Cost Calculation: Comparing Windows Lu Sang, Michael Haberl, Raphael Ullmann
Cost Calculation: Comparing Windows Lu Sang, Michael Haberl, Raphael Ullmann
❞ ✏ ñ ✏ ✁ ❞ ✏ ñ ✏ ✁ r✁ s Cost Calculation: Cross Correlation Lu Sang, Michael Haberl, Raphael Ullmann
ñ ✏ ✁ ❞ ✏ ñ ✏ ✁ r✁ s Cost Calculation: Cross Correlation ❞ ✏ Lu Sang, Michael Haberl, Raphael Ullmann
ñ ✏ ✁ ❞ ✏ ñ ✏ ✁ r✁ s Cost Calculation: Cross Correlation ❞ ✏ Lu Sang, Michael Haberl, Raphael Ullmann
❞ ✏ ñ ✏ ✁ r✁ s Cost Calculation: Cross Correlation ❞ ✏ Total sum = 2 ñ Costs ✏ ✁ 2 Lu Sang, Michael Haberl, Raphael Ullmann
❞ ✏ ñ ✏ ✁ r✁ s Cost Calculation: Cross Correlation ❞ ✏ Total sum = 2 ñ Costs ✏ ✁ 2 Lu Sang, Michael Haberl, Raphael Ullmann
ñ ✏ ✁ r✁ s Cost Calculation: Cross Correlation ❞ ✏ Total sum = 2 ñ Costs ✏ ✁ 2 ❞ ✏ Lu Sang, Michael Haberl, Raphael Ullmann
r✁ s Cost Calculation: Cross Correlation ❞ ✏ Total sum = 2 ñ Costs ✏ ✁ 2 ❞ ✏ Total sum = 4 ñ Costs ✏ ✁ 4 Lu Sang, Michael Haberl, Raphael Ullmann
Cost Calculation: Cross Correlation ❞ ✏ Total sum = 2 ñ Costs ✏ ✁ 2 ❞ ✏ Total sum = 4 ñ Costs ✏ ✁ 4 Normalization and Zero Mean: Values in r✁ 1 , 1 s Lu Sang, Michael Haberl, Raphael Ullmann
Cost Calculation: Result Lu Sang, Michael Haberl, Raphael Ullmann
Cost Calculation: Result Lu Sang, Michael Haberl, Raphael Ullmann
Cost Calculation: Result • Error Rate of NCC: 33 . 09% Lu Sang, Michael Haberl, Raphael Ullmann
Test Data Figure: Left Image Figure: Right Image Lu Sang, Michael Haberl, Raphael Ullmann
Test Data • Middlebury Dataset • Ground Truth • Leaderboard Lu Sang, Michael Haberl, Raphael Ullmann
Algorithm Defect Lu Sang, Michael Haberl, Raphael Ullmann
Overview Pre-processing Cost Calculation Pyramid Scheme Force Local Consistency Penalty Terms Median Filter Lu Sang, Michael Haberl, Raphael Ullmann
Pyramid Scheme Lu Sang, Michael Haberl, Raphael Ullmann
Pyramid Scheme Lu Sang, Michael Haberl, Raphael Ullmann
Pyramid Scheme Lu Sang, Michael Haberl, Raphael Ullmann
Pyramid Scheme: Results Figure: Results of NCC Figure: Results of Pyramid Scheme • Error Rate of NCC: 33 . 09% • Error Rate of Pyramid Scheme: 28 . 1% Lu Sang, Michael Haberl, Raphael Ullmann
Overview Pre-processing Cost Calculation Pyramid Scheme Force Local Consistency Penalty Terms Median Filter Lu Sang, Michael Haberl, Raphael Ullmann
Force Local Consistency • Interpret Cross Correlation as confidence indicator • Use only pixels with high confidence • Replace low confidence by values with high confidence in the window Lu Sang, Michael Haberl, Raphael Ullmann
Force Local Consistency: Confidence Map Figure: Results of NCC Figure: Confidence Map • Black point: trustworthy pixel with correct disparity ( C ♣ p q ➙ T ). • Red point: unreliable pixel with violated disparity ( C ♣ p q ➔ T ). Lu Sang, Michael Haberl, Raphael Ullmann
Force Local Consistency Original C ♣ p ✶ q P W Disparity all violated Violated Unique p ✶ Disparity unique ➔ T max p ✶ P W C ♣ p ✶ q C ♣ p q C ♣ p ✶ q not unique ➙ T Correct Closest p ✶ Disparity 1 Mark all violated pixels. Lu Sang, Michael Haberl, Raphael Ullmann
Force Local Consistency Original C ♣ p ✶ q P W Disparity all violated Violated Unique p ✶ Disparity unique ➔ T max p ✶ P W C ♣ p ✶ q C ♣ p q C ♣ p ✶ q not unique ➙ T Correct Closest p ✶ Disparity 1 Mark all violated pixels. 2 Find the pixel with max NCC coefficient. Lu Sang, Michael Haberl, Raphael Ullmann
Force Local Consistency Original C ♣ p ✶ q P W Disparity all violated Violated Unique p ✶ Disparity unique ➔ T max p ✶ P W C ♣ p ✶ q C ♣ p q C ♣ p ✶ q not unique ➙ T Correct Closest p ✶ Disparity 1 Mark all violated pixels. 2 Find the pixel with max NCC coefficient. 3 Replace the disparity value by using the disparity of new pixel. Lu Sang, Michael Haberl, Raphael Ullmann
Force Local Consistency: Results Figure: Results of NCC Figure: Results of FLC • Error Rate of NCC: 33 . 09% • Error Rate of FLC: 26 . 90% Lu Sang, Michael Haberl, Raphael Ullmann
Overview Pre-processing Cost Calculation Pyramid Scheme Force Local Consistency Penalty Terms Median Filter Lu Sang, Michael Haberl, Raphael Ullmann
Local Penalty Local Penalty 1 Smoothness check 2 Search four directions 3 Punish on NCC coefficient 4 Iterate Lu Sang, Michael Haberl, Raphael Ullmann
Overview Pre-processing Cost Calculation Pyramid Scheme Force Local Consistency Penalty Terms Median Filter Lu Sang, Michael Haberl, Raphael Ullmann
Post Processing: Median Filter ñ Lu Sang, Michael Haberl, Raphael Ullmann
Final Results Figure: Results of FLC Figure: After Post-processing • Error Rate of FLC: 26 . 90% • Error Rate of FLC + Median filter: 21 . 10% Lu Sang, Michael Haberl, Raphael Ullmann
Overview Problem Description Algorithms Results Lu Sang, Michael Haberl, Raphael Ullmann
Recommend
More recommend
