Evaluation of Different Methods for Using Colour Information in - - PowerPoint PPT Presentation

▶
evaluation of different methods for using colour
SMART_READER_LITE
LIVE PREVIEW

Evaluation of Different Methods for Using Colour Information in - - PowerPoint PPT Presentation

Mar 10, 2023 296 likes •881 views

Evaluation of Different Methods for Using Colour Information in Global Stereo Matching Approaches Michael Bleyer 1 , Sylvie Chambon 2 , Uta Poppe 1 and Margrit Gelautz 1 1 Vienna University of Technology, Austria 2 Laboratoire Central des Ponts et

slide-1
SLIDE 1

Evaluation of Different Methods for Using Colour Information in Global Stereo Matching Approaches

Michael Bleyer1, Sylvie Chambon2, Uta Poppe1 and Margrit Gelautz1

1Vienna University of Technology, Austria 2Laboratoire Central des Ponts et Chaussées, Nantes, France

slide-2
SLIDE 2

Evaluation of different methods for using colour information in global stereo matching approaches

Dense Stereo Matching

(Left Image) (Right Image)

slide-3
SLIDE 3

Evaluation of different methods for using colour information in global stereo matching approaches

Dense Stereo Matching

(Left Image) (Right Image) (Disparity Map)

slide-4
SLIDE 4

Evaluation of different methods for using colour information in global stereo matching approaches

Structure

  • Introduction
  • Benchmark design
  • Evaluated energy functions
  • Applied optimization methods
  • Parameter estimation
  • Results
  • Conclusions
slide-5
SLIDE 5

Evaluation of different methods for using colour information in global stereo matching approaches

I ntroduction

  • Evaluation of stereo energy functions.
  • Two key questions:
  • Does colour help to improve the performance of global stereo

methods?

  • What is the best method for using colour? (Colour system, Pixel

difference)

  • Observation:
  • Colour is expected to reduce matching ambiguities.
  • However, a lot of researchers do not want to use colour

information.

slide-6
SLIDE 6

Evaluation of different methods for using colour information in global stereo matching approaches

I ntroduction

  • Evaluation of stereo energy functions.
  • Two key questions:
  • Does colour help to improve the performance of global stereo

methods?

  • What is the best method for using colour? (Colour system, Pixel

difference)

  • Observation:
  • Colour is expected to reduce matching ambiguities.
  • However, a lot of researchers do not want to use colour

information.

(Left I mage) (Right I mage)

slide-7
SLIDE 7

Evaluation of different methods for using colour information in global stereo matching approaches

I ntroduction

  • Evaluation of stereo energy functions.
  • Two key questions:
  • Does colour help to improve the performance of global stereo

methods?

  • What is the best method for using colour? (Colour system, Pixel

difference)

  • Observation:
  • Colour is expected to reduce matching ambiguities.
  • However, a lot of researchers do not want to use colour

information.

(Left I mage) (Right I mage)

slide-8
SLIDE 8

Evaluation of different methods for using colour information in global stereo matching approaches

I ntroduction

  • Evaluation of stereo energy functions.
  • Two key questions:
  • Does colour help to improve the performance of global stereo

methods?

  • What is the best method for using colour? (Colour system, Pixel

difference)

  • Observation:
  • Colour is expected to reduce matching ambiguities.
  • However, a lot of researchers do not want to use colour

information.

slide-9
SLIDE 9

Evaluation of different methods for using colour information in global stereo matching approaches

Energy Functions

slide-10
SLIDE 10

Evaluation of different methods for using colour information in global stereo matching approaches

Energy Functions

  • Data term
  • Photo consistency assumption
  • Computes colour difference

between corresponding pixels

  • Focus of this study
slide-11
SLIDE 11

Evaluation of different methods for using colour information in global stereo matching approaches

Energy Functions

  • Smoothness term
  • Smoothness assumption
  • Penalizes neighbouring pixels

assigned to different disparities

slide-12
SLIDE 12

Evaluation of different methods for using colour information in global stereo matching approaches

Data Term – Colour Spaces

  • 10 different choices evaluated:
  • Primary systems:
  • RGB, XYZ;
  • Luminance-chrominance systems:
  • LUV, LAB, AC1C2, YC1C2;
  • Perceptual systems:
  • HSI ;
  • Statistical independent component systems:
  • I 1I 2I 3, H1H2H3;
  • Use of intensity values only:
  • Grey;
slide-13
SLIDE 13

Evaluation of different methods for using colour information in global stereo matching approaches

Data Term – Difference Measurements

  • 2 choices evaluated:
  • L1 distance (Sum-of-absolute-differences)
  • L2 distance (Euclidean distance)
  • Special treatment for HSI and Grey.
  • In total, 18 different energy functions evaluated in this study.
slide-14
SLIDE 14

Evaluation of different methods for using colour information in global stereo matching approaches

Smoothness Term

slide-15
SLIDE 15

Evaluation of different methods for using colour information in global stereo matching approaches

Smoothness Term

Modified Potts model

slide-16
SLIDE 16

Evaluation of different methods for using colour information in global stereo matching approaches

Smoothness Term

Modified Potts model

slide-17
SLIDE 17

Evaluation of different methods for using colour information in global stereo matching approaches

Smoothness Term

Modified Potts model Weighted by intensity gradient

slide-18
SLIDE 18

Evaluation of different methods for using colour information in global stereo matching approaches

Energy Optimization

  • Computing energy minimum is known to be NP-hard.
  • 2 methods for approximation:
  • Graph-cuts (Alpha-expansion framework):
  • Standard method for energy functions of this type
  • Dynamic programming-based method:
  • Optimizes energy function on a tree structure via DP
  • Two spanning trees generated for each pixel
  • Computation time less than a second

p p

slide-19
SLIDE 19

Evaluation of different methods for using colour information in global stereo matching approaches

Parameter Estimation

  • Two important parameters (P1 and P2) in the energy

function:

  • Balance data and smoothness terms
  • Balance affected by the use of different data terms
  • For fairness, optimize parameter settings for each of the

18 energy functions separately

  • Approximately, 100 combinations of P1 and P2 tested
slide-20
SLIDE 20

Evaluation of different methods for using colour information in global stereo matching approaches

The 2003 Sets

(Left Image) (Ground Truth)

  • Currently used in the Middlebury Stereo Vision Benchmark
slide-21
SLIDE 21

Evaluation of different methods for using colour information in global stereo matching approaches

The 2003 Sets

(Graph-Cut Method - L1 Distance)

slide-22
SLIDE 22

Evaluation of different methods for using colour information in global stereo matching approaches

The 2003 Sets

(Graph-Cut Method - L1 Distance)

  • Test sets
slide-23
SLIDE 23

Evaluation of different methods for using colour information in global stereo matching approaches

The 2003 Sets

(Graph-Cut Method - L1 Distance)

  • Error metric: Percentage of

unoccluded pixels having a disparity error > 1 pixel

slide-24
SLIDE 24

Evaluation of different methods for using colour information in global stereo matching approaches

The 2003 Sets

(Graph-Cut Method - L1 Distance)

  • 3 selected colour spaces
slide-25
SLIDE 25

Evaluation of different methods for using colour information in global stereo matching approaches

The 2003 Sets

(Graph-Cut Method - L1 Distance)

slide-26
SLIDE 26

Evaluation of different methods for using colour information in global stereo matching approaches

The 2003 Sets

(Dynamic Programming Method - L1 Distance)

slide-27
SLIDE 27

Evaluation of different methods for using colour information in global stereo matching approaches

The 2003 Sets

(Dynamic Programming Method - L1 Distance)

  • Good option not to use colour at all. (not

very intuitive)

  • Potential reason why researchers do not use

colour.

slide-28
SLIDE 28

Evaluation of different methods for using colour information in global stereo matching approaches

The 2005 Sets

  • More complex in terms of geometry, occlusions and untextured regions
slide-29
SLIDE 29

Evaluation of different methods for using colour information in global stereo matching approaches

The 2005 Sets

(Graph-Cut Method - L1 Distance)

slide-30
SLIDE 30

Evaluation of different methods for using colour information in global stereo matching approaches

The 2005 Sets

(Dynamic Programming Method - L1 Distance)

slide-31
SLIDE 31

Evaluation of different methods for using colour information in global stereo matching approaches

The 2006 Sets

slide-32
SLIDE 32

Evaluation of different methods for using colour information in global stereo matching approaches

The 2006 Sets

(Graph-Cut Method - L1 Distance)

slide-33
SLIDE 33

Evaluation of different methods for using colour information in global stereo matching approaches

The 2006 Sets

(Dynamic Programming Method - L1 Distance)

slide-34
SLIDE 34

Evaluation of different methods for using colour information in global stereo matching approaches

The 2006 Sets

(Dynamic Programming Method - L1 Distance)

  • Colour clearly improves the results
  • LUV outperforms RGB
slide-35
SLIDE 35

Evaluation of different methods for using colour information in global stereo matching approaches

Quantitative Results – L1 Distance

(Dynamic Programming) (Graph-Cuts)

slide-36
SLIDE 36

Evaluation of different methods for using colour information in global stereo matching approaches

Quantitative Results – L1 Distance

(Dynamic Programming) (Graph-Cuts)

  • Error percentage in unoccluded

regions (averaged over all test sets)

slide-37
SLIDE 37

Evaluation of different methods for using colour information in global stereo matching approaches

Quantitative Results – L1 Distance

(Dynamic Programming) (Graph-Cuts)

  • Relative rank in comparison against

competing colour systems (averaged

  • ver all test sets)
  • Table sorted according to this error

measurement

slide-38
SLIDE 38

Evaluation of different methods for using colour information in global stereo matching approaches

Quantitative Results – L1 Distance

(Dynamic Programming) (Graph-Cuts)

  • Luminance-chrominance systems
slide-39
SLIDE 39

Evaluation of different methods for using colour information in global stereo matching approaches

Quantitative Results – L1 Distance

(Dynamic Programming) (Graph-Cuts)

  • I1I2I3
slide-40
SLIDE 40

Evaluation of different methods for using colour information in global stereo matching approaches

Quantitative Results – L1 Distance

(Dynamic Programming) (Graph-Cuts)

  • RGB
slide-41
SLIDE 41

Evaluation of different methods for using colour information in global stereo matching approaches

Quantitative Results – L1 Distance

(Dynamic Programming) (Graph-Cuts)

  • H1H2H3, XYZ, LAB
slide-42
SLIDE 42

Evaluation of different methods for using colour information in global stereo matching approaches

Quantitative Results – L1 Distance

(Dynamic Programming) (Graph-Cuts)

  • Grey
slide-43
SLIDE 43

Evaluation of different methods for using colour information in global stereo matching approaches

Quantitative Results – L1 Distance

(Dynamic Programming) (Graph-Cuts)

  • HSI
slide-44
SLIDE 44

Evaluation of different methods for using colour information in global stereo matching approaches

Quantitative Results – L1 Distance

(Dynamic Programming) (Graph-Cuts)

  • 25.4% less

errors

  • 29.0% less

errors

slide-45
SLIDE 45

Evaluation of different methods for using colour information in global stereo matching approaches

Quantitative Results – L1 Distance

(Dynamic Programming) (Graph-Cuts)

  • 14.8% less

errors

  • 17.0% less

errors

slide-46
SLIDE 46

Evaluation of different methods for using colour information in global stereo matching approaches

Quantitative Results – L1 Distance

(Dynamic Programming) (Graph-Cuts)

  • 0.7% less errors for DP
slide-47
SLIDE 47

Evaluation of different methods for using colour information in global stereo matching approaches

Quantitative Results – L1 Distance

(Dynamic Programming) (Graph-Cuts)

  • 0.7% less errors for DP
  • There seems to lie more

potential in the energy modelling component than in energy optimization.

slide-48
SLIDE 48

Evaluation of different methods for using colour information in global stereo matching approaches

Quantitative Results – L1 vs L2

(Graph-Cuts – L2) (Graph-Cuts – L1)

slide-49
SLIDE 49

Evaluation of different methods for using colour information in global stereo matching approaches

Quantitative Results – L1 vs L2

(Graph-Cuts – L2) (Graph-Cuts – L1)

  • 4.8% less errors for L1
  • The choice of the colour system

seems to be more important than the difference method.

slide-50
SLIDE 50

Evaluation of different methods for using colour information in global stereo matching approaches

Example: Dolls Test Set – Graph-Cuts

(Grey - Disparity) (Left I mage) (LUV - Disparity)

slide-51
SLIDE 51

Evaluation of different methods for using colour information in global stereo matching approaches

Example: Dolls Test Set – Graph-Cuts

(Grey - Errors) (Left I mage) (LUV - Errors)

slide-52
SLIDE 52

Evaluation of different methods for using colour information in global stereo matching approaches

Example: Dolls Test Set – Graph-Cuts

(Grey - Errors) (Left I mage) (LUV - Errors)

slide-53
SLIDE 53

Evaluation of different methods for using colour information in global stereo matching approaches

Example: Dolls Test Set – Dynamic Programming

(Grey - Disparity) (Left I mage) (LUV - Disparity)

slide-54
SLIDE 54

Evaluation of different methods for using colour information in global stereo matching approaches

Example: Dolls Test Set – Dynamic Programming

(Left I mage) (Grey - Errors) (LUV - Errors)

slide-55
SLIDE 55

Evaluation of different methods for using colour information in global stereo matching approaches

Example: Dolls Test Set – Dynamic Programming

(Left I mage) (Grey - Errors) (LUV - Errors)

slide-56
SLIDE 56

Evaluation of different methods for using colour information in global stereo matching approaches

Conclusions (1)

  • Investigation of the role of colour in global stereo

methods

  • 18 energy functions tested with 2 optimization

algorithms on 30 ground truth images

  • Colour does not always improve results. (Current

Middlebury evaluation set)

  • Performance improvement of 25% achieved by using

the best-performing colour system instead of grey- scale matching

slide-57
SLIDE 57

Evaluation of different methods for using colour information in global stereo matching approaches

Conclusions (2)

  • Luminance-chrominance systems have shown the best
  • results. (relationship to human perception)
  • RGB only gives average results. (most popular colour

system)

  • Choice of colour system is probably more important

than difference method or optimization. (It is worth paying more attention to data term.)

slide-58
SLIDE 58

Evaluation of different methods for using colour information in global stereo matching approaches

The End Thank You