Color Segmentation Based Depth Filtering Dipl.-Math. Michael - - PowerPoint PPT Presentation

▶
color segmentation based depth filtering
SMART_READER_LITE
LIVE PREVIEW

Color Segmentation Based Depth Filtering Dipl.-Math. Michael - - PowerPoint PPT Presentation

Sep 19, 2023 298 likes •614 views

Color Segmentation Based Depth Filtering Dipl.-Math. Michael Schmeing Prof. Dr. Xiaoyi Jiang Institute of Computer Science University of Mnster, Germany Michael Schmeing, CPVR@Uni Mnster WDIA 2012 11.11.2012, Tsukuba, Japan Overview

slide-1
SLIDE 1

WDIA 2012 – 11.11.2012, Tsukuba, Japan Michael Schmeing, CPVR@Uni Münster

Color Segmentation Based Depth Filtering

Dipl.-Math. Michael Schmeing

  • Prof. Dr. Xiaoyi Jiang

Institute of Computer Science University of Münster, Germany

slide-2
SLIDE 2

WDIA 2012 – 11.11.2012, Tsukuba, Japan Michael Schmeing, CPVR@Uni Münster

Overview

  • Depth Filtering
  • Our Approach
  • Results

– Qualitative – Quantitative analysis method

  • Conclusion
slide-3
SLIDE 3

WDIA 2012 – 11.11.2012, Tsukuba, Japan Michael Schmeing, CPVR@Uni Münster

Depth Filtering

  • Depth generation methods:

– Active

  • Laser Scanner
  • ToF
  • Structured Light

– Passive

  • Depth from stereo
  • Depth from motion
  • Depth from X
  • There are no perfect depth maps
slide-4
SLIDE 4

WDIA 2012 – 11.11.2012, Tsukuba, Japan Michael Schmeing, CPVR@Uni Münster

Example depth map

  • Kinect (structured light camera)

Imperfect edges Occlusions

slide-5
SLIDE 5

WDIA 2012 – 11.11.2012, Tsukuba, Japan Michael Schmeing, CPVR@Uni Münster

OUR APPROACH

slide-6
SLIDE 6

WDIA 2012 – 11.11.2012, Tsukuba, Japan Michael Schmeing, CPVR@Uni Münster

Our Approach

  • Focuses on edge restoration
  • Takes edge information of associated color stream
  • Workflow:

1. Occlusion Filling 2. Segmentation of color stream 3. Computation of representative depth map 4. Edge restoration 5. Post processing

slide-7
SLIDE 7

WDIA 2012 – 11.11.2012, Tsukuba, Japan Michael Schmeing, CPVR@Uni Münster

Occlusion Filling

  • Normalized convolution
slide-8
SLIDE 8

WDIA 2012 – 11.11.2012, Tsukuba, Japan Michael Schmeing, CPVR@Uni Münster

Color Segmentation

  • Edge information is taken from an oversegmentation

(superpixel segmentation)

  • We take Watershed segmentation because

– Fast – Compact segments – Segments of approx. the same size (except thin “edge segments”)

  • Color Segmentation:

– Preprocessing of color stream (bilateral filter because of noise) – Apply Watershed – Cluster Splitting

slide-9
SLIDE 9

WDIA 2012 – 11.11.2012, Tsukuba, Japan Michael Schmeing, CPVR@Uni Münster

Watershed Segmentation

  • Idea of Watershed:

– Interpret Grayscale image as relief – Place water sources on it – Flood relief and draw borders where lakes meet – Apply Bilateral Filter prior to reduce noise

slide-10
SLIDE 10

WDIA 2012 – 11.11.2012, Tsukuba, Japan Michael Schmeing, CPVR@Uni Münster

Watershed Color Segmentation

slide-11
SLIDE 11

WDIA 2012 – 11.11.2012, Tsukuba, Japan Michael Schmeing, CPVR@Uni Münster

Projected Color Segmentation in Depth

slide-12
SLIDE 12

WDIA 2012 – 11.11.2012, Tsukuba, Japan Michael Schmeing, CPVR@Uni Münster

Representative Depth Map

  • Compute a representative depth value for each segment
slide-13
SLIDE 13

WDIA 2012 – 11.11.2012, Tsukuba, Japan Michael Schmeing, CPVR@Uni Münster

Edge Restoration

  • Use representative depth map to enhance edges:
  • Outliers are corrected by depth values of the representative

depth map

  • Postprocessing: Bilateral Filter
slide-14
SLIDE 14

WDIA 2012 – 11.11.2012, Tsukuba, Japan Michael Schmeing, CPVR@Uni Münster

RESULTS

slide-15
SLIDE 15

WDIA 2012 – 11.11.2012, Tsukuba, Japan Michael Schmeing, CPVR@Uni Münster

Original Depth Map

slide-16
SLIDE 16

WDIA 2012 – 11.11.2012, Tsukuba, Japan Michael Schmeing, CPVR@Uni Münster

Normalized Convolution [9]

slide-17
SLIDE 17

WDIA 2012 – 11.11.2012, Tsukuba, Japan Michael Schmeing, CPVR@Uni Münster

Berdnikov et al. [6]

slide-18
SLIDE 18

WDIA 2012 – 11.11.2012, Tsukuba, Japan Michael Schmeing, CPVR@Uni Münster

Wasza et al. [7]

slide-19
SLIDE 19

WDIA 2012 – 11.11.2012, Tsukuba, Japan Michael Schmeing, CPVR@Uni Münster

Our method

slide-20
SLIDE 20

WDIA 2012 – 11.11.2012, Tsukuba, Japan Michael Schmeing, CPVR@Uni Münster

Qualitative Results

Input depth map Our method

slide-21
SLIDE 21

WDIA 2012 – 11.11.2012, Tsukuba, Japan Michael Schmeing, CPVR@Uni Münster

Quantitative Results - Method

  • Test sequence: Clear foreground and background
  • Other geometry is possible
slide-22
SLIDE 22

WDIA 2012 – 11.11.2012, Tsukuba, Japan Michael Schmeing, CPVR@Uni Münster

Quantitative Results - Method

  • Color frames define a clustering into foreground and

background

  • Depth frames define a clustering into foreground and

background

  • Perfect depth map -> Same clusterings
  • Measure cluster similarity using Rand Index

– Gives values between 0 and 1

slide-23
SLIDE 23

WDIA 2012 – 11.11.2012, Tsukuba, Japan Michael Schmeing, CPVR@Uni Münster

Quantitative Results

slide-24
SLIDE 24

WDIA 2012 – 11.11.2012, Tsukuba, Japan Michael Schmeing, CPVR@Uni Münster

Quantitative Results

  • Test sequence 2:
slide-25
SLIDE 25

WDIA 2012 – 11.11.2012, Tsukuba, Japan Michael Schmeing, CPVR@Uni Münster

Quantitative Results

Sequence 1 Sequence 2 Our method 0.9865 0.9778 Berdnikov [6] 0.9118 0.9129 Knutsson [9] 0.8952 0.9120 Wasza [7] 0.8899 0.9121 Mean Rand Index Values

slide-26
SLIDE 26

WDIA 2012 – 11.11.2012, Tsukuba, Japan Michael Schmeing, CPVR@Uni Münster

CONCLUSION

slide-27
SLIDE 27

WDIA 2012 – 11.11.2012, Tsukuba, Japan Michael Schmeing, CPVR@Uni Münster

Conclusion

  • We presented a new method for depth map enhancement
  • Special focus on edge restoration
  • We introduced a new method to quantify our results
  • Our method shows promising results and outperforms others

in terms of Rand Index values

  • Future Work:

– Add a temporal component – Make color segmentation temporal stable

slide-28
SLIDE 28

WDIA 2012 – 11.11.2012, Tsukuba, Japan Michael Schmeing, CPVR@Uni Münster

References

  • [1] Fehn, C., de la Barre, R., Pastoor, R.S.: Interactive 3-DTV-Concepts and Key
  • Technologies. Proceedings of the IEEE 94 (2006) 524
  • [6] Berdnikov, Y., Vatolin, D.: Real-time Depth Map Occlusion Filling and Scene

Background Restoration for Projected-Pattern-based Depth Camera. In: 21th International Conference on Computer Graphics and Vision (GraphiCon2011). (2011)

  • [7] Wasza, J., Bauer, S., Hornegger, J.: Real-time Preprocessing for Dense 3-D Range

Imaging on the GPU: Defect Interpolation, Bilateral Temporal Averaging and Guided

  • Filtering. In: IEEE International Conference on Computer Vision Workshops (ICCV

Workshops). (2011)

  • [9] Knutsson, H., Westin, C.F.: Normalized and Differential Convolution. In: 1993 IEEE

Computer Society Conference on Computer Vision and Pattern Recognition, 1993. (1993)

  • [12] Beucher, S., Lantuejoul, C.: Use of Watersheds in Contour Detection. In:

International Workshop on Image Processing: Real-time Edge and Motion Detection/Estimation, Rennes, France. (1979)

  • [13] Rand, W.M.: Objective Criteria for the Evaluation of Clustering Methods. Journal of

the American Statistical Association 66(336) (1971) pp. 846

slide-29
SLIDE 29

WDIA 2012 – 11.11.2012, Tsukuba, Japan Michael Schmeing, CPVR@Uni Münster

Thank you. Questions?