3D Rotation-Invariant Description from Tensor Operation on Spherical - PowerPoint PPT Presentation

Motivations and Proposal Spherical HOG Regional Description Experiment and Application Conclusion 3D Rotation-Invariant Description from Tensor Operation on Spherical HOG Field Kun Liu 1 , 3 Henrik Skibbe 1 , 3 Thorsten Schmidt 1 , 3 Thomas Blein 2 , 3 Klaus Palme 2 , 3 Olaf Ronneberger 1 , 3 Department of Computer Science, Univ. of Freiburg 1 Institute of Biology II - Botany, Univ. of Freiburg 2 BIOSS Centre for Biological Signalling Studies, Univ. of Freiburg 3 Freiburg, Germany This study was supported by the Excellence Initiative of the German Federal and State Governments ( EXC 294) and BMBF Project ”New Methods in Systems Biology” (SYSTEC) BMVC 2011

Motivations and Proposal Spherical HOG Regional Description Experiment and Application Conclusion Motivations and Proposal 1 2 HOG as Continuous Angular Signal 3 Regional Description Experiment and Application 4 Conclusion 5

Motivations and Proposal Spherical HOG Regional Description Experiment and Application Conclusion Motivations Rotational-invariance is important for many applications with 3D volumetric data. 3D microscopic images of pollen 3D shape models

Motivations and Proposal Spherical HOG Regional Description Experiment and Application Conclusion Rotation-invariance from pose normalization Pose normalization in 2D SIFT 3D pose is more complicated 2D → 3D: 2 more angles to be determined Pose normalization becomes more complicated and less reliable

Motivations and Proposal Spherical HOG Regional Description Experiment and Application Conclusion Rotation-invariance from Fourier analysis Spherical Harmonics → Analytical rotational-invariance [Q. Wang, et al , Rotational Invariance based on Fourier Analysis in Polar and Spherical Coordinates. IEEE Transactions on PAMI, 2009] Our contribution: HOG + Spherical Harmonics → robust 3D rotation-invariant descriptions

Motivations and Proposal Spherical HOG Regional Description Experiment and Application Conclusion Proposal: Spherical HOG Feature + Regional Description image gradient histogram regional description → → → → using Spherical Harmonics for features in the spherical coordinates

Motivations and Proposal Spherical HOG Regional Description Experiment and Application Conclusion 2D HOG as continuous circular signals Discrete Histogram Continuous Histogram Rotation can be easily addressed in Fourier space.

Motivations and Proposal Spherical HOG Regional Description Experiment and Application Conclusion The continuous histogram in Fourier space e i nφ Fourier basis Fourier coefficients � π : c n =  f ( φ ) e − i nφ dφ π 2 π π  gradient orientation φ

Motivations and Proposal Spherical HOG Regional Description Experiment and Application Conclusion The continuous histogram in Fourier space e i nφ Fourier basis Fourier coefficients � π : c n =  f ( φ ) e − i nφ dφ π 2 π π  gradient orientation φ = c  + c  e iφ + c  e iφ + ... Fourier ( ) = [ c  , c  , c  · · · ]

Motivations and Proposal Spherical HOG Regional Description Experiment and Application Conclusion The continuous histogram in Fourier space e i nφ Fourier basis Fourier coefficients � π : c n =  f ( φ ) e − i nφ dφ π 2 π π  gradient orientation φ = c  + c  e iφ + c  e iφ + ... Fourier ( ) = [ c  , c  , c  · · · ] Under a rotation (of angle θ ), → Fourier ( ) = [ c ′  , c ′  , c ′  · · · ] n = e − i nθ c n c ′

Motivations and Proposal Spherical HOG Regional Description Experiment and Application Conclusion Circles → Spheres, Fourier basis → Spherical Harmonics Analogously to the Fourier basis e i nφ , the wave functions on a sphere are called spherical harmonics 0 1 ℓ 2 3 4 . . . -4 -3 -2 -1 0 1 2 3 4 m

Motivations and Proposal Spherical HOG Regional Description Experiment and Application Conclusion Expansion on spheres → Spherical Harmonic Coefficients = ˆ + ˆ + ˆ + ˆ f  f  f  f   · −  ·  ·  · + · · · ˆ f ℓ m are complex-valued coefficients: f 0 ˆ f 1 ˆ − 1 , ˆ f 1 0 , ˆ f 1 f 2 ˆ − 2 , ˆ f 2 − 1 , ˆ f 2 0 , ˆ f 2 1 , ˆ f 2 0 , 1 , 2 , · · · ���� � �� � � �� � band 0: ˆ band 1: ˆ band 2: ˆ f  f  f  The coefficients in the same band transform together under rotations.

Motivations and Proposal Spherical HOG Regional Description Experiment and Application Conclusion Rotation in Spherical Harmonic Space ˆ f ℓ Multiplying the coefficients with Wigner-D Matrices D ℓ ( g ) ( g represents the rotation) D ℓ ( g )ˆ f ℓ

Motivations and Proposal Spherical HOG Regional Description Experiment and Application Conclusion 3D HOG represented in Spherical Harmonic space Take an individual gradient as a Dirac function on sphere Project it onto Spherical Harmonics

Motivations and Proposal Spherical HOG Regional Description Experiment and Application Conclusion 3D HOG represented in Spherical Harmonic space → Take an individual gradient as a Dirac function on sphere Project it onto Spherical Harmonics Spatial aggregation → spatial smoothing on spherical harmonic coefficients

Motivations and Proposal Spherical HOG Regional Description Experiment and Application Conclusion Regional description of HOG arrangement grid sampling shell sampling + expansion M. Kazhdan, et al , Rotation invariant spherical harmonic representation of 3D shape descriptors, 2003.

Motivations and Proposal Spherical HOG Regional Description Experiment and Application Conclusion 2D example: radial sampling + Fourier expansion pixel-wise HOG in Fourier space

Motivations and Proposal Spherical HOG Regional Description Experiment and Application Conclusion 2D example: radial sampling + Fourier expansion pixel-wise HOG in Fourier space ↓ radial sampling → (in each channel)

Motivations and Proposal Spherical HOG Regional Description Experiment and Application Conclusion 2D example: radial sampling + Fourier expansion pixel-wise HOG in Fourier space ↓ radial sampling → (in each channel) ↓ Fourier expansion Fourier ( ) = [ cc  , cc  , cc  · · · ] (at each sampled radius) { cc n } has simple rotation behaviour

Motivations and Proposal Spherical HOG Regional Description Experiment and Application Conclusion How to create rotation-invariance f ′ = rotate f by angle θ image f a ′ = e − i mθ a a feature b ′ = e − i nθ b another feature b

Motivations and Proposal Spherical HOG Regional Description Experiment and Application Conclusion How to create rotation-invariance f ′ = rotate f by angle θ image f a ′ = e − i mθ a a feature b ′ = e − i nθ b another feature b || a || 2 = aa || a ′ || 2 = ae i mθ e − i mθ a = || a || 2 energy a ′ b ′ = e i ( m − n ) θ ab coupled value ab Energy is rotation-invariant. ab is rotation-invariant if m = n .

Motivations and Proposal Spherical HOG Regional Description Experiment and Application Conclusion Solution for the 3D HOG Field Spherical → → Tensorial Expansion Dense HOG feature in SH space shell sampling The dense HOG features in Spherical Harmonics space need the Spherical Tensorial expansion. [M. Reisert and H. Burkhardt. Spherical tensor calculus for local adaptive filtering, 2009 ] Rotation-invariance: coupling two expansion coefficients which transform with the same Wigner-D matrices.

Motivations and Proposal Spherical HOG Regional Description Experiment and Application Conclusion Summary of approach → Representing HOG in Spherical Harmonics space Describing local region by shell-sampling + expansion Coupling the output of the same rotation behaviour

Motivations and Proposal Spherical HOG Regional Description Experiment and Application Conclusion Evaluation on Princeton Shape Benchmark Method Nearest Neighbour(%) First Tier(%) Second Tier(%) E-measure(%) DCG(%) HOG-ST 67.4 37.4 47.6 28.0 63.8 SH 56.0 28.4 37.6 22.3 56.0 StrT-ST 61.7 30.7 39.6 23.2 58.2 BoF SHcorr 62.4 / / / / HOG align 58 27 35 21 55 Using the evaluation tools from the benchmark

Motivations and Proposal Spherical HOG Regional Description Experiment and Application Conclusion Evaluation on Princeton Shape Benchmark Method Nearest Neighbour(%) First Tier(%) Second Tier(%) E-measure(%) DCG(%) HOG-ST 67.4 37.4 47.6 28.0 63.8 SH 56.0 28.4 37.6 22.3 56.0 StrT-ST 61.7 30.7 39.6 23.2 58.2 BoF SHcorr 62.4 / / / / HOG align 58 27 35 21 55 HOG-ST: Spherical HOG + shell-wise tensorial expansion. SH: Spherical Harmonics Descriptor. [P. Shilane, et al , 2004.] StrT-ST: Structure Tensor + shell-wise tensorial expansion. [H. Skibbe, et al , 2009. ] BoF SHcorr : Bag-of-features approach with Spherical Correlation for feature comparison. [J. Fehr, et al , 2009. ] HOG align : HOG features on pose-normalized 3D shapes. [M. Scherer, et al , 2010. ]