Windowed All- k NN Search over Multidimensional Array Data from - PowerPoint PPT Presentation

Windowed All- k NN Search over Multidimensional Array Data from Medical Imaging GTC 2016 – San Jose, CA, USA Dimitrios Floros 1 Alexandros-Stavros Iliopoulos 2 Nikos Pitsianis 1 , 2 Xiaobai Sun 2 1 Department of Electrical and Computer Engineering, Aristotle University of Thessaloniki 2 Department of Computer Science, Duke University April 06, 2016 Floros , Iliopoulos, Pitsianis, Sun (AUTh|Duke) Windowed All- k NN Search GTC16 April 06, 2016 1 / 35

1 Introduction Windowed all- k NN among pixels in image objects 2 Fast calculation of pairwise distances Single object Multiple objects Distance matrix compression 3 Rapid k -nearest neighbors selection 4 GPU experiments 5 k NN classification 6 Recap – discussion Floros , Iliopoulos, Pitsianis, Sun (AUTh|Duke) Windowed All- k NN Search GTC16 April 06, 2016 2 / 35

1 Introduction Windowed all- k NN among pixels in image objects 2 Fast calculation of pairwise distances Single object Multiple objects Distance matrix compression 3 Rapid k -nearest neighbors selection 4 GPU experiments 5 k NN classification 6 Recap – discussion Floros , Iliopoulos, Pitsianis, Sun (AUTh|Duke) Windowed All- k NN Search GTC16 April 06, 2016 3 / 35

Windowed all- k NN • Data objects: Q image arrays, M × N pixels, F features (similarly for 3D-image arrays) F F F · · · · · · b b b U U U M M M V V V N N N • Every pixel is a query pixel (all- k NN) • Find k -nearest neighbors for each pixel within a U × V window: – affinity in the feature space, R F – proximity (window) in the spatial domain, Ω ⊕ ℳ × 𝒪 Floros , Iliopoulos, Pitsianis, Sun (AUTh|Duke) Windowed All- k NN Search GTC16 April 06, 2016 4 / 35

Applications: hyperspectral/multispectral imaging Left cheek Right Cheek Forehead 0.4 0.4 0.3 0.3 0.2 0.2 0.2 0.1 Amplitude Amplitude Amplitude 0.1 0 0 -0.1 0 -0.2 -0.2 -0.1 -0.3 -0.4 -0.2 -0.4 700 800 900 1000 1100 700 800 900 1000 1100 700 800 900 1000 1100 Wavelength (nm) Wavelength (nm) Wavelength (nm) Left Eye Right Eye Mouth 0.4 0.3 0.3 0.3 0.2 0.2 0.2 0.1 0.1 Amplitude Amplitude Amplitude 0 0.1 0 -0.1 -0.1 0 -0.2 -0.2 -0.1 -0.3 -0.3 -0.2 -0.4 -0.4 sample bands from single HSI face object 1 700 800 900 1000 1100 700 800 900 1000 1100 700 800 900 1000 1100 Wavelength (nm) Wavelength (nm) Wavelength (nm) ( Q = 148 objects) 1 Denes et al . CMU-RI-TR-02-25 , 2002 Floros , Iliopoulos, Pitsianis, Sun (AUTh|Duke) Windowed All- k NN Search GTC16 April 06, 2016 5 / 35

Applications: respiratory deformation vector fields t 4 t 3 . . . t 2 t T t 1 respiratory trajectories as voxel-wise features XCAT 1 4D-CT coronal slice and super-imposed deformation field vectors 1 Segars et al . Medical Physics 35, 2008 Floros , Iliopoulos, Pitsianis, Sun (AUTh|Duke) Windowed All- k NN Search GTC16 April 06, 2016 6 / 35

1 Introduction Windowed all- k NN among pixels in image objects 2 Fast calculation of pairwise distances Single object Multiple objects Distance matrix compression 3 Rapid k -nearest neighbors selection 4 GPU experiments 5 k NN classification 6 Recap – discussion Floros , Iliopoulos, Pitsianis, Sun (AUTh|Duke) Windowed All- k NN Search GTC16 April 06, 2016 7 / 35

Pairwise distance calculation • Pairwise distance functions for two feature vectors, x and y ‖ x ⊗ y ‖ 2 2 = ⟨ x , x ⟩ + ⟨ y , y ⟩ ⊗ 2 ⟨ x , y ⟩ ⊙ 0 ⟨ x , y ⟩ d cos ( x , y ) = 1 ⊗ ⟨ x , x ⟩⟨ y , y ⟩ ∈ ( 0 , 2 ) √︁ • Windowed k NN ⊗ ⊃ k nearest (w.r.t. feature-space) pixels within window • ⟨ x , y ⟩ inner products are the most expensive operations • Symmetry issues – arithmetic complexity (factor of 2) – data locality (factor of ? ) Floros , Iliopoulos, Pitsianis, Sun (AUTh|Duke) Windowed All- k NN Search GTC16 April 06, 2016 8 / 35

1 Introduction Windowed all- k NN among pixels in image objects 2 Fast calculation of pairwise distances Single object Multiple objects Distance matrix compression 3 Rapid k -nearest neighbors selection 4 GPU experiments 5 k NN classification 6 Recap – discussion Floros , Iliopoulos, Pitsianis, Sun (AUTh|Duke) Windowed All- k NN Search GTC16 April 06, 2016 9 / 35

Single-object distance matrix X (image array) D (nest-banded distance matrix) F b U M V N • ( i , j ) block in D corresponds to rows i and j in X • MN ≤ UV non-zero entries in D , instead of ( MN ) 2 Floros , Iliopoulos, Pitsianis, Sun (AUTh|Duke) Windowed All- k NN Search GTC16 April 06, 2016 10 / 35

Single-object distance matrix X (image array) D (nest-banded distance matrix) F b U M V N • ( i , j ) block in D corresponds to rows i and j in X • MN ≤ UV non-zero entries in D , instead of ( MN ) 2 Floros , Iliopoulos, Pitsianis, Sun (AUTh|Duke) Windowed All- k NN Search GTC16 April 06, 2016 10 / 35

Two basic module options: introduction baseline : ( i , j ) ⊂ ⊃ ( u i , v j ) distances sliding window ( 𝒱 i , 𝒲 j ) sliding image ( ℳ u , 𝒪 v ) each query pixel ( i , j ) ∈ ( ℳ , 𝒪 ) each window offset ( u , v ) ∈ ( 𝒱 , 𝒲 ) Floros , Iliopoulos, Pitsianis, Sun (AUTh|Duke) Windowed All- k NN Search GTC16 April 06, 2016 11 / 35

Baseline (each query pixel – each window offset) ( u i , v j ) ∈ ( i + 𝒱 , j + 𝒲 ) : D [( i , j ), ( u i , v j )] = ⟨ X (:, i , j ), X (:, u i , v j ) ⟩ M + 2 b M M × N N + 2 b N M × N f o r ( i , j ) = ( 1 :m, 1 : n ) % loop over query p i x e l s f o r (u , v ) = ( i+U, j+V) % loop over window p i x e l s around query D( i , j , u , v ) = X( : , i , j ) ' ∗ X( : , u , v ) ; % vector - v e c t o r product Floros , Iliopoulos, Pitsianis, Sun (AUTh|Duke) Windowed All- k NN Search GTC16 April 06, 2016 12 / 35

Baseline (each query pixel – each window offset) ( u i , v j ) ∈ ( i + 𝒱 , j + 𝒲 ) : D [( i , j ), ( u i , v j )] = ⟨ X (:, i , j ), X (:, u i , v j ) ⟩ M × N M × N f o r ( i , j ) = ( 1 :m, 1 : n ) % loop over query p i x e l s f o r (u , v ) = ( i+U, j+V) % loop over window p i x e l s around query D( i , j , u , v ) = X( : , i , j ) ' ∗ X( : , u , v ) ; % vector - v e c t o r product Floros , Iliopoulos, Pitsianis, Sun (AUTh|Duke) Windowed All- k NN Search GTC16 April 06, 2016 12 / 35

Sliding window (each query pixel) ( 𝒱 i , 𝒲 j ) ⊕ ( i + 𝒱 , j + 𝒲 ) : D [( i , j ), ( 𝒱 i , 𝒲 j )] = ⟨ X (:, i , j ), X (:, 𝒱 i , 𝒲 j ) ⟩ f o r ( i , j ) = ( 1 :m, 1 : n ) % loop over query p i x e l s D( i , j , i+U, j+V) = X( : , i , j ) ' ∗ X( : , i+U, j+V ) ; % vector - matrix ( p i x e l - window ) product Floros , Iliopoulos, Pitsianis, Sun (AUTh|Duke) Windowed All- k NN Search GTC16 April 06, 2016 13 / 35

Sliding image (each window offset) ∑︂ ( u , v ) ∈ ( 𝒱 , 𝒲 ) : D [( ℳ , 𝒪 ), ( ℳ + u , 𝒪 + v )] = X ( f , ℳ , 𝒪 ) * X ( f , ℳ + u , 𝒪 + v ) f f o r (u , v ) = (U,V) % loop over window o f f s e t s Y = X( : ,M,N) . ∗ X( : ,M +u ,N+v ) ; % Hadamard matrix product ( query and (u , v ) - o f f s e t p i x e l s ) D(M,N,M +u ,N +v ) = sum(Y , 1 ) ; % r e d u c t i o n along f e a t u r e dimension Floros , Iliopoulos, Pitsianis, Sun (AUTh|Duke) Windowed All- k NN Search GTC16 April 06, 2016 14 / 35

Two basic module options: summary baseline : ( i , j ) ⊂ ⊃ ( u i , v j ) distances sliding window ( 𝒱 i , 𝒲 j ) sliding image ( ℳ u , 𝒪 v ) each query pixel ( i , j ) ∈ ( ℳ , 𝒪 ) each window offset ( u , v ) ∈ ( 𝒱 , 𝒲 ) Hadamard array product & reduction along features matrix-vector product – ( F × M × N ) * ( F × M × N ) – ( 1 × F ) ≤ ( F × UV ) – UV products + UV F -vector reductions – MN products – easy to exploit symmetry – difficult to exploit symmetry – good for low F – good for higher F Floros , Iliopoulos, Pitsianis, Sun (AUTh|Duke) Windowed All- k NN Search GTC16 April 06, 2016 15 / 35

1 Introduction Windowed all- k NN among pixels in image objects 2 Fast calculation of pairwise distances Single object Multiple objects Distance matrix compression 3 Rapid k -nearest neighbors selection 4 GPU experiments 5 k NN classification 6 Recap – discussion Floros , Iliopoulos, Pitsianis, Sun (AUTh|Duke) Windowed All- k NN Search GTC16 April 06, 2016 16 / 35

Multiple objects F F F · · · · · · b b b U U U M M M V V V N N N Find the k -nearest neighbors of each pixel inside a window amongst all objects, inclusively or exclusively. Floros , Iliopoulos, Pitsianis, Sun (AUTh|Duke) Windowed All- k NN Search GTC16 April 06, 2016 17 / 35

Two basic multi-object modules: introduction baseline : ( i , j , q 1 ) ⊂ ⊃ ( u , v , q 2 ) distances each multi-object query pixel ( i , j , 𝒭 ) sliding images ( ℳ u , 𝒪 v , 𝒭 ) each multi-object window offset ( u , v , 𝒭 ) each object and window offset ( u , v , q ) ∈ ( 𝒱 , 𝒲 , 𝒭 ) sliding windows ( 𝒱 i , 𝒲 j , 𝒭 ) each query pixel set ( i , j , 𝒭 ) ; ( i , j ) ∈ ( ℳ , 𝒪 ) Floros , Iliopoulos, Pitsianis, Sun (AUTh|Duke) Windowed All- k NN Search GTC16 April 06, 2016 18 / 35