Dihedral Groups and Spatio-Chromatic Filter Systems Reiner Lenz, - PowerPoint PPT Presentation

Dihedral Groups and Spatio-Chromatic Filter Systems Reiner Lenz, Martin Solli Linköping University reiner.lenz@liu.se; martin.solli@liu.se FWS-2010-Ilmenau Reiner Lenz

Take-Home-Message Can you see the irreducible representations of dihedral groups? Can you hear the shape of a drum? Feynman FWS-2010-Ilmenau Reiner Lenz

The Algebra of Digital Color Images 1. Algebra, Filter design from first principles 2. Visual properties of filters 3. Discrimination of visual classes, Image retrieval, Emotions FWS-2010-Ilmenau Reiner Lenz

Digital Color Images • Consists of pixels • Pixels are located on a grid • Pixel value is a 3-D vector FWS-2010-Ilmenau Reiner Lenz

What can we do with them? Transform the grid k ● 90 degree rotation reflection on diagonal Form the group D(4) FWS-2010-Ilmenau Reiner Lenz

What can we do with them? Transform the channels B k ● 120 degree rotation reflection on symmetry axis Form the group D(3) = S(3) permutation group R G FWS-2010-Ilmenau Reiner Lenz

Dimensionality of Pattern Space Take N points invariant under D(4) and the 3 color channels invariant under D(3) Coordinates are (x,c) = (position, channel) The possible value distributions form a 3N – dimensional space P P of patterns p(x,c) This space is invariant under D(4) and D(3) Rot, Green = 12 Blue = 24 RGB = 48 FWS-2010-Ilmenau Reiner Lenz

Decomposition of Pattern Space Representation theory of finite groups: The space can be split into invariant subspaces of minimum dimension P = P 0 + P 1 + … + P K P = P 0 + P 1 + … + P K p = (a 01 b 01 + … + a 0k(0) b 0k(0) ) + … + (a K1 b K1 + … + a Kk(K) b Kk(K) ) a kl = p’b kl a kl are coefficients, b kl are “basis patterns” FWS-2010-Ilmenau Reiner Lenz

Some Properties The spaces P P k have dimensions 1, 2 or 4 The “basis patterns” or “filter functions” consist of 0,1 and -1 The “feature vectors ” ( a n1 …a nk(n) ) have simple transformation properties under D(4)xD(3) transformations of the pattern p (steerable filters) The norm of the “feature vectors ” r n = ||( a n1 …a nk(n) )|| is invariant under D(4)xD(3) transformations FWS-2010-Ilmenau Reiner Lenz

Simplest example: 2x2 Color Channels 2x2 = 4 pixels 3x4 = 12 dimensionial vectors p Step 1: Combine channels in R+G+B and (2-D vector (R-G,R+G-2B) FWS-2010-Ilmenau Reiner Lenz

Spatial - Intensity four pixel vector with intensity values is multiplied by The first is an averaging filter (1D subspace) The second is a line filter (1D subspace) The third+fourth are x- and y-gradients (2D subspace) FWS-2010-Ilmenau Reiner Lenz

Spatial – Color Channels 4 pixels and 2 channels = 8 – dimensional vector Theory gives the decomposition in invariant subspaces of dimensions up to four FWS-2010-Ilmenau Reiner Lenz

Implementation Compute + + + - + + + - This is the “FFT” form of the computation FWS-2010-Ilmenau Reiner Lenz

4x4 Window 4x4x3 = 48 dimensions FWS-2010-Ilmenau Reiner Lenz

Signatures From 4x4x3 = 48D RGB distribution Compute 48 new features (multiplication with a square matrix) Collect the 48 features in 24 sub-vectors For every sub-vector compute the norms r 1 … r 24 These 24 norms are collected in the signature vector FWS-2010-Ilmenau Reiner Lenz

Application to Image Retrieval Databases: Real-world databases from Picsearch search engine a) Objects like beach Monet and Warhol 320 000 images b) Emotions like colorful and elegant 1.2 million images Download at http://diameter.itn.liu.se/emodb FWS-2010-Ilmenau Reiner Lenz

Descriptors 1. An image consists of B blocks of size 4x4 2. Every block gives a 24D signature vector 3. Describe the image by the histograms over these signature vectors FWS-2010-Ilmenau Reiner Lenz

Characteristic Images Intensity line 2x2 FWS-2010-Ilmenau Reiner Lenz

Intensity Edge FWS-2010-Ilmenau Reiner Lenz

Color Edge FWS-2010-Ilmenau Reiner Lenz

Homogeneous Color FWS-2010-Ilmenau Reiner Lenz

Visual properties of Colorful/Elegant FWS-2010-Ilmenau Reiner Lenz

Training of Support-Vector-Machines for two-class discrimination Use different combinations of signature vectors train a SVM with half of the data and two classes Use the output of the SVM as indicator of the class membership FWS-2010-Ilmenau Reiner Lenz

Warhol-Monet FWS-2010-Ilmenau Reiner Lenz

Intensity-Line-2x2 FWS-2010-Ilmenau Reiner Lenz

FWS-2010-Ilmenau Reiner Lenz Edge-Intensity-2x2

FWS-2010-Ilmenau Reiner Lenz Homogeneous color 2x2

FWS-2010-Ilmenau Reiner Lenz Color Edge 2x2

FWS-2010-Ilmenau Reiner Lenz All Color

FWS-2010-Ilmenau Reiner Lenz Colorful-Elegant