Intro to harmonic analysis on groups Risi Kondor . The Fourier - - PowerPoint PPT Presentation

Intro to harmonic analysis on groups

Risi Kondor

.

The Fourier series (1807)

Any (sufficiently smooth) function f on the unit circle (equivalently, any

2π–periodic f) can be decomposed into a sum of sinusoidal waves f(x) =

∞

∑

k=−∞

cn eikx cn = 1 2π ∫ 2π f(x) e−ikx dx.

• Workhorse of much of applied mathematics.
• Exact conditions under which it works get messy.

E.g., for f ∈ L2([0, 2π)) almost everywhere convergence proved only in 1966 (Carleson).

2/37 .

2/37

.

The Fourier transform

f(x) = ∫

• f(k) e2πikx dk
• f(k) =

∫ f(x) e−2πikx dx

• Duality between time domain and Fourier domain (wave/particle duality in

quantum mechanics)

• Heisenberg uncertainty principle
• Easily generalizes to Rp.

3/37 .

3/37

.

The discrete Fourier transform (DFT)

f(x) =

n−1

∑

k=0

• f(k) e2πikx/n
• f(k) = 1

n

n−1

∑

x=0

f(x) e−2πikx/n

• Unitary transform Cn → Cn (with appropriate normalization).
• Can be seen as discretized version of Fourier series, or as the Fourier

transform on {0, 1, 2, . . . , n−1}.

• Foundation of all of digital signal processing.
• Fast Fourier transforms reduce computation time from O(n2) to

O(n log n) [Cooley & Tukey, 1965].

4/37 .

4/37

Underlying principles

.

• 1. Analytic

Take a measurable space X, a space of functions on X, say L2(X), and a self-adjoint smoothing operator Υ. For example, on X = Rp, Υ may be the time t diffusion operator

(Υf)(x) = 1 √ 4πt ∫ f(y) e−∥ x−y ∥2/(4t)dy.

Question: How does Υ filter L2(X) into a nested sequence of spaces

WΩ = { f ∈ L2(X) | |⟨f, Υf⟩ / ⟨f, f⟩| ≤ Ω } ?

6/37 .

6/37

.

• 2. Algebraic

Now let a group G act on X inducing linear operators Tg : L2(X) → L2(X). E.g., on on X = Rp,

(Tgf)(x) = f(x − g) g ∈ Rp.

Question: What are the smallest spaces fixed by these operators,

Tg(V ) = V ∀g ∈ G ?

7/37 .

7/37

.

On Rp we are lucky because these two notions match up:

• The diffusion operator is et∇2, where ∇2 is the Laplacian

∇2 = ∂2 ∂x2

1

+ ∂2 ∂x2

2

+ . . . + ∂2 ∂x2

p

.

• The e2πik·x Fourier basis functions are eigenfunctions of both ∆ and Tg:
• ∇2e2πik·x = −4π2∥k∥2e2πik·x,
• Tg e2πik·x = e2πik·g e2πik·x.
• Therefore
• WΩ = { f |

f(k) = 0 if ∥k∥2 ≥ Ω } (band-limited functions)

• Vκ = { f |

f(k) = 0 if k ̸= κ } (isotypics)

Question: Does this correspondence hold more generally?

8/37 .

8/37

.

Fourier analysis on graphs

On a finite graph G, the analog of ∆ is the graph Laplacian

[L]i,j =      1 i ∼ j − di i = j

• therwise.

It does lead to a natural measure of smoothness:

f⊤Lf = − ∑

i∼j

(fi − fj)2.

Analyzing functions in terms of the eigenfunctions of L is called spectral graph theory. However (in general) on graphs there is no analog of translation.

9/37 .

9/37

More properties of the FT on R

.

• The Fourier transform is
• Linear
• Invertible
• ∫

f(x)g(x)∗dx = ∫ f(k) g(k)∗dk

(Parseval thm)

Therefore, it is essentially a unitary change of basis.

11/37 .

11/37

.

• Diagonalizes the derivative operator:

g(x) = d dx f(x) = ⇒

• g(k) = 2πik

f(k).

• Diagonalizes the Laplacian:

g(x) = d2 dx2 f(x) = ⇒

• g(k) = −4πk2

f(k).

12/37 .

12/37

.

• Translation theorem:

g(x) = f(x−t) = ⇒

• g(k) = e−2πikt

f(k)

• Scaling theorem:

g(x) = f(λx) = ⇒

• f′(k) = |λ|−1

f(k/λ)

13/37 .

13/37

.

• Convolution theorem:

(f ∗ g)(x) = ∫ f(x−y)g(y)dy = ⇒ f ∗g(k) = f(k) · g(k)

• Cross-correlation theorem:

(f ⋆ g)(x) = ∫ f(y)∗g(x+y)dy = ⇒ f ⋆g(k) = f(k)∗· g(k)

• Autocorrelation:

h(x) = ∫ f(y)∗f(x+y)dy = ⇒ h(k) = ∥ f(k)∥2

14/37 .

14/37

Fourier analysis on compact groups

.

Fourier tranform on R

• f(k) =

∫ f(x) e−2πikx dx

Observation: χk(x) = e−2πikx are exactly the characters of R.

16/37 .

16/37

.

Locally Compact Abelian groups

The Fourier transform of a function on an LCA group G with Haar measure µ is

• f(χ) =

∫

G

f(x) χ(x) dµ χ ∈ G.

• The dual object is itself a group: T ↔ Z, R ↔ R, and for finite groups
• G ∼

= G (Pontryagin duality).

• This covers the Fourier series and the Fourier transform.

17/37 .

17/37

.

Compact non-Abelian groups

The Fourier transform of a function on a compact group G with Haar measure µ is

• f(ρ) =

∫

G

f(x) ρ(x) dµ(x) ρ ∈ R,

where R is a complete set of inequivalent irreducible representations (irreps).

• Now the dual object is no longer a group, but a set of representations

(Tannaka–Krein duality).

• If G is finite, R is finite. If G is compact, R is countable.
• Each Fourier component

f(ρ) is a matrix.

In the following, we will always assume that each ρ is unitary. Every representation is over C.

18/37 .

18/37

Properties

.

Invertibility

Forward transform:

• f(ρ) =

∫

G

f(x) ρ(x) dµ(x) ρ ∈ R.

Inverse transform:

f(x) = 1 µ(G) ∑

ρ∈R

dρ tr [

• f(ρ) ρ(x−1)

] x ∈ G.

• Just as before (with respect to the appropriate scaled matrix norms), this

transform is unitary.

• The eρ

i,j(x) =

√ dρ [ρ(x)]i,j functions form an orthogonormal basis

(Peter-Weyl theorem).

20/37 .

20/37

.

Left-translation

• Theorem. Given f : G → C and t ∈ G, define ft(x) = f(t−1x). Then
• ft(ρ) = ρ(t) ·

f(ρ) ρ ∈ R.

Proof.

∫ ft(x) ρ(x) dµ(x) = ∫ f(t−1x) ρ(x) dµ(x) = ∫ f(x) ρ(tx) dµ(x) = ∫ f(x) ρ(t) ρ(x) dµ(x) = ρ(t) f(ρ)

21/37 .

21/37

.

Left-translation

• Convolution theorem:

(f ∗ g)(x) = ∫ f(xy−1)g(y)dµ(y) = ⇒ f ∗g(ρ) = f(ρ) · g(ρ)

• Cross-correlation theorem:

(f ⋆ g)(x) = ∫ f(xy)g(y)∗dµ(y) = ⇒ f ⋆g(ρ) = f(ρ) · g(ρ)†

22/37 .

22/37

.

Right-translation

• Theorem. Given f : G → C and t ∈ G, define f(t)(x) = f(xt−1). Then
• f(t)(ρ) =

f(ρ) · ρ(t) ρ ∈ R.

Proof.

∫ f(t)(x) ρ(x) dµ(x) = ∫ f(xt−1) ρ(x) dµ(x) = ∫ f(x) ρ(xt) dµ(x) = ∫ f(x) ρ(x) ρ(t) dµ(x) = f(ρ) ρ(t)

23/37 .

23/37

.

Invariant subspaces

• To left-translation:

Wρ,j = span{ eρ

i,j | j = 1, . . . , dρ }

ρ ∈ R j = 1, . . . , dρ.

• To right-translation:

Wρ,i = span{ eρ

i,j | i = 1, . . . , dρ }

ρ ∈ R i = 1, . . . , dρ.

• To left- and right-translation:

Vρ = span{ eρ

i,j | i, j = 1, . . . , dρ }

ρ ∈ R.

24/37 .

24/37

.

The group algebra

The group algebra C[G] is a space with orthonormal basis { ex | x ∈ G } and a notion of multipilication defined by

exey = exy ∀x, y ∈ G.

Letting ⟨f, ex⟩ = f(x) and extending to the rest of C[G] by linearity, for any

f, g ∈ C[G], (fg)(x) = ∫ f(xy−1)g(y)dµ(y) = (f ∗ g)(x).

The group algebra of any compact group is semi-simple, i.e., it decomposes into a direct sum of simple algebras.

25/37 .

25/37

.

The group algebra

• The group algebra decomposes into a sum of simple algebras:

C[G] = ⊕

ρ

Vρ.

(1) Each Vρ is called an isotypic, and corresponds to a single Fourier matrix

• f(ρ). This decomposition is unique.
• Each Vρ further decomposes into a sum of dρ left G–modules

Vρ = W ρ

1 ⊕ W ρ 2 ⊕ . . . ⊕ W ρ dρ

(2) corresponding to each column of

f(ρ). This decomposition is not unique,

i.e., it depends on the choice of R. The Fourier transform is a projection of f onto a basis adapted to (1) and (2).

26/37 .

26/37

Fourier analysis on homogeneous spaces

.

Group actions

• So far we have considered:
• f is a function on a compact group G.
• G acts on G by t: x → tx inducing f → f t, where f t(x) = f(t−1x)

(similarly for the right-action and right-translation).

• In practice it is often more common that:
• f is a function on a set X.
• G acts on X transitively by t: x → tx, inducing f → f t, where

f t(x) = f(t−1x).

Example: The rotation group SO(3) and the sphere S2. The symmetric group acting on a matrix by permuting rows/columns.

28/37 .

28/37

.

Homogeneous spaces

Assume that G acts on X transitively by x → gx.

• Pick some x0 ∈ X.
• The subset of G fixing x0 is a subgroup H of G.
• Each set gH = { gh | h ∈ H } is called a left H–coset.
• The set of left H–cosets we denote G/H.
• { gH | gH ∈ G/H } forms a partition of G.
• yx0 = y′x0 if and only if y, y′ belong to the same coset.
• Therefore, we have a bijection

X ↔ G/H. X is called a homogeneous space of G.

Example: S2 ∼ SO(3)/SO(2).

29/37 .

29/37

.

FT on homogeneous spaces

Now L(X) is only a G-module, not a C[G] algebra. However, we can still ask, how it decomposes into a sum of invariant modules.

• Definition. The Fourier transform of f : X → C is the FT of the induced

function f ↑G(g) = f(gx0), i.e.,

• f(ρ) =

∫

G

f(gx0) ρ(g) dµ(g) ρ ∈ R

30/37 .

30/37

.

Adapted bases

We say that the repesentation ρ of G is adapted to H ≤ G, if ρ↓H is of the block diagonal form

ρ↓H(h) = ⊕

ρ′∈Rρ

ρ′(h) h ∈ H

(3) for some multiset Rρ of irreps of H. We use mtr(ρ) to denote the multiplicity of the trivial representation in the decomposition (3).

31/37 .

31/37

.

FT on homogeneous spaces

• Theorem. If f : X → C and

f is expressed in a basis adapted to H ≤ G,

then

f(ρ) has at most mtr(ρ) non-zero columns.

Proof.

• f(ρ) =

∫

g∈G/H

∫

h∈H

f(gx0) ρ(gh) dµ(g)dµ(h) = [∫

g∈G/H

f(gx0) ρ(g) dµ(g) ][∫

h∈H

ρ(h)dµ(h)

• ]

32/37 .

32/37

.

Proof (continued)

∫

h∈H

ρ(h)dµ(h) = ⊕

ρ′∈R

∫

h∈H

ρ′(h)dµ(h)

If ρ′ is the trivial irrep of H, then

∫

h∈H ρ′(h)dµ(h) = µ(H). However, by

• rthogonality of the Fourier basis functions, for any other irrep,

∫

h∈H ρ′(h)dµ(h) = 0.

33/37 .

33/37

.

Example

The rotation group SO(3) is parametrized by the Euler angles (θ, ϕ, ψ), and the irreps are given by the D(0), D(1), D(2), . . . Wigner matrices, where

[D(ℓ)]m,m′ = e−im′ψ Y m

ℓ (θ, ϕ),

m, m′ = −ℓ, . . . , ℓ

where

Y m

ℓ (θ, ϕ) =

√ 2ℓ + 1 4π (ℓ − m)! (ℓ + m)! P m

l (cos θ) eimϕ,

are the spherical harmonics. Clearly, the Wigner matrices are adapted to the subgroup SO(2) that rotates ψ. In particular,

D(ℓ) ↓SO(2) (ψ) =

ℓ

⊕

m′=−ℓ

χm′(ψ) χm′(ψ) = e−im′ψ,

so the multiplicity of the trivial irrep χ0 is 1.

34/37 .

34/37

.

Example

The Fourier transform of f : S2 → C is

• f(ℓ) =

∫

SO(3)

f(Rx0) D(ℓ)(R) dµ(R) ℓ = 0, 1, 2, . . . ,

but by our theorem only the middle column of each of these matrices is non-zero, which yields exactly the celebrated spherical harmonic expansion

f(θ, ϕ) =

∞

∑

ℓ=0 m

∑

m=−ℓ

fm

ℓ Y m ℓ (θ, ϕ).

35/37 .

35/37

.

Example

36/37 .

36/37

.

APPLICATIONS

• Invariants to group actions:
• Computer vision
• Graph invariants
• Band–limited approximations on Sn:
• Multi-object tracking
• Wavelets on Sn
• Learning on Sn:
• Ranking problems
• Optimization on Sn:
• Fast QAP solvers.

37/37 .

37/37