Invariants Risi Kondor . A classical problem Let g R , and f : R - - PowerPoint PPT Presentation

Invariants

Risi Kondor

.

A classical problem

Let g ∈ R, and f : R → C. The real numbers, as a group acts on the space of functions on R by translation:

g: f → f′

where

f′(x) = f(x − g).

Question: How do we construct functionals Υ[f] that are invariant to this action, i.e., for which Υ[f] = Υ[f′] for any f and any g? Many applications in signal processing, image analysis, etc..

2/25 .

2/25

.

Solution 1: Autocorrelation

The autocorrelation of f is

a(x) = ∫ f(x + y)f(y)dy.

Tells us how much f changes when we translate it by an amount y. Clearly invariant to translation:

3/25 .

3/25

.

Solution 2: Power spectrum

The power spectrum of f is

• a(ω) =

∫

• f(ω)∗

f(ω) dx = ∫ | f(ω) |2 dx.

Literally measures the amount of energy in each Fourier mode. Clearly invariant to translation:

• af′(ω) =

∫ (e2πiωg f(ω))∗(e2πiωg f(ω)) dx = ∫

• f(ω)∗

f(ω) dx = af(ω).

• In fact, the power spectrum is just the Fourier transform of the

autocorrelation.

• Their common limitation: lose all the information in the phase.

4/25 .

4/25

.

3: Triple correlation & bispectrum

The triple correlation of f is

b(x1, x2) = ∫ f(y − x1) f(y − x2) f(y) dy

The bispectrum of f is

• b(k1, k2) =

f(k1)∗ f(k2)∗ f(k1 + k2).

5/25 .

5/25

.

Reconstructing f from

b

• b(k1, k2) =

f(k1)∗ f(k2)∗ f(k1 + k2).

Use the following algorithm to recover f from

b:

1.

f(0) = b(0, 0)1/3

2.

f(1) = eiφ √

• b(0, 1)/

f(0) → indeterminacy in φ inevitable

3.

• f(k + 1) =
• b(1, k)
• f(1)∗

f(k)∗ k = 2, 3, . . .

• If

f(k)̸= 0 for any k, then b uniquely determines f up to translation.

6/25 .

6/25

Invariants on groups

.

General setup

• G is a group acting on a set X transitively. This means that each g ∈ G is a

map g: X → X, sending

g: x → gx.

• L(X) is a space of functions on X. The action of G on X extends to

functions in L(X) by

g: f → fg

where

(fg)(x) = f(g−1x).

(Assume that gf ∈ L(X) for all f ∈ L(X) and g ∈ G.)

• A functional Υ[f] is an invariant to this action if

Υ[f] = Υ[fg] ∀f ∈ L(X), ∀g ∈ G.

8/25 .

8/25

.

Examples

• The rotation group SO(3) acts on the sphere S2 by x → Rx. Consider

L2(S2)...

• The symmetric group acts on the adjacency matrix of a graph by

(i, j) → (σ(i), σ(j)).

9/25 .

9/25

.

The Fourier Transform

Restrict ourselves for now to finite X and finite G. Recall that f induces a function f ↑G (g) = f(gx0), and the Fourier transform of f is

• f(ρ) =

∑

g∈G

f(g) ρ(g) ρ ∈ RG.

Moreover, if ft(g) = f(t−1g), then

• ft(ρ) = ρ(t) ·

f(ρ).

10/25 .

10/25

.

Noncommutative power spectrum

The power spectrum of a function f : X → C is

• a(ρ) =

f(ρ)† · f(ρ).

Clearly invariant because

• fτ(ρ)† ·

fτ(ρ) = (ρρ(t) · f(ρ))†(ρρ(t) · f(ρ)) = f(ρ)† · f(ρ).

The power spectrum is the FT of the (flipped) autocorrelation function

a(h) = ∑

g∈G

f(gh−1)f(g).

11/25 .

11/25

.

The noncommutative bispectrum

Recall the Clebsch-Gordan decomposition

ρ1(σ) ⊗ ρ2(σ) = Cρ1,ρ2 [ ⊕

ρ∈Rρ1,ρ2 c(ρ1,ρ2,ρ)

⊕

i=1

ρ(σ) ] C†

ρ1,ρ2.

The bispectrum:

• bf(ρ1, ρ2) = C†

ρ1,ρ2

[

• f(ρ1) ⊗

f(ρ2) ]† Cρ1,ρ2 ⊕

ρ∈Λρ1,ρ2 c(ρ1,ρ2,ρ)

⊕

i=1

• f(ρ)

The bispectrum is the FT of the triple correlation

b(h1, h2) = ∑

g∈G

f(gh−1

1 )f(gh−1 2 )f(g).

12/25 .

12/25

.

Completeness result

Theorem [Kakarala, 1992]. Let f and f′ be a pair of complex valued integrable functions on a compact group G. Assume that

f(ρ) is invertible for

each ρ ∈ R. Then f′ = fz for some z ∈ G if and only if

bf(ρ1, ρ2) = bf′(ρ1, ρ2) for all ρ1, ρ2 ∈ R.

• Generalizes to any Tatsuuma duality group (ISO(n), ihomog. Lorentz

Group, etc.)

13/25 .

13/25

.

The skew spectrum

The skew spectrum of f : Sn → C is the collection of matrices

• qh(ρ) =

r†

h(ρ) ·

f(ρ), ρ ∈ RG,

• ∈G,

with rh(g) = f(gh)f(g). Unitarily equivalent to the bispectrum, but sometimes easier to compute [K., 2007]

14/25 .

14/25

Optimization problems

Risi Kondor

.

The Quadratic Assignment Problem

Given A, A′ ∈ Rn×n, the Quadaratic Assignment Problem is to solve maximize

σ∈Sn

f(σ) f(σ) =

n

∑

i,j=1

Aσ(i),σ(j) A′

i,j.

Equivalently, maximize

σ∈Sn

tr(PσAP ⊤

σ A′)

[Pσ]i,j = { 1

if σ(j) = i else.

• Can interpret A and A′ as the adjacency matrices of two (weighted) graphs.
• The QAP aims to find the “best way to match” A to A′.
• Most graph-to-graph matching problems reduce to special cases of the

QAP: traveling salesman, (sub)-graph isomorphism, graph partitioning, etc.

• → NP-hard. Also hard in practice. No PTAS until recently.
• Sometimes written in minimization form.

16/25 .

16/25

.

Classical approach: branch&bound

• Subdivide in the form of a tree based on where vertex 1 is mapped, where

vertex 2 is mapped, etc.. Each node corresponds to a coset τi1,...,ikSn−k.

• Define bounds

Bi1,...,ik ≥ max

σ∈τi1,...,ikSn−k f(σ).

• Do a directed depth-first search: follow most promising branch, go down to

leaf, then backtrack, but never follow branches which are guaranteed to be worse that optimum so far.

• Hard to come up with theoretical performance guarantees. Empirical

performance depends critically on the tightness of the bounds.

17/25 .

17/25

The Fourier approach to the QAP

.

Fundamental Observation

Consider the Fourier transform of the objective function:

• f(λ) =

∑

σ∈Sn

f(σ) ρλ(σ).

Theorem [Rockmore et al., 2002]. If f is the QAP objective function, then

• f(λ) = 0 unless λ ∈

{

, , ,

}

. Proposition [K., 2010].

f ( )

and

f ( )

are rank one matrices. Question: Why is this?

19/25 .

19/25

.

Graph Functions

Sn acts transitively on the off-diagonal entries of A, so it induces a function gA(σ) = Aσ(n),σ(n−1)

with the property π ◦(gA) = g(π◦A) (note gA(στ) = gA(σ) ∀τ ∈ Sn−2).

20/25 .

20/25

.

Graph correlation

• Proposition. The QAP objective function can be written in the form

f(σ) = 1 (n − 2)! ∑

τ∈Sn

gA(στ) gA′(τ).

Proof.

n

∑

i,j=1

Aσ(i),σ(j) A′i,j = 1 (n−2)! ∑

π∈Sn

Aσπ(n),σπ(n−1) A′

π(n),π(n−1) =

1 (n−2)! ∑

π∈Sn

gA(σπ)gA′(π).

21/25 .

21/25

.

Graph correlation

• Proposition. The QAP objective function can be written in the form

f(σ) = 1 (n − 2)! ∑

τ∈Sn

gA(στ) gA′(τ).

• Corollary. The Fourier transform of the QAP objective is of the form
• f(λ) =

1 (n − 2)! gA(λ) · ( gA′(λ))⊤, λ ⊢ n.

(1) In particular,

f(λ) is identically zero unless λ = (n), (n−1, 1), (n−2, 2) or (n−2, 1, 1), and f((n−2, 2)) and f((n−2, 1, 1)) are dyadic (rank one)

matrices. Question: Is this useful for anything?

22/25 .

22/25

.

Fourier space branch & bound

• First compute

gA and gA′ → O(n2) space, O(n3) time.

• Compute
• f. → O(n2) space, O(n3) time.
• Do branch and bound on the same coset tree as before.
• Compute bounds directly in Fourier space.

23/25 .

23/25

.

Restriction to cosets in Fourier space

• fi(λ) =

∑

λ′∈λn

dλ′ nλ [ ρλ(in)⊤ · f(λ) ]

λ′

24/25 .

24/25

.

Fourier space bounds

• Proposition. For any f : Sn → R

max

σ∈Sn f(σ) ≤ 1

n! ∑

λ⊢n

dλ∥ f(λ)∥tr.

Proof.

f(σ) = 1 n! ∑

λ⊢n

dλ tr [ f(λ) · ρλ(σ)−1]

For any orthogonal matrix O, tr(MO) ≤ ∥M∥tr.

• This bound is computed in time O(n3) (requires an SVD).

25/25 .

25/25