SLIDE 1
Approximating Orthogonal Matrices with Effective Givens - - PowerPoint PPT Presentation
Approximating Orthogonal Matrices with Effective Givens - - PowerPoint PPT Presentation
Approximating Orthogonal Matrices with Effective Givens Factorization Thomas Frerix Technical University of Munich joint work with Joan Bruna (NYU) Poster #164 Givens Factorization of Orthogonal Matrices 1 0 0 0
SLIDE 2
SLIDE 3
Givens Factorization of Orthogonal Matrices
G T(i, j, α) =
1 ··· ··· ··· 0
. . . ... . . . . . . . . .
0 ··· cos(α) ··· − sin(α) ··· 0
. . . . . . ... . . . . . .
0 ··· sin(α) ··· cos(α) ··· 0
. . . . . . . . . ... . . .
0 ··· ··· ··· 1
Exact Givens Factorization U = G1 . . . GN N = d(d − 1) 2
SLIDE 4
Approximate Givens Factorization
Approximate Givens Factorization U ≈ G1 . . . GN N ≪ d(d − 1) 2 computationally hard problem
SLIDE 5
Approximate Givens Factorization
Approximate Givens Factorization U ≈ G1 . . . GN N ≪ d(d − 1) 2 computationally hard problem Our Questions in this Context
- 1. Which orthogonal matrices can be effectively approximated?
(not all of them)
SLIDE 6
Approximate Givens Factorization
Approximate Givens Factorization U ≈ G1 . . . GN N ≪ d(d − 1) 2 computationally hard problem Our Questions in this Context
- 1. Which orthogonal matrices can be effectively approximated?
(not all of them)
- 2. Which principles are behind effective approximation algorithms?
(sparsity-inducing algorithms)
SLIDE 7
Motivation: Unitary Basis Transform / FFT
Advantageous Setting Once computed, applied many times
SLIDE 8
Motivation: Unitary Basis Transform / FFT
Advantageous Setting Once computed, applied many times Unitary Basis Transform FFT: O
- d2
→ O
- d log(d)
SLIDE 9
Motivation: Unitary Basis Transform / FFT
Advantageous Setting Once computed, applied many times Unitary Basis Transform FFT: O
- d2
→ O
- d log(d)
- Application: Graph Fourier Transform
SLIDE 10
Which Matrices can be Effectively Approximated?
Theorem Let ǫ > 0. If N =o
- d2/ log(d)
- , then as d → ∞,
µ
- U ∈ U(d)
- inf
G1...GN
U −
- n
Gn2 ≤ ǫ → 0 , where µ is the Haar measure over U(d).
SLIDE 11
Which Matrices can be Effectively Approximated?
Theorem Let ǫ > 0. If N =o
- d2/ log(d)
- , then as d → ∞,
µ
- U ∈ U(d)
- inf
G1...GN
U −
- n
Gn2 ≤ ǫ → 0 , where µ is the Haar measure over U(d).
- proof is based on an ǫ-covering argument
- suggests computational-to-statistical gap together with experimental results (details
at poster)
SLIDE 12
K-planted Distribution over SO(d)
Sample U = G1 . . . GK
- choose subspace (ik, jk) uniformly with replacement
- choose rotation angle αk ∈ [0, 2π) uniformly
SLIDE 13
K-planted Distribution over SO(d)
Sample U = G1 . . . GK
- choose subspace (ik, jk) uniformly with replacement
- choose rotation angle αk ∈ [0, 2π) uniformly
0.2 0.4 0.6 0.8 1 0.2 0.4 0.6 0.8 1
K/d log2(d) ||U||0/d2
256 512 1024
K-planted matrices quickly become dense
SLIDE 14
Minimizing Sparsity-Inducing Norms over O(d)
G T
N . . . G T N U ≈ I
ˆ U = G1 . . . GN
SLIDE 15
Minimizing Sparsity-Inducing Norms over O(d)
G T
N . . . G T N U ≈ I
ˆ U = G1 . . . GN Approximation criterion
- U − ˆ
U
- F,sym := min
P∈Pd
- U − ˆ
UP
- F
SLIDE 16
Minimizing Sparsity-Inducing Norms over O(d)
G T
N . . . G T N U ≈ I
ˆ U = G1 . . . GN Approximation criterion
- U − ˆ
U
- F,sym := min
P∈Pd
- U − ˆ
UP
- F
Better functions to be minimized greedily? f (U) := d−1U1 = d−1
d
- i,j=1
- Uij
SLIDE 17
Minimizing Sparsity-Inducing Norms over O(d)
G T
N . . . G T N U ≈ I
ˆ U = G1 . . . GN Approximation criterion
- U − ˆ
U
- F,sym := min
P∈Pd
- U − ˆ
UP
- F
Better functions to be minimized greedily? f (U) := d−1U1 = d−1
d
- i,j=1
- Uij
- Non-convex greedy step
- global optimum in O(d2) amortized time complexity
SLIDE 18