From moments to sparse representations, a geometric, algebraic and - PowerPoint PPT Presentation

From moments to sparse representations, a geometric, algebraic and algorithmic viewpoint Bernard Mourrain Inria Méditerranée, Sophia Antipolis Bernard.Mourrain@inria.fr

Sparse representation problems

Sparse representation of sequences Given a sequence of values σ 0 , σ 1 , . . . , σ s ∈ C , find/guess the values of σ n for all n ∈ N . ☞ Find r ∈ N , ω i , ξ i ∈ C such that σ n = � r 1 ω i ξ i n , for all n ∈ N . Example: 0, 1, 1, 2, 3, 5, 8, 13, . . . . Solution: ◮ Find a recurrence relation valid for the first terms: σ k + 2 − σ k + 1 − σ k = 0. √ √ ◮ Find the roots ξ 1 = 1 + 5 , ξ 2 = 1 − 5 (golden numbers) of the characteristic 2 2 polynomial: x 2 − x − 1 = 0. √ √ ) n − 5 ( 1 + 5 ( 1 − 1 5 1 5 ) n . ◮ Deduce σ n = √ √ 2 2 1

Sparse representation of signals Given a function or signal f ( t ) : decompose it as r ′ r ( a i cos ( µ i t ) + b i sin ( µ i t )) e ν i t = � � ω i e ζ i t f ( t ) = i = 1 i = 1 2

Prony’s method (1795) For the signal f ( t ) = � r i = 1 ω i e ζ i t , ( ω i , ζ i ∈ C ), • Evaluate f at 2 r regularly spaced points: σ 0 := f ( 0 ) , σ 1 := f ( 1 ) , . . . • Compute a non-zero element p = [ p 0 , . . . , p r ] in the kernel:     σ 0 σ 1 . . . σ r p 0 σ 1 σ r + 1 p 1         = 0 . . .     . . . . . .         σ r − 1 . . . σ 2 r − 1 σ 2 r − 1 p r • Compute the roots ξ 1 = e ζ 1 , . . . , ξ r = e ζ r of p ( x ) := � r i = 0 p i x i . • Solve the system       1 . . . . . . 1 ω 1 σ 0 ξ 1 ξ r ω 2 σ 1             = . . . . .       . . . . . . . .             ξ r − 1 ξ r − 1 . . . . . . ω r σ r − 1 3 r 1

Symmetric tensor decomposition and Waring problem (1770) Symmetric tensor decomposition problem: Given a homogeneous polynomial ψ of degree d in the variables x = ( x 0 , x 1 , . . . , x n ) with coefficients ∈ K : � d � � x α , ψ ( x ) = σ α α | α | = d find a minimal decomposition of ψ of the form r � ω i ( ξ i , 0 x 0 + ξ i , 1 x 1 + · · · + ξ i , n x n ) d ψ ( x ) = i = 1 n + 1 spanning disctint lines, ω i ∈ K . with ξ i = ( ξ i , 0 , ξ i , 1 , . . . , ξ i , n ) ∈ K The minimal r in such a decomposition is called the rank of ψ . 4

Sylvester approach (1851) Theorem The binary form ψ ( x 0 , x 1 ) = � d � d x d − i � x i i = 0 σ i 1 can be decomposed as a i 0 sum of r distinct powers of linear forms r � ω k ( α k x 0 + β k x 1 ) d ψ = k = 1 0 + p 1 x r − 1 iff there exists a polynomial p ( x 0 , x 1 ) := p 0 x r x 1 + · · · + p r x r 1 s.t. 0     σ 0 σ 1 . . . σ r p 0 σ 1 σ r + 1 p 1         = 0 . . .     . . . . . .         σ d − r . . . σ d − 1 σ d p r and of the form p = c � r k = 1 ( β k x 0 − α k x 1 ) with ( α k : β k ) distinct. 5

Sparse interpolation Given a black-box polynomial function f ( x ) find what are the terms inside from output values. ☞ Find r ∈ N , ω i ∈ C , α i ∈ N such that f ( x ) = � r i = 1 ω i x α i . 6

• Choose ϕ ∈ C • Compute the sequence of terms σ 0 = f ( 1 ) , . . . , σ 2 r − 1 = f ( ϕ 2 r − 1 ) ; • Construct the matrix H = [ σ i + j ] and its kernel p = [ p 0 , . . . , p r ] s.t.     σ 0 σ 1 . . . σ r p 0 σ 1 σ r + 1 p 1         = 0 . . .     . . . . . .         σ r − 1 . . . σ 2 r − 1 σ 2 r − 1 p r i = 0 p i x i and • Compute the roots ξ 1 = ϕ α 1 , . . . , ξ r = ϕ α r of p ( x ) := � r deduce the exponents α i = log ϕ ( ξ i ) . • Deduce the weights W = [ ω i ] by solving V Ξ W = [ σ 0 , . . . , σ r − 1 ] where V Ξ is the Vandermonde system of the roots ξ 1 , . . . , ξ r . 7

Decoding − − − − − − − − − − → An algebraic code: E = { c ( f ) = [ f ( ξ 1 ) , . . . , f ( ξ m )] | f ∈ K [ x ]; deg ( f ) ≤ d } . Encoding messages using the dual code: C = E ⊥ = { c | c · [ f ( ξ 1 ) , . . . , f ( ξ m )] = 0 ∀ f ∈ V = � x a � ⊂ F [ x ] } Message received: r = m + e for m ∈ C where e = [ ω 1 , . . . , ω m ] is an error with ω j � = 0 for j = i 1 , . . . , i r and ω j = 0 otherwise. ☞ Find the error e . 8

Berlekamp-Massey method (1969) • Compute the syndrome σ k = c ( x k ) · r = c ( x k ) · e = � r j = 1 ω i j ξ k i j . • Compute the matrix     σ 0 σ 1 . . . σ r p 0 σ 1 σ r + 1 p 1         = 0 . . .     . . . . . .         σ r − 1 . . . σ 2 r − 1 σ 2 r − 1 p r and its kernel p = [ p 0 , . . . , p r ] . • Compute the roots of the error locator polynomial i = 0 p i x i = p r p ( x ) = � r � r j = 1 ( x − ξ i j ) . • Deduce the errors ω i j . 9

Simultaneous decomposition Simultaneous decomposition problem Given symmetric tensors ψ 1 , . . . , ψ m of order d 1 , . . . , d m , find a simultaneous decomposition of the form r � ω l , i ( ξ i , 0 x 0 + ξ i , 1 x 1 + · · · + ξ i , n x n ) d l ψ l = i = 1 n + 1 and ω l , i ∈ K for where ξ i = ( ξ i , 0 , . . . , ξ i , n ) span distinct lines in K l = 1 , . . . , m . 10

Proposition (One dimensional decomposition) Let ψ l = � d l x d l − i � d l � x i i = 0 σ 1 , i 1 ∈ K [ x 0 , x 1 ] d l for l = 1 , . . . , m . i 0 0 + p 1 x r − 1 If there exists a polynomial p ( x 0 , x 1 ) := p 0 x r x 1 + · · · + p r x r 1 s.t. 0   σ 1 , 0 σ 1 , 1 . . . σ 1 , r σ 1 , 1 σ 1 , r + 1     . .  . .  . .     p 0    σ 1 , d 1 − r . . . σ 1 , d 1 − 1 σ 1 , d 1    p 1   . .   . .   = 0   . . .   .   .     σ m , 0 σ m , 1 . . . σ m , r     p r    σ m , 1 σ r + 1    . .   . .   . .   σ m , d m − r . . . σ m , d m − 1 σ m , d m of the form p = c � r k = 1 ( β k x 0 − α k x 1 ) with [ α k : β k ] distinct, then r � ω i , l ( α l x 0 + β l x 1 ) d l ψ l = i = 1 for ω i , l ∈ K and l = 1 , . . . , m . 11

Duality

Dual of polynomial rings For R = K [ x ] = K [ x 1 , . . . , x n ] = { p = � α ∈ A p α x α , p α ∈ K } , K [ x ] ∗ = Hom K ( K [ x ] , K ) The element σ ∈ R ∗ : p ∈ R �→ � σ | p � ∈ K is a linear functional on R . The coefficients � σ | x α � = σ α ∈ K , α ∈ N n are the moments of σ . Examples: • p �→ coefficient of x α in p • e ζ : p �→ p ( ζ ) for ζ = ( ζ 1 , . . . , ζ n ) ∈ K n . • For K = R , Ω ⊂ R n compact, � � Ω : p �→ Ω p ( x ) d x Structure of K [ x ] -module: p ⋆ σ ∈ R ∗ : q �→ � σ | p q � . Example: For p , q ∈ R , p ⋆ e ζ : q �→ � e ζ | p q � = p ( ζ ) � e ζ | q � ⇒ p ⋆ e ζ = p ( ζ ) e ζ Property: For p , q ∈ R , σ ∈ R ∗ , p ⋆ ( q ⋆ σ ) = p q ⋆ σ = q ⋆ ( p ⋆ σ ) . 12

Linear functionals as sequences Correspondence: σ ∈ K [ x ] ∗ ≡ ( σ α ) α ∈ N n ∈ K N n sequence indexed by α = ( α 1 , . . . , α n ) ∈ N n with σ α = � σ | x α � . p α x α ∈ R �→ � σ | p � = � � σ α p α ∈ K σ : p = α α Example: e ζ ≡ ( ζ α ) α ∈ K N n where ζ α = ζ α 1 1 · · · ζ α n n . Structure of K [ x ] -module: α ∈ A p α x α ∈ R , σ ≡ ( σ α ) α ∈ N n ∈ K N n , β ∈ N n For p = � � ( p ⋆ σ ) β = p α σ α + β α ∈ A (correlation sequence). 13

Linear functionals as series Correspondence: σ ∈ K [ x ] ∗ ≡ y α α ∈ N n σ α z α ∈ K [[ z 1 , . . . , z n ]] σ ( y ) = � α ! ∈ K [[ y 1 , . . . , y n ]] σ ( z ) = � α ∈ N n σ α with σ α = � σ | x α � , α ! = � α i ! for α = ( α 1 , . . . , α n ) ∈ N n . Example: α ζ α y α α ! = e ζ · y ∈ K [[ y ]] e ζ ( z ) = � α ζ α z α = 1 e ζ ( y ) = � i = 1 ( 1 − ζ i z i ) ∈ K [[ z ]] � n y α ◮ For p = � α p α ∈ R , σ ( y ) = � α ∈ N n σ α α ! ∈ K [[ y ]] , � σ | p � = � α σ α p α ◮ The basis dual to ( x α ) is ( y α α ! ) α ∈ N n (resp. ( z α ) α ∈ N n ) x ( p )( 0 ) , � z α | p � = coeff. of x α in p . ◮ For p ∈ R , α ∈ N n , � y α | p � = ∂ α Structure of R -module: y α − e 1 α 1 > 0 σ α z α − e 1 x 1 ⋆ σ ( y ) = � α 1 > 0 σ α x 1 ⋆ σ ( z ) = � ( α − e 1 )! π + ( z − 1 = ∂ y 1 ( σ ( y )) = 1 σ ( z )) π + ( p ( z − 1 1 , . . . , z − 1 p ⋆ σ = p ( ∂ 1 , . . . , ∂ n )( σ )( y ) p ⋆ σ = n ) σ ( z )) 14