Application of K ahler manifold to signal processing and Bayesian - - PowerPoint PPT Presentation

Review on K¨ ahler manifold K¨ ahlerian information geometry for signal processing Geometric shrinkage priors Example: ARFIMA Conclusion

Application of K¨ ahler manifold to signal processing and Bayesian inference

Jaehyung Choi∗,1,2 Andrew P. Mullhaupt1

• 1. Department of Applied Mathematics and Statistics
• 2. Department of Physics and Astronomy

SUNY at Stony Brook

September 23, 2014 MaxEnt 2014, Amboise

Jaehyung Choi∗,1,2Andrew P. Mullhaupt1 Application of K¨ ahler manifold to signal processing and Bayesian

Review on K¨ ahler manifold K¨ ahlerian information geometry for signal processing Geometric shrinkage priors Example: ARFIMA Conclusion

Table of contents

1 Review on K¨

ahler manifold

2 K¨

ahlerian information geometry for signal processing

3 Geometric shrinkage priors 4 Example: ARFIMA 5 Conclusion

Jaehyung Choi∗,1,2Andrew P. Mullhaupt1 Application of K¨ ahler manifold to signal processing and Bayesian

Review on K¨ ahler manifold K¨ ahlerian information geometry for signal processing Geometric shrinkage priors Example: ARFIMA Conclusion

K¨ ahler manifold and information geometry

Implications of K¨ ahler manifold differential geometry, algebraic geometry

Jaehyung Choi∗,1,2Andrew P. Mullhaupt1 Application of K¨ ahler manifold to signal processing and Bayesian

Review on K¨ ahler manifold K¨ ahlerian information geometry for signal processing Geometric shrinkage priors Example: ARFIMA Conclusion

K¨ ahler manifold and information geometry

Implications of K¨ ahler manifold differential geometry, algebraic geometry superstring theory and supergravity in theoretical physics

Jaehyung Choi∗,1,2Andrew P. Mullhaupt1 Application of K¨ ahler manifold to signal processing and Bayesian

Review on K¨ ahler manifold K¨ ahlerian information geometry for signal processing Geometric shrinkage priors Example: ARFIMA Conclusion

K¨ ahler manifold and information geometry

Implications of K¨ ahler manifold differential geometry, algebraic geometry superstring theory and supergravity in theoretical physics Information Geometry

Jaehyung Choi∗,1,2Andrew P. Mullhaupt1 Application of K¨ ahler manifold to signal processing and Bayesian

Review on K¨ ahler manifold K¨ ahlerian information geometry for signal processing Geometric shrinkage priors Example: ARFIMA Conclusion

K¨ ahler manifold and information geometry

Implications of K¨ ahler manifold differential geometry, algebraic geometry superstring theory and supergravity in theoretical physics Information Geometry Barbaresco (2006, 2012, 2014): K¨ ahler manifold and Koszul information geometry

Jaehyung Choi∗,1,2Andrew P. Mullhaupt1 Application of K¨ ahler manifold to signal processing and Bayesian

Review on K¨ ahler manifold K¨ ahlerian information geometry for signal processing Geometric shrinkage priors Example: ARFIMA Conclusion

K¨ ahler manifold and information geometry

Implications of K¨ ahler manifold differential geometry, algebraic geometry superstring theory and supergravity in theoretical physics Information Geometry Barbaresco (2006, 2012, 2014): K¨ ahler manifold and Koszul information geometry Zhang and Li (2013): symplectic and K¨ ahler structures in divergence function

Jaehyung Choi∗,1,2Andrew P. Mullhaupt1 Application of K¨ ahler manifold to signal processing and Bayesian

Review on K¨ ahler manifold K¨ ahlerian information geometry for signal processing Geometric shrinkage priors Example: ARFIMA Conclusion

K¨ ahler manifold

Definition The K¨ ahler manifold is the Hermitian manifold with the closed K¨ ahler two-form. In the metric expression, gij = g¯

i¯ j = 0

∂igj¯

k = ∂jgi¯ k = 0

Any advantages? Let’s discuss later.

Jaehyung Choi∗,1,2Andrew P. Mullhaupt1 Application of K¨ ahler manifold to signal processing and Bayesian

Review on K¨ ahler manifold K¨ ahlerian information geometry for signal processing Geometric shrinkage priors Example: ARFIMA Conclusion

Linear systems and information geometry

Linear systems are described by the transfer function h(w; ξ) y(w) = h(w; ξ)x(w; ξ) where input x and output y. The metric tensor for the filter gµν(ξ) = 1 2π π

−π

(∂µ log S)(∂ν log S)dw where S(w; ξ) = |h(w; ξ)|2.

Jaehyung Choi∗,1,2Andrew P. Mullhaupt1 Application of K¨ ahler manifold to signal processing and Bayesian

Review on K¨ ahler manifold K¨ ahlerian information geometry for signal processing Geometric shrinkage priors Example: ARFIMA Conclusion

z-transformation h(z; ξ) = ∞

r=0 hr(ξ)z−r

log h(z; ξ) = log h0 + log (1 +

∞

• r=1

hr h0 z−r) = log h0 +

∞

• r=1

ηrz−r The metric tensor in terms of transfer function gµν = 1 2πi

• |z|=1

∂µ

• log h + log ¯

h

• ∂ν
• log h + log ¯

h dz z where µ, ν run holomorphic and anti-holomorphic indices.

Jaehyung Choi∗,1,2Andrew P. Mullhaupt1 Application of K¨ ahler manifold to signal processing and Bayesian

Review on K¨ ahler manifold K¨ ahlerian information geometry for signal processing Geometric shrinkage priors Example: ARFIMA Conclusion

The metric tensors in holomorphic and anti-holomorphic coordinates gij(ξ) = 1 2πi

• |z|=1

∂i log h(z; ξ)∂j log h(z; ξ)dz z gi¯

j(ξ) =

1 2πi

• |z|=1

∂i log h(z; ξ)∂¯

j log ¯

h(¯ z; ¯ ξ)dz z The metric tensor gij = ∂i log h0∂j log h0 gi¯

j = ∂i log h0∂¯ j log ¯

h0 +

∞

• r=1

∂iηr∂¯

j ¯

ηr

Jaehyung Choi∗,1,2Andrew P. Mullhaupt1 Application of K¨ ahler manifold to signal processing and Bayesian

Review on K¨ ahler manifold K¨ ahlerian information geometry for signal processing Geometric shrinkage priors Example: ARFIMA Conclusion

K¨ ahler manifold for signal processing

Theorem Given a holomorphic transfer function h(z; ξ), the information geometry of a signal processing model is K¨ ahler manifold if and

• nly if h0 is a constant in ξ.

(⇒) If the geometry is K¨ ahler, it should be Hermitian imposing gij = ∂i log (h0)∂j log (h0) = 0 → h0 constant in ξ (⇐) If h0 is a constant in ξ, the metric tensor is given in gij = 0 and gi¯

j = ∞

• r=1

∂iηr∂¯

j ¯

ηr → Hermitian The K¨ ahler two-form is closed : Ω = igi¯

jdξi ∧ d ¯

ξj

Jaehyung Choi∗,1,2Andrew P. Mullhaupt1 Application of K¨ ahler manifold to signal processing and Bayesian

Review on K¨ ahler manifold K¨ ahlerian information geometry for signal processing Geometric shrinkage priors Example: ARFIMA Conclusion

K¨ ahler potential for signal processing

On the K¨ ahler manifold, the metric tensor is gi¯

j = ∂i∂¯ jK

where the K¨ ahler potential K. Corollary Given K¨ ahler geometry, the K¨ ahler potential of the geometry is the square of the Hardy norm of the log-transfer function. K = 1 2πi

• |z|=1
• log h(z; ξ)
• log h(z; ξ)

∗ dz z = || log h(z; ξ)||2

H2

Jaehyung Choi∗,1,2Andrew P. Mullhaupt1 Application of K¨ ahler manifold to signal processing and Bayesian

Review on K¨ ahler manifold K¨ ahlerian information geometry for signal processing Geometric shrinkage priors Example: ARFIMA Conclusion

Benefits of K¨ ahlerian information geometry

• 1. Calculation of geometric objects is simplified.

gi¯

j = ∂i∂¯ jK, Γij,¯ k = ∂i∂j∂¯ kK

Ri

j ¯ mn = ∂ ¯ mΓi jn, Ri¯ j = −∂i∂¯ j log G

• 2. Easy α-generalization and linear order correction in α

Γ(α) = Γ + αT, R(α) = R + α∂T

• 3. Submanifolds of K¨

ahler is K¨ ahler.

• 4. Laplace-Beltrami operator:∆ = 2gi¯

j∂i∂¯ j

Jaehyung Choi∗,1,2Andrew P. Mullhaupt1 Application of K¨ ahler manifold to signal processing and Bayesian

Review on K¨ ahler manifold K¨ ahlerian information geometry for signal processing Geometric shrinkage priors Example: ARFIMA Conclusion

Komaki’s shrinkage prior for Bayesian inference

Komaki (2006): The difference in risk functions is given by E(DKL(p(y|ξ)||pπJ(y|x(N)))|ξ)) − E(DKL(p(y|ξ)||pπI (y|x(N)))|ξ)) = 1 2N2 gij∂i log πI πJ

• ∂j log

πI πJ

• − 1

N2 πJ πI ∆ πI πJ

• + o(N−2)

If ψ = πI/πJ is superharmonic, pπI outperforms pπJ. Superharmonic prior πI, Jeffreys prior πJ Superharmonicity of functions is hard to check. In particular, in high-dimensional curved geometry!

Jaehyung Choi∗,1,2Andrew P. Mullhaupt1 Application of K¨ ahler manifold to signal processing and Bayesian

Review on K¨ ahler manifold K¨ ahlerian information geometry for signal processing Geometric shrinkage priors Example: ARFIMA Conclusion

Geometric priors

Theorem On a K¨ ahler manifold, a positive function ψ = Ψ(u∗ − κ(ξ, ¯ ξ)) is a superharmonic prior function if κ(ξ, ¯ ξ) is (sub)harmonic, bounded above by u∗, and Ψ is concave decreasing: Ψ′(τ) > 0, Ψ′′(τ) < 0. The ans¨ atze for Ψ: Ψ1(τ) = τ a, Ψ2(τ) = log (1 + τ a) (τ > 0, 0 < a ≤ 1) The ans¨ atze for κ: κ1 = K, κ2 =

∞

• r=0

ar|hr(ξ)|2, κ3 =

n

• i=1

bi|ξi|2 (ar > 0, bi > 0)

Jaehyung Choi∗,1,2Andrew P. Mullhaupt1 Application of K¨ ahler manifold to signal processing and Bayesian

Review on K¨ ahler manifold K¨ ahlerian information geometry for signal processing Geometric shrinkage priors Example: ARFIMA Conclusion

Algorithm for geometric priors

The algorithm for finding geometric priors is the following:

1 Check whether the geometry is K¨

ahler.

2 Check the superharmonicity of prior function ψ. 3 If (sub)harmonic, plug it into the theorem to get

superharmonic functions and move to the next step.

4 If superharmonic, multiply the Jeffreys prior and set it as the

shrinkage prior.

5 Do Bayesian inference. Jaehyung Choi∗,1,2Andrew P. Mullhaupt1 Application of K¨ ahler manifold to signal processing and Bayesian

Review on K¨ ahler manifold K¨ ahlerian information geometry for signal processing Geometric shrinkage priors Example: ARFIMA Conclusion

ARFIMA

The transfer function of ARFIMA: h(z; ξ) = (1 − µ1z−1)(1 − µ2z−1) · · · (1 − µqz−1) (1 − λ1z−1)(1 − λ2z−1) · · · (1 − λpz−1)(1 − z−1)d The K¨ ahler potential: K =

∞

• n=1
• d + (µn

1 + · · · + µn q) − (λn 1 + · · · + λn p)

n

• 2

The metric tesnor of ARFIMA: gi¯

j =

   

π2 6 1 ¯ λj log (1 − ¯

λj) − 1

¯ µj log (1 − ¯

µj)

1 λi log (1 − λi) 1 1−λi ¯ λj

−

1 1−λi ¯ µj

− 1

µi log (1 − µi)

−

1 1−µi ¯ λj 1 1−µi ¯ µj

   

Jaehyung Choi∗,1,2Andrew P. Mullhaupt1 Application of K¨ ahler manifold to signal processing and Bayesian

Review on K¨ ahler manifold K¨ ahlerian information geometry for signal processing Geometric shrinkage priors Example: ARFIMA Conclusion

Conclusion

K¨ ahler manifold: information geometry for signal processing K¨ ahler potential: square of Hardy norm of log-transfer function Several computational benefits exist on the K¨ ahler manifold. In particular, Komaki priors are easy to build. An algorithm and ans¨ atze for Komaki priors are introduced.

Jaehyung Choi∗,1,2Andrew P. Mullhaupt1 Application of K¨ ahler manifold to signal processing and Bayesian

Review on K¨ ahler manifold K¨ ahlerian information geometry for signal processing Geometric shrinkage priors Example: ARFIMA Conclusion

Choi, J. and Mullhaupt, A. P., K¨ ahlerian information geometry for signal processing, arXiv:1404.2006 Choi, J. and Mullhaupt, A. P., Geometric shrinkage priors for K¨ ahlerian signal filters, arXiv:1408.6800 Amari, S. and Nagaoka, H., Methods of information geometry, Oxford University Press (2000) Barbaresco, F., Information intrinsic geometric flows, AIP

• Conf. Proc. 872 (2006) 211-218

Barbaresco, F., Information Geometry of Covariance Matrix: Cartan-Siegel Homogeneous Bounded Domains, Mostow/Berger Fibration and Fr´ echet Median, Matrix Information Geometry, Bhatia, R., Nielsen, F., Eds., Springer (2012) 199-256

Jaehyung Choi∗,1,2Andrew P. Mullhaupt1 Application of K¨ ahler manifold to signal processing and Bayesian

Review on K¨ ahler manifold K¨ ahlerian information geometry for signal processing Geometric shrinkage priors Example: ARFIMA Conclusion

Barbaresco, F., Koszul Information Geometry and Souriau Geometric Temperature/Capacity of Lie Group Thermodynamics, Entropy 16 (2014) 4521-4565 Komaki, F., Shrinkage priors for Bayesian prediction, Ann. Statistics 34 (2006) 808-819 Ravishanker, N., Melnick, E. L., and Tsai, C., Differential geometry of ARMA models, Journal of Time Series Analysis 11 (1990) 259-274 Ravishanker, N., Differential geometry of ARFIMA processes, Communications in Statistics - Theory and Methods 30 (2001) 1889-1902 Tanaka, F. and Komaki, F., A superharmonic prior for the autoregressive process of the second order, Journal of Time Series Analysis 29 (2008) 444-452

Jaehyung Choi∗,1,2Andrew P. Mullhaupt1 Application of K¨ ahler manifold to signal processing and Bayesian

Review on K¨ ahler manifold K¨ ahlerian information geometry for signal processing Geometric shrinkage priors Example: ARFIMA Conclusion

Tanaka, F., Superharmonic priors for autoregressive models, Mathematical Engineering Technical Reports, University of Tokyo (2009) Zhang, J. and Li, F., Symplectic and K¨ ahler Structures on Statistical Manifolds Induced from Divergence Functions, Geometric Science of Information 8085 (2013) 595-603

Jaehyung Choi∗,1,2Andrew P. Mullhaupt1 Application of K¨ ahler manifold to signal processing and Bayesian