A Users Guide to Riemannian Newton-Type Methods on Manifolds Felipe - - PowerPoint PPT Presentation

Motivation Outline

A User’s Guide to Riemannian Newton-Type Methods on Manifolds

Felipe Álvarez

Departamento de Ingeniería Matemática Centro de Modelamiento Matemático (CNRS UMI 2807) Universidad de Chile

In collaboration with: J. Bolte, J. Munier, J. López

Sixièmes Journées Franco-Chiliennes d’Optimisation Université du Sud Toulon-Var Mai 19-21, 2008

http://www.dim.uchile.cl/~falvarez Felipe Álvarez (DIM) A User’s Guide to Riemannian Newton-Type Methods 1/23

Motivation Outline

Motivation: Nonlinear equations in a manifold

Goal: find p∗ ∈ M satisfying F(p∗) = 0 ∈ Tp∗M M is a connected and n-dimensional differentiable manifold. TpM ≃ Rn is the tangent space of M at p: If c(t) is a curve passing through p at t = 0 then ˙ c(0) ∈ TpM. F : M → TM is a continuously differentiable vector field: M ∋ p → F(p) ∈ TpM

Felipe Álvarez (DIM) A User’s Guide to Riemannian Newton-Type Methods 2/23

Motivation Outline

Motivation: Nonlinear equations in a manifold

Goal: find p∗ ∈ M satisfying F(p∗) = 0 ∈ Tp∗M M is a connected and n-dimensional differentiable manifold. TpM ≃ Rn is the tangent space of M at p: If c(t) is a curve passing through p at t = 0 then ˙ c(0) ∈ TpM. F : M → TM is a continuously differentiable vector field: M ∋ p → F(p) ∈ TpM

Felipe Álvarez (DIM) A User’s Guide to Riemannian Newton-Type Methods 2/23

Motivation Outline

Motivation: Nonlinear equations in a manifold

Goal: find p∗ ∈ M satisfying F(p∗) = 0 ∈ Tp∗M M is a connected and n-dimensional differentiable manifold. TpM ≃ Rn is the tangent space of M at p: If c(t) is a curve passing through p at t = 0 then ˙ c(0) ∈ TpM. F : M → TM is a continuously differentiable vector field: M ∋ p → F(p) ∈ TpM

Felipe Álvarez (DIM) A User’s Guide to Riemannian Newton-Type Methods 2/23

Motivation Outline

Motivation: Nonlinear equations in a manifold

Goal: find p∗ ∈ M satisfying F(p∗) = 0 ∈ Tp∗M M is a connected and n-dimensional differentiable manifold. TpM ≃ Rn is the tangent space of M at p: If c(t) is a curve passing through p at t = 0 then ˙ c(0) ∈ TpM. F : M → TM is a continuously differentiable vector field: M ∋ p → F(p) ∈ TpM

Felipe Álvarez (DIM) A User’s Guide to Riemannian Newton-Type Methods 2/23

Motivation Outline

Example 1: Rayleigh’s quotient on the sphere

M = Sn = {x ∈ Rn+1 | xTx = 1} (unit sphere in Rn+1). TxM = {v ∈ Rn+1 | xTv = 0}. F(x) = Ax − q(x)x with A ∈ Rn×n being symmetric and positive definite. q(x) = xTAx. xTF(x) = 0 ⇒ F(x) ∈ TxM. F(x∗) = 0 iff x∗ is an eigenvector of A with q(x∗) the corresponding eigenvalue.

Felipe Álvarez (DIM) A User’s Guide to Riemannian Newton-Type Methods 3/23

Motivation Outline

Example 2: Stiefel manifold

M = Sn,k = {Y ∈ Rn×k | Y TY = Ik}. TYSn,k = {∆ ∈ Rn×k | ∆TY + Y T∆ = 0}. If k = 1 then Sn,1 = Sn−1. If k = n then Sn,n = On the orthogonal group. TInOn = {∆ ∈ Rn×n | ∆T = −∆}. dim Sn,k = nk − 1

2k(k + 1).

F(Y) = AY − YY TAY F(Y ∗) = 0 iff the columns of Y ∗ are eigenvectors of A.

Felipe Álvarez (DIM) A User’s Guide to Riemannian Newton-Type Methods 4/23

Motivation Outline

Example 2: Stiefel manifold

M = Sn,k = {Y ∈ Rn×k | Y TY = Ik}. TYSn,k = {∆ ∈ Rn×k | ∆TY + Y T∆ = 0}. If k = 1 then Sn,1 = Sn−1. If k = n then Sn,n = On the orthogonal group. TInOn = {∆ ∈ Rn×n | ∆T = −∆}. dim Sn,k = nk − 1

2k(k + 1).

F(Y) = AY − YY TAY F(Y ∗) = 0 iff the columns of Y ∗ are eigenvectors of A.

Felipe Álvarez (DIM) A User’s Guide to Riemannian Newton-Type Methods 4/23

Motivation Outline

Example 2: Stiefel manifold

M = Sn,k = {Y ∈ Rn×k | Y TY = Ik}. TYSn,k = {∆ ∈ Rn×k | ∆TY + Y T∆ = 0}. If k = 1 then Sn,1 = Sn−1. If k = n then Sn,n = On the orthogonal group. TInOn = {∆ ∈ Rn×n | ∆T = −∆}. dim Sn,k = nk − 1

2k(k + 1).

F(Y) = AY − YY TAY F(Y ∗) = 0 iff the columns of Y ∗ are eigenvectors of A.

Felipe Álvarez (DIM) A User’s Guide to Riemannian Newton-Type Methods 4/23

Motivation Outline

Example 2: Stiefel manifold

M = Sn,k = {Y ∈ Rn×k | Y TY = Ik}. TYSn,k = {∆ ∈ Rn×k | ∆TY + Y T∆ = 0}. If k = 1 then Sn,1 = Sn−1. If k = n then Sn,n = On the orthogonal group. TInOn = {∆ ∈ Rn×n | ∆T = −∆}. dim Sn,k = nk − 1

2k(k + 1).

F(Y) = AY − YY TAY F(Y ∗) = 0 iff the columns of Y ∗ are eigenvectors of A.

Felipe Álvarez (DIM) A User’s Guide to Riemannian Newton-Type Methods 4/23

Motivation Outline

Solving nonlinear equations: Euclidean case

Goal: find p∗ ∈ Ω such that F(p∗) = 0 ∈ Rn , where Ω is open and F : Ω ⊂ Rn → Rn is a C1 vector field. Newton’s method: F(pk) + F ′(pk)(pk+1 − pk) = 0.

−1 −0.5 0.5 1 1.5 2 2.5 −4 −2 2 4 6 8 10

pk+1 pk p*

Felipe Álvarez (DIM) A User’s Guide to Riemannian Newton-Type Methods 5/23

Motivation Outline

Outline

1

Abstract differential geometry setting for R-Newton

2

Other explicit examples

Felipe Álvarez (DIM) A User’s Guide to Riemannian Newton-Type Methods 6/23

Abstract differential geometry setting for R-Newton Other explicit examples

Outline

1

Abstract differential geometry setting for R-Newton

2

Other explicit examples

Felipe Álvarez (DIM) A User’s Guide to Riemannian Newton-Type Methods 7/23

Abstract differential geometry setting for R-Newton Other explicit examples

Metric framework

M is endowed with a Riemannian metric g: v2

p = g(p)(v, v) for v ∈ TpM.

Riemannian distance d : M × M → [0, +∞): d(p, q) = inf{ b

a˙

c(t)c(t)dt | c : [a, b] → M, c(a) = p, c(b) = q} Assumption: (M, d) is a complete metric space. Covariant derivative: F ′(p)v := ∇vF(p) = (∇YF)(p), v ∈ TpM , where Y is any vector field on M satisfying v = Y(p). ∇ is the Riemannian (or Levi-Civita) connection on (M, g).

Felipe Álvarez (DIM) A User’s Guide to Riemannian Newton-Type Methods 8/23

Abstract differential geometry setting for R-Newton Other explicit examples

Metric framework

M is endowed with a Riemannian metric g: v2

p = g(p)(v, v) for v ∈ TpM.

Riemannian distance d : M × M → [0, +∞): d(p, q) = inf{ b

a˙

c(t)c(t)dt | c : [a, b] → M, c(a) = p, c(b) = q} Assumption: (M, d) is a complete metric space. Covariant derivative: F ′(p)v := ∇vF(p) = (∇YF)(p), v ∈ TpM , where Y is any vector field on M satisfying v = Y(p). ∇ is the Riemannian (or Levi-Civita) connection on (M, g).

Felipe Álvarez (DIM) A User’s Guide to Riemannian Newton-Type Methods 8/23

Abstract differential geometry setting for R-Newton Other explicit examples

Metric framework

M is endowed with a Riemannian metric g: v2

p = g(p)(v, v) for v ∈ TpM.

Riemannian distance d : M × M → [0, +∞): d(p, q) = inf{ b

a˙

c(t)c(t)dt | c : [a, b] → M, c(a) = p, c(b) = q} Assumption: (M, d) is a complete metric space. Covariant derivative: F ′(p)v := ∇vF(p) = (∇YF)(p), v ∈ TpM , where Y is any vector field on M satisfying v = Y(p). ∇ is the Riemannian (or Levi-Civita) connection on (M, g).

Felipe Álvarez (DIM) A User’s Guide to Riemannian Newton-Type Methods 8/23

Abstract differential geometry setting for R-Newton Other explicit examples

Metric framework

M is endowed with a Riemannian metric g: v2

p = g(p)(v, v) for v ∈ TpM.

Riemannian distance d : M × M → [0, +∞): d(p, q) = inf{ b

a˙

c(t)c(t)dt | c : [a, b] → M, c(a) = p, c(b) = q} Assumption: (M, d) is a complete metric space. Covariant derivative: F ′(p)v := ∇vF(p) = (∇YF)(p), v ∈ TpM , where Y is any vector field on M satisfying v = Y(p). ∇ is the Riemannian (or Levi-Civita) connection on (M, g).

Felipe Álvarez (DIM) A User’s Guide to Riemannian Newton-Type Methods 8/23

Abstract differential geometry setting for R-Newton Other explicit examples

Metric framework

M is endowed with a Riemannian metric g: v2

p = g(p)(v, v) for v ∈ TpM.

Riemannian distance d : M × M → [0, +∞): d(p, q) = inf{ b

a˙

c(t)c(t)dt | c : [a, b] → M, c(a) = p, c(b) = q} Assumption: (M, d) is a complete metric space. Covariant derivative: F ′(p)v := ∇vF(p) = (∇YF)(p), v ∈ TpM , where Y is any vector field on M satisfying v = Y(p). ∇ is the Riemannian (or Levi-Civita) connection on (M, g).

Felipe Álvarez (DIM) A User’s Guide to Riemannian Newton-Type Methods 8/23

Abstract differential geometry setting for R-Newton Other explicit examples

Exponential Map

Geodesic: a curve γ : (a, b) → M with ∇ ˙

γ ˙

γ = 0. If γi are the coordinates of γ, d2γk dt2 +

n

• i,j=1

Γk

ij

dγi dt dγj dt = 0; k = 1, . . . , n, where Γk

i,j are the Christoffel symbols.

Exponential map: expp : TpM → M is defined by setting expp[v] = γ(1), where γ : R → M is the geodesic with γ(0) = p and ˙ γ(0) = v.

Felipe Álvarez (DIM) A User’s Guide to Riemannian Newton-Type Methods 9/23

Abstract differential geometry setting for R-Newton Other explicit examples

Exponential Map

Geodesic: a curve γ : (a, b) → M with ∇ ˙

γ ˙

γ = 0. If γi are the coordinates of γ, d2γk dt2 +

n

• i,j=1

Γk

ij

dγi dt dγj dt = 0; k = 1, . . . , n, where Γk

i,j are the Christoffel symbols.

Exponential map: expp : TpM → M is defined by setting expp[v] = γ(1), where γ : R → M is the geodesic with γ(0) = p and ˙ γ(0) = v.

Felipe Álvarez (DIM) A User’s Guide to Riemannian Newton-Type Methods 9/23

Abstract differential geometry setting for R-Newton Other explicit examples

Exponential Map

Geodesic: a curve γ : (a, b) → M with ∇ ˙

γ ˙

γ = 0. If γi are the coordinates of γ, d2γk dt2 +

n

• i,j=1

Γk

ij

dγi dt dγj dt = 0; k = 1, . . . , n, where Γk

i,j are the Christoffel symbols.

Exponential map: expp : TpM → M is defined by setting expp[v] = γ(1), where γ : R → M is the geodesic with γ(0) = p and ˙ γ(0) = v.

Felipe Álvarez (DIM) A User’s Guide to Riemannian Newton-Type Methods 9/23

Abstract differential geometry setting for R-Newton Other explicit examples

Riemannian Newton’s method (Shub ’86)

1

Data: Given pk with F(pk) = 0 and F ′(pk) nondegenerate.

2

Newton’s correction: Find vk ∈ Tpk M s.t. F ′(pk)vk = −F(pk).

3

Update: Set pk+1 = exppk [vk] .

pk pk+1

Tp

k

Sn

−X’(pk)−1X(pk) Felipe Álvarez (DIM) A User’s Guide to Riemannian Newton-Type Methods 10/23

Abstract differential geometry setting for R-Newton Other explicit examples

Rayleigh quotient (continuation)

Manifold: M = Sn = {x ∈ Rn+1 | xTx = 1} (unit sphere in Rn+1). Tangent space: TxSn = {v ∈ Rn+1 | xTv = 0}. Metric: g(v, w) = vTw. Exponential map: expp[v] = p cos(v) + v v sin(v) Vector field: F(x) = Ax − q(x)x with q(x) = xTAx. Newton direction at p: v = −p + 1 pTw w where (A − q(p)I)w = p

Felipe Álvarez (DIM) A User’s Guide to Riemannian Newton-Type Methods 11/23

Abstract differential geometry setting for R-Newton Other explicit examples

Rayleigh quotient (continuation)

Manifold: M = Sn = {x ∈ Rn+1 | xTx = 1} (unit sphere in Rn+1). Tangent space: TxSn = {v ∈ Rn+1 | xTv = 0}. Metric: g(v, w) = vTw. Exponential map: expp[v] = p cos(v) + v v sin(v) Vector field: F(x) = Ax − q(x)x with q(x) = xTAx. Newton direction at p: v = −p + 1 pTw w where (A − q(p)I)w = p

Felipe Álvarez (DIM) A User’s Guide to Riemannian Newton-Type Methods 11/23

Abstract differential geometry setting for R-Newton Other explicit examples

Rayleigh quotient (continuation)

Manifold: M = Sn = {x ∈ Rn+1 | xTx = 1} (unit sphere in Rn+1). Tangent space: TxSn = {v ∈ Rn+1 | xTv = 0}. Metric: g(v, w) = vTw. Exponential map: expp[v] = p cos(v) + v v sin(v) Vector field: F(x) = Ax − q(x)x with q(x) = xTAx. Newton direction at p: v = −p + 1 pTw w where (A − q(p)I)w = p

Felipe Álvarez (DIM) A User’s Guide to Riemannian Newton-Type Methods 11/23

Abstract differential geometry setting for R-Newton Other explicit examples

Rayleigh quotient (continuation)

Manifold: M = Sn = {x ∈ Rn+1 | xTx = 1} (unit sphere in Rn+1). Tangent space: TxSn = {v ∈ Rn+1 | xTv = 0}. Metric: g(v, w) = vTw. Exponential map: expp[v] = p cos(v) + v v sin(v) Vector field: F(x) = Ax − q(x)x with q(x) = xTAx. Newton direction at p: v = −p + 1 pTw w where (A − q(p)I)w = p

Felipe Álvarez (DIM) A User’s Guide to Riemannian Newton-Type Methods 11/23

Abstract differential geometry setting for R-Newton Other explicit examples

Rayleigh quotient (continuation)

A = B B B B B B @ 1 1 1 1 1 1 1 2 3 4 5 6 1 3 6 10 15 21 1 4 10 20 35 56 1 5 15 35 70 126 1 6 21 56 126 252 1 C C C C C C A

1 2 3 4 5 6 7 20 40 60 80 100 120 140 160

Iteration ||Ax−(xTAx)x||

Felipe Álvarez (DIM) A User’s Guide to Riemannian Newton-Type Methods 12/23

Abstract differential geometry setting for R-Newton Other explicit examples

Some references

UDRISTE ’94 (VOL. 297, KLUWER): Convexity and optimization on manifolds, including R-Newton. SMITH ’94 (FIELDS INSTITUTE COMM.): Existence of a basin of attraction for quadratic convergence. EDELMAN, ARIAS & SMITH ’98 (SIAM J. ON MATRIX ANAL. AND APPL): Matrix orthogonality constrains. FERREIRA & SVAITER ’02 (J. OF COMPLEXITY): Kantorovich-type proximity test for quadratic convergence under a Lipschitz condition. DEDIEU, PRIOURET & MALAJOVICH ’03 (IMA J.NUMER. ANAL.): Smale-type proximity test for analytic vector fields. A., BOLTE & MUNIER (FOUND. COMP. MATHEMATICS ’08): General and unifying proximity test.

Felipe Álvarez (DIM) A User’s Guide to Riemannian Newton-Type Methods 13/23

Abstract differential geometry setting for R-Newton Other explicit examples

Outline

1

Abstract differential geometry setting for R-Newton

2

Other explicit examples

Felipe Álvarez (DIM) A User’s Guide to Riemannian Newton-Type Methods 14/23

Abstract differential geometry setting for R-Newton Other explicit examples

Positivity constraints

M = Rn

++ and g(p)(u, v) = n k=1 ukvk/h2 k(pk) where

hi : R++ → R++ differentiable function. Isometry: γ = γ(t) geodesic

⇐ ⇒

solution

• 1

hi(γk)dγk = akt + bk

For the barrier φ(p) = − n

k=1 log(pk),

∇2φ(p) =diag(h2

1(p1), . . . , h2 n(pn)) where hk(pk) = pk.

Thus, γ geodesics γ(0) = p, ˙ γ(0) = v

• ⇐

⇒ γk(t) = pk exp(t vk

pk )

Felipe Álvarez (DIM) A User’s Guide to Riemannian Newton-Type Methods 15/23

Abstract differential geometry setting for R-Newton Other explicit examples

Positivity constraints

M = Rn

++ and g(p)(u, v) = n k=1 ukvk/h2 k(pk) where

hi : R++ → R++ differentiable function. Isometry: γ = γ(t) geodesic

⇐ ⇒

solution

• 1

hi(γk)dγk = akt + bk

For the barrier φ(p) = − n

k=1 log(pk),

∇2φ(p) =diag(h2

1(p1), . . . , h2 n(pn)) where hk(pk) = pk.

Thus, γ geodesics γ(0) = p, ˙ γ(0) = v

• ⇐

⇒ γk(t) = pk exp(t vk

pk )

Felipe Álvarez (DIM) A User’s Guide to Riemannian Newton-Type Methods 15/23

Abstract differential geometry setting for R-Newton Other explicit examples

Positivity constraints

M = Rn

++ and g(p)(u, v) = n k=1 ukvk/h2 k(pk) where

hi : R++ → R++ differentiable function. Isometry: γ = γ(t) geodesic

⇐ ⇒

solution

• 1

hi(γk)dγk = akt + bk

For the barrier φ(p) = − n

k=1 log(pk),

∇2φ(p) =diag(h2

1(p1), . . . , h2 n(pn)) where hk(pk) = pk.

Thus, γ geodesics γ(0) = p, ˙ γ(0) = v

• ⇐

⇒ γk(t) = pk exp(t vk

pk )

Felipe Álvarez (DIM) A User’s Guide to Riemannian Newton-Type Methods 15/23

Abstract differential geometry setting for R-Newton Other explicit examples

Positivity constraints

M = Rn

++ and g(p)(u, v) = n k=1 ukvk/h2 k(pk) where

hi : R++ → R++ differentiable function. Isometry: γ = γ(t) geodesic

⇐ ⇒

solution

• 1

hi(γk)dγk = akt + bk

For the barrier φ(p) = − n

k=1 log(pk),

∇2φ(p) =diag(h2

1(p1), . . . , h2 n(pn)) where hk(pk) = pk.

Thus, γ geodesics γ(0) = p, ˙ γ(0) = v

• ⇐

⇒ γk(t) = pk exp(t vk

pk )

Felipe Álvarez (DIM) A User’s Guide to Riemannian Newton-Type Methods 15/23

Abstract differential geometry setting for R-Newton Other explicit examples

Relative interior of the unitary simplex

M = ∆n−1

++ = {p ∈ Rn | n

• i=1

pi = 1, pi > 0, i = 1, . . . , n}, Tangent space: Tp∆n−1 = {v ∈ Rn |

n

• i=1

vi = 0} Metric: g(p)(u, v) = (1 − 1

n) n

• k=1

ukvk h2

k(pk).

γ = γ(t) geodesics ⇐

⇒

solution d

dt

• 1

hk(γk) dγk dt − 1 n

n

i=1 1 hi(γi) dγi dt

• = 0

For hk(pk) = pk γ geodesics γ(0) = p, ˙ γ(0) = v

• ⇐

⇒ γk(t) = pk exp(t vk

pk )

n

i=1 pi exp(t vi pi )

Felipe Álvarez (DIM) A User’s Guide to Riemannian Newton-Type Methods 16/23

Abstract differential geometry setting for R-Newton Other explicit examples

Relative interior of the unitary simplex

M = ∆n−1

++ = {p ∈ Rn | n

• i=1

pi = 1, pi > 0, i = 1, . . . , n}, Tangent space: Tp∆n−1 = {v ∈ Rn |

n

• i=1

vi = 0} Metric: g(p)(u, v) = (1 − 1

n) n

• k=1

ukvk h2

k(pk).

γ = γ(t) geodesics ⇐

⇒

solution d

dt

• 1

hk(γk) dγk dt − 1 n

n

i=1 1 hi(γi) dγi dt

• = 0

For hk(pk) = pk γ geodesics γ(0) = p, ˙ γ(0) = v

• ⇐

⇒ γk(t) = pk exp(t vk

pk )

n

i=1 pi exp(t vi pi )

Felipe Álvarez (DIM) A User’s Guide to Riemannian Newton-Type Methods 16/23

Abstract differential geometry setting for R-Newton Other explicit examples

Relative interior of the unitary simplex

M = ∆n−1

++ = {p ∈ Rn | n

• i=1

pi = 1, pi > 0, i = 1, . . . , n}, Tangent space: Tp∆n−1 = {v ∈ Rn |

n

• i=1

vi = 0} Metric: g(p)(u, v) = (1 − 1

n) n

• k=1

ukvk h2

k(pk).

γ = γ(t) geodesics ⇐

⇒

solution d

dt

• 1

hk(γk) dγk dt − 1 n

n

i=1 1 hi(γi) dγi dt

• = 0

For hk(pk) = pk γ geodesics γ(0) = p, ˙ γ(0) = v

• ⇐

⇒ γk(t) = pk exp(t vk

pk )

n

i=1 pi exp(t vi pi )

Felipe Álvarez (DIM) A User’s Guide to Riemannian Newton-Type Methods 16/23

Abstract differential geometry setting for R-Newton Other explicit examples

The Stiefel manifold (continuation)

Sn,k = {A ∈ Rn×k : ATA = Ik} Tangent space: TASn,k = {∆ ∈ Rn×k : ∆TA + AT∆ = 0} Euclidian Metric: g(∆1, ∆2) = trace (∆T

1 ∆2)

The equation that described the geodesics is given by ¨ Y(t) + Y(t)( ˙ Y(t)T ˙ Y(t)) = 0, and the corresponding exponential map is: expA[∆] = (A ∆) exp AT∆ −∆T∆ Ip AT∆

• I2p,p exp(−AT∆),

where I2p,p = I2p 0p

• and exp(B) = I + B + 1

2B2 + . . ..

Felipe Álvarez (DIM) A User’s Guide to Riemannian Newton-Type Methods 17/23

Abstract differential geometry setting for R-Newton Other explicit examples

The Stiefel manifold (continuation)

Sn,k = {A ∈ Rn×k : ATA = Ik} Tangent space: TASn,k = {∆ ∈ Rn×k : ∆TA + AT∆ = 0} Euclidian Metric: g(∆1, ∆2) = trace (∆T

1 ∆2)

The equation that described the geodesics is given by ¨ Y(t) + Y(t)( ˙ Y(t)T ˙ Y(t)) = 0, and the corresponding exponential map is: expA[∆] = (A ∆) exp AT∆ −∆T∆ Ip AT∆

• I2p,p exp(−AT∆),

where I2p,p =

• I2p

0p

• and exp(B) = I + B + 1

2B2 + . . ..

Felipe Álvarez (DIM) A User’s Guide to Riemannian Newton-Type Methods 17/23

Abstract differential geometry setting for R-Newton Other explicit examples

Stiefel manifold (continuation)

Manifold: Sn,k = {Y ∈ Rn×k : Y TY = Ik}. Tangent space: TYSn,k = {∆ ∈ Rn×k : ∆TY + Y T∆ = 0}. Euclidian Metric: g(∆1, ∆2) = trace (∆T

1 ∆2).

Exponential map: expY[∆] = (Y ∆) exp Y T∆ −∆T∆ Ip Y T∆

• I2p,p exp(−Y T∆)

Vector field: V(Y) = BY − Y(Y TBY) Newton direction ∆ at Y must satisfy: B∆ − ∆Y TBY − 1

2Y(Y TB∆ + ∆TBY + ∆TYY TBY + Y TBYY T∆) =

−V(Y)

Felipe Álvarez (DIM) A User’s Guide to Riemannian Newton-Type Methods 18/23

Abstract differential geometry setting for R-Newton Other explicit examples

Stiefel manifold (continuation)

Manifold: Sn,k = {Y ∈ Rn×k : Y TY = Ik}. Tangent space: TYSn,k = {∆ ∈ Rn×k : ∆TY + Y T∆ = 0}. Euclidian Metric: g(∆1, ∆2) = trace (∆T

1 ∆2).

Exponential map: expY[∆] = (Y ∆) exp Y T∆ −∆T∆ Ip Y T∆

• I2p,p exp(−Y T∆)

Vector field: V(Y) = BY − Y(Y TBY) Newton direction ∆ at Y must satisfy: B∆ − ∆Y TBY − 1

2Y(Y TB∆ + ∆TBY + ∆TYY TBY + Y TBYY T∆) =

−V(Y)

Felipe Álvarez (DIM) A User’s Guide to Riemannian Newton-Type Methods 18/23

Abstract differential geometry setting for R-Newton Other explicit examples

Stiefel manifold (continuation)

Manifold: Sn,k = {Y ∈ Rn×k : Y TY = Ik}. Tangent space: TYSn,k = {∆ ∈ Rn×k : ∆TY + Y T∆ = 0}. Euclidian Metric: g(∆1, ∆2) = trace (∆T

1 ∆2).

Exponential map: expY[∆] = (Y ∆) exp Y T∆ −∆T∆ Ip Y T∆

• I2p,p exp(−Y T∆)

Vector field: V(Y) = BY − Y(Y TBY) Newton direction ∆ at Y must satisfy: B∆ − ∆Y TBY − 1

2Y(Y TB∆ + ∆TBY + ∆TYY TBY + Y TBYY T∆) =

−V(Y)

Felipe Álvarez (DIM) A User’s Guide to Riemannian Newton-Type Methods 18/23

Abstract differential geometry setting for R-Newton Other explicit examples

Stiefel manifold (continuation)

Manifold: Sn,k = {Y ∈ Rn×k : Y TY = Ik}. Tangent space: TYSn,k = {∆ ∈ Rn×k : ∆TY + Y T∆ = 0}. Euclidian Metric: g(∆1, ∆2) = trace (∆T

1 ∆2).

Exponential map: expY[∆] = (Y ∆) exp Y T∆ −∆T∆ Ip Y T∆

• I2p,p exp(−Y T∆)

Vector field: V(Y) = BY − Y(Y TBY) Newton direction ∆ at Y must satisfy: B∆ − ∆Y TBY − 1

2Y(Y TB∆ + ∆TBY + ∆TYY TBY + Y TBYY T∆) =

−V(Y)

Felipe Álvarez (DIM) A User’s Guide to Riemannian Newton-Type Methods 18/23

Abstract differential geometry setting for R-Newton Other explicit examples

Stiefel Manifold (continuation)

B =       1 1 1 1 1 1 2 3 4 5 1 3 6 10 15 1 4 10 20 35 1 5 15 35 70       , Y0 =       0.1947 −0.6155 −0.7448 −0.4268 0.5660 −0.4579 0.7236 0.1412 0.2122 −0.5050 −0.5012 0.3901 0.0355 0.1723 −0.1958       and S5,3.

Felipe Álvarez (DIM) A User’s Guide to Riemannian Newton-Type Methods 19/23

Abstract differential geometry setting for R-Newton Other explicit examples

Stiefel Manifold (continuation)

B =       1 1 1 1 1 1 2 3 4 5 1 3 6 10 15 1 4 10 20 35 1 5 15 35 70       , Y0 =       0.1947 −0.6155 −0.7448 −0.4268 0.5660 −0.4579 0.7236 0.1412 0.2122 −0.5050 −0.5012 0.3901 0.0355 0.1723 −0.1958       and S5,3.

1 1.5 2 2.5 3 3.5 4 4.5 5 1 2 3 4 5 6 7 8

Iteration ||BY−YYTBY||

Felipe Álvarez (DIM) A User’s Guide to Riemannian Newton-Type Methods 19/23

Abstract differential geometry setting for R-Newton Other explicit examples

Stiefel Manifold (continuation)

B =       1 1 1 1 1 1 2 3 4 5 1 3 6 10 15 1 4 10 20 35 1 5 15 35 70       , Y ∗ =       0.1795 −0.5882 −0.7500 −0.4682 0.6017 −0.4265 0.7062 0.1923 0.2209 −0.4847 −0.4777 0.4030 0.1222 0.1635 −0.2108      

1 1.5 2 2.5 3 3.5 4 4.5 5 1 2 3 4 5 6 7 8

Iteration ||BY−YYTBY||

Felipe Álvarez (DIM) A User’s Guide to Riemannian Newton-Type Methods 20/23

Abstract differential geometry setting for R-Newton Other explicit examples

Cone of Positive Semidefinite Matrices

Manifold: M = Sn

++ cone of symmetric positive definite matrices.

Tangent space: TPSn

++ ≃ Sn (Sn space symmetric matrices).

Hessian Metric: g(∆1, ∆2) = trace(∇2ϕ(P)∆1∆2) with ϕ barrier

• n Sn

++.

Taking ϕ(P) = − log(det(P)): g(∆1, ∆2) = trace (P−1∆1P−1∆2) Goal: minimize f : Sn → R over M Exponential map: expP[∆] = P1/2 exp(P−1/2∆P−1/2)P1/2 Vector field: V(P) = P∇f(P)P Newton direction ∆ at P must satisfy: V ′(P)∆ = ∇2f(P)∆ + 1

2(P−1∆∇f(P) + ∇f(P)∆P−1) = −P∇f(P)P

Felipe Álvarez (DIM) A User’s Guide to Riemannian Newton-Type Methods 21/23

Abstract differential geometry setting for R-Newton Other explicit examples

Cone of Positive Semidefinite Matrices

Manifold: M = Sn

++ cone of symmetric positive definite matrices.

Tangent space: TPSn

++ ≃ Sn (Sn space symmetric matrices).

Hessian Metric: g(∆1, ∆2) = trace(∇2ϕ(P)∆1∆2) with ϕ barrier

• n Sn

++.

Taking ϕ(P) = − log(det(P)): g(∆1, ∆2) = trace (P−1∆1P−1∆2) Goal: minimize f : Sn → R over M Exponential map: expP[∆] = P1/2 exp(P−1/2∆P−1/2)P1/2 Vector field: V(P) = P∇f(P)P Newton direction ∆ at P must satisfy: V ′(P)∆ = ∇2f(P)∆ + 1

2(P−1∆∇f(P) + ∇f(P)∆P−1) = −P∇f(P)P

Felipe Álvarez (DIM) A User’s Guide to Riemannian Newton-Type Methods 21/23

Abstract differential geometry setting for R-Newton Other explicit examples

Cone of Positive Semidefinite Matrices

Manifold: M = Sn

++ cone of symmetric positive definite matrices.

Tangent space: TPSn

++ ≃ Sn (Sn space symmetric matrices).

Hessian Metric: g(∆1, ∆2) = trace(∇2ϕ(P)∆1∆2) with ϕ barrier

• n Sn

++.

Taking ϕ(P) = − log(det(P)): g(∆1, ∆2) = trace (P−1∆1P−1∆2) Goal: minimize f : Sn → R over M Exponential map: expP[∆] = P1/2 exp(P−1/2∆P−1/2)P1/2 Vector field: V(P) = P∇f(P)P Newton direction ∆ at P must satisfy: V ′(P)∆ = ∇2f(P)∆ + 1

2(P−1∆∇f(P) + ∇f(P)∆P−1) = −P∇f(P)P

Felipe Álvarez (DIM) A User’s Guide to Riemannian Newton-Type Methods 21/23

Abstract differential geometry setting for R-Newton Other explicit examples

Cone of Positive Semidefinite Matrices

Manifold: M = Sn

++ cone of symmetric positive definite matrices.

Tangent space: TPSn

++ ≃ Sn (Sn space symmetric matrices).

Hessian Metric: g(∆1, ∆2) = trace(∇2ϕ(P)∆1∆2) with ϕ barrier

• n Sn

++.

Taking ϕ(P) = − log(det(P)): g(∆1, ∆2) = trace (P−1∆1P−1∆2) Goal: minimize f : Sn → R over M Exponential map: expP[∆] = P1/2 exp(P−1/2∆P−1/2)P1/2 Vector field: V(P) = P∇f(P)P Newton direction ∆ at P must satisfy: V ′(P)∆ = ∇2f(P)∆ + 1

2(P−1∆∇f(P) + ∇f(P)∆P−1) = −P∇f(P)P

Felipe Álvarez (DIM) A User’s Guide to Riemannian Newton-Type Methods 21/23

Abstract differential geometry setting for R-Newton Other explicit examples

Unitary Generalized Simplex on Sn

++ Manifold: M = {P ∈ Sn : Trace(P) = 1, P ∈ Sn

++} .

Tangent space: TPM = {∆ ∈ Sn : Trace(∆) = n

i=1 λi(∆) = 0}.

ϕ(P) = − log(det(P)). Exponential map: expP[∆] =

1

Trace(P exp(P−1/2∆P−1/2))P1/2 exp(P−1/2∆P−1/2)P1/2

Felipe Álvarez (DIM) A User’s Guide to Riemannian Newton-Type Methods 22/23

Abstract differential geometry setting for R-Newton Other explicit examples

Unitary Generalized Simplex on Sn

++ Manifold: M = {P ∈ Sn : Trace(P) = 1, P ∈ Sn

++} .

Tangent space: TPM = {∆ ∈ Sn : Trace(∆) = n

i=1 λi(∆) = 0}.

ϕ(P) = − log(det(P)). Exponential map: expP[∆] =

1

Trace(P exp(P−1/2∆P−1/2))P1/2 exp(P−1/2∆P−1/2)P1/2

Felipe Álvarez (DIM) A User’s Guide to Riemannian Newton-Type Methods 22/23

Abstract differential geometry setting for R-Newton Other explicit examples

Open problems

Drop out completeness ? Manifolds with boundary. Globalization: pk+1 = exppk[tkvk] for some scalar parameter tk > 0.

Felipe Álvarez (DIM) A User’s Guide to Riemannian Newton-Type Methods 23/23