Hierarchical Clustering on Special Manifolds Motivation Background - - PowerPoint PPT Presentation

Hierarchical Clustering on Special Manifolds Markos & Menexes Motivation Background

Manifolds Special Manifolds

HCA on Special Manifolds

Distance & Mean Intrinsic Case Extrinsic Case HCA Algorithms

Experiments Summary & Future Work References

Hierarchical Clustering on Special Manifolds

Angelos Markos1 George Menexes2

1Democritus University of Thrace, Greece 2Aristotle University of Thessaloniki, Greece

February 11th - CARME 2011

Markos & Menexes Hierarchical Clustering on Special Manifolds

Hierarchical Clustering on Special Manifolds Markos & Menexes Motivation Background

Manifolds Special Manifolds

HCA on Special Manifolds

Distance & Mean Intrinsic Case Extrinsic Case HCA Algorithms

Experiments Summary & Future Work References

Motivation

Let Xi, i = 1, . . . , n denote n matrices of size (m × k) such that X′X = Ik. The rows of each Xi correspond to a low-dimensional representation of different “views” of the same object and the columns refer to the same

• variables. The whole dataset can be seen as a collection of
• rthonormal bases or subspaces.

For example, suppose we have multiple images of the same person where each image is represented by a vector. A low-dimensional representation of this image set spans a subspace in the so-called image space. We address the problem of clustering sets of objects, where object-specific subspaces instead of vectors are compared.We should take into account the specific geometry of the space of orthonormal matrices.

Markos & Menexes Hierarchical Clustering on Special Manifolds

Hierarchical Clustering on Special Manifolds Markos & Menexes Motivation Background

Manifolds Special Manifolds

HCA on Special Manifolds

Distance & Mean Intrinsic Case Extrinsic Case HCA Algorithms

Experiments Summary & Future Work References

Motivation

Let Xi, i = 1, . . . , n denote n matrices of size (m × k) such that X′X = Ik. The rows of each Xi correspond to a low-dimensional representation of different “views” of the same object and the columns refer to the same

• variables. The whole dataset can be seen as a collection of
• rthonormal bases or subspaces.

For example, suppose we have multiple images of the same person where each image is represented by a vector. A low-dimensional representation of this image set spans a subspace in the so-called image space. We address the problem of clustering sets of objects, where object-specific subspaces instead of vectors are compared.We should take into account the specific geometry of the space of orthonormal matrices.

Markos & Menexes Hierarchical Clustering on Special Manifolds

Hierarchical Clustering on Special Manifolds Markos & Menexes Motivation Background

Manifolds Special Manifolds

HCA on Special Manifolds

Distance & Mean Intrinsic Case Extrinsic Case HCA Algorithms

Experiments Summary & Future Work References

Motivation

Let Xi, i = 1, . . . , n denote n matrices of size (m × k) such that X′X = Ik. The rows of each Xi correspond to a low-dimensional representation of different “views” of the same object and the columns refer to the same

• variables. The whole dataset can be seen as a collection of
• rthonormal bases or subspaces.

For example, suppose we have multiple images of the same person where each image is represented by a vector. A low-dimensional representation of this image set spans a subspace in the so-called image space. We address the problem of clustering sets of objects, where object-specific subspaces instead of vectors are compared.We should take into account the specific geometry of the space of orthonormal matrices.

Markos & Menexes Hierarchical Clustering on Special Manifolds

Hierarchical Clustering on Special Manifolds Markos & Menexes Motivation Background

Manifolds Special Manifolds

HCA on Special Manifolds

Distance & Mean Intrinsic Case Extrinsic Case HCA Algorithms

Experiments Summary & Future Work References

Motivation

Let Xi, i = 1, . . . , n denote n matrices of size (m × k) such that X′X = Ik. The rows of each Xi correspond to a low-dimensional representation of different “views” of the same object and the columns refer to the same

• variables. The whole dataset can be seen as a collection of
• rthonormal bases or subspaces.

For example, suppose we have multiple images of the same person where each image is represented by a vector. A low-dimensional representation of this image set spans a subspace in the so-called image space. We address the problem of clustering sets of objects, where object-specific subspaces instead of vectors are compared.We should take into account the specific geometry of the space of orthonormal matrices.

Markos & Menexes Hierarchical Clustering on Special Manifolds

Hierarchical Clustering on Special Manifolds Markos & Menexes Motivation Background

Manifolds Special Manifolds

HCA on Special Manifolds

Distance & Mean Intrinsic Case Extrinsic Case HCA Algorithms

Experiments Summary & Future Work References

Motivation

Let Xi, i = 1, . . . , n denote n matrices of size (m × k) such that X′X = Ik. The rows of each Xi correspond to a low-dimensional representation of different “views” of the same object and the columns refer to the same

• variables. The whole dataset can be seen as a collection of
• rthonormal bases or subspaces.

For example, suppose we have multiple images of the same person where each image is represented by a vector. A low-dimensional representation of this image set spans a subspace in the so-called image space. We address the problem of clustering sets of objects, where object-specific subspaces instead of vectors are compared.We should take into account the specific geometry of the space of orthonormal matrices.

Markos & Menexes Hierarchical Clustering on Special Manifolds

Hierarchical Clustering on Special Manifolds Markos & Menexes Motivation Background

Manifolds Special Manifolds

HCA on Special Manifolds

Distance & Mean Intrinsic Case Extrinsic Case HCA Algorithms

Experiments Summary & Future Work References

Outline

1 Motivation 2 Background

Manifolds Special Manifolds

3 HCA on Special Manifolds

Distance & Mean Intrinsic Case Extrinsic Case HCA Algorithms

4 Experiments 5 Summary & Future Work 6 References

Markos & Menexes Hierarchical Clustering on Special Manifolds

Hierarchical Clustering on Special Manifolds Markos & Menexes Motivation Background

Manifolds Special Manifolds

HCA on Special Manifolds

Distance & Mean Intrinsic Case Extrinsic Case HCA Algorithms

Experiments Summary & Future Work References

Manifolds (Boothby, 2002)

A manifold is a topological space that is locally similar (homeomorphic) to an open set in a Euclidean space. The shortest distance between two points is a geodesic distance.

Figure: A two dimensional manifold embedded in R3

Markos & Menexes Hierarchical Clustering on Special Manifolds

Hierarchical Clustering on Special Manifolds Markos & Menexes Motivation Background

Manifolds Special Manifolds

HCA on Special Manifolds

Distance & Mean Intrinsic Case Extrinsic Case HCA Algorithms

Experiments Summary & Future Work References Markos & Menexes Hierarchical Clustering on Special Manifolds

Hierarchical Clustering on Special Manifolds Markos & Menexes Motivation Background

Manifolds Special Manifolds

HCA on Special Manifolds

Distance & Mean Intrinsic Case Extrinsic Case HCA Algorithms

Experiments Summary & Future Work References

Differentiable Manifolds (Boothby, 2002)

For differentiable manifolds, it is possible to define the derivatives of the curves on the manifold. The derivatives at a point X on the manifold M lie in a vector space TX, which is the tangent space at that point.

Figure: Basic geometry of a manifold and its tangent space at a point

Markos & Menexes Hierarchical Clustering on Special Manifolds

Hierarchical Clustering on Special Manifolds Markos & Menexes Motivation Background

Manifolds Special Manifolds

HCA on Special Manifolds

Distance & Mean Intrinsic Case Extrinsic Case HCA Algorithms

Experiments Summary & Future Work References

Riemannian Manifolds 1/4

A Riemmanian manifold can be defined as a manifold with an inner product structure (Riemmanian metric) at each point (Chikuse, 2003). The inner product induces a norm for the tangent vectors in the tangent space.

Figure: Basic geometry of a manifold and its tangent space at a point

Markos & Menexes Hierarchical Clustering on Special Manifolds

Hierarchical Clustering on Special Manifolds Markos & Menexes Motivation Background

Manifolds Special Manifolds

HCA on Special Manifolds

Distance & Mean Intrinsic Case Extrinsic Case HCA Algorithms

Experiments Summary & Future Work References

Riemannian Manifolds 2/4

A geodesic is a smooth curve that locally joins their points along the shortest path. The length of the geodesic is defined to be the Riemannian distance between the two points.

Figure: Basic geometry of a manifold and its tangent space at a point

Markos & Menexes Hierarchical Clustering on Special Manifolds

Hierarchical Clustering on Special Manifolds Markos & Menexes Motivation Background

Manifolds Special Manifolds

HCA on Special Manifolds

Distance & Mean Intrinsic Case Extrinsic Case HCA Algorithms

Experiments Summary & Future Work References

Riemannian Manifolds 3/4

The exponential map, expX : TX → M, maps the vector y in the tangent space to the point on the manifold reached by the geodesic after unit time expX(y) = 1.

Figure: Basic geometry of a manifold and its tangent space at a point

Markos & Menexes Hierarchical Clustering on Special Manifolds

Hierarchical Clustering on Special Manifolds Markos & Menexes Motivation Background

Manifolds Special Manifolds

HCA on Special Manifolds

Distance & Mean Intrinsic Case Extrinsic Case HCA Algorithms

Experiments Summary & Future Work References

Riemannian Manifolds 4/4

The inverse exponential mapping logX : M → TX takes the point Y on the manifold and returns it on the tangent space. It is uniquely defined only around the neighborhood of the point X.

Figure: Basic geometry of a manifold and its tangent space at a point

Markos & Menexes Hierarchical Clustering on Special Manifolds

Hierarchical Clustering on Special Manifolds Markos & Menexes Motivation Background

Manifolds Special Manifolds

HCA on Special Manifolds

Distance & Mean Intrinsic Case Extrinsic Case HCA Algorithms

Experiments Summary & Future Work References

Outline

1 Motivation 2 Background

Manifolds Special Manifolds

3 HCA on Special Manifolds

Distance & Mean Intrinsic Case Extrinsic Case HCA Algorithms

4 Experiments 5 Summary & Future Work 6 References

Markos & Menexes Hierarchical Clustering on Special Manifolds

Hierarchical Clustering on Special Manifolds Markos & Menexes Motivation Background

Manifolds Special Manifolds

HCA on Special Manifolds

Distance & Mean Intrinsic Case Extrinsic Case HCA Algorithms

Experiments Summary & Future Work References

Special Manifolds

We consider two differentiable manifolds with well-defined mathematical properties, the Stiefel and Grassmann

• manifolds. A good introduction to their geometry can be

found in Edelman et al. (1999). In terms of their (differential) topology, the special manifolds can be described

• a. as embedded submanifolds of the real Euclidean space
• b. as quotient spaces of the orthogonal group under

different equivalence relations. The equivalence classes on special manifolds induce some nice mathematical properties and make geodesic distance computable.

Markos & Menexes Hierarchical Clustering on Special Manifolds

Hierarchical Clustering on Special Manifolds Markos & Menexes Motivation Background

Manifolds Special Manifolds

HCA on Special Manifolds

Distance & Mean Intrinsic Case Extrinsic Case HCA Algorithms

Experiments Summary & Future Work References

Special Manifolds

We consider two differentiable manifolds with well-defined mathematical properties, the Stiefel and Grassmann

• manifolds. A good introduction to their geometry can be

found in Edelman et al. (1999). In terms of their (differential) topology, the special manifolds can be described

• a. as embedded submanifolds of the real Euclidean space
• b. as quotient spaces of the orthogonal group under

different equivalence relations. The equivalence classes on special manifolds induce some nice mathematical properties and make geodesic distance computable.

Markos & Menexes Hierarchical Clustering on Special Manifolds

Hierarchical Clustering on Special Manifolds Markos & Menexes Motivation Background

Manifolds Special Manifolds

HCA on Special Manifolds

Distance & Mean Intrinsic Case Extrinsic Case HCA Algorithms

Experiments Summary & Future Work References

Special Manifolds

We consider two differentiable manifolds with well-defined mathematical properties, the Stiefel and Grassmann

• manifolds. A good introduction to their geometry can be

found in Edelman et al. (1999). In terms of their (differential) topology, the special manifolds can be described

• a. as embedded submanifolds of the real Euclidean space
• b. as quotient spaces of the orthogonal group under

different equivalence relations. The equivalence classes on special manifolds induce some nice mathematical properties and make geodesic distance computable.

Markos & Menexes Hierarchical Clustering on Special Manifolds

Hierarchical Clustering on Special Manifolds Markos & Menexes Motivation Background

Manifolds Special Manifolds

HCA on Special Manifolds

Distance & Mean Intrinsic Case Extrinsic Case HCA Algorithms

Experiments Summary & Future Work References

Special Manifolds

We consider two differentiable manifolds with well-defined mathematical properties, the Stiefel and Grassmann

• manifolds. A good introduction to their geometry can be

found in Edelman et al. (1999). In terms of their (differential) topology, the special manifolds can be described

• a. as embedded submanifolds of the real Euclidean space
• b. as quotient spaces of the orthogonal group under

different equivalence relations. The equivalence classes on special manifolds induce some nice mathematical properties and make geodesic distance computable.

Markos & Menexes Hierarchical Clustering on Special Manifolds

Hierarchical Clustering on Special Manifolds Markos & Menexes Motivation Background

Manifolds Special Manifolds

HCA on Special Manifolds

Distance & Mean Intrinsic Case Extrinsic Case HCA Algorithms

Experiments Summary & Future Work References

Special Manifolds

We consider two differentiable manifolds with well-defined mathematical properties, the Stiefel and Grassmann

• manifolds. A good introduction to their geometry can be

found in Edelman et al. (1999). In terms of their (differential) topology, the special manifolds can be described

• a. as embedded submanifolds of the real Euclidean space
• b. as quotient spaces of the orthogonal group under

different equivalence relations. The equivalence classes on special manifolds induce some nice mathematical properties and make geodesic distance computable.

Markos & Menexes Hierarchical Clustering on Special Manifolds

Hierarchical Clustering on Special Manifolds Markos & Menexes Motivation Background

Manifolds Special Manifolds

HCA on Special Manifolds

Distance & Mean Intrinsic Case Extrinsic Case HCA Algorithms

Experiments Summary & Future Work References

The Stiefel Manifold

Let Vk,m denote the Stiefel manifold which is the space of k orthonormal vectors in Rm represented by the set of m × k (k ≤ m) matrices X such that X′X = Ik, where Ik is the k × k identity matrix (Chikuse, 2003). For m = k, Vk,m is the orthogonal group O(m) of m × m

• rthonormal matrices.

The Stiefel manifold may be thought of as the quotient space O(m)/O(m − k) with respect to the group of left-orthogonal transformations X → HX for H ∈ O(m).

Markos & Menexes Hierarchical Clustering on Special Manifolds

Hierarchical Clustering on Special Manifolds Markos & Menexes Motivation Background

Manifolds Special Manifolds

HCA on Special Manifolds

Distance & Mean Intrinsic Case Extrinsic Case HCA Algorithms

Experiments Summary & Future Work References

The Stiefel Manifold

Let Vk,m denote the Stiefel manifold which is the space of k orthonormal vectors in Rm represented by the set of m × k (k ≤ m) matrices X such that X′X = Ik, where Ik is the k × k identity matrix (Chikuse, 2003). For m = k, Vk,m is the orthogonal group O(m) of m × m

• rthonormal matrices.

The Stiefel manifold may be thought of as the quotient space O(m)/O(m − k) with respect to the group of left-orthogonal transformations X → HX for H ∈ O(m).

Markos & Menexes Hierarchical Clustering on Special Manifolds

Hierarchical Clustering on Special Manifolds Markos & Menexes Motivation Background

Manifolds Special Manifolds

HCA on Special Manifolds

Distance & Mean Intrinsic Case Extrinsic Case HCA Algorithms

Experiments Summary & Future Work References

The Stiefel Manifold

Let Vk,m denote the Stiefel manifold which is the space of k orthonormal vectors in Rm represented by the set of m × k (k ≤ m) matrices X such that X′X = Ik, where Ik is the k × k identity matrix (Chikuse, 2003). For m = k, Vk,m is the orthogonal group O(m) of m × m

• rthonormal matrices.

The Stiefel manifold may be thought of as the quotient space O(m)/O(m − k) with respect to the group of left-orthogonal transformations X → HX for H ∈ O(m).

Markos & Menexes Hierarchical Clustering on Special Manifolds

Hierarchical Clustering on Special Manifolds Markos & Menexes Motivation Background

Manifolds Special Manifolds

HCA on Special Manifolds

Distance & Mean Intrinsic Case Extrinsic Case HCA Algorithms

Experiments Summary & Future Work References

The Stiefel Manifold

Let Vk,m denote the Stiefel manifold which is the space of k orthonormal vectors in Rm represented by the set of m × k (k ≤ m) matrices X such that X′X = Ik, where Ik is the k × k identity matrix (Chikuse, 2003). For m = k, Vk,m is the orthogonal group O(m) of m × m

• rthonormal matrices.

The Stiefel manifold may be thought of as the quotient space O(m)/O(m − k) with respect to the group of left-orthogonal transformations X → HX for H ∈ O(m).

Markos & Menexes Hierarchical Clustering on Special Manifolds

Hierarchical Clustering on Special Manifolds Markos & Menexes Motivation Background

Manifolds Special Manifolds

HCA on Special Manifolds

Distance & Mean Intrinsic Case Extrinsic Case HCA Algorithms

Experiments Summary & Future Work References

The Stiefel Manifold

Let Vk,m denote the Stiefel manifold which is the space of k orthonormal vectors in Rm represented by the set of m × k (k ≤ m) matrices X such that X′X = Ik, where Ik is the k × k identity matrix (Chikuse, 2003). For m = k, Vk,m is the orthogonal group O(m) of m × m

• rthonormal matrices.

The Stiefel manifold may be thought of as the quotient space O(m)/O(m − k) with respect to the group of left-orthogonal transformations X → HX for H ∈ O(m).

Markos & Menexes Hierarchical Clustering on Special Manifolds

Hierarchical Clustering on Special Manifolds Markos & Menexes Motivation Background

Manifolds Special Manifolds

HCA on Special Manifolds

Distance & Mean Intrinsic Case Extrinsic Case HCA Algorithms

Experiments Summary & Future Work References

The Grassmann Manifold

The Grassmann manifold Gk,m is the space whose points are k-dimensional linear subspaces of Rm (k-planes in Rm, containing the origin). To each k-plane ν in Gk,m corresponds a unique m × m orthogonal projection matrix P idempotent of rank k onto ν. If the columns of an m × k matrix Y span ν, then Y Y ′ = P. (Mardia & Jupp, 2009) The Grassmann manifold can be identified by a quotient representation O(m)/O(k) × O(m − k). Using the quotient representation of Stiefel manifolds, Gk,m = Vk,m/O(k) with respect to the group of right-orthogonal transformations X → XH for H ∈ O(k). We can represent subspaces using their unique orthogonal projections

Markos & Menexes Hierarchical Clustering on Special Manifolds

Hierarchical Clustering on Special Manifolds Markos & Menexes Motivation Background

Manifolds Special Manifolds

HCA on Special Manifolds

Distance & Mean Intrinsic Case Extrinsic Case HCA Algorithms

Experiments Summary & Future Work References

The Grassmann Manifold

The Grassmann manifold Gk,m is the space whose points are k-dimensional linear subspaces of Rm (k-planes in Rm, containing the origin). To each k-plane ν in Gk,m corresponds a unique m × m orthogonal projection matrix P idempotent of rank k onto ν. If the columns of an m × k matrix Y span ν, then Y Y ′ = P. (Mardia & Jupp, 2009) The Grassmann manifold can be identified by a quotient representation O(m)/O(k) × O(m − k). Using the quotient representation of Stiefel manifolds, Gk,m = Vk,m/O(k) with respect to the group of right-orthogonal transformations X → XH for H ∈ O(k). We can represent subspaces using their unique orthogonal projections

Markos & Menexes Hierarchical Clustering on Special Manifolds

Hierarchical Clustering on Special Manifolds Markos & Menexes Motivation Background

Manifolds Special Manifolds

HCA on Special Manifolds

Distance & Mean Intrinsic Case Extrinsic Case HCA Algorithms

Experiments Summary & Future Work References

The Grassmann Manifold

The Grassmann manifold Gk,m is the space whose points are k-dimensional linear subspaces of Rm (k-planes in Rm, containing the origin). To each k-plane ν in Gk,m corresponds a unique m × m orthogonal projection matrix P idempotent of rank k onto ν. If the columns of an m × k matrix Y span ν, then Y Y ′ = P. (Mardia & Jupp, 2009) The Grassmann manifold can be identified by a quotient representation O(m)/O(k) × O(m − k). Using the quotient representation of Stiefel manifolds, Gk,m = Vk,m/O(k) with respect to the group of right-orthogonal transformations X → XH for H ∈ O(k). We can represent subspaces using their unique orthogonal projections

Markos & Menexes Hierarchical Clustering on Special Manifolds

Hierarchical Clustering on Special Manifolds Markos & Menexes Motivation Background

Manifolds Special Manifolds

HCA on Special Manifolds

Distance & Mean Intrinsic Case Extrinsic Case HCA Algorithms

Experiments Summary & Future Work References

The Grassmann Manifold

The Grassmann manifold Gk,m is the space whose points are k-dimensional linear subspaces of Rm (k-planes in Rm, containing the origin). To each k-plane ν in Gk,m corresponds a unique m × m orthogonal projection matrix P idempotent of rank k onto ν. If the columns of an m × k matrix Y span ν, then Y Y ′ = P. (Mardia & Jupp, 2009) The Grassmann manifold can be identified by a quotient representation O(m)/O(k) × O(m − k). Using the quotient representation of Stiefel manifolds, Gk,m = Vk,m/O(k) with respect to the group of right-orthogonal transformations X → XH for H ∈ O(k). We can represent subspaces using their unique orthogonal projections

Markos & Menexes Hierarchical Clustering on Special Manifolds

Hierarchical Clustering on Special Manifolds Markos & Menexes Motivation Background

Manifolds Special Manifolds

HCA on Special Manifolds

Distance & Mean Intrinsic Case Extrinsic Case HCA Algorithms

Experiments Summary & Future Work References

The Grassmann Manifold

The Grassmann manifold Gk,m is the space whose points are k-dimensional linear subspaces of Rm (k-planes in Rm, containing the origin). To each k-plane ν in Gk,m corresponds a unique m × m orthogonal projection matrix P idempotent of rank k onto ν. If the columns of an m × k matrix Y span ν, then Y Y ′ = P. (Mardia & Jupp, 2009) The Grassmann manifold can be identified by a quotient representation O(m)/O(k) × O(m − k). Using the quotient representation of Stiefel manifolds, Gk,m = Vk,m/O(k) with respect to the group of right-orthogonal transformations X → XH for H ∈ O(k). We can represent subspaces using their unique orthogonal projections

Markos & Menexes Hierarchical Clustering on Special Manifolds

Hierarchical Clustering on Special Manifolds Markos & Menexes Motivation Background

Manifolds Special Manifolds

HCA on Special Manifolds

Distance & Mean Intrinsic Case Extrinsic Case HCA Algorithms

Experiments Summary & Future Work References

The Grassmann Manifold

The Grassmann manifold Gk,m is the space whose points are k-dimensional linear subspaces of Rm (k-planes in Rm, containing the origin). To each k-plane ν in Gk,m corresponds a unique m × m orthogonal projection matrix P idempotent of rank k onto ν. If the columns of an m × k matrix Y span ν, then Y Y ′ = P. (Mardia & Jupp, 2009) The Grassmann manifold can be identified by a quotient representation O(m)/O(k) × O(m − k). Using the quotient representation of Stiefel manifolds, Gk,m = Vk,m/O(k) with respect to the group of right-orthogonal transformations X → XH for H ∈ O(k). We can represent subspaces using their unique orthogonal projections

Markos & Menexes Hierarchical Clustering on Special Manifolds

Hierarchical Clustering on Special Manifolds Markos & Menexes Motivation Background

Manifolds Special Manifolds

HCA on Special Manifolds

Distance & Mean Intrinsic Case Extrinsic Case HCA Algorithms

Experiments Summary & Future Work References

The Grassmann Manifold

The Grassmann manifold Gk,m is the space whose points are k-dimensional linear subspaces of Rm (k-planes in Rm, containing the origin). To each k-plane ν in Gk,m corresponds a unique m × m orthogonal projection matrix P idempotent of rank k onto ν. If the columns of an m × k matrix Y span ν, then Y Y ′ = P. (Mardia & Jupp, 2009) The Grassmann manifold can be identified by a quotient representation O(m)/O(k) × O(m − k). Using the quotient representation of Stiefel manifolds, Gk,m = Vk,m/O(k) with respect to the group of right-orthogonal transformations X → XH for H ∈ O(k). We can represent subspaces using their unique orthogonal projections

Markos & Menexes Hierarchical Clustering on Special Manifolds

Hierarchical Clustering on Special Manifolds Markos & Menexes Motivation Background

Manifolds Special Manifolds

HCA on Special Manifolds

Distance & Mean Intrinsic Case Extrinsic Case HCA Algorithms

Experiments Summary & Future Work References

Statistics on Special Manifolds - Applications

Statistics on Riemannian manifolds have found wide applicability in Shape Analysis (Goodall & Mardia 1999, Patrangenaru & Mardia 2003). The Grassmann manifold structure of the affine shape space was exploited in Begelfor & Werman (2006) to perform affine invariant clustering of shapes. Srivasatava & Klassen (2004) exploited the geometry of the Grassmann manifold for subspace tracking in array signal processing applications. Turaga & Srivastava (2010) showed how a large class of problems drawn from face, activity, and object recognition can be recast as statistical inference problems on the Stiefel and/or Grassmann manifolds.

Markos & Menexes Hierarchical Clustering on Special Manifolds

Hierarchical Clustering on Special Manifolds Markos & Menexes Motivation Background

Manifolds Special Manifolds

HCA on Special Manifolds

Distance & Mean Intrinsic Case Extrinsic Case HCA Algorithms

Experiments Summary & Future Work References

Statistics on Special Manifolds - Applications

Statistics on Riemannian manifolds have found wide applicability in Shape Analysis (Goodall & Mardia 1999, Patrangenaru & Mardia 2003). The Grassmann manifold structure of the affine shape space was exploited in Begelfor & Werman (2006) to perform affine invariant clustering of shapes. Srivasatava & Klassen (2004) exploited the geometry of the Grassmann manifold for subspace tracking in array signal processing applications. Turaga & Srivastava (2010) showed how a large class of problems drawn from face, activity, and object recognition can be recast as statistical inference problems on the Stiefel and/or Grassmann manifolds.

Markos & Menexes Hierarchical Clustering on Special Manifolds

Hierarchical Clustering on Special Manifolds Markos & Menexes Motivation Background

Manifolds Special Manifolds

HCA on Special Manifolds

Distance & Mean Intrinsic Case Extrinsic Case HCA Algorithms

Experiments Summary & Future Work References

Statistics on Special Manifolds - Applications

Statistics on Riemannian manifolds have found wide applicability in Shape Analysis (Goodall & Mardia 1999, Patrangenaru & Mardia 2003). The Grassmann manifold structure of the affine shape space was exploited in Begelfor & Werman (2006) to perform affine invariant clustering of shapes. Srivasatava & Klassen (2004) exploited the geometry of the Grassmann manifold for subspace tracking in array signal processing applications. Turaga & Srivastava (2010) showed how a large class of problems drawn from face, activity, and object recognition can be recast as statistical inference problems on the Stiefel and/or Grassmann manifolds.

Markos & Menexes Hierarchical Clustering on Special Manifolds

Hierarchical Clustering on Special Manifolds Markos & Menexes Motivation Background

Manifolds Special Manifolds

HCA on Special Manifolds

Distance & Mean Intrinsic Case Extrinsic Case HCA Algorithms

Experiments Summary & Future Work References

Statistics on Special Manifolds - Applications

Statistics on Riemannian manifolds have found wide applicability in Shape Analysis (Goodall & Mardia 1999, Patrangenaru & Mardia 2003). The Grassmann manifold structure of the affine shape space was exploited in Begelfor & Werman (2006) to perform affine invariant clustering of shapes. Srivasatava & Klassen (2004) exploited the geometry of the Grassmann manifold for subspace tracking in array signal processing applications. Turaga & Srivastava (2010) showed how a large class of problems drawn from face, activity, and object recognition can be recast as statistical inference problems on the Stiefel and/or Grassmann manifolds.

Markos & Menexes Hierarchical Clustering on Special Manifolds

Hierarchical Clustering on Special Manifolds Markos & Menexes Motivation Background

Manifolds Special Manifolds

HCA on Special Manifolds

Distance & Mean Intrinsic Case Extrinsic Case HCA Algorithms

Experiments Summary & Future Work References

Statistics on Special Manifolds - Applications

Statistics on Riemannian manifolds have found wide applicability in Shape Analysis (Goodall & Mardia 1999, Patrangenaru & Mardia 2003). The Grassmann manifold structure of the affine shape space was exploited in Begelfor & Werman (2006) to perform affine invariant clustering of shapes. Srivasatava & Klassen (2004) exploited the geometry of the Grassmann manifold for subspace tracking in array signal processing applications. Turaga & Srivastava (2010) showed how a large class of problems drawn from face, activity, and object recognition can be recast as statistical inference problems on the Stiefel and/or Grassmann manifolds.

Markos & Menexes Hierarchical Clustering on Special Manifolds

Hierarchical Clustering on Special Manifolds Markos & Menexes Motivation Background

Manifolds Special Manifolds

HCA on Special Manifolds

Distance & Mean Intrinsic Case Extrinsic Case HCA Algorithms

Experiments Summary & Future Work References

Statistics on Special Manifolds - Applications

Statistics on Riemannian manifolds have found wide applicability in Shape Analysis (Goodall & Mardia 1999, Patrangenaru & Mardia 2003). The Grassmann manifold structure of the affine shape space was exploited in Begelfor & Werman (2006) to perform affine invariant clustering of shapes. Srivasatava & Klassen (2004) exploited the geometry of the Grassmann manifold for subspace tracking in array signal processing applications. Turaga & Srivastava (2010) showed how a large class of problems drawn from face, activity, and object recognition can be recast as statistical inference problems on the Stiefel and/or Grassmann manifolds.

Markos & Menexes Hierarchical Clustering on Special Manifolds

Hierarchical Clustering on Special Manifolds Markos & Menexes Motivation Background

Manifolds Special Manifolds

HCA on Special Manifolds

Distance & Mean Intrinsic Case Extrinsic Case HCA Algorithms

Experiments Summary & Future Work References

Statistics on Special Manifolds - Applications

Statistics on Riemannian manifolds have found wide applicability in Shape Analysis (Goodall & Mardia 1999, Patrangenaru & Mardia 2003). The Grassmann manifold structure of the affine shape space was exploited in Begelfor & Werman (2006) to perform affine invariant clustering of shapes. Srivasatava & Klassen (2004) exploited the geometry of the Grassmann manifold for subspace tracking in array signal processing applications. Turaga & Srivastava (2010) showed how a large class of problems drawn from face, activity, and object recognition can be recast as statistical inference problems on the Stiefel and/or Grassmann manifolds.

Markos & Menexes Hierarchical Clustering on Special Manifolds

Hierarchical Clustering on Special Manifolds Markos & Menexes Motivation Background

Manifolds Special Manifolds

HCA on Special Manifolds

Distance & Mean Intrinsic Case Extrinsic Case HCA Algorithms

Experiments Summary & Future Work References

Hierarchical Clustering on Special Manifolds

Need to define: a distance metric applicable to subspaces (points on Gk,m). Single, complete and average linkage hierarchical clustering. a suitable notion of the mean on Riemannian manifolds (intrinsic or extrinsic). Centroid-linkage, Ward-like Clustering (?).

Markos & Menexes Hierarchical Clustering on Special Manifolds

Hierarchical Clustering on Special Manifolds Markos & Menexes Motivation Background

Manifolds Special Manifolds

HCA on Special Manifolds

Distance & Mean Intrinsic Case Extrinsic Case HCA Algorithms

Experiments Summary & Future Work References

Hierarchical Clustering on Special Manifolds

Need to define: a distance metric applicable to subspaces (points on Gk,m). Single, complete and average linkage hierarchical clustering. a suitable notion of the mean on Riemannian manifolds (intrinsic or extrinsic). Centroid-linkage, Ward-like Clustering (?).

Markos & Menexes Hierarchical Clustering on Special Manifolds

Hierarchical Clustering on Special Manifolds Markos & Menexes Motivation Background

Manifolds Special Manifolds

HCA on Special Manifolds

Distance & Mean Intrinsic Case Extrinsic Case HCA Algorithms

Experiments Summary & Future Work References

Outline

1 Motivation 2 Background

Manifolds Special Manifolds

3 HCA on Special Manifolds

Distance & Mean Intrinsic Case Extrinsic Case HCA Algorithms

4 Experiments 5 Summary & Future Work 6 References

Markos & Menexes Hierarchical Clustering on Special Manifolds

Hierarchical Clustering on Special Manifolds Markos & Menexes Motivation Background

Manifolds Special Manifolds

HCA on Special Manifolds

Distance & Mean Intrinsic Case Extrinsic Case HCA Algorithms

Experiments Summary & Future Work References

Measuring the Distance on Manifolds

Traditional distance measures, such as the Euclidean measure, are not reasonable to use when measuring distances between spaces. This point has been either missed or ignored in many simulation studies where inappropriate distance measures have been used (Larsson & Villani, 2001). The concept of principal angles is fundamental to understand the measure of closeness or similarity between two subspaces. Principal angles reflect the closeness of two subspaces in each individual dimension, while subspace distances reflect the distance of two subspaces along the Grassmann manifold or in embedding space. Distances on Gk,m have clear geometrical interpretation.

Markos & Menexes Hierarchical Clustering on Special Manifolds

Hierarchical Clustering on Special Manifolds Markos & Menexes Motivation Background

Manifolds Special Manifolds

HCA on Special Manifolds

Distance & Mean Intrinsic Case Extrinsic Case HCA Algorithms

Experiments Summary & Future Work References

Measuring the Distance on Manifolds

Traditional distance measures, such as the Euclidean measure, are not reasonable to use when measuring distances between spaces. This point has been either missed or ignored in many simulation studies where inappropriate distance measures have been used (Larsson & Villani, 2001). The concept of principal angles is fundamental to understand the measure of closeness or similarity between two subspaces. Principal angles reflect the closeness of two subspaces in each individual dimension, while subspace distances reflect the distance of two subspaces along the Grassmann manifold or in embedding space. Distances on Gk,m have clear geometrical interpretation.

Markos & Menexes Hierarchical Clustering on Special Manifolds

Hierarchical Clustering on Special Manifolds Markos & Menexes Motivation Background

Manifolds Special Manifolds

HCA on Special Manifolds

Distance & Mean Intrinsic Case Extrinsic Case HCA Algorithms

Experiments Summary & Future Work References

Measuring the Distance on Manifolds

Traditional distance measures, such as the Euclidean measure, are not reasonable to use when measuring distances between spaces. This point has been either missed or ignored in many simulation studies where inappropriate distance measures have been used (Larsson & Villani, 2001). The concept of principal angles is fundamental to understand the measure of closeness or similarity between two subspaces. Principal angles reflect the closeness of two subspaces in each individual dimension, while subspace distances reflect the distance of two subspaces along the Grassmann manifold or in embedding space. Distances on Gk,m have clear geometrical interpretation.

Markos & Menexes Hierarchical Clustering on Special Manifolds

Hierarchical Clustering on Special Manifolds Markos & Menexes Motivation Background

Manifolds Special Manifolds

HCA on Special Manifolds

Distance & Mean Intrinsic Case Extrinsic Case HCA Algorithms

Experiments Summary & Future Work References

Measuring the Distance on Manifolds

Traditional distance measures, such as the Euclidean measure, are not reasonable to use when measuring distances between spaces. This point has been either missed or ignored in many simulation studies where inappropriate distance measures have been used (Larsson & Villani, 2001). The concept of principal angles is fundamental to understand the measure of closeness or similarity between two subspaces. Principal angles reflect the closeness of two subspaces in each individual dimension, while subspace distances reflect the distance of two subspaces along the Grassmann manifold or in embedding space. Distances on Gk,m have clear geometrical interpretation.

Markos & Menexes Hierarchical Clustering on Special Manifolds

Hierarchical Clustering on Special Manifolds Markos & Menexes Motivation Background

Manifolds Special Manifolds

HCA on Special Manifolds

Distance & Mean Intrinsic Case Extrinsic Case HCA Algorithms

Experiments Summary & Future Work References

Measuring the Distance on Manifolds

Traditional distance measures, such as the Euclidean measure, are not reasonable to use when measuring distances between spaces. This point has been either missed or ignored in many simulation studies where inappropriate distance measures have been used (Larsson & Villani, 2001). The concept of principal angles is fundamental to understand the measure of closeness or similarity between two subspaces. Principal angles reflect the closeness of two subspaces in each individual dimension, while subspace distances reflect the distance of two subspaces along the Grassmann manifold or in embedding space. Distances on Gk,m have clear geometrical interpretation.

Markos & Menexes Hierarchical Clustering on Special Manifolds

Hierarchical Clustering on Special Manifolds Markos & Menexes Motivation Background

Manifolds Special Manifolds

HCA on Special Manifolds

Distance & Mean Intrinsic Case Extrinsic Case HCA Algorithms

Experiments Summary & Future Work References

Principal angles

Let X and Y be two orthonormal matrices of size m by k. The principal angles or canonical angles 0 ≤ θ1 ≤ . . . θm ≤ π/2 between span(X) and span(Y ) are defined recursively by cos(θk) = max

uk∈span(X)

max

vk∈span(Y ) u′ kvk

subject to u′

kuk = 1, v′ kvk = 1,

u′

kui = 0, v′ kvi = 0, (i = 1, . . . , k − 1).

The principal angles can be computed from the SVD of X′Y (Bj¨

• rck & Golub, 1973),

X′Y = U(cos Θ)V ′ where U = [u1 . . . um], V = [v1 . . . vm], and cos Θ is the diagonal matrix cos Θ = diag(cos θ1 . . . cos θm).

Markos & Menexes Hierarchical Clustering on Special Manifolds

Hierarchical Clustering on Special Manifolds Markos & Menexes Motivation Background

Manifolds Special Manifolds

HCA on Special Manifolds

Distance & Mean Intrinsic Case Extrinsic Case HCA Algorithms

Experiments Summary & Future Work References

Principal angles

Let X and Y be two orthonormal matrices of size m by k. The principal angles or canonical angles 0 ≤ θ1 ≤ . . . θm ≤ π/2 between span(X) and span(Y ) are defined recursively by cos(θk) = max

uk∈span(X)

max

vk∈span(Y ) u′ kvk

subject to u′

kuk = 1, v′ kvk = 1,

u′

kui = 0, v′ kvi = 0, (i = 1, . . . , k − 1).

The principal angles can be computed from the SVD of X′Y (Bj¨

• rck & Golub, 1973),

X′Y = U(cos Θ)V ′ where U = [u1 . . . um], V = [v1 . . . vm], and cos Θ is the diagonal matrix cos Θ = diag(cos θ1 . . . cos θm).

Markos & Menexes Hierarchical Clustering on Special Manifolds

Hierarchical Clustering on Special Manifolds Markos & Menexes Motivation Background

Manifolds Special Manifolds

HCA on Special Manifolds

Distance & Mean Intrinsic Case Extrinsic Case HCA Algorithms

Experiments Summary & Future Work References

Outline

1 Motivation 2 Background

Manifolds Special Manifolds

3 HCA on Special Manifolds

Distance & Mean Intrinsic Case Extrinsic Case HCA Algorithms

4 Experiments 5 Summary & Future Work 6 References

Markos & Menexes Hierarchical Clustering on Special Manifolds

Hierarchical Clustering on Special Manifolds Markos & Menexes Motivation Background

Manifolds Special Manifolds

HCA on Special Manifolds

Distance & Mean Intrinsic Case Extrinsic Case HCA Algorithms

Experiments Summary & Future Work References

The geodesic distance

The geodesic distance or arc length is derived from the intrinsic geometry of Grassmann manifold. It is the length of the geodesic curve connecting two subspaces along the Grassmannian surface. dg(X, Y ) = q

• i=1

θ2

i

1/2 = θ2 Instead of using only the first principal angle, the full geometry

• f manifolds is taken into account. However, this distance is

not differentiable everywhere.

Markos & Menexes Hierarchical Clustering on Special Manifolds

Hierarchical Clustering on Special Manifolds Markos & Menexes Motivation Background

Manifolds Special Manifolds

HCA on Special Manifolds

Distance & Mean Intrinsic Case Extrinsic Case HCA Algorithms

Experiments Summary & Future Work References

The Karcher Mean

The Karcher Mean (Karcher, 1977) is an intrinsic mean on manifolds which minimizes the sum of squared geodesic distances. Algorithm (Begelfor & Werman 2006) Input: Points p1, . . . , pn ∈ G(k, m), ǫ (machine zero) Output: Karcher mean q

1 Set q = p1 2 Find A = 1 n

n

i=1 Logq(pi) 3 A < ǫ return q else, go to Step 4. 4 Find the SVD UΣV T = A and update

q → qV cos(Σ) + U sin(Σ) Go to Step 2. The Karcher Mean is unique if the points are clustered close together on the manifold (Berger, 2003).

Markos & Menexes Hierarchical Clustering on Special Manifolds

Hierarchical Clustering on Special Manifolds Markos & Menexes Motivation Background

Manifolds Special Manifolds

HCA on Special Manifolds

Distance & Mean Intrinsic Case Extrinsic Case HCA Algorithms

Experiments Summary & Future Work References

The Karcher Mean

The Karcher Mean (Karcher, 1977) is an intrinsic mean on manifolds which minimizes the sum of squared geodesic distances. Algorithm (Begelfor & Werman 2006) Input: Points p1, . . . , pn ∈ G(k, m), ǫ (machine zero) Output: Karcher mean q

1 Set q = p1 2 Find A = 1 n

n

i=1 Logq(pi) 3 A < ǫ return q else, go to Step 4. 4 Find the SVD UΣV T = A and update

q → qV cos(Σ) + U sin(Σ) Go to Step 2. The Karcher Mean is unique if the points are clustered close together on the manifold (Berger, 2003).

Markos & Menexes Hierarchical Clustering on Special Manifolds

Hierarchical Clustering on Special Manifolds Markos & Menexes Motivation Background

Manifolds Special Manifolds

HCA on Special Manifolds

Distance & Mean Intrinsic Case Extrinsic Case HCA Algorithms

Experiments Summary & Future Work References

The Karcher Mean

The Karcher Mean (Karcher, 1977) is an intrinsic mean on manifolds which minimizes the sum of squared geodesic distances. Algorithm (Begelfor & Werman 2006) Input: Points p1, . . . , pn ∈ G(k, m), ǫ (machine zero) Output: Karcher mean q

1 Set q = p1 2 Find A = 1 n

n

i=1 Logq(pi) 3 A < ǫ return q else, go to Step 4. 4 Find the SVD UΣV T = A and update

q → qV cos(Σ) + U sin(Σ) Go to Step 2. The Karcher Mean is unique if the points are clustered close together on the manifold (Berger, 2003).

Markos & Menexes Hierarchical Clustering on Special Manifolds

Hierarchical Clustering on Special Manifolds Markos & Menexes Motivation Background

Manifolds Special Manifolds

HCA on Special Manifolds

Distance & Mean Intrinsic Case Extrinsic Case HCA Algorithms

Experiments Summary & Future Work References

The Karcher Mean

The Karcher Mean (Karcher, 1977) is an intrinsic mean on manifolds which minimizes the sum of squared geodesic distances. Algorithm (Begelfor & Werman 2006) Input: Points p1, . . . , pn ∈ G(k, m), ǫ (machine zero) Output: Karcher mean q

1 Set q = p1 2 Find A = 1 n

n

i=1 Logq(pi) 3 A < ǫ return q else, go to Step 4. 4 Find the SVD UΣV T = A and update

q → qV cos(Σ) + U sin(Σ) Go to Step 2. The Karcher Mean is unique if the points are clustered close together on the manifold (Berger, 2003).

Markos & Menexes Hierarchical Clustering on Special Manifolds

Hierarchical Clustering on Special Manifolds Markos & Menexes Motivation Background

Manifolds Special Manifolds

HCA on Special Manifolds

Distance & Mean Intrinsic Case Extrinsic Case HCA Algorithms

Experiments Summary & Future Work References

The Karcher Mean

The Karcher Mean (Karcher, 1977) is an intrinsic mean on manifolds which minimizes the sum of squared geodesic distances. Algorithm (Begelfor & Werman 2006) Input: Points p1, . . . , pn ∈ G(k, m), ǫ (machine zero) Output: Karcher mean q

1 Set q = p1 2 Find A = 1 n

n

i=1 Logq(pi) 3 A < ǫ return q else, go to Step 4. 4 Find the SVD UΣV T = A and update

q → qV cos(Σ) + U sin(Σ) Go to Step 2. The Karcher Mean is unique if the points are clustered close together on the manifold (Berger, 2003).

Markos & Menexes Hierarchical Clustering on Special Manifolds

Hierarchical Clustering on Special Manifolds Markos & Menexes Motivation Background

Manifolds Special Manifolds

HCA on Special Manifolds

Distance & Mean Intrinsic Case Extrinsic Case HCA Algorithms

Experiments Summary & Future Work References

The Karcher Mean

The Karcher Mean (Karcher, 1977) is an intrinsic mean on manifolds which minimizes the sum of squared geodesic distances. Algorithm (Begelfor & Werman 2006) Input: Points p1, . . . , pn ∈ G(k, m), ǫ (machine zero) Output: Karcher mean q

1 Set q = p1 2 Find A = 1 n

n

i=1 Logq(pi) 3 A < ǫ return q else, go to Step 4. 4 Find the SVD UΣV T = A and update

q → qV cos(Σ) + U sin(Σ) Go to Step 2. The Karcher Mean is unique if the points are clustered close together on the manifold (Berger, 2003).

Markos & Menexes Hierarchical Clustering on Special Manifolds

Hierarchical Clustering on Special Manifolds Markos & Menexes Motivation Background

Manifolds Special Manifolds

HCA on Special Manifolds

Distance & Mean Intrinsic Case Extrinsic Case HCA Algorithms

Experiments Summary & Future Work References

The Karcher Mean

The Karcher Mean (Karcher, 1977) is an intrinsic mean on manifolds which minimizes the sum of squared geodesic distances. Algorithm (Begelfor & Werman 2006) Input: Points p1, . . . , pn ∈ G(k, m), ǫ (machine zero) Output: Karcher mean q

1 Set q = p1 2 Find A = 1 n

n

i=1 Logq(pi) 3 A < ǫ return q else, go to Step 4. 4 Find the SVD UΣV T = A and update

q → qV cos(Σ) + U sin(Σ) Go to Step 2. The Karcher Mean is unique if the points are clustered close together on the manifold (Berger, 2003).

Markos & Menexes Hierarchical Clustering on Special Manifolds

Hierarchical Clustering on Special Manifolds Markos & Menexes Motivation Background

Manifolds Special Manifolds

HCA on Special Manifolds

Distance & Mean Intrinsic Case Extrinsic Case HCA Algorithms

Experiments Summary & Future Work References

Outline

1 Motivation 2 Background

Manifolds Special Manifolds

3 HCA on Special Manifolds

Distance & Mean Intrinsic Case Extrinsic Case HCA Algorithms

4 Experiments 5 Summary & Future Work 6 References

Markos & Menexes Hierarchical Clustering on Special Manifolds

Hierarchical Clustering on Special Manifolds Markos & Menexes Motivation Background

Manifolds Special Manifolds

HCA on Special Manifolds

Distance & Mean Intrinsic Case Extrinsic Case HCA Algorithms

Experiments Summary & Future Work References

The projection metric 1/2

When Gk,m is defined as a submanifold of Euclidean space via a projection embedding, the projection metric on Gk,m is given in terms of the principal angles by (Edelman et al., 1999): dP (X, Y ) = sin θ2

Markos & Menexes Hierarchical Clustering on Special Manifolds

Hierarchical Clustering on Special Manifolds Markos & Menexes Motivation Background

Manifolds Special Manifolds

HCA on Special Manifolds

Distance & Mean Intrinsic Case Extrinsic Case HCA Algorithms

Experiments Summary & Future Work References

The projection metric 2/2

dP (X, Y ) is the distance proposed by Larsson & Villani (2001) as a measure of the distance between two cointegration spaces. dP (X, Y ) =

• m − tr(XX′Y Y ′) =
• m − X′Y 2

F

Note that tr(XX′Y Y ′) corresponds to a scalar product between two positive semidefinite matrices. d2

P (X, Y ) = tr(X⊥X⊥′Y Y ′) = COI(X⊥, Y ), where COI

is the co-inertia criterion, the numerator of both the RV and Tucker’s congruence coefficient for positive semidefinite matrices (Dray, 2008).

Markos & Menexes Hierarchical Clustering on Special Manifolds

Hierarchical Clustering on Special Manifolds Markos & Menexes Motivation Background

Manifolds Special Manifolds

HCA on Special Manifolds

Distance & Mean Intrinsic Case Extrinsic Case HCA Algorithms

Experiments Summary & Future Work References

The projection metric 2/2

dP (X, Y ) is the distance proposed by Larsson & Villani (2001) as a measure of the distance between two cointegration spaces. dP (X, Y ) =

• m − tr(XX′Y Y ′) =
• m − X′Y 2

F

Note that tr(XX′Y Y ′) corresponds to a scalar product between two positive semidefinite matrices. d2

P (X, Y ) = tr(X⊥X⊥′Y Y ′) = COI(X⊥, Y ), where COI

is the co-inertia criterion, the numerator of both the RV and Tucker’s congruence coefficient for positive semidefinite matrices (Dray, 2008).

Markos & Menexes Hierarchical Clustering on Special Manifolds

Hierarchical Clustering on Special Manifolds Markos & Menexes Motivation Background

Manifolds Special Manifolds

HCA on Special Manifolds

Distance & Mean Intrinsic Case Extrinsic Case HCA Algorithms

Experiments Summary & Future Work References

The projection metric 2/2

dP (X, Y ) is the distance proposed by Larsson & Villani (2001) as a measure of the distance between two cointegration spaces. dP (X, Y ) =

• m − tr(XX′Y Y ′) =
• m − X′Y 2

F

Note that tr(XX′Y Y ′) corresponds to a scalar product between two positive semidefinite matrices. d2

P (X, Y ) = tr(X⊥X⊥′Y Y ′) = COI(X⊥, Y ), where COI

is the co-inertia criterion, the numerator of both the RV and Tucker’s congruence coefficient for positive semidefinite matrices (Dray, 2008).

Markos & Menexes Hierarchical Clustering on Special Manifolds

Hierarchical Clustering on Special Manifolds Markos & Menexes Motivation Background

Manifolds Special Manifolds

HCA on Special Manifolds

Distance & Mean Intrinsic Case Extrinsic Case HCA Algorithms

Experiments Summary & Future Work References

The projection metric 2/2

dP (X, Y ) is the distance proposed by Larsson & Villani (2001) as a measure of the distance between two cointegration spaces. dP (X, Y ) =

• m − tr(XX′Y Y ′) =
• m − X′Y 2

F

Note that tr(XX′Y Y ′) corresponds to a scalar product between two positive semidefinite matrices. d2

P (X, Y ) = tr(X⊥X⊥′Y Y ′) = COI(X⊥, Y ), where COI

is the co-inertia criterion, the numerator of both the RV and Tucker’s congruence coefficient for positive semidefinite matrices (Dray, 2008).

Markos & Menexes Hierarchical Clustering on Special Manifolds

Hierarchical Clustering on Special Manifolds Markos & Menexes Motivation Background

Manifolds Special Manifolds

HCA on Special Manifolds

Distance & Mean Intrinsic Case Extrinsic Case HCA Algorithms

Experiments Summary & Future Work References

The projection metric 2/2

dP (X, Y ) is the distance proposed by Larsson & Villani (2001) as a measure of the distance between two cointegration spaces. dP (X, Y ) =

• m − tr(XX′Y Y ′) =
• m − X′Y 2

F

Note that tr(XX′Y Y ′) corresponds to a scalar product between two positive semidefinite matrices. d2

P (X, Y ) = tr(X⊥X⊥′Y Y ′) = COI(X⊥, Y ), where COI

is the co-inertia criterion, the numerator of both the RV and Tucker’s congruence coefficient for positive semidefinite matrices (Dray, 2008).

Markos & Menexes Hierarchical Clustering on Special Manifolds

Hierarchical Clustering on Special Manifolds Markos & Menexes Motivation Background

Manifolds Special Manifolds

HCA on Special Manifolds

Distance & Mean Intrinsic Case Extrinsic Case HCA Algorithms

Experiments Summary & Future Work References

Other Subspace distances 1/2

As a slight variation of the projection metric, we may also consider the so called chordal Frobenius distance (Edelman et. al, 1999) or Procrustes metric (Chikuse, 2003), given by: dF (X, Y ) = 2 sin 1 2θ2 Note that the geodesic, projection and Procrustes metrics are asymptotically equivalent for small principal angles i.e. these embeddings are isometries (Edelman et al., 1999).

Markos & Menexes Hierarchical Clustering on Special Manifolds

Hierarchical Clustering on Special Manifolds Markos & Menexes Motivation Background

Manifolds Special Manifolds

HCA on Special Manifolds

Distance & Mean Intrinsic Case Extrinsic Case HCA Algorithms

Experiments Summary & Future Work References

Other Subspace distances 2/2

Max Correlation (Golub & Van Loan, 1996) dmax(X, Y ) = XX′ − Y Y ′2 = sin(θ1) dmax(X, Y ) is a distance based on only the largest canonical correlation cos θ1 (or the smallest principal angle θ1) Fubiny-Study metric (Edelman et al., 1999) dF S(X, Y ) = arccos|detX′Y | = arccos(

• i

cos θi) Binet-Cauchy metric (Wolf & Shashua, 2003) dBC(X, Y ) =

• 1 −
• i

cos2 θi 1/2

Markos & Menexes Hierarchical Clustering on Special Manifolds

Hierarchical Clustering on Special Manifolds Markos & Menexes Motivation Background

Manifolds Special Manifolds

HCA on Special Manifolds

Distance & Mean Intrinsic Case Extrinsic Case HCA Algorithms

Experiments Summary & Future Work References

Other Subspace distances 2/2

Max Correlation (Golub & Van Loan, 1996) dmax(X, Y ) = XX′ − Y Y ′2 = sin(θ1) dmax(X, Y ) is a distance based on only the largest canonical correlation cos θ1 (or the smallest principal angle θ1) Fubiny-Study metric (Edelman et al., 1999) dF S(X, Y ) = arccos|detX′Y | = arccos(

• i

cos θi) Binet-Cauchy metric (Wolf & Shashua, 2003) dBC(X, Y ) =

• 1 −
• i

cos2 θi 1/2

Markos & Menexes Hierarchical Clustering on Special Manifolds

Hierarchical Clustering on Special Manifolds Markos & Menexes Motivation Background

Manifolds Special Manifolds

HCA on Special Manifolds

Distance & Mean Intrinsic Case Extrinsic Case HCA Algorithms

Experiments Summary & Future Work References

Other Subspace distances 2/2

Max Correlation (Golub & Van Loan, 1996) dmax(X, Y ) = XX′ − Y Y ′2 = sin(θ1) dmax(X, Y ) is a distance based on only the largest canonical correlation cos θ1 (or the smallest principal angle θ1) Fubiny-Study metric (Edelman et al., 1999) dF S(X, Y ) = arccos|detX′Y | = arccos(

• i

cos θi) Binet-Cauchy metric (Wolf & Shashua, 2003) dBC(X, Y ) =

• 1 −
• i

cos2 θi 1/2

Markos & Menexes Hierarchical Clustering on Special Manifolds

Hierarchical Clustering on Special Manifolds Markos & Menexes Motivation Background

Manifolds Special Manifolds

HCA on Special Manifolds

Distance & Mean Intrinsic Case Extrinsic Case HCA Algorithms

Experiments Summary & Future Work References

An Extrinsic Mean

Chikuse (2003) proposed an extrinsic mean which minimizes the sum of squared projection distances on Gk,m: Given a set of matrices Pi = XiX′

i on Gk,m, for

Xi ∈ Vk,m, i = 1, . . . , n, a natural mean ˆ P ∈ Gk,m is defined by minimizing:

n

• i=1

(trIk − trPi ˆ P) Letting the spectral decomposition of S = n

i=1 Pi be

S = HDsH′, where H ∈ O(m), Ds = diag(s1, . . . , sm), s1 > . . . sm > 0, and putting H = (H1H2), with H1 being m × k, we obtain ˆ P = H1H′

1 and min = kn − k i=1 si.

Markos & Menexes Hierarchical Clustering on Special Manifolds

Hierarchical Clustering on Special Manifolds Markos & Menexes Motivation Background

Manifolds Special Manifolds

HCA on Special Manifolds

Distance & Mean Intrinsic Case Extrinsic Case HCA Algorithms

Experiments Summary & Future Work References

An Extrinsic Mean

Chikuse (2003) proposed an extrinsic mean which minimizes the sum of squared projection distances on Gk,m: Given a set of matrices Pi = XiX′

i on Gk,m, for

Xi ∈ Vk,m, i = 1, . . . , n, a natural mean ˆ P ∈ Gk,m is defined by minimizing:

n

• i=1

(trIk − trPi ˆ P) Letting the spectral decomposition of S = n

i=1 Pi be

S = HDsH′, where H ∈ O(m), Ds = diag(s1, . . . , sm), s1 > . . . sm > 0, and putting H = (H1H2), with H1 being m × k, we obtain ˆ P = H1H′

1 and min = kn − k i=1 si.

Markos & Menexes Hierarchical Clustering on Special Manifolds

Hierarchical Clustering on Special Manifolds Markos & Menexes Motivation Background

Manifolds Special Manifolds

HCA on Special Manifolds

Distance & Mean Intrinsic Case Extrinsic Case HCA Algorithms

Experiments Summary & Future Work References

Outline

1 Motivation 2 Background

Manifolds Special Manifolds

3 HCA on Special Manifolds

Distance & Mean Intrinsic Case Extrinsic Case HCA Algorithms

4 Experiments 5 Summary & Future Work 6 References

Markos & Menexes Hierarchical Clustering on Special Manifolds

Hierarchical Clustering on Special Manifolds Markos & Menexes Motivation Background

Manifolds Special Manifolds

HCA on Special Manifolds

Distance & Mean Intrinsic Case Extrinsic Case HCA Algorithms

Experiments Summary & Future Work References

HCA Algorithms 1/2 (Rencher, 2002)

Single-Link. Defines the distance between any two clusters r and s as the minimum distance between them: d(r, s) = min(d(xri, ysj)), i ∈ (1, . . . nr), i ∈ (1, . . . ns) Complete-Link. Defines the distance as the maximum distance between them: d(r, s) = max(d(xri, ysj)), i ∈ (1, . . . nr), i ∈ (1, . . . ns)

Markos & Menexes Hierarchical Clustering on Special Manifolds

Hierarchical Clustering on Special Manifolds Markos & Menexes Motivation Background

Manifolds Special Manifolds

HCA on Special Manifolds

Distance & Mean Intrinsic Case Extrinsic Case HCA Algorithms

Experiments Summary & Future Work References

HCA Algorithms 1/2 (Rencher, 2002)

Single-Link. Defines the distance between any two clusters r and s as the minimum distance between them: d(r, s) = min(d(xri, ysj)), i ∈ (1, . . . nr), i ∈ (1, . . . ns) Complete-Link. Defines the distance as the maximum distance between them: d(r, s) = max(d(xri, ysj)), i ∈ (1, . . . nr), i ∈ (1, . . . ns)

Markos & Menexes Hierarchical Clustering on Special Manifolds

Hierarchical Clustering on Special Manifolds Markos & Menexes Motivation Background

Manifolds Special Manifolds

HCA on Special Manifolds

Distance & Mean Intrinsic Case Extrinsic Case HCA Algorithms

Experiments Summary & Future Work References

HCA Algorithms 2/2

• Average. Uses the average distance between all pairs of
• bjects in any two clusters:

d(r, s) = 1 nrns

nr

• i=1

ns

• i=1

d(xri, xsj)

• Centroid. Uses a Grassmannian distance between the

centroids of the two clusters, e.g.: d(r, s) = d2

g(¯

xr, ¯ xs) (squared geodesic) d(r, s) = d2

P (¯

xr, ¯ xs) (squared projection metric) where ¯ x is either the Karcher or the Extrinsic mean.

Markos & Menexes Hierarchical Clustering on Special Manifolds

Hierarchical Clustering on Special Manifolds Markos & Menexes Motivation Background

Manifolds Special Manifolds

HCA on Special Manifolds

Distance & Mean Intrinsic Case Extrinsic Case HCA Algorithms

Experiments Summary & Future Work References

HCA Algorithms 2/2

• Average. Uses the average distance between all pairs of
• bjects in any two clusters:

d(r, s) = 1 nrns

nr

• i=1

ns

• i=1

d(xri, xsj)

• Centroid. Uses a Grassmannian distance between the

centroids of the two clusters, e.g.: d(r, s) = d2

g(¯

xr, ¯ xs) (squared geodesic) d(r, s) = d2

P (¯

xr, ¯ xs) (squared projection metric) where ¯ x is either the Karcher or the Extrinsic mean.

Markos & Menexes Hierarchical Clustering on Special Manifolds

Hierarchical Clustering on Special Manifolds Markos & Menexes Motivation Background

Manifolds Special Manifolds

HCA on Special Manifolds

Distance & Mean Intrinsic Case Extrinsic Case HCA Algorithms

Experiments Summary & Future Work References

A Toy Example

• 15 Original matrices of size 14 × 6, orthonormalized via

P = X(X′X)1/2

• 15 Geographic Areas
• 14 Crop Farming Systems
• 6 Outputs and Inputs [Height, Fertilizer (MJ/ha), Labor

(MJ/ha), Machinery (MJ/ha), Fuel (MJ/ha), Transportation (MJ/ha)]

• Three groups with high, medium, and low within-group, low

between-group correlations

• Distance and Mean: a.Karcher mean with squared geodesic

distance b.squared projection metric with the Extrinsic mean Experiments were performed in Matlab, http://utopia.duth.gr/~amarkos/subspacehca

Markos & Menexes Hierarchical Clustering on Special Manifolds

Hierarchical Clustering on Special Manifolds Markos & Menexes Motivation Background

Manifolds Special Manifolds

HCA on Special Manifolds

Distance & Mean Intrinsic Case Extrinsic Case HCA Algorithms

Experiments Summary & Future Work References

A Toy Example

• 15 Original matrices of size 14 × 6, orthonormalized via

P = X(X′X)1/2

• 15 Geographic Areas
• 14 Crop Farming Systems
• 6 Outputs and Inputs [Height, Fertilizer (MJ/ha), Labor

(MJ/ha), Machinery (MJ/ha), Fuel (MJ/ha), Transportation (MJ/ha)]

• Three groups with high, medium, and low within-group, low

between-group correlations

• Distance and Mean: a.Karcher mean with squared geodesic

distance b.squared projection metric with the Extrinsic mean Experiments were performed in Matlab, http://utopia.duth.gr/~amarkos/subspacehca

Markos & Menexes Hierarchical Clustering on Special Manifolds

Hierarchical Clustering on Special Manifolds Markos & Menexes Motivation Background

Manifolds Special Manifolds

HCA on Special Manifolds

Distance & Mean Intrinsic Case Extrinsic Case HCA Algorithms

Experiments Summary & Future Work References

A Toy Example

• 15 Original matrices of size 14 × 6, orthonormalized via

P = X(X′X)1/2

• 15 Geographic Areas
• 14 Crop Farming Systems
• 6 Outputs and Inputs [Height, Fertilizer (MJ/ha), Labor

(MJ/ha), Machinery (MJ/ha), Fuel (MJ/ha), Transportation (MJ/ha)]

• Three groups with high, medium, and low within-group, low

between-group correlations

• Distance and Mean: a.Karcher mean with squared geodesic

distance b.squared projection metric with the Extrinsic mean Experiments were performed in Matlab, http://utopia.duth.gr/~amarkos/subspacehca

Markos & Menexes Hierarchical Clustering on Special Manifolds

Hierarchical Clustering on Special Manifolds Markos & Menexes Motivation Background

Manifolds Special Manifolds

HCA on Special Manifolds

Distance & Mean Intrinsic Case Extrinsic Case HCA Algorithms

Experiments Summary & Future Work References

The 20NG Dataset (1/4)

Settings of the experiment: The 20News-Group collection consists of newsgroup documents that were manually classified into 20 different categories, related (e.g. comp.sys.ibm.pc.hardware / comp.sys.mac.hardware) or not (e.g misc.forsale / soc.religion.christian). http://people.csail.mit.edu/jrennie/20Newsgroups/. Each category includes 1, 000 documents, for a total collection size of about 20, 000 documents. We consider a particular instance of a Semantic Space, the Hyperspace Analogue to Language (HAL). The HAL space is created through the co-occurrence statistics within a corpus of documents (see Lund & Burgess, 1996).

Markos & Menexes Hierarchical Clustering on Special Manifolds

Hierarchical Clustering on Special Manifolds Markos & Menexes Motivation Background

Manifolds Special Manifolds

HCA on Special Manifolds

Distance & Mean Intrinsic Case Extrinsic Case HCA Algorithms

Experiments Summary & Future Work References

The 20NG Dataset (1/4)

Settings of the experiment: The 20News-Group collection consists of newsgroup documents that were manually classified into 20 different categories, related (e.g. comp.sys.ibm.pc.hardware / comp.sys.mac.hardware) or not (e.g misc.forsale / soc.religion.christian). http://people.csail.mit.edu/jrennie/20Newsgroups/. Each category includes 1, 000 documents, for a total collection size of about 20, 000 documents. We consider a particular instance of a Semantic Space, the Hyperspace Analogue to Language (HAL). The HAL space is created through the co-occurrence statistics within a corpus of documents (see Lund & Burgess, 1996).

Markos & Menexes Hierarchical Clustering on Special Manifolds

Hierarchical Clustering on Special Manifolds Markos & Menexes Motivation Background

Manifolds Special Manifolds

HCA on Special Manifolds

Distance & Mean Intrinsic Case Extrinsic Case HCA Algorithms

Experiments Summary & Future Work References

The 20NG Dataset (2/4)

Procedure:

• combine the documents of each category (20 sets)
• compute the SS representation of each document set

(co-occurence matrix based on the HAL model)

• Agglomerative HCA (square projection metric, extrinsic mean,

centroid linkage)

Markos & Menexes Hierarchical Clustering on Special Manifolds

Hierarchical Clustering on Special Manifolds Markos & Menexes Motivation Background

Manifolds Special Manifolds

HCA on Special Manifolds

Distance & Mean Intrinsic Case Extrinsic Case HCA Algorithms

Experiments Summary & Future Work References

The 20NG Dataset (3/4)

Results: 6 - cluster solution C1: comp.graphics, comp.os.ms-windows.misc, comp.sys.ibm.pc.hardware, comp.sys.mac.hardware, comp.windows.x C2: rec.autos, rec.motorcycles, rec.sport.baseball, rec.sport.hockey C3: sci.crypt, sci.electronics, sci.med, sci.space C4: misc.forsale C5: talk.politics.misc, talk.politics.guns, talk.politics.mideast C6: talk.religion.misc, alt.atheism, soc.religion.christian

Markos & Menexes Hierarchical Clustering on Special Manifolds

Hierarchical Clustering on Special Manifolds Markos & Menexes Motivation Background

Manifolds Special Manifolds

HCA on Special Manifolds

Distance & Mean Intrinsic Case Extrinsic Case HCA Algorithms

Experiments Summary & Future Work References

The 20NG Dataset (4/4)

Results: 4 - cluster solution C1: comp. & sci. (computers & science) C2: talk. (politics & religion) C3: rec. (sports) C4: misc.forsale

Markos & Menexes Hierarchical Clustering on Special Manifolds

Hierarchical Clustering on Special Manifolds Markos & Menexes Motivation Background

Manifolds Special Manifolds

HCA on Special Manifolds

Distance & Mean Intrinsic Case Extrinsic Case HCA Algorithms

Experiments Summary & Future Work References

Summary

Affine invariance can be treated robustly and effectively using a Riemmanian framework, by viewing subspaces as points on special manifolds. New geometric insights in designing data analysis algorithms that incorporate the geometry manifolds. We reviewed actual distance measures and notions of the mean naturally available on the Grassmann manifold and defined algorithms for hierarchical clustering on the Grassmann manifold, providing empirical results.

Markos & Menexes Hierarchical Clustering on Special Manifolds

Hierarchical Clustering on Special Manifolds Markos & Menexes Motivation Background

Manifolds Special Manifolds

HCA on Special Manifolds

Distance & Mean Intrinsic Case Extrinsic Case HCA Algorithms

Experiments Summary & Future Work References

Summary

Affine invariance can be treated robustly and effectively using a Riemmanian framework, by viewing subspaces as points on special manifolds. New geometric insights in designing data analysis algorithms that incorporate the geometry manifolds. We reviewed actual distance measures and notions of the mean naturally available on the Grassmann manifold and defined algorithms for hierarchical clustering on the Grassmann manifold, providing empirical results.

Markos & Menexes Hierarchical Clustering on Special Manifolds

Hierarchical Clustering on Special Manifolds Markos & Menexes Motivation Background

Manifolds Special Manifolds

HCA on Special Manifolds

Distance & Mean Intrinsic Case Extrinsic Case HCA Algorithms

Experiments Summary & Future Work References

Summary

Affine invariance can be treated robustly and effectively using a Riemmanian framework, by viewing subspaces as points on special manifolds. New geometric insights in designing data analysis algorithms that incorporate the geometry manifolds. We reviewed actual distance measures and notions of the mean naturally available on the Grassmann manifold and defined algorithms for hierarchical clustering on the Grassmann manifold, providing empirical results.

Markos & Menexes Hierarchical Clustering on Special Manifolds

Hierarchical Clustering on Special Manifolds Markos & Menexes Motivation Background

Manifolds Special Manifolds

HCA on Special Manifolds

Distance & Mean Intrinsic Case Extrinsic Case HCA Algorithms

Experiments Summary & Future Work References

Future Directions

Methods which rely on distance matrices or on centroids (e.g. MDS, k-means) Interesting applications (further experiments) Define a non-linear extension using the kernel trick Relaxing the orthonormality condition (the invariant property is lost, a uniform way of choosing the basis is then needed)

Markos & Menexes Hierarchical Clustering on Special Manifolds

Hierarchical Clustering on Special Manifolds Markos & Menexes Motivation Background

Manifolds Special Manifolds

HCA on Special Manifolds

Distance & Mean Intrinsic Case Extrinsic Case HCA Algorithms

Experiments Summary & Future Work References

Future Directions

Methods which rely on distance matrices or on centroids (e.g. MDS, k-means) Interesting applications (further experiments) Define a non-linear extension using the kernel trick Relaxing the orthonormality condition (the invariant property is lost, a uniform way of choosing the basis is then needed)

Markos & Menexes Hierarchical Clustering on Special Manifolds

Hierarchical Clustering on Special Manifolds Markos & Menexes Motivation Background

Manifolds Special Manifolds

HCA on Special Manifolds

Distance & Mean Intrinsic Case Extrinsic Case HCA Algorithms

Experiments Summary & Future Work References

Future Directions

Methods which rely on distance matrices or on centroids (e.g. MDS, k-means) Interesting applications (further experiments) Define a non-linear extension using the kernel trick Relaxing the orthonormality condition (the invariant property is lost, a uniform way of choosing the basis is then needed)

Markos & Menexes Hierarchical Clustering on Special Manifolds

Hierarchical Clustering on Special Manifolds Markos & Menexes Motivation Background

Manifolds Special Manifolds

HCA on Special Manifolds

Distance & Mean Intrinsic Case Extrinsic Case HCA Algorithms

Experiments Summary & Future Work References

Future Directions

Methods which rely on distance matrices or on centroids (e.g. MDS, k-means) Interesting applications (further experiments) Define a non-linear extension using the kernel trick Relaxing the orthonormality condition (the invariant property is lost, a uniform way of choosing the basis is then needed)

Markos & Menexes Hierarchical Clustering on Special Manifolds

Hierarchical Clustering on Special Manifolds Markos & Menexes Motivation Background

Manifolds Special Manifolds

HCA on Special Manifolds

Distance & Mean Intrinsic Case Extrinsic Case HCA Algorithms

Experiments Summary & Future Work References

References 1/4

Barg, A. and Nogin, D. (2002). Bounds on packings of spheres in the Grassmann manifold. IEEE Trans. Information Theory, 48(9), 2450-2454. Begelfor E. & Werman M. (2006). Affine invariance

• revisited. In Proc. IEEE Conf. on Computer Vision and

Pattern Recognition, New York, NY, 2, 2087-2094. Berger, M. (2003). A Panoramic View of Riemannian

• Geometry. Springer, Berlin.

Bj¨

• rck, A. & Golub, G.H. (1973) Numerical methods for

computing the angles between linear subspaces, Math.

• Comp. 27, 579-594

Boothby, W.M. (2002). An Introduction to Differentiable Manifolds and Riemannian Geometry. Academic Press, 2002.

Markos & Menexes Hierarchical Clustering on Special Manifolds

Hierarchical Clustering on Special Manifolds Markos & Menexes Motivation Background

Manifolds Special Manifolds

HCA on Special Manifolds

Distance & Mean Intrinsic Case Extrinsic Case HCA Algorithms

Experiments Summary & Future Work References

References 1/4

Barg, A. and Nogin, D. (2002). Bounds on packings of spheres in the Grassmann manifold. IEEE Trans. Information Theory, 48(9), 2450-2454. Begelfor E. & Werman M. (2006). Affine invariance

• revisited. In Proc. IEEE Conf. on Computer Vision and

Pattern Recognition, New York, NY, 2, 2087-2094. Berger, M. (2003). A Panoramic View of Riemannian

• Geometry. Springer, Berlin.

Bj¨

• rck, A. & Golub, G.H. (1973) Numerical methods for

computing the angles between linear subspaces, Math.

• Comp. 27, 579-594

Boothby, W.M. (2002). An Introduction to Differentiable Manifolds and Riemannian Geometry. Academic Press, 2002.

Markos & Menexes Hierarchical Clustering on Special Manifolds

Hierarchical Clustering on Special Manifolds Markos & Menexes Motivation Background

Manifolds Special Manifolds

HCA on Special Manifolds

Distance & Mean Intrinsic Case Extrinsic Case HCA Algorithms

Experiments Summary & Future Work References

References 1/4

Barg, A. and Nogin, D. (2002). Bounds on packings of spheres in the Grassmann manifold. IEEE Trans. Information Theory, 48(9), 2450-2454. Begelfor E. & Werman M. (2006). Affine invariance

• revisited. In Proc. IEEE Conf. on Computer Vision and

Pattern Recognition, New York, NY, 2, 2087-2094. Berger, M. (2003). A Panoramic View of Riemannian

• Geometry. Springer, Berlin.

Bj¨

• rck, A. & Golub, G.H. (1973) Numerical methods for

computing the angles between linear subspaces, Math.

• Comp. 27, 579-594

Boothby, W.M. (2002). An Introduction to Differentiable Manifolds and Riemannian Geometry. Academic Press, 2002.

Markos & Menexes Hierarchical Clustering on Special Manifolds

Hierarchical Clustering on Special Manifolds Markos & Menexes Motivation Background

Manifolds Special Manifolds

HCA on Special Manifolds

Distance & Mean Intrinsic Case Extrinsic Case HCA Algorithms

Experiments Summary & Future Work References

References 1/4

Barg, A. and Nogin, D. (2002). Bounds on packings of spheres in the Grassmann manifold. IEEE Trans. Information Theory, 48(9), 2450-2454. Begelfor E. & Werman M. (2006). Affine invariance

• revisited. In Proc. IEEE Conf. on Computer Vision and

Pattern Recognition, New York, NY, 2, 2087-2094. Berger, M. (2003). A Panoramic View of Riemannian

• Geometry. Springer, Berlin.

Bj¨

• rck, A. & Golub, G.H. (1973) Numerical methods for

computing the angles between linear subspaces, Math.

• Comp. 27, 579-594

Boothby, W.M. (2002). An Introduction to Differentiable Manifolds and Riemannian Geometry. Academic Press, 2002.

Markos & Menexes Hierarchical Clustering on Special Manifolds

Hierarchical Clustering on Special Manifolds Markos & Menexes Motivation Background

Manifolds Special Manifolds

HCA on Special Manifolds

Distance & Mean Intrinsic Case Extrinsic Case HCA Algorithms

Experiments Summary & Future Work References

References 1/4

Barg, A. and Nogin, D. (2002). Bounds on packings of spheres in the Grassmann manifold. IEEE Trans. Information Theory, 48(9), 2450-2454. Begelfor E. & Werman M. (2006). Affine invariance

• revisited. In Proc. IEEE Conf. on Computer Vision and

Pattern Recognition, New York, NY, 2, 2087-2094. Berger, M. (2003). A Panoramic View of Riemannian

• Geometry. Springer, Berlin.

Bj¨

• rck, A. & Golub, G.H. (1973) Numerical methods for

computing the angles between linear subspaces, Math.

• Comp. 27, 579-594

Boothby, W.M. (2002). An Introduction to Differentiable Manifolds and Riemannian Geometry. Academic Press, 2002.

Markos & Menexes Hierarchical Clustering on Special Manifolds

Hierarchical Clustering on Special Manifolds Markos & Menexes Motivation Background

Manifolds Special Manifolds

HCA on Special Manifolds

Distance & Mean Intrinsic Case Extrinsic Case HCA Algorithms

Experiments Summary & Future Work References

References 2/4

Conway, J., Hardin, R. & Sloane, N. (1996). Packing lines, planes, etc.: Packings in Grassmannian spaces. Experimental Mathematics, 5, 139-159. Edelman, A., Arias, T. & Smith, S. (1998). The geometry

• f algorithms with orthogonality constrains. SIAM J.

Matrix Anal. Appl, 20(2), 303-353. Goodall, C. R. & Mardia, K.V. (1999). Projective shape analysis, Journal of Computational and Graphical Statistics, 8(2), 143-168. Karcher, H. (1977). Riemannian center of mass and mollifier smoothing, Communications on Pure and Applied Mathematics, 30, 509-541. Larsson, R. & Villani, M. (2001). A distance measure between cointegration spaces, Economics Letters, 70(1), 21–27.

Markos & Menexes Hierarchical Clustering on Special Manifolds

Hierarchical Clustering on Special Manifolds Markos & Menexes Motivation Background

Manifolds Special Manifolds

HCA on Special Manifolds

Distance & Mean Intrinsic Case Extrinsic Case HCA Algorithms

Experiments Summary & Future Work References

References 2/4

Conway, J., Hardin, R. & Sloane, N. (1996). Packing lines, planes, etc.: Packings in Grassmannian spaces. Experimental Mathematics, 5, 139-159. Edelman, A., Arias, T. & Smith, S. (1998). The geometry

• f algorithms with orthogonality constrains. SIAM J.

Matrix Anal. Appl, 20(2), 303-353. Goodall, C. R. & Mardia, K.V. (1999). Projective shape analysis, Journal of Computational and Graphical Statistics, 8(2), 143-168. Karcher, H. (1977). Riemannian center of mass and mollifier smoothing, Communications on Pure and Applied Mathematics, 30, 509-541. Larsson, R. & Villani, M. (2001). A distance measure between cointegration spaces, Economics Letters, 70(1), 21–27.

Markos & Menexes Hierarchical Clustering on Special Manifolds

Hierarchical Clustering on Special Manifolds Markos & Menexes Motivation Background

Manifolds Special Manifolds

HCA on Special Manifolds

Distance & Mean Intrinsic Case Extrinsic Case HCA Algorithms

Experiments Summary & Future Work References

References 2/4

Conway, J., Hardin, R. & Sloane, N. (1996). Packing lines, planes, etc.: Packings in Grassmannian spaces. Experimental Mathematics, 5, 139-159. Edelman, A., Arias, T. & Smith, S. (1998). The geometry

• f algorithms with orthogonality constrains. SIAM J.

Matrix Anal. Appl, 20(2), 303-353. Goodall, C. R. & Mardia, K.V. (1999). Projective shape analysis, Journal of Computational and Graphical Statistics, 8(2), 143-168. Karcher, H. (1977). Riemannian center of mass and mollifier smoothing, Communications on Pure and Applied Mathematics, 30, 509-541. Larsson, R. & Villani, M. (2001). A distance measure between cointegration spaces, Economics Letters, 70(1), 21–27.

Markos & Menexes Hierarchical Clustering on Special Manifolds

Hierarchical Clustering on Special Manifolds Markos & Menexes Motivation Background

Manifolds Special Manifolds

HCA on Special Manifolds

Distance & Mean Intrinsic Case Extrinsic Case HCA Algorithms

Experiments Summary & Future Work References

References 2/4

Conway, J., Hardin, R. & Sloane, N. (1996). Packing lines, planes, etc.: Packings in Grassmannian spaces. Experimental Mathematics, 5, 139-159. Edelman, A., Arias, T. & Smith, S. (1998). The geometry

• f algorithms with orthogonality constrains. SIAM J.

Matrix Anal. Appl, 20(2), 303-353. Goodall, C. R. & Mardia, K.V. (1999). Projective shape analysis, Journal of Computational and Graphical Statistics, 8(2), 143-168. Karcher, H. (1977). Riemannian center of mass and mollifier smoothing, Communications on Pure and Applied Mathematics, 30, 509-541. Larsson, R. & Villani, M. (2001). A distance measure between cointegration spaces, Economics Letters, 70(1), 21–27.

Markos & Menexes Hierarchical Clustering on Special Manifolds

Hierarchical Clustering on Special Manifolds Markos & Menexes Motivation Background

Manifolds Special Manifolds

HCA on Special Manifolds

Distance & Mean Intrinsic Case Extrinsic Case HCA Algorithms

Experiments Summary & Future Work References

References 2/4

Conway, J., Hardin, R. & Sloane, N. (1996). Packing lines, planes, etc.: Packings in Grassmannian spaces. Experimental Mathematics, 5, 139-159. Edelman, A., Arias, T. & Smith, S. (1998). The geometry

• f algorithms with orthogonality constrains. SIAM J.

Matrix Anal. Appl, 20(2), 303-353. Goodall, C. R. & Mardia, K.V. (1999). Projective shape analysis, Journal of Computational and Graphical Statistics, 8(2), 143-168. Karcher, H. (1977). Riemannian center of mass and mollifier smoothing, Communications on Pure and Applied Mathematics, 30, 509-541. Larsson, R. & Villani, M. (2001). A distance measure between cointegration spaces, Economics Letters, 70(1), 21–27.

Markos & Menexes Hierarchical Clustering on Special Manifolds

Hierarchical Clustering on Special Manifolds Markos & Menexes Motivation Background

Manifolds Special Manifolds

HCA on Special Manifolds

Distance & Mean Intrinsic Case Extrinsic Case HCA Algorithms

Experiments Summary & Future Work References

References 3/4

Lund, K., Burgess, C. (1996). Producing High-dimensional Semantic Spaces from Lexical Co-occurrence. Behavior Research Methods 28(2), 203-208. Rencher, A.C. (2002). Methods of Multivariate Analysis, 2nd ed. New York: John Wiley & Sons. Srivasatava, A. & Klassen, E. (2004). Bayesian geometric subspace tracking, Advances in Applied Probability, 36, 43–56. Wang, L., Wang, X., & Feng, J. (2006). Subspace Distance Analysis with Application to Adaptive Bayesian Algorithm for Face Recognition. Pattern Recognition, 39(3).

Markos & Menexes Hierarchical Clustering on Special Manifolds

Hierarchical Clustering on Special Manifolds Markos & Menexes Motivation Background

Manifolds Special Manifolds

HCA on Special Manifolds

Distance & Mean Intrinsic Case Extrinsic Case HCA Algorithms

Experiments Summary & Future Work References

References 3/4

Lund, K., Burgess, C. (1996). Producing High-dimensional Semantic Spaces from Lexical Co-occurrence. Behavior Research Methods 28(2), 203-208. Rencher, A.C. (2002). Methods of Multivariate Analysis, 2nd ed. New York: John Wiley & Sons. Srivasatava, A. & Klassen, E. (2004). Bayesian geometric subspace tracking, Advances in Applied Probability, 36, 43–56. Wang, L., Wang, X., & Feng, J. (2006). Subspace Distance Analysis with Application to Adaptive Bayesian Algorithm for Face Recognition. Pattern Recognition, 39(3).

Markos & Menexes Hierarchical Clustering on Special Manifolds

Hierarchical Clustering on Special Manifolds Markos & Menexes Motivation Background

Manifolds Special Manifolds

HCA on Special Manifolds

Distance & Mean Intrinsic Case Extrinsic Case HCA Algorithms

Experiments Summary & Future Work References

References 3/4

Lund, K., Burgess, C. (1996). Producing High-dimensional Semantic Spaces from Lexical Co-occurrence. Behavior Research Methods 28(2), 203-208. Rencher, A.C. (2002). Methods of Multivariate Analysis, 2nd ed. New York: John Wiley & Sons. Srivasatava, A. & Klassen, E. (2004). Bayesian geometric subspace tracking, Advances in Applied Probability, 36, 43–56. Wang, L., Wang, X., & Feng, J. (2006). Subspace Distance Analysis with Application to Adaptive Bayesian Algorithm for Face Recognition. Pattern Recognition, 39(3).

Markos & Menexes Hierarchical Clustering on Special Manifolds

Hierarchical Clustering on Special Manifolds Markos & Menexes Motivation Background

Manifolds Special Manifolds

HCA on Special Manifolds

Distance & Mean Intrinsic Case Extrinsic Case HCA Algorithms

Experiments Summary & Future Work References

References 3/4

Lund, K., Burgess, C. (1996). Producing High-dimensional Semantic Spaces from Lexical Co-occurrence. Behavior Research Methods 28(2), 203-208. Rencher, A.C. (2002). Methods of Multivariate Analysis, 2nd ed. New York: John Wiley & Sons. Srivasatava, A. & Klassen, E. (2004). Bayesian geometric subspace tracking, Advances in Applied Probability, 36, 43–56. Wang, L., Wang, X., & Feng, J. (2006). Subspace Distance Analysis with Application to Adaptive Bayesian Algorithm for Face Recognition. Pattern Recognition, 39(3).

Markos & Menexes Hierarchical Clustering on Special Manifolds

Hierarchical Clustering on Special Manifolds Markos & Menexes Motivation Background

Manifolds Special Manifolds

HCA on Special Manifolds

Distance & Mean Intrinsic Case Extrinsic Case HCA Algorithms

Experiments Summary & Future Work References

References 4/4

Wolf, L., & Shashua, A. (2003). Learning over sets using kernel principal angles. J. Mach. Learn. Res., 4, 913-931. Wong, Y.-C. (1967). Differential geometry of Grassmann

• manifolds. Proc. of the Nat. Acad. of Sci., 57, 589-594.

Markos & Menexes Hierarchical Clustering on Special Manifolds

Hierarchical Clustering on Special Manifolds Markos & Menexes Motivation Background

Manifolds Special Manifolds

HCA on Special Manifolds

Distance & Mean Intrinsic Case Extrinsic Case HCA Algorithms

Experiments Summary & Future Work References

References 4/4

Wolf, L., & Shashua, A. (2003). Learning over sets using kernel principal angles. J. Mach. Learn. Res., 4, 913-931. Wong, Y.-C. (1967). Differential geometry of Grassmann

• manifolds. Proc. of the Nat. Acad. of Sci., 57, 589-594.

Markos & Menexes Hierarchical Clustering on Special Manifolds

Hierarchical Clustering on Special Manifolds Markos & Menexes Motivation Background

Manifolds Special Manifolds

HCA on Special Manifolds

Distance & Mean Intrinsic Case Extrinsic Case HCA Algorithms

Experiments Summary & Future Work References

Thank you for your attention!

Markos & Menexes Hierarchical Clustering on Special Manifolds

Hierarchical Clustering on Special Manifolds Markos & Menexes Motivation Background

Manifolds Special Manifolds

HCA on Special Manifolds

Distance & Mean Intrinsic Case Extrinsic Case HCA Algorithms

Experiments Summary & Future Work References

Bonus Slide - The HAL Model

A window of text is passed over each document in the collection in order to capture co-occurrences of words. The length of the window is set to l: a typical value of l is 10; different values capture different levels of relationship between words. Words that co-occur into a window do so with a strength inversely proportional to the distance between the two co-occurring words. By sliding the window over the whole collection and recording the co-occurrence values, a co-occurrence matrix A can be created. Since in our approach, we are not interested in the order

• f the co-occurrences therefore we can compute a

symmetric matrix by means of S = AA′ and then

• rthonormalize the columns.

For a similar experiment see Zuccon (2009).

Markos & Menexes Hierarchical Clustering on Special Manifolds

Hierarchical Clustering on Special Manifolds Markos & Menexes Motivation Background

Manifolds Special Manifolds

HCA on Special Manifolds

Distance & Mean Intrinsic Case Extrinsic Case HCA Algorithms

Experiments Summary & Future Work References

Bonus Slide - The HAL Model

A window of text is passed over each document in the collection in order to capture co-occurrences of words. The length of the window is set to l: a typical value of l is 10; different values capture different levels of relationship between words. Words that co-occur into a window do so with a strength inversely proportional to the distance between the two co-occurring words. By sliding the window over the whole collection and recording the co-occurrence values, a co-occurrence matrix A can be created. Since in our approach, we are not interested in the order

• f the co-occurrences therefore we can compute a

symmetric matrix by means of S = AA′ and then

• rthonormalize the columns.

For a similar experiment see Zuccon (2009).

Markos & Menexes Hierarchical Clustering on Special Manifolds

Hierarchical Clustering on Special Manifolds Markos & Menexes Motivation Background

Manifolds Special Manifolds

HCA on Special Manifolds

Distance & Mean Intrinsic Case Extrinsic Case HCA Algorithms

Experiments Summary & Future Work References

Bonus Slide - The HAL Model

A window of text is passed over each document in the collection in order to capture co-occurrences of words. The length of the window is set to l: a typical value of l is 10; different values capture different levels of relationship between words. Words that co-occur into a window do so with a strength inversely proportional to the distance between the two co-occurring words. By sliding the window over the whole collection and recording the co-occurrence values, a co-occurrence matrix A can be created. Since in our approach, we are not interested in the order

• f the co-occurrences therefore we can compute a

symmetric matrix by means of S = AA′ and then

• rthonormalize the columns.

For a similar experiment see Zuccon (2009).

Markos & Menexes Hierarchical Clustering on Special Manifolds

Hierarchical Clustering on Special Manifolds Markos & Menexes Motivation Background

Manifolds Special Manifolds

HCA on Special Manifolds

Distance & Mean Intrinsic Case Extrinsic Case HCA Algorithms

Experiments Summary & Future Work References

Bonus Slide - The HAL Model

A window of text is passed over each document in the collection in order to capture co-occurrences of words. The length of the window is set to l: a typical value of l is 10; different values capture different levels of relationship between words. Words that co-occur into a window do so with a strength inversely proportional to the distance between the two co-occurring words. By sliding the window over the whole collection and recording the co-occurrence values, a co-occurrence matrix A can be created. Since in our approach, we are not interested in the order

• f the co-occurrences therefore we can compute a

symmetric matrix by means of S = AA′ and then

• rthonormalize the columns.

For a similar experiment see Zuccon (2009).

Markos & Menexes Hierarchical Clustering on Special Manifolds