Separation and convexity properties of hierarchical and non - - PowerPoint PPT Presentation

Plan

Separation and convexity properties of hierarchical and non hierarchical clustering

Patrice Bertrand1

1CEREMADE, Universit´

e Paris-Dauphine, Paris, France

Joint work with Jean Diatta 2

2 LIM, Universit´

e de La R´ eunion, Saint-Denis, France

• P. Bertrand

Plan

Plan

1

Background

2

Ternary separation and convexity

3

Characterizations of clustering structures

4

Application to Cluster Analysis

• P. Bertrand

Plan

Plan

1

Background

2

Ternary separation and convexity

3

Characterizations of clustering structures

4

Application to Cluster Analysis

• P. Bertrand

Plan

Plan

1

Background

2

Ternary separation and convexity

3

Characterizations of clustering structures

4

Application to Cluster Analysis

• P. Bertrand

Plan

Plan

1

Background

2

Ternary separation and convexity

3

Characterizations of clustering structures

4

Application to Cluster Analysis

• P. Bertrand

Background Separation and convexity Characterizations Application to Cluster Analysis

◮ Multi-level clustering structures

• Hierarchies

Johnson (1967), Benz´ ecri (1973)

• Weak Hierarchies

Bandelt & Dress (1989, 1994), Diatta & Fichet (1994, 1998), Bertrand & Janowitz (2002)

• Pyramids (or pseudo-hierarchies)

Diday (1984, 1986), Fichet (1984, 1986)

• Paired hierarchies

Bertrand (2002, 2008), Bertrand & Brucker (2007)

• P. Bertrand

Background Separation and convexity Characterizations Application to Cluster Analysis

Definitions

A pair {A, B} ⊆ E (ground set) is said to be ◮ hierarchical: A ∩ B ∈ {A, B, ∅} If {A, B} is not hierarchical, then A and B cross each other

A B

We use the following terminology for F ⊆ 2E: ◮ set-system: {∅} / ∈ F and E ∈ F ◮ total: for all x ∈ E, {x} ⊆ F ◮ closed: F is closed under non empty intersections: ∀G ⊆ F,

• G ∈ F ∪ {∅}

◮ (strongly) hierarchical: each pair {X, Y} ⊆ F is hierarchical

• P. Bertrand

Background Separation and convexity Characterizations Application to Cluster Analysis

Weak hierarchies

A collection F ⊆ 2E is said to be weakly hierarchical if ∀X, Y, Z ∈ F, X ∩ Y ∩ Z ∈ {X ∩ Y, Y ∩ Z, X ∩ Z} nsc There are no A1, A2, A3 ∈ F and a1, a2, a3 ∈ E s.t. ai ∈ Aj ⇐ ⇒ i = j Forbidden configuration:

A

1 3 2

A A

1 3

a a

2

a

• P. Bertrand

Background Separation and convexity Characterizations Application to Cluster Analysis

Paired hierarchies

A collection F ⊆ 2E is called paired hierarchical if each F-member crosses at most one F-member nsc ◮ ∀X, Y, Z ∈ F, at least 2 of {X, Y}, {Y, Z}, {X, Z} are hierarchical ◮ ”X crosses Y” defines an equivalence relation whose class sizes are at most 2 ◮ Forbidden configurations:

(a) (b) (c)

The term paired-hierarchy is used since { G : G is a class} is a hierarchy

• P. Bertrand

Background Separation and convexity Characterizations Application to Cluster Analysis

Examples and counter-examples

Paired-hierarchies

b a d c

a b c d

a b c d

c b a d

Weak-hierarchies

c b d a

c b a d

• P. Bertrand

Background Separation and convexity Characterizations Application to Cluster Analysis

Correspondences between dissimilarities and multi-level clustering structures d (dissimilarity on E) ← → (F, f) (F ⊆ 2E and f : F → R+ being increasing) φ : (F, f) → φ(F, f) with φ(F, f)(x, y) = min{f(A) : a, b ∈ A, A ∈ F} Conversely, each dissimilarity d is associated with: ◮ Dd(x, y): closed ball of center x ∈ E and radius r = d(x, y) Dd(x, y) = {z ∈ E : d(z, x) ≤ d(x, y)} ◮ Bd(x, y): 2-ball generated by x, y ∈ E, in the sense of d Bd(x, y) = Dd(x, y) ∩ Dd(y, x) = { z ∈ E : max{d(z, x), d(z, y)} ≤ d(x, y)}

y x

• P. Bertrand

Background Separation and convexity Characterizations Application to Cluster Analysis

Separation relation

A ternary relation designates any subset of E3 A (ternary) separation relation is a ternary relation of the form: ◮ Given F ⊆ 2E, the ternary separation relation s(F) is defined by (x, y, z) ∈ s(F) if it exists a F-member which contains x and y but not z.

In what follows, we will write simply xyz ∈ s(F) in place of (x, y, z) ∈ s(F)

• P. Bertrand

Background Separation and convexity Characterizations Application to Cluster Analysis

Convexity

Abstract convexity (van de Vel, cf. early 1950s).

◮ A collection C ⊆ 2E is called a convexity on E if ∅, E ∈ C and C is closed both under intersections and nested unions. (E, C ) is called a convex structure or a convexity space. Convex set : any member of C ◮ ∀A ⊆ E, conv C(A) = A C = {C : A ⊆ C ∈ C }, is called the (convex) hull of A. ◮ Notations: a, b := a, bC ◮ Segment joining a and b: the 2-polytope conv({a, b}) ◮ , C : (a, b) ∈ E2 → a, bC ∈ 2E is called the segment operator of the convexity C .

• P. Bertrand

Background Separation and convexity Characterizations Application to Cluster Analysis

Arity The arity of C is ≤ n if for all C ∈ C and F ⊆ C with #F ≤ n, we have: F = conv(F) ⊆ C Rank A ⊆ E is called convexly independent if a / ∈ A \ {a} for all a ∈ A The rank of a convex structure (E, C ) is defined as the maximum size

• f a convexly independent set.

Interval operator I : E × E → 2E is called an interval operator on E if ∀a, b ∈ E, a, b ∈ I(a, b) = I(b, a). I(a, b): interval between a and b; (E, I): interval space. Example: , C : (a, b) ∈ E2 → a, bC of any convexity C

• P. Bertrand

Background Separation and convexity Characterizations Application to Cluster Analysis

Notations GI := {C ⊆ E | ∀x, y ∈ C, I(x, y) ⊆ C} is the convexity induced by I G , C := interval convexity induced by the segment operator , C a, bI segment between a and b in the sense of the convexity GI. Properties (Calder (1971)) ◮ ∀a, b ∈ E, I(a, b) ⊆ a, bI ◮ A convexity is induced by an interval operator iff its arity is ≤ 2. ◮ The hull of a set A in an interval space is given by A =

∞

• k=0

Ak, where A0 = A and for all k ∈ N, Ak+1 = {I(a, a′) | a, a′ ∈ Ak}.

• P. Bertrand

Background Separation and convexity Characterizations Application to Cluster Analysis

Convexity induced by , C

Lemma 1

Let C be a convexity on E. (i) , C and , , C coincide. (ii) We have: {a, bC | a, b ∈ E} ⊆ C ⊆ G , C , where the two inclusions may be strict. Remark It is easily checked that: xyz ∈ s(C ) ⇐ ⇒ z ∈ x, yC .

• P. Bertrand

Background Separation and convexity Characterizations Application to Cluster Analysis

Interval operators and Cluster Analysis

◮ Bd and Dd are two interval operators defined on E

Lemma 2

For all dissimilarity d on E and all x, y ∈ E, there exist u, v ∈ E such that: x , yBd = u , vBd = Bd(u, v).

• P. Bertrand

Background Separation and convexity Characterizations Application to Cluster Analysis

Separation, Interval operators and Weak Hierarchies

◮ Bandelt and Dress (1994): A set-system C is weakly hierarchical iff for all x1, x2, x3 distinct in E, s(C ) does not contains both x1x2x3, x2x3x1 and x3x1x2 ◮ Let I be an interval operator on E, and let (W) No x, y, z ∈ E exist s.t. x / ∈ I(y, z), y / ∈ I(x, z) and z / ∈ I(x, y).

• P. Bertrand

Background Separation and convexity Characterizations Application to Cluster Analysis

Proposition 3

Let I be an interval operator and let (i) I satisfies (W) (ii) , I satisfies (W) (iii) GI is weakly hierarchical (iv) GI is of rank at most 2, i.e. if ∅ A ⊆ E, then AI is of the form a, bI for some a, b ∈ A. Then (i) ⇒ (ii) ⇔ (iii) ⇔ (iv)

• P. Bertrand

Background Separation and convexity Characterizations Application to Cluster Analysis

Corollary 4

If the interval operator I satisfies (W), then GI = {a, bI | a, b ∈ E}

Definition 5 (k-ball)

Let A ⊆ E with #A = k > 2, and denote Bd

A = {x ∈ E | ∀a ∈ A, d(a, x) ≤ diamd A}.

Proposition 6

If (C , f) is an indexed closed weak-hierarchical set system s.t. f −1(0) = {X ∈ C | f(X) = 0} is a partition of E, then Bφ((C ,f))

A

= AC ∪{∅}, for all nonempty subset A of E.

• P. Bertrand

Background Separation and convexity Characterizations Application to Cluster Analysis

Notation 7

B(C ,f) := Bφ((C ,f))

Corollary 8

Let C be a set-system on E. The following are equivalent: (i) C is closed and weakly hierarchical (ii) C ∪ {∅} = GI for some interval operator I satisfying (W) (iii) C ∪ {∅} = GB(C,f) for some index f on C satisfying f −1(0) = E (iv) C ∪ {∅} = GB(C,f) for all index f on C satisfying f −1(0) = E Criterion to recognize whether a set system is weakly hierarchical: define f by f(A) := | A | −1

• P. Bertrand

Background Separation and convexity Characterizations Application to Cluster Analysis

Separation relation, Interval operators, Hierarchies

Proposition 9

For all collection C of subsets of E, the following are equivalent: (i) C is hierarchical; (ii) For all x, y, z ∈ E, xyz ∈ s(C ) ⇒

• yzx /

∈ s(C ) and zxy / ∈ s(C )

• .

Definition 10

Let I be an interval operator on E, we denote: (H) For all x, y, z ∈ E, either I(x, y) ⊆ I(x, z) or I(x, z) ⊆ I(x, y).

Remark 11

Clearly, (H) ⇒ (W) and Dd satisfies (H)

• P. Bertrand

Background Separation and convexity Characterizations Application to Cluster Analysis

Proposition 12

Let I be any interval operator on E, and denote: (i) I satisfies the condition (H) (ii) The segment operator , I satisfies the condition (H) (iii) GI is a hierarchical set system Then (i) ⇒ (ii) ⇔ (iii)

• P. Bertrand

Background Separation and convexity Characterizations Application to Cluster Analysis

Notation 13

Dd

A

:= {x ∈ E | ∃ a ∈ A s.t. d(a, x) ≤ diamd A} D(C ,f)(x, y) := Dφ((C ,f))(x, y) for any ∅ A ⊆ E and any x, y ∈ E.

Proposition 14

If d is an ultrametric, then Dd

A = Bd A

Corollary 15

If (C , f) is an indexed hierarchy on E such that f −1(0) = {X ∈ C | f(X) = 0} is a partition of E, then ∀A ∈ 2E \ {∅}, D(C ,f)

A

= AC ∪{∅}.

• P. Bertrand

Background Separation and convexity Characterizations Application to Cluster Analysis

Corollary 16

Let C be a set-system on E. The following are equivalent: (i) C is hierarchical (ii) C ∪ {∅} = GI for some interval operator I satisfying (H) (iii) C ∪ {∅} = GD(C,f) for some index f on C s. t. f −1(0) = E (iv) C ∪ {∅} = GD(C,f) for all index f on C s.t. f −1(0) = E Criterion to recognize whether a set system is hierarchical: define f by f(A) := | A | −1

• P. Bertrand

Background Separation and convexity Characterizations Application to Cluster Analysis

Interval operators, separation relation and paired hierarchies

Definition 17 (property P[C ])

For all A ∈ F, all distinct elements x, y of A and all u, v / ∈ A,

• uyx ∈ s(C ) ⇒ vxy /

∈ s(C )

• and
• uyv ∈ s(C ) ⇒ vyx /

∈ s(C )

• Proposition 18

A set-system C is hierarchical iff P[C ] is true

• P. Bertrand

Background Separation and convexity Characterizations Application to Cluster Analysis

Definition 19

Given an interval operator I on E, let (P) be defined as: (P) No x, y, u, v ∈ E, with x = y, satisfy both u, v / ∈ I(x, y) and either

• x /

∈ I(y, u) and y / ∈ I(x, v)

• r
• v /

∈ I(y, u) and x / ∈ I(y, v)

• .

Proposition 20

Let C be a closed set-system on E. The following assertions are equivalent: (i) C is hierarchical (ii) C ∪ {∅} = GI for some interval operator I satisfying (P)

• P. Bertrand

Background Separation and convexity Characterizations Application to Cluster Analysis

Application to Cluster Analysis

Algorithm of construction of the interval convexity GI

Denote I an interval operator on E and E := {e1, . . . , en} Denote Gk

I the convexity induced on Ek := {e1, . . . , ek} (k ≤ n) by the

restriction of I to Ek

1 Put G0 I = {∅} 2 Assume Gk I is known for k < n. For each C ∈ Gk I ,

☛ ✡ ✟ ✠

2a mark C iff ek+1 / ∈ I(e, e′) for all e, e′ ∈ C

☛ ✡ ✟ ✠

2b mark C ∪ {ek+1} iff I(e, ek+1) ⊆ C ∪ {ek+1} for all e ∈ C ∪ {ek+1}

☛ ✡ ✟ ✠

2c add to Gk+1

I

any subset that was marked either by

☛ ✡ ✟ ✠

2a or by

☛ ✡ ✟ ✠

2b

3 If k < n, increment the value of k and repeat step 2

• P. Bertrand

Background Separation and convexity Characterizations Application to Cluster Analysis

Some Discussion

1 Characterizations of clustering structures in terms of ternary

separation relations and (abstract) convexity

2 Dd satisfies (H) and Bd satisfies (W): is there some similar map

ϕp

d, that can be derived from any dissimilarity d, satisfying (P)?

Rmk: Bd(a, b) ⊆ ϕp

d(a, b) ⊆ Dd(a, b) must hold, for all a, b ∈ E

3 Separation properties

?

← → convexity properties

• P. Bertrand