[PPT] - MixtComp software: Model-based clustering/imputation with mixed PowerPoint Presentation

SLIDE 1

Classifications(s): overview Mixture model solution Estimation Clustering with MixtComp Imputation with MixtComp Conclusion

MixtComp software: Model-based clustering/imputation with mixed data, missing data and uncertain data

https://modal-research.lille.inria.fr/BigStat/

Christophe Biernacki

(with Thibault Deregnaucourt and Vincent Kubicki)

Tutorial in MissData Conference June, 17th 2015

1/58

SLIDE 2

Classifications(s): overview Mixture model solution Estimation Clustering with MixtComp Imputation with MixtComp Conclusion

Take-home message

Imputation: should take into account the final analysis purpose Clustering: no imputation is needed in the model-based context Mixture models: flexible enough for accurate multiple imputation

MixtComp software

Clustering/imputation for mixed data

2/58

SLIDE 3

Classifications(s): overview Mixture model solution Estimation Clustering with MixtComp Imputation with MixtComp Conclusion

Outline

1 Classifications(s): overview 2 Mixture model solution 3 Estimation 4 Clustering with MixtComp 5 Imputation with MixtComp 6 Conclusion

3/58

SLIDE 4

Classifications(s): overview Mixture model solution Estimation Clustering with MixtComp Imputation with MixtComp Conclusion

Today’s data (1/2)

Today, it is easy to collect many features, so it favors data variety and/or mixed data missing data uncertainty (or interval data)

Mixed, missing, uncertain

Observed individuals xO ∈ X ? 0.5 ? 5 0.3 0.1 green 3 0.3 0.6 {red,green} 3 0.9 [0.25 0.45] red ? ↓ ↓ ↓ ↓ continuous continuous categorical integer

4/58

SLIDE 5

Classifications(s): overview Mixture model solution Estimation Clustering with MixtComp Imputation with MixtComp Conclusion

Today’s data (2/2)

And also Ranking data Directional data Ordinal data Functional data Graphical data . . .

5/58

SLIDE 6

Classifications(s): overview Mixture model solution Estimation Clustering with MixtComp Imputation with MixtComp Conclusion

Supervised classification (1/3)

Data: learning dataset D = (xO, z)

n individuals: x = (x1, . . . , xn) = (xO, xM) belonging to a space X Observed individuals

xO

Missing individuals

xM

Partition in K groups G1, . . . , GK : z = (z1, . . . , zn), zi = (zi1, . . . , ziK)′ xi ∈ Gk ⇔ zih = I{h=k}

Aim: estimation of an allocation rule r from D r : X − → {1, . . . , K} xO

n+1

− → r(xO

n+1).

6/58

SLIDE 7

Classifications(s): overview Mixture model solution Estimation Clustering with MixtComp Imputation with MixtComp Conclusion

Supervised classification (2/3)

Mixed, missing, uncertain

Individuals xO Partition z ⇔ Group ? 0.5 red 5 1 ⇔ G2 0.3 0.1 green 3 1 ⇔ G1 0.3 0.6 {red,green} 3 1 ⇔ G1 0.9 [0.25 0.45] red ? 1 ⇔ G3 ↓ ↓ ↓ ↓ continuous continuous categorical integer

7/58

SLIDE 8

Classifications(s): overview Mixture model solution Estimation Clustering with MixtComp Imputation with MixtComp Conclusion

Supervised classification (3/3)

−6 −4 −2 2 4 6 8 −6 −4 −2 2 4 6 −4 −2 2 4 6 −5 −4 −3 −2 −1 1 2 3 4 5

1 2 3

(xO, z) and xO

n+1

ˆ r and ˆ zn+1

− →

?

8/58

SLIDE 9

Classifications(s): overview Mixture model solution Estimation Clustering with MixtComp Imputation with MixtComp Conclusion

Semi-supervised classification (1/3)

Data: learning dataset D = (xO, zO)

n individuals: x = (x1, . . . , xn) = (xO, xM) belonging to a space X Observed individuals

xO

Missing individuals

xM

Partition: z = (z1, . . . , zn) = (zO, zM) Observed partition

zO

Missing partition

zM

Aim: estimation of an allocation rule r from D r : X − → {1, . . . , K} xO

n+1

− → r(xO

n+1).

Idea: x is cheaper than z so #zM ≫ #zO

9/58

SLIDE 10

Classifications(s): overview Mixture model solution Estimation Clustering with MixtComp Imputation with MixtComp Conclusion

Semi-supervised classification (2/3)

Mixed, missing, uncertain

Individuals xO Partition zO ⇔ Group ? 0.5 red 5 ? ? ⇔ G2 or G3 0.3 0.1 green 3 1 ⇔ G1 0.3 0.6 {red,green} 3 ? ? ? ⇔ ??? 0.9 [0.25 0.45] red ? 1 ⇔ G3 ↓ ↓ ↓ ↓ continuous continuous categorical integer

10/58

SLIDE 11

Classifications(s): overview Mixture model solution Estimation Clustering with MixtComp Imputation with MixtComp Conclusion

Semi-supervised classification (3/3)

−6 −4 −2 2 4 6 8 −6 −4 −2 2 4 6 −4 −2 2 4 6 −5 −4 −3 −2 −1 1 2 3 4 5

1 2 3

(xO, zO) and xO

n+1

ˆ r and ˆ zn+1

− →

?

11/58

SLIDE 12

Classifications(s): overview Mixture model solution Estimation Clustering with MixtComp Imputation with MixtComp Conclusion

Unsupervised classification (1/3)

Data: learning dataset D = xO, so zO = ∅ Aim: estimation of the partition z and the number of groups K Also known as: clustering

12/58

SLIDE 13

Classifications(s): overview Mixture model solution Estimation Clustering with MixtComp Imputation with MixtComp Conclusion

Unsupervised classification (2/3)

Mixed, missing, uncertain

Individuals xO Partition zO ⇔ Group ? 0.5 red 5 ? ? ? ⇔ ??? 0.3 0.1 green 3 ? ? ? ⇔ ??? 0.3 0.6 {red,green} 3 ? ? ? ⇔ ??? 0.9 [0.25 0.45] red ? ? ? ? ⇔ ??? ↓ ↓ ↓ ↓ continuous continuous categorical integer

13/58

SLIDE 14

Classifications(s): overview Mixture model solution Estimation Clustering with MixtComp Imputation with MixtComp Conclusion

Unsupervised classification (3/3)

−6 −4 −2 2 4 6 −5 −4 −3 −2 −1 1 2 3 4 5 −6 −4 −2 2 4 6 8 −6 −4 −2 2 4 6

xO (xO, ˆ z)

− →

14/58

SLIDE 15

Classifications(s): overview Mixture model solution Estimation Clustering with MixtComp Imputation with MixtComp Conclusion

Traditional solutions (1/3)

Two main frameworks Generative models

Model p(x,

z)

Thus direct model for p(x) =

z p(x, z)

Easy to take into account some missing

z and x

Predictive models

Model p(z| x) or sometimes 1{p(z|x)>1/2} or also ranking on p(z| x) Avoid asumptions on p(x), thus avoids associated error model difficult to take into account some missing

z and x

15/58

SLIDE 16

Classifications(s): overview Mixture model solution Estimation Clustering with MixtComp Imputation with MixtComp Conclusion

Traditional solutions (2/3)

No mixed, missing or uncertain data: Supervised classification1

Generative models: linear/quadratic discriminant analysis Predictive models: logistic regression, support vector machines (SVM), k nearest neighbourhood, classification trees. . .

Semi-supervised classification2

Generative models: mixture models Predictive models: low density separation (transductive SVM), graph-based methods. . .

Unsupervised classification3

Generative models: k-means like criteria, hierarchical clustering, mixture models Predictive models: -

1Govaert et al., Data Analysis, Chap.6, 2009 2Chapelle et al., Semi-supervised learning, 2006 3Govaert et al., Data Analysis, Chap.7-9, 2009 16/58

SLIDE 17

Classifications(s): overview Mixture model solution Estimation Clustering with MixtComp Imputation with MixtComp Conclusion

Traditional solutions (3/3)

But more complex with mixed, missing or uncertain data. . . Missing/uncertain data: multiple imputation is possible but it should ideally take into account the classification purpose at hand Mixed data: some heuristic methods with recoding How to marry the classification aim with mixed, missing or uncertain data?

17/58

SLIDE 18

Classifications(s): overview Mixture model solution Estimation Clustering with MixtComp Imputation with MixtComp Conclusion

Outline

1 Classifications(s): overview 2 Mixture model solution 3 Estimation 4 Clustering with MixtComp 5 Imputation with MixtComp 6 Conclusion

18/58

SLIDE 19

Classifications(s): overview Mixture model solution Estimation Clustering with MixtComp Imputation with MixtComp Conclusion

Density estimation (1/2)

Data: learning dataset D = xO, so

zO = ∅

Aim: estimation of the distribution p(x) Extension easy to: D = (xO, zO) with

zO = ∅

Useful for: data imputation and multi-purpose classification!

19/58

SLIDE 20

Classifications(s): overview Mixture model solution Estimation Clustering with MixtComp Imputation with MixtComp Conclusion

Density estimation (2/2)

−6 −4 −2 2 4 6 −5 −4 −3 −2 −1 1 2 3 4 5 −6 −4 −2 2 4 6 8 −6 −4 −2 2 4 6 0.05 0.1 0.15 0.2 0.25 0.3 0.35

y x density

xO ˆ p(x)

− →

20/58

SLIDE 21

Classifications(s): overview Mixture model solution Estimation Clustering with MixtComp Imputation with MixtComp Conclusion

The mixture model answer in {∅,semi,un} classification

Rigorous definition of a group: x1 ∈ Gk ⇔ x1 is a realization of X1 ∼ pk(x1) Mixture formulation: X1|Z1k=1 ∼ pk(x1) Z1 ∼ MultK (1, π1, . . . , πK

π

) Joint and marginal (or mixture) distributions: (X1, Z1) ∼

K

k=1

[πkpk(x1)]z1k X1 ∼ p(x1) =

K

k=1

πkpk(x1) Maximum a posteriori (MAP): with tk(xO

1 ) = p(Z1k = 1|xO 1 ) = πkpk(xO

1 )

p(xO

1 )

r(xO

1 ) = arg

max

k={1,...,K} tk(xO 1 )

21/58

SLIDE 22

Classifications(s): overview Mixture model solution Estimation Clustering with MixtComp Imputation with MixtComp Conclusion

The mixture model answer for imputation

Straightforward also, for instance by the mode ˆ

xM = arg max xM p( xM| xO)

Other possibilities, depending on the data type: mean, etc.

Distribution p(

xM|xO)

It allows also to perform a specific multiple imputation!

22/58

SLIDE 23

Classifications(s): overview Mixture model solution Estimation Clustering with MixtComp Imputation with MixtComp Conclusion

The mixture model answer in density estimation

Mixture models: extremely flexible family of distributions

50 100 150 200 250 100 200 300 400 500 600

Niveaux de gris n* frequence

50 100 150 200 250 1 2 3 4 5 6 7 8 9 x 10

−3

Niveaux de gris Densite

Mixture of mixture models: flexibility for groups also

−6 −4 −2 2 4 6 8 10 −6 −4 −2 2 4 6 group 1 group 1 group 1 group 1 group 2 group 2 x1 x2 −6 −4 −2 2 4 6 8 10 −6 −4 −2 2 4 6 x1 x2

23/58

SLIDE 24

Classifications(s): overview Mixture model solution Estimation Clustering with MixtComp Imputation with MixtComp Conclusion

Parametric mixture model

Parametric assumption: pk(x1) = p(x1; αk) thus p(x1) = p(x1; θ) =

K

k=1

πkp(x1; αk) Mixture parameter: θ = (π, α) with α = (α1, . . . , αK ) Model: it includes both the family p(·; αk) and the number of groups K m = {p(x1; θ) : θ ∈ Θ} The number of free continuous parameters is given by ν = dim(Θ)

24/58

SLIDE 25

Classifications(s): overview Mixture model solution Estimation Clustering with MixtComp Imputation with MixtComp Conclusion

Mixed data: conditional independence everywhere

The aim is to combine continuous, categorical and integer data

x1 = ( xcont

1

,

xcat

1

,

xint

1 )

The proposed solution is to mixed all types by inter conditional independence p(x1; αk) = p(xcont

1

; αcont

k

) × p(xcat

1 ; αcat k ) × p(xint 1 ; αint k )

In addition, for symmetry between types, intra conditional independence for each type

25/58

SLIDE 26

Classifications(s): overview Mixture model solution Estimation Clustering with MixtComp Imputation with MixtComp Conclusion

Continuous: Gaussian mixture model

p(·; αcont

k

) = Nd(µk, Σk

diagonal

)

−2 2 4 6 −4 −2 2 4 0.02 0.04 0.06 0.08 0.1 0.12 x2 x1 f(x) −2 −1 1 2 3 4 0.05 0.1 0.15 0.2 0.25 0.3 0.35 x f(x) component 1 component 2 mixture density

26/58

SLIDE 27

Classifications(s): overview Mixture model solution Estimation Clustering with MixtComp Imputation with MixtComp Conclusion

Categorical: latent class model

categorical variables: d variables with mj modalities each, xj

i ∈ {0, 1}mj and

xjh

i

= 1 ⇔ variable j of xi takes modality h Intra conditional independence: p(xcat

i

; αcat

k ) = d

j=1

mj

h=1

(αjh

k )xjh

i

and αjh

k = p(X jh i

= 1|Zik = 1) with αk = (αjh

k ; j = 1, . . . , d; h = 1, . . . , mj)

27/58

SLIDE 28

Classifications(s): overview Mixture model solution Estimation Clustering with MixtComp Imputation with MixtComp Conclusion

Integer: Poisson mixture model

integer variables: d variables xj

i ∈ N

Intra conditional independence: p(xint

i

; αint

k ) = d

j=1

(αj

k)xj

i

αj

k!

e−αj

k 28/58

SLIDE 29

Classifications(s): overview Mixture model solution Estimation Clustering with MixtComp Imputation with MixtComp Conclusion

Outline

1 Classifications(s): overview 2 Mixture model solution 3 Estimation 4 Clustering with MixtComp 5 Imputation with MixtComp 6 Conclusion

29/58

SLIDE 30

Classifications(s): overview Mixture model solution Estimation Clustering with MixtComp Imputation with MixtComp Conclusion

Sampling assumptions

True distribution: D ∼ p(D) Model distribution: (xi, zi) i.i.d. ∼ p(x1, z1; θ) Gap between both, but flexibiliy: θ∗ = arg min

θ∈Θ KL(p, pθ)

where KL(p, pθ) = ED′[ln p(D′) − ln p(D′; θ)]

30/58

SLIDE 31

Classifications(s): overview Mixture model solution Estimation Clustering with MixtComp Imputation with MixtComp Conclusion

Observed-data log-likelihood estimation of θ

Principle: MLE ˆ θ = arg max

θ∈Θ ℓ(θ; D)

with observed log-likelihood ℓ(θ; D) = ln p(D; θ) = ln

xM
zM

p(

x, z; θ)d xM

Consistency: we have ˆ θ a.s. − → θ∗ Algorithm: SEM

31/58

SLIDE 32

Classifications(s): overview Mixture model solution Estimation Clustering with MixtComp Imputation with MixtComp Conclusion

SEM algorithm

Initialisation: θ(0) Iteration nb q:

E-step: compute conditional probabilities p(xM,

zM|D; θ(q))

S-step: draw (xM(q),

zM(q)) from p(xM, zM|D; θ(q))

M-step: maximize θ(q+1) = arg maxθ ln p(xO,

zO, xM(q), zM(q); θ)

Stopping rule: iteration number

Properties

simplicity because of conditional independence classical M steps avoids local maxima the mean of the sequence (θ(q)) approximates ˆ θ the variance of the sequence (θ(q)) gives confidence intervals

32/58

SLIDE 33

Classifications(s): overview Mixture model solution Estimation Clustering with MixtComp Imputation with MixtComp Conclusion

SE algorithm

A SE algorithm estimates then (

xM, zM)

Iteration nb q:

E-step: compute conditional probabilities p(xM,

zM|D; ˆ

θ) S-step: draw (xM(q),

zM(q)) from p(xM, zM|D; ˆ

θ)

Stopping rule: iteration number

Properties

simplicity because of conditional independence the mean/mode of the sequence (

xM(q), zM(q)) estimates ( xM, zM)

confidence intervals are also derived

33/58

SLIDE 34

Classifications(s): overview Mixture model solution Estimation Clustering with MixtComp Imputation with MixtComp Conclusion

Estimating K

Density estimation purpose: BIC = ln p(

xO, zO; ˆ

θ) − nb param. 2 ln(n) Clustering purpose: ICL = ln p(

xO, zO, ˆ zM; ˆ

θ) − nb param. 2 ln(n)

−1 1 2 3 4 5 6 −3 −2 −1 1 2 3 x1 x2

ˆ K 1 2 3 4 5 BIC . 60 . 32 8 ICL . 100 . . .

34/58

SLIDE 35

Classifications(s): overview Mixture model solution Estimation Clustering with MixtComp Imputation with MixtComp Conclusion

What about the process that causes missing data?

35/58

SLIDE 36

Classifications(s): overview Mixture model solution Estimation Clustering with MixtComp Imputation with MixtComp Conclusion

Outline

1 Classifications(s): overview 2 Mixture model solution 3 Estimation 4 Clustering with MixtComp 5 Imputation with MixtComp 6 Conclusion

36/58

SLIDE 37

Classifications(s): overview Mixture model solution Estimation Clustering with MixtComp Imputation with MixtComp Conclusion

Prostate cancer data4 (1/2)

Individuals: 506 patients with prostatic cancer grouped on clinical criteria into two Stages 3 and 4 of the disease Variables: d = 12 pre-trial variates were measured on each patient, composed by eight continuous variables (age, weight, systolic blood pressure, diastolic blood pressure, serum haemoglobin, size of primary tumour, index of tumour stage and histolic grade, serum prostatic acid phosphatase) and four categorical variables with various numbers of levels (performance rating, cardiovascular disease history, electrocardiogram code, bone metastases) Some missing data: 62 missing values (≈ 1%)

4Byar DP, Green SB (1980): Bulletin Cancer, Paris 67:477-488 37/58

SLIDE 38

Classifications(s): overview Mixture model solution Estimation Clustering with MixtComp Imputation with MixtComp Conclusion

Prostate cancer data (2/2)

38/58

SLIDE 39

Classifications(s): overview Mixture model solution Estimation Clustering with MixtComp Imputation with MixtComp Conclusion

Aim

We forget the classes (Stages of the desease) for performing clustering

Questions

How many clusters? Which partition? Visually not so easy. . .

−80 −60 −40 −20 20 40 60 −50 −40 −30 −20 −10 10 20 30 40

1st axis PCA 2nd axis PCA Continuous data

−2.5 −2 −1.5 −1 −0.5 0.5 1 −2 −1 1 2 3 4 5

1st axis MCA 2nd axis MCA Categorical data

39/58

SLIDE 40

Classifications(s): overview Mixture model solution Estimation Clustering with MixtComp Imputation with MixtComp Conclusion

Create an account in MixtComp

https://modal-research.lille.inria.fr/BigStat/ See documentation at https://modal.lille.inria.fr/wikimodal/doku.php?id=mixtcomp

40/58

SLIDE 41

Classifications(s): overview Mixture model solution Estimation Clustering with MixtComp Imputation with MixtComp Conclusion

Variable descriptor file: descriptor.csv

41/58

SLIDE 42

Classifications(s): overview Mixture model solution Estimation Clustering with MixtComp Imputation with MixtComp Conclusion

Syntax/allowed missing data

42/58

SLIDE 43

Classifications(s): overview Mixture model solution Estimation Clustering with MixtComp Imputation with MixtComp Conclusion

Data file: data.csv

43/58

SLIDE 44

Classifications(s): overview Mixture model solution Estimation Clustering with MixtComp Imputation with MixtComp Conclusion

Number of clusters file: param.ini

44/58

SLIDE 45

Classifications(s): overview Mixture model solution Estimation Clustering with MixtComp Imputation with MixtComp Conclusion

Input file: *.zip

descriptor.csv + data.csv + param.ini = NameYouWant.zip

45/58

SLIDE 46

Classifications(s): overview Mixture model solution Estimation Clustering with MixtComp Imputation with MixtComp Conclusion

Learn!

46/58

SLIDE 47

Classifications(s): overview Mixture model solution Estimation Clustering with MixtComp Imputation with MixtComp Conclusion

Output zip file

47/58

SLIDE 48

Classifications(s): overview Mixture model solution Estimation Clustering with MixtComp Imputation with MixtComp Conclusion

Output R format

48/58

SLIDE 49

Classifications(s): overview Mixture model solution Estimation Clustering with MixtComp Imputation with MixtComp Conclusion

Two strategies in competition

Strategy “mice5 + MixtComp”: MixtComp on the dataset completed by mice > data.imp=mice(data) > data.comp.mice=complete(data.imp) Strategy “full MixtComp”: MixtComp on the observed (no completed) dataset

5http://cran.r-project.org/web/packages/mice/mice.pdf 49/58

SLIDE 50

Classifications(s): overview Mixture model solution Estimation Clustering with MixtComp Imputation with MixtComp Conclusion

Choosing K with the ICL criterion

1 2 3 4 5 6 7 −12600 −12500 −12400 −12300 K ICL 1 2 3 4 5 6 7 −12550 −12450 −12350 −12250 K ICL

mice + MixtComp full MixtComp ˆ K = 7 ˆ K = 2 . . . may lose some cluster information when imputation before clustering

50/58

SLIDE 51

Classifications(s): overview Mixture model solution Estimation Clustering with MixtComp Imputation with MixtComp Conclusion

Partition quality with K = 2

Strategy mice + MixtComp full MixtComp % misclassified 12.8 8.1 To be compared also to missing data removal: 475 patients with non-missing data MixtComp for clustering possibility to consider continuous, categorical or mixed data Strategy continuous only categorical only mixed cont/cat % misclassified 9.46 47.16 8.63 risk of information lost when removing missing data lines/columns avoid to complete missing data (imputation depends on the purpose)

51/58

SLIDE 52

Classifications(s): overview Mixture model solution Estimation Clustering with MixtComp Imputation with MixtComp Conclusion

And for supervised classification?

Use now the predict functionality of MixtComp descriptor.csv + data.csv +

utput.RData

(from previous learn. . . ) = NameYouWant.zip Then same output format as the learn functionality of MixtComp

52/58

SLIDE 53

Classifications(s): overview Mixture model solution Estimation Clustering with MixtComp Imputation with MixtComp Conclusion

Outline

1 Classifications(s): overview 2 Mixture model solution 3 Estimation 4 Clustering with MixtComp 5 Imputation with MixtComp 6 Conclusion

53/58

SLIDE 54

Classifications(s): overview Mixture model solution Estimation Clustering with MixtComp Imputation with MixtComp Conclusion

Cancer dataset with more missing data

Add artificially ≈ 30% missing data with a MCAR design Then compare two strategies of imputation: Strategy “mice”: dataset completed by mice > data.imp=mice(data) > data.comp.mice=complete(data.imp) Strategy “full MixtComp”: MixtComp on the observed (no completed) dataset

1 2 3 4 5 6 −8780 −8760 −8740 −8720 −8700 −8680 −8660 −8640 K ICL 1 2 3 4 5 6 −8750 −8700 −8650 −8600 −8550 K BIC

ICL BIC ˆ K = 2 ˆ K = 4

54/58

SLIDE 55

Classifications(s): overview Mixture model solution Estimation Clustering with MixtComp Imputation with MixtComp Conclusion

Output multiple imputation by MixtComp

cont. cat.

55/58

SLIDE 56

Classifications(s): overview Mixture model solution Estimation Clustering with MixtComp Imputation with MixtComp Conclusion

Imputation accuracy

Continuous variables: mean of absolute difference between x and ˆ x var. mice MixtComp (K = 2) MixtComp (K = 4) Age 8.907143 5.546571 5.526861 Wt 13.51656 9.779485 9.731182 SBP 2.103226 1.788152 1.795820 DBP 1.317568 1.165201 1.169672 HG 21.67568 14.83514 14.51291 SZ 1.714899 1.160546 1.158105 SG 1.979866 1.386841 1.416053 AP 1.359299 1.027513 1.009126 Global mean 6.5718 4.5862 4.5400 Categorical variable: mean of the proportion of difference between x and ˆ x var. mice MixtComp (K = 2) MixtComp (K = 4) PF 0.1904762 0.0952381 0.0952381 HX 0.4121622 0.4391892 0.4121622 EKG 0.7564103 0.6858974 0.7179487 BM 0.1081081 0.1486486 0.1216216 Global mean 0.3668 0.3422 0.3367

56/58

SLIDE 57

Classifications(s): overview Mixture model solution Estimation Clustering with MixtComp Imputation with MixtComp Conclusion

Outline

1 Classifications(s): overview 2 Mixture model solution 3 Estimation 4 Clustering with MixtComp 5 Imputation with MixtComp 6 Conclusion

57/58

SLIDE 58

Classifications(s): overview Mixture model solution Estimation Clustering with MixtComp Imputation with MixtComp Conclusion

Conclusion

Clustering: work directly on observed (not imputed) data Imputation: possible since flexibility of mixture models for density estimation MixtComp: clustering and/or imputation for mixed data

Now: continuous, categorical, integer Next: ordinal, ranks, functional, directional

58/58