On the Use of NMF and curvHDR to Cluster Flow Cytometry Data e M. - - PowerPoint PPT Presentation

On the Use of NMF and curvHDR to Cluster Flow Cytometry Data

Jos´ e M. Maisog1,2, Andrea A. Barbo2, George Luta2

1Medical Numerics, Inc., Germantown, MD 20876 2Department of Biostatistics, Bioinformatics, and Biomathematics,

Georgetown University Medical Center, Washington, DC 20057

FlowCAP Summit, September 21-22, 2010

Outline

1

Non-Negative Matrix Factorization

NMF and curvHDR September, 2010 2 / 21

Outline

1

Non-Negative Matrix Factorization

2

curvHDR

NMF and curvHDR September, 2010 2 / 21

Outline

1

Non-Negative Matrix Factorization

2

curvHDR

3

Strategy for FlowCAP Challenge 2

NMF and curvHDR September, 2010 2 / 21

Outline

1

Non-Negative Matrix Factorization

2

curvHDR

3

Strategy for FlowCAP Challenge 2

4

Discussion

NMF and curvHDR September, 2010 2 / 21

Non-Negative Matrix Factorization

NMF

A relatively new method of matrix decomposition [LS99]

NMF and curvHDR September, 2010 3 / 21

Non-Negative Matrix Factorization

NMF

A relatively new method of matrix decomposition [LS99] Given Y , an M × N non-negative matrix

NMF and curvHDR September, 2010 3 / 21

Non-Negative Matrix Factorization

NMF

A relatively new method of matrix decomposition [LS99] Given Y , an M × N non-negative matrix Find W and H such that:

Y ≈ W × H

NMF and curvHDR September, 2010 3 / 21

Non-Negative Matrix Factorization

NMF

A relatively new method of matrix decomposition [LS99] Given Y , an M × N non-negative matrix Find W and H such that:

Y ≈ W × H W is M × k, H is k × N

NMF and curvHDR September, 2010 3 / 21

Non-Negative Matrix Factorization

NMF

A relatively new method of matrix decomposition [LS99] Given Y , an M × N non-negative matrix Find W and H such that:

Y ≈ W × H W is M × k, H is k × N W and H are non-negative

NMF and curvHDR September, 2010 3 / 21

Non-Negative Matrix Factorization

NMF

A relatively new method of matrix decomposition [LS99] Given Y , an M × N non-negative matrix Find W and H such that:

Y ≈ W × H W is M × k, H is k × N W and H are non-negative

Must specify k (cf. k-means clustering)

NMF and curvHDR September, 2010 3 / 21

Non-Negative Matrix Factorization

NMF

A relatively new method of matrix decomposition [LS99] Given Y , an M × N non-negative matrix Find W and H such that:

Y ≈ W × H W is M × k, H is k × N W and H are non-negative

Must specify k (cf. k-means clustering) Dimensionality Reduction: k < M, N

NMF and curvHDR September, 2010 3 / 21

Non-Negative Matrix Factorization

NMF

A relatively new method of matrix decomposition [LS99] Given Y , an M × N non-negative matrix Find W and H such that:

Y ≈ W × H W is M × k, H is k × N W and H are non-negative

Must specify k (cf. k-means clustering) Dimensionality Reduction: k < M, N There are multiple variations, e.g. different optimization criteria

NMF and curvHDR September, 2010 3 / 21

Non-Negative Matrix Factorization

NMF: Algorithm

les Variables (e.g., genes)

H = + E W • H

Samples

Y W

(Based on a figure from [You09])

NMF and curvHDR September, 2010 4 / 21

Non-Negative Matrix Factorization

NMF: Algorithm

les Variables (e.g., genes)

H = + E W • H

Samples

Y W

(Based on a figure from [You09]) Initialize W and H with random values.

NMF and curvHDR September, 2010 4 / 21

Non-Negative Matrix Factorization

NMF: Algorithm

les Variables (e.g., genes)

H = + E W • H

Samples

Y W

(Based on a figure from [You09]) Initialize W and H with random values. Optimize so that (yij − whij)2 is minimized.

NMF and curvHDR September, 2010 4 / 21

Non-Negative Matrix Factorization

NMF: Algorithm

les Variables (e.g., genes)

H = + E W • H

Samples

Y W

(Based on a figure from [You09]) Initialize W and H with random values. Optimize so that (yij − whij)2 is minimized. The k rows of H define “metagenes”, while the ith row of W represents the “metagene expression pattern of the corresponding sample” [Dev08]

NMF and curvHDR September, 2010 4 / 21

Non-Negative Matrix Factorization

NMF Results are “Sparse”

NMF has decomposed the face data into discrete “parts.” (Lee and Seung, Nature 1999 Oct 21;401(6755):788-91)

NMF and curvHDR September, 2010 5 / 21

Non-Negative Matrix Factorization

PCA of Face Data

Principal components are “holistic” rather than discrete “parts.” (Lee and Seung, Nature 1999 Oct 21;401(6755):788-91)

NMF and curvHDR September, 2010 6 / 21

Non-Negative Matrix Factorization

Comparison of Matrix Decomposition Methods

Method Constraints Basis Encodings PCA/SVD components are orthogonal non-sparse non-sparse ICA statistically independent components sparse non-sparse NMF data and factors are non-negative sparse sparse

NMF and curvHDR September, 2010 7 / 21

curvHDR

curvHDR

Unsupervised clustering with unknown number of clusters [NLW10]

NMF and curvHDR September, 2010 8 / 21

curvHDR

curvHDR

Unsupervised clustering with unknown number of clusters [NLW10] Algorithm:

Remove excess boundary points and other debris

NMF and curvHDR September, 2010 8 / 21

curvHDR

curvHDR

Unsupervised clustering with unknown number of clusters [NLW10] Algorithm:

Remove excess boundary points and other debris Obtain significant high negative curvature regions [DCKW08]

NMF and curvHDR September, 2010 8 / 21

curvHDR

curvHDR

Unsupervised clustering with unknown number of clusters [NLW10] Algorithm:

Remove excess boundary points and other debris Obtain significant high negative curvature regions [DCKW08] Replace each of the significant curvature regions by their convex hull

NMF and curvHDR September, 2010 8 / 21

curvHDR

curvHDR

Unsupervised clustering with unknown number of clusters [NLW10] Algorithm:

Remove excess boundary points and other debris Obtain significant high negative curvature regions [DCKW08] Replace each of the significant curvature regions by their convex hull Grow each convex hull by a factor G.

NMF and curvHDR September, 2010 8 / 21

curvHDR

curvHDR

Unsupervised clustering with unknown number of clusters [NLW10] Algorithm:

Remove excess boundary points and other debris Obtain significant high negative curvature regions [DCKW08] Replace each of the significant curvature regions by their convex hull Grow each convex hull by a factor G. Obtain a kernel density estimate for data within each grown region [DCKW08]

NMF and curvHDR September, 2010 8 / 21

curvHDR

curvHDR

Unsupervised clustering with unknown number of clusters [NLW10] Algorithm:

Remove excess boundary points and other debris Obtain significant high negative curvature regions [DCKW08] Replace each of the significant curvature regions by their convex hull Grow each convex hull by a factor G. Obtain a kernel density estimate for data within each grown region [DCKW08] The curvHDR gate is the union of the level-τ high density regions (HDRs).

NMF and curvHDR September, 2010 8 / 21

curvHDR

curvHDR

Unsupervised clustering with unknown number of clusters [NLW10] Algorithm:

Remove excess boundary points and other debris Obtain significant high negative curvature regions [DCKW08] Replace each of the significant curvature regions by their convex hull Grow each convex hull by a factor G. Obtain a kernel density estimate for data within each grown region [DCKW08] The curvHDR gate is the union of the level-τ high density regions (HDRs).

Currently only the 2D version is implemented, but a 3D version will be released soon

NMF and curvHDR September, 2010 8 / 21

curvHDR

curvHDR: Illustration

(Naumann et al., BMC Bioinformatics 2010 Jan 22;11:44)

NMF and curvHDR September, 2010 9 / 21

curvHDR

GvHD Data, Sample #7, HDR Level = 0.1

• ●
• ●
• 100

200 300 400 500 200 400 600 800

curvHDR filter with HDR level= 0.1

SSC.H FL2.H

NMF and curvHDR September, 2010 10 / 21

curvHDR

GvHD Data, Sample #7, HDR Level = 0.2

• ●
• ●
• 100

200 300 400 500 200 400 600 800

curvHDR filter with HDR level= 0.2

SSC.H FL2.H

NMF and curvHDR September, 2010 11 / 21

Strategy for FlowCAP Challenge 2

Processing Pipeline

NMF

input data clustered data

Variance Normalization asinh curvHDR

All computations performed in R.

NMF and curvHDR September, 2010 12 / 21

Strategy for FlowCAP Challenge 2

Processing Pipeline

NMF

input data clustered data

Variance Normalization asinh curvHDR

All computations performed in R. Normalize variance of variables (channels) to 1

NMF and curvHDR September, 2010 12 / 21

Strategy for FlowCAP Challenge 2

Processing Pipeline

NMF

input data clustered data

Variance Normalization asinh curvHDR

All computations performed in R. Normalize variance of variables (channels) to 1 Use NMF to reduce dimensionality to 2 http://cran.r-project.org/web/packages/NMF/index.html

NMF and curvHDR September, 2010 12 / 21

Strategy for FlowCAP Challenge 2

Processing Pipeline

NMF

input data clustered data

Variance Normalization asinh curvHDR

All computations performed in R. Normalize variance of variables (channels) to 1 Use NMF to reduce dimensionality to 2 http://cran.r-project.org/web/packages/NMF/index.html NMF algorithm: “left matrix” version of Kim and Park’s Alternating Least Squares method [KP07]

NMF and curvHDR September, 2010 12 / 21

Strategy for FlowCAP Challenge 2

Processing Pipeline

NMF

input data clustered data

Variance Normalization asinh curvHDR

All computations performed in R. Normalize variance of variables (channels) to 1 Use NMF to reduce dimensionality to 2 http://cran.r-project.org/web/packages/NMF/index.html NMF algorithm: “left matrix” version of Kim and Park’s Alternating Least Squares method [KP07] Transform the NMF encodings (the W matrix) with the inverse hyperbolic sine function: asinh(x) = ln(x + √ x2 + 1)

NMF and curvHDR September, 2010 12 / 21

Strategy for FlowCAP Challenge 2

Processing Pipeline

NMF

input data clustered data

Variance Normalization asinh curvHDR

All computations performed in R. Normalize variance of variables (channels) to 1 Use NMF to reduce dimensionality to 2 http://cran.r-project.org/web/packages/NMF/index.html NMF algorithm: “left matrix” version of Kim and Park’s Alternating Least Squares method [KP07] Transform the NMF encodings (the W matrix) with the inverse hyperbolic sine function: asinh(x) = ln(x + √ x2 + 1) Then use 2D version of curvHDR to perform clustering http://www.uow.edu.au/˜mwand/Rpacks.html

NMF and curvHDR September, 2010 12 / 21

Strategy for FlowCAP Challenge 2

Example of NMF-curvHDR Results

• ●
• 0.0

0.5 1.0 1.5 2.0 0.0 0.5 1.0 1.5 2.0

GvHD, sample #7, HDR Level = 0.1

NMF Factor #1 NMF Factor #2

NMF and curvHDR September, 2010 13 / 21

Discussion

Discussion

NMF is a relatively new matrix decomposition method that has been used in molecular pattern discovery and cross-platform and cross-species characterization (especially in gene and protein microarray data), biomedical informatics (e.g., text mining), and magnetic resonance spectroscopy [Dev08]

NMF and curvHDR September, 2010 14 / 21

Discussion

Discussion

NMF is a relatively new matrix decomposition method that has been used in molecular pattern discovery and cross-platform and cross-species characterization (especially in gene and protein microarray data), biomedical informatics (e.g., text mining), and magnetic resonance spectroscopy [Dev08] NMF may have utility in flow cytometry data

NMF and curvHDR September, 2010 14 / 21

Discussion

Discussion

NMF is a relatively new matrix decomposition method that has been used in molecular pattern discovery and cross-platform and cross-species characterization (especially in gene and protein microarray data), biomedical informatics (e.g., text mining), and magnetic resonance spectroscopy [Dev08] NMF may have utility in flow cytometry data Could have used some other method (e.g., PCA or ICA) for dimensionality reduction, but decided to use NMF because of its good performance with other types of data (out of curiosity, too).

NMF and curvHDR September, 2010 14 / 21

Discussion

Discussion

NMF is a relatively new matrix decomposition method that has been used in molecular pattern discovery and cross-platform and cross-species characterization (especially in gene and protein microarray data), biomedical informatics (e.g., text mining), and magnetic resonance spectroscopy [Dev08] NMF may have utility in flow cytometry data Could have used some other method (e.g., PCA or ICA) for dimensionality reduction, but decided to use NMF because of its good performance with other types of data (out of curiosity, too). curvHDR is an even newer method than NMF, specifically for clustering flow cytometry data

NMF and curvHDR September, 2010 14 / 21

Discussion

END

NMF and curvHDR September, 2010 15 / 21

Discussion

References I

[Bre01] L Breiman. Random forests. Machine Learning, 45(1):5–32, March 2001. [DCKW08] T Duong, A Cowling, I Koch, and MP Wand. Feature significance for multivariate kernel density estimation. Computational Statistics and Data Analysis, 52:42254242, May 2008. [Dev08] K Devarajan. Nonnegative matrix factorization: an analytical and interpretive tool in computational biology. PLoS Computational Biology, 4(7):e1000029, July 2008. [GEGMMI08] LA Garcia-Escudero, A Gordaliza, C Matran, and A Mayo-Iscar. A general trimming approach to robust cluster analysis. Annals of Statistics, 36(3):1324–1345, 2008. [KP07] H Kim and H Park. Sparse non-negative matrix factorizations via alternating non-negativity-constrained least squares for microarray data analysis. Bioinformatics, 23(12):1495–502, June 2007.

NMF and curvHDR September, 2010 16 / 21

Discussion

References II

[LS99] DD Lee and HS Seung. Learning the parts of objects by non-negative matrix factorization. Nature, 401(6755):788–91, October 1999. [NLW10] U Naumann, G Luta, and MP Wand. The curvHDR method for gating flow cytometry samples. BMC Bioinformatics, 22:44, January 2010. [You09] SS Young. Pre-processing HCS data using non-negative matrix

• factorization. In Midwest Biopharmaceutical Statistics Workshop,

2009.

NMF and curvHDR September, 2010 17 / 21

Discussion

Strategy for FlowCAP Challenge 3: NMF

Use NMF to define clusters, set k equal to desired number of clusters

NMF and curvHDR September, 2010 18 / 21

Discussion

Strategy for FlowCAP Challenge 3: NMF

Use NMF to define clusters, set k equal to desired number of clusters Problem #1: NMF is usually used to reduce dimensionality in high-dimensional data

Sometimes the desired number of clusters is actually greater than the dimensionality

NMF and curvHDR September, 2010 18 / 21

Discussion

Strategy for FlowCAP Challenge 3: NMF

Use NMF to define clusters, set k equal to desired number of clusters Problem #1: NMF is usually used to reduce dimensionality in high-dimensional data

Sometimes the desired number of clusters is actually greater than the dimensionality E.g., after removing the FS and SS variables, the GvHD data had only four variables (FL1.H, FL2.H,FL3.H,, and FL4.H)

NMF and curvHDR September, 2010 18 / 21

Discussion

Strategy for FlowCAP Challenge 3: NMF

Use NMF to define clusters, set k equal to desired number of clusters Problem #1: NMF is usually used to reduce dimensionality in high-dimensional data

Sometimes the desired number of clusters is actually greater than the dimensionality E.g., after removing the FS and SS variables, the GvHD data had only four variables (FL1.H, FL2.H,FL3.H,, and FL4.H) But GvHD samples #7 and #9 had five groups by manual analysis!

NMF and curvHDR September, 2010 18 / 21

Discussion

Strategy for FlowCAP Challenge 3: NMF

Use NMF to define clusters, set k equal to desired number of clusters Problem #1: NMF is usually used to reduce dimensionality in high-dimensional data

Sometimes the desired number of clusters is actually greater than the dimensionality E.g., after removing the FS and SS variables, the GvHD data had only four variables (FL1.H, FL2.H,FL3.H,, and FL4.H) But GvHD samples #7 and #9 had five groups by manual analysis! Possible kludge: add an extra column for these cases.

NMF and curvHDR September, 2010 18 / 21

Discussion

Strategy for FlowCAP Challenge 3: NMF

Use NMF to define clusters, set k equal to desired number of clusters Problem #1: NMF is usually used to reduce dimensionality in high-dimensional data

Sometimes the desired number of clusters is actually greater than the dimensionality E.g., after removing the FS and SS variables, the GvHD data had only four variables (FL1.H, FL2.H,FL3.H,, and FL4.H) But GvHD samples #7 and #9 had five groups by manual analysis! Possible kludge: add an extra column for these cases.

Problem #2: algorithm fails if there is a row in the data matrix whose entries are all zero

This situation does indeed arise in the flowCAP data sets after the FS and SS variables are removed.

NMF and curvHDR September, 2010 18 / 21

Discussion

Strategy for FlowCAP Challenge 3: NMF

Use NMF to define clusters, set k equal to desired number of clusters Problem #1: NMF is usually used to reduce dimensionality in high-dimensional data

Sometimes the desired number of clusters is actually greater than the dimensionality E.g., after removing the FS and SS variables, the GvHD data had only four variables (FL1.H, FL2.H,FL3.H,, and FL4.H) But GvHD samples #7 and #9 had five groups by manual analysis! Possible kludge: add an extra column for these cases.

Problem #2: algorithm fails if there is a row in the data matrix whose entries are all zero

This situation does indeed arise in the flowCAP data sets after the FS and SS variables are removed. Possible kludge: if there’s a row whose entries are all zero, add a small amount of non-negative noise to the entries of that row.

NMF and curvHDR September, 2010 18 / 21

Discussion

Strategy for FlowCAP Challenge 3: Trimmed Clustering

Instead, try trimmed clustering method (tCLUST; [GEGMMI08])

NMF and curvHDR September, 2010 19 / 21

Discussion

Strategy for FlowCAP Challenge 3: Trimmed Clustering

Instead, try trimmed clustering method (tCLUST; [GEGMMI08]) Similar to curvHDR, a fraction of the observations are discarded (“trimmed”) and considered not part of any group.

NMF and curvHDR September, 2010 19 / 21

Discussion

Strategy for FlowCAP Challenge 3: Trimmed Clustering

Instead, try trimmed clustering method (tCLUST; [GEGMMI08]) Similar to curvHDR, a fraction of the observations are discarded (“trimmed”) and considered not part of any group. Unlike curvHDR, the number of desired clusters is specified as input to the algorithm.

NMF and curvHDR September, 2010 19 / 21

Discussion

Strategy for FlowCAP Challenge 3: Trimmed Clustering

Instead, try trimmed clustering method (tCLUST; [GEGMMI08]) Similar to curvHDR, a fraction of the observations are discarded (“trimmed”) and considered not part of any group. Unlike curvHDR, the number of desired clusters is specified as input to the algorithm. http://cran.r-project.org/web/packages/tclust/index.html

NMF and curvHDR September, 2010 19 / 21

Discussion

Example of TCLUST Results (GvHD data, sample #7)

• ●
• ●
• ●
• ●
• ●
• ●
• ●
• ●
• ●
• ●
• ●
• ●
• ●
• ●
• ●
• ●
• ●
• ●
• ●
• ●
• ●
• ●
• ●
• ●
• ●
• ●
• ●
• ●
• ●
• ●
• ●
• ●
• ●
• ●
• ●
• ●
• ●
• ●
• ●
• ●
• ●
• ●
• ●
• ●
• ●
• ●
• ●
• ●
• ●
• Classification

k = 5, alpha = 0.05 First discriminant coord. Second discriminant coord.

k = 5, alpha = 0.05

NMF and curvHDR September, 2010 20 / 21

Discussion

Strategy for FlowCAP Challenge 4: Random Forests

Classification and Regression Tree (CART), recursive partitioning (similar to Decision Trees and C4.5)

NMF and curvHDR September, 2010 21 / 21

Discussion

Strategy for FlowCAP Challenge 4: Random Forests

Classification and Regression Tree (CART), recursive partitioning (similar to Decision Trees and C4.5)

Interpretability

NMF and curvHDR September, 2010 21 / 21

Discussion

Strategy for FlowCAP Challenge 4: Random Forests

Classification and Regression Tree (CART), recursive partitioning (similar to Decision Trees and C4.5)

Interpretability Can handle heterogeneous data (continuous/interval, categorical,

• rdinal)

NMF and curvHDR September, 2010 21 / 21

Discussion

Strategy for FlowCAP Challenge 4: Random Forests

Classification and Regression Tree (CART), recursive partitioning (similar to Decision Trees and C4.5)

Interpretability Can handle heterogeneous data (continuous/interval, categorical,

• rdinal)

Can handle missing data

NMF and curvHDR September, 2010 21 / 21

Discussion

Strategy for FlowCAP Challenge 4: Random Forests

Classification and Regression Tree (CART), recursive partitioning (similar to Decision Trees and C4.5)

Interpretability Can handle heterogeneous data (continuous/interval, categorical,

• rdinal)

Can handle missing data

“Random forests” [Bre01] is an ensemble extension of Classification and Regression Tree (CART) approach

NMF and curvHDR September, 2010 21 / 21

Discussion

Strategy for FlowCAP Challenge 4: Random Forests

Classification and Regression Tree (CART), recursive partitioning (similar to Decision Trees and C4.5)

Interpretability Can handle heterogeneous data (continuous/interval, categorical,

• rdinal)

Can handle missing data

“Random forests” [Bre01] is an ensemble extension of Classification and Regression Tree (CART) approach http://cran.r-project.org/web/packages/randomForest/

NMF and curvHDR September, 2010 21 / 21