cutting the dendrogram through permutation tests
play

Cutting the dendrogram through permutation tests Dario Bruzzese - PowerPoint PPT Presentation

Feb 07, 2023 •23 likes •693 views

Cutting the dendrogram through permutation tests Dario Bruzzese Domenico Vistocco dbruzzes@unina.it vistocco@unicas.it Department of Department of Preventive Medical Sciences Economics U NIVERSITY OF N APLES ITALY U NIVERSITY OF C ASSINO

  1. Cutting the dendrogram through permutation tests Dario Bruzzese Domenico Vistocco dbruzzes@unina.it vistocco@unicas.it Department of Department of Preventive Medical Sciences Economics U NIVERSITY OF N APLES ITALY U NIVERSITY OF C ASSINO ITALY Dario Bruzzese, Domenico Vistocco () Compstat 2010 1 / 19

  2. La Carte Motivation 1 The stairstep-like permutation procedure 2 Notation The outline Some results 3 Real datasets Synthetic dataset ToDo List 4 Dario Bruzzese, Domenico Vistocco () Compstat 2010 2 / 19

  3. La Carte Motivation 1 The stairstep-like permutation procedure 2 Notation The outline Some results 3 Real datasets Synthetic dataset ToDo List 4 Dario Bruzzese, Domenico Vistocco () Compstat 2010 3 / 19

  4. Motivation Automatically determine the optimal cut-off level of a dendrogram Explore partitions different from those allowed by an horizontal cut Dario Bruzzese, Domenico Vistocco () Compstat 2010 4 / 19

  5. Motivation Automatically determine the optimal cut-off level of a dendrogram Explore partitions different from those allowed by an horizontal cut The rep1HighNoise dataset Yeung KY, Medvedovic M, Bumgarner KY: Clustering gene-expression data with repeated measurements. Genome Biology, 2003, 4:R34 n = 200 p = 20 Dario Bruzzese, Domenico Vistocco () Compstat 2010 4 / 19

  6. Motivation Automatically determine the optimal cut-off level of a dendrogram Explore partitions different from those allowed by an horizontal cut Horizontal cut k = 3 Dario Bruzzese, Domenico Vistocco () Compstat 2010 4 / 19

  7. Motivation Automatically determine the optimal cut-off level of a dendrogram Explore partitions different from those allowed by an horizontal cut An alternative cut k = 3 Dario Bruzzese, Domenico Vistocco () Compstat 2010 4 / 19

  8. La Carte Motivation 1 The stairstep-like permutation procedure 2 Notation The outline Some results 3 Real datasets Synthetic dataset ToDo List 4 Dario Bruzzese, Domenico Vistocco () Compstat 2010 5 / 19

  9. Notation Let: Dario Bruzzese, Domenico Vistocco () Compstat 2010 6 / 19

  10. Notation Let: n the number of objects to classify; � � �� �� � � Dario Bruzzese, Domenico Vistocco () Compstat 2010 6 / 19

  11. Notation Let: n the number of objects to classify; C k L and C k R the two classes merged at level k (k=1,...,n-1) Dario Bruzzese, Domenico Vistocco () Compstat 2010 6 / 19

  12. Notation Let: n the number of objects to classify; C k L and C k R the two classes merged at level k (k=1,...,n-1) C 1 C 1 L R Dario Bruzzese, Domenico Vistocco () Compstat 2010 6 / 19

  13. Notation Let: n the number of objects to classify; C k L and C k R the two classes merged at level k (k=1,...,n-1) C 2 C 2 L R Dario Bruzzese, Domenico Vistocco () Compstat 2010 6 / 19

  14. Notation Let: n the number of objects to classify; C k L and C k R the two classes merged at level k (k=1,...,n-1) C 3 C 3 L R Dario Bruzzese, Domenico Vistocco () Compstat 2010 6 / 19

  15. Notation Let: n the number of objects to classify; C k L and C k R the two classes merged at level k (k=1,...,n-1) � � C k L ∪ C k h the height necessary to merge R C k L and C k R Dario Bruzzese, Domenico Vistocco () Compstat 2010 6 / 19

  16. Notation � � C 1 L ∪ C 1 h R Let: n the number of objects to classify; C k L and C k R the two classes merged at level k (k=1,...,n-1) � � C k L ∪ C k h the height necessary to merge R C k L and C k R C 1 C 1 L R Dario Bruzzese, Domenico Vistocco () Compstat 2010 6 / 19

  17. Notation Let: n the number of objects to classify; C k L and C k R the two classes merged at level k (k=1,...,n-1) � � C 2 L ∪ C 2 h � � C k L ∪ C k R h the height necessary to merge R C k L and C k R C 2 C 2 L R Dario Bruzzese, Domenico Vistocco () Compstat 2010 6 / 19

  18. Notation Let: n the number of objects to classify; C k L and C k R the two classes merged at level k (k=1,...,n-1) � � C 3 L ∪ C 3 � � C k L ∪ C k h h the height necessary to merge R R C k L and C k R C 3 C 3 L R Dario Bruzzese, Domenico Vistocco () Compstat 2010 6 / 19

  19. Notation Let: n the number of objects to classify; C k L and C k R the two classes merged at level k (k=1,...,n-1) � � C k L ∪ C k h the height necessary to merge R C k L and C k R � � C k the height at which C k h j has been obtained j (j ∈ { L, R }) Dario Bruzzese, Domenico Vistocco () Compstat 2010 6 / 19

  20. Notation Let: n the number of objects to classify; C k L and C k R the two classes merged at level k (k=1,...,n-1) � � C k L ∪ C k � � h the height necessary to merge C 1 h R L C k L and C k R � � C k the height at which C k h j has been obtained j (j ∈ { L, R }) C 1 L Dario Bruzzese, Domenico Vistocco () Compstat 2010 6 / 19

  21. Notation Let: n the number of objects to classify; C k L and C k R the two classes merged at level k (k=1,...,n-1) � � C 1 � � h C k L ∪ C k h the height necessary to merge R R C k L and C k R � � C k the height at which C k h j has been obtained j (j ∈ { L, R }) C 1 R Dario Bruzzese, Domenico Vistocco () Compstat 2010 6 / 19

  22. Notation Let: n the number of objects to classify; C k L and C k R the two classes merged at level k (k=1,...,n-1) � � C k L ∪ C k h the height necessary to merge R � � C 2 C k L and C k h L R � � C k the height at which C k h j has been obtained j (j ∈ { L, R }) C 2 L Dario Bruzzese, Domenico Vistocco () Compstat 2010 6 / 19

  23. Notation Let: n the number of objects to classify; C k L and C k R the two classes merged at level k (k=1,...,n-1) � � C k L ∪ C k h the height necessary to merge R � � C 2 h C k L and C k R R � � C k the height at which C k h j has been obtained j (j ∈ { L, R }) C 2 R Dario Bruzzese, Domenico Vistocco () Compstat 2010 6 / 19

  24. Notation Let: n the number of objects to classify; C k L and C k R the two classes merged at level k (k=1,...,n-1) � � C k L ∪ C k h the height necessary to merge R C k L and C k � � C 3 h R L � � C k the height at which C k h j has been obtained j (j ∈ { L, R }) C 3 L Dario Bruzzese, Domenico Vistocco () Compstat 2010 6 / 19

  25. Notation Let: n the number of objects to classify; C k L and C k R the two classes merged at level k (k=1,...,n-1) � � C k L ∪ C k h the height necessary to merge � � R C 3 h R C k L and C k R � � C k the height at which C k h j has been obtained j (j ∈ { L, R }) C 3 R Dario Bruzzese, Domenico Vistocco () Compstat 2010 6 / 19

  26. The algorithm - Pseudo Code Input : A dataset and its related dendrogram Output : A partition of the dataset Dario Bruzzese, Domenico Vistocco () Compstat 2010 7 / 19

  27. The algorithm - Pseudo Code Input : A dataset and its related dendrogram Output : A partition of the dataset initialization: aggregationLevelsToVisit ← h ( C 1 L ∪ C 1 R ) permClusters ← [ ] i ← 1 Dario Bruzzese, Domenico Vistocco () Compstat 2010 7 / 19

  28. The algorithm - Pseudo Code Input : A dataset and its related dendrogram Output : A partition of the dataset initialization: aggregationLevelsToVisit ← h ( C 1 L ∪ C 1 R ) permClusters ← [ ] i ← 1 repeat if C i L ≡ C i R then add C i L ∪ C i R to permClusters else add h ( C i L ) and h ( C i R ) to aggregationLevelsToVisit sort aggregationLevelsToVisit in descending order end Dario Bruzzese, Domenico Vistocco () Compstat 2010 7 / 19

  29. The algorithm - Pseudo Code Input : A dataset and its related dendrogram Output : A partition of the dataset initialization: aggregationLevelsToVisit ← h ( C 1 L ∪ C 1 R ) permClusters ← [ ] i ← 1 repeat if C i L ≡ C i R then add C i L ∪ C i R to permClusters else add h ( C i L ) and h ( C i R ) to aggregationLevelsToVisit sort aggregationLevelsToVisit in descending order end remove the first element from aggregationLevelsToVisit i ← i+1 Dario Bruzzese, Domenico Vistocco () Compstat 2010 7 / 19

  30. The algorithm - Pseudo Code Input : A dataset and its related dendrogram Output : A partition of the dataset initialization: aggregationLevelsToVisit ← h ( C 1 L ∪ C 1 R ) permClusters ← [ ] i ← 1 repeat if C i L ≡ C i R then add C i L ∪ C i R to permClusters else add h ( C i L ) and h ( C i R ) to aggregationLevelsToVisit sort aggregationLevelsToVisit in descending order end remove the first element from aggregationLevelsToVisit i ← i+1 until aggregationLevelsToVisit is empty Dario Bruzzese, Domenico Vistocco () Compstat 2010 7 / 19

  31. The algorithm - The outline Initialization i ← 0 aggregationLevelsToVisit h ( C 1 L ∪ C 1 R ) permClusters Dario Bruzzese, Domenico Vistocco () Compstat 2010 8 / 19

  32. The algorithm - The outline � � C 1 L ∪ C 1 h R Iteration i ← 1 aggregationLevelsToVisit h ( C 1 L ∪ C 1 R ) permClusters Dario Bruzzese, Domenico Vistocco () Compstat 2010 8 / 19

  33. The algorithm - The outline Iteration i ← 1 aggregationLevelsToVisit h ( C 1 L ∪ C 1 R ) permClusters C 1 C 1 L R clusters to compare H 0 : C 1 L ≡ C 1 R �→ reject Dario Bruzzese, Domenico Vistocco () Compstat 2010 8 / 19

Download Presentation
Download Policy: The content available on the website is offered to you 'AS IS' for your personal information and use only. It cannot be commercialized, licensed, or distributed on other websites without prior consent from the author. To download a presentation, simply click this link. If you encounter any difficulties during the download process, it's possible that the publisher has removed the file from their server.

Recommend

Stairstep-like dendrogram cut: a permutation test approach Dario Bruzzese Domenico Vistocco

Stairstep-like dendrogram cut: a permutation test approach Dario Bruzzese Domenico Vistocco

Stairstep-like dendrogram cut: a permutation test approach Dario Bruzzese Domenico Vistocco dbruzzes@unina.it vistocco@unicas.it Department of

1.46k views • 102 slides
Stairstep-like dendrogram cut: a permutation test approach Dario Bruzzese Umberto Giani

Stairstep-like dendrogram cut: a permutation test approach Dario Bruzzese Umberto Giani

Stairstep-like dendrogram cut: a permutation test approach Dario Bruzzese Umberto Giani Domenico Vistocco dbruzzes@unina.it ugiani@unina.it vistocco@unicas.it -

1.51k views • 84 slides
Nonparametric hypothesis tests and permutation tests 1.7 & 2.3. Probability Generating

Nonparametric hypothesis tests and permutation tests 1.7 & 2.3. Probability Generating

Nonparametric hypothesis tests and permutation tests 1.7 & 2.3. Probability Generating Functions 3.8.3. Wilcoxon Signed Rank Test 3.8.2. Mann-Whitney Test Prof. Tesler Math 283 Fall 2018 Prof. Tesler Wilcoxon and Mann-Whitney Tests

554 views • 36 slides
Lecture 21: Bootstrap and Permutation Tests The bootstrap Bootstrapping generally refers to

Lecture 21: Bootstrap and Permutation Tests The bootstrap Bootstrapping generally refers to

Mathematical Tools for Neuroscience (NEU 314) Princeton University, Spring 2016 Jonathan Pillow Lecture 21: Bootstrap and Permutation Tests The bootstrap Bootstrapping generally refers to statistical approach to quantifying uncertainty by

345 views • 3 slides
Permutation tests for coefficients of variation in general one-way ANOVA models Markus Pauly 1 and

Permutation tests for coefficients of variation in general one-way ANOVA models Markus Pauly 1 and

Permutation tests for coefficients of variation in general one-way ANOVA models Markus Pauly 1 and ukasz Smaga 2 1 Faculty of Statistics TU Dortmund University Dortmund, Germany 2 Faculty of Mathematics and Computer Science Adam Mickiewicz

360 views • 34 slides
Permutation tests Fabian Pedregosa October 3, 2017 Data Science Learn2Launch, UC Berkeley

Permutation tests Fabian Pedregosa October 3, 2017 Data Science Learn2Launch, UC Berkeley

Permutation tests Fabian Pedregosa October 3, 2017 Data Science Learn2Launch, UC Berkeley Announcements Next week is the first presentation! 1. 10 min presentation (by teams) + 5 min questions 2. At least: objective of the project,

531 views • 33 slides
On Synchronized Permutation tests in Two-way ANOVA Dario Basso 1 , Luigi Salmaso 1 , Fortunato

On Synchronized Permutation tests in Two-way ANOVA Dario Basso 1 , Luigi Salmaso 1 , Fortunato

On Synchronized Permutation tests in Two-way ANOVA Dario Basso 1 , Luigi Salmaso 1 , Fortunato Pesarin 2 1 : Department of Management and Engineering, University of Padova 2 : Department of Statistics, University of Padova dario@stat.unipd.it

717 views • 29 slides
Constraint-based Bayesian Network Learning with Permutation Tests Marco Scutari

Constraint-based Bayesian Network Learning with Permutation Tests Marco Scutari

Constraint-based Bayesian Network Learning with Permutation Tests Marco Scutari marco.scutari@stat.unipd.it Adriana Brogini brogini@stat.unipd.it Department of Statistical Sciences University of Padova June 15, 2010 Marco Scutari and

404 views • 15 slides
Lecture 4: Permutation Methods Applied Statistics 2014 1 / 21 Randomization Model Population

Lecture 4: Permutation Methods Applied Statistics 2014 1 / 21 Randomization Model Population

Randomization Model Population Model Rank Tests Assignment Lecture 4: Permutation Methods Applied Statistics 2014 1 / 21 Randomization Model Population Model Rank Tests Assignment Permutation Methods Non-parametric methods for testing

351 views • 23 slides
Dendrogram-based Algorithm for Dominated Graph Flooding Claude

Dendrogram-based Algorithm for Dominated Graph Flooding Claude

Dendrogram-based Algorithm for Dominated Graph Flooding Claude Tadonki Joint work with Fernand Meyer and Franois Irigoin C entre de R echerche en I nformatique - C entre de M orphologie M athmatique Mines

293 views • 18 slides
J , the all- 1 matrix. (The corresponding only G -invariant association scheme is the trivial A

J , the all- 1 matrix. (The corresponding only G -invariant association scheme is the trivial A

Association schemes and Permutation groups permutation groups Let G be a permutation group on . Then the characteristic functions of the orbits of G on Peter J Cameron G

44 views • 3 slides
The diameter of permutation groups permutation groups H. A. Helfgott February 2017 The

The diameter of permutation groups permutation groups H. A. Helfgott February 2017 The

The diameter of permutation groups H. A. Helfgott Introduction Diameter bounds New work on The diameter of permutation groups permutation groups H. A. Helfgott February 2017 The diameter of Cayley graphs permutation groups H. A.

751 views • 64 slides
Table Saw BEST PRACTICES 2018 Table Saw Purpose NCUs Table Saw Purposes Cutting

Table Saw BEST PRACTICES 2018 Table Saw Purpose NCUs Table Saw Purposes Cutting

Table Saw BEST PRACTICES 2018 Table Saw Purpose NCUs Table Saw Purposes Cutting Plywood Cutting aluminum sheeting Cutting Lumber Cutting high density foam Best Practice - Tooling Inspection Set-up Operation Storage

645 views • 15 slides
Metal Cutting (Machining) Metal cutting , commonly called machining , is the removal of unwanted

Metal Cutting (Machining) Metal cutting , commonly called machining , is the removal of unwanted

Metal Cutting (Machining) Metal cutting , commonly called machining , is the removal of unwanted portions from a block of material in the form of chips so as to obtain a finished product of desired size, shape, and finish. 1 Metal Cutting

633 views • 52 slides
Statistics on permutation tableaux Pawel Hitczenko Drexel University parts based on joint work

Statistics on permutation tableaux Pawel Hitczenko Drexel University parts based on joint work

Statistics on permutation tableaux Pawel Hitczenko Drexel University parts based on joint work with Sylvie Corteel (Paris-Sud) and parts with Svante Janson (Uppsala) LIPN, February 5, 2008 Permutation tableaux Permutation tableau T : a Ferrers

1.11k views • 50 slides
Growth in permutation groups and linear New work on algebraic groups permutation groups H. A.

Growth in permutation groups and linear New work on algebraic groups permutation groups H. A.

Growth in permutation groups and linear algebraic groups H. A. Helfgott Introduction Diameter bounds Growth in permutation groups and linear New work on algebraic groups permutation groups H. A. Helfgott September 2018 Growth in

697 views • 65 slides
CA in a cervical LN. UROTHELIAL CARCINOMA (Prim. or Metastatic site) Challenges: - Poorly

CA in a cervical LN. UROTHELIAL CARCINOMA (Prim. or Metastatic site) Challenges: - Poorly

BEST PRACTICES IN THE APPLICATION OF IMMUNOHISTOCHEMISTRY TO DIAGNOSTIC UROLOGIC PATHOLOGY: LESSONS FROM USES & ABUSES The slides and syllabus are provided here exclusively for educational purposes and cannot be reproduced or used

648 views • 42 slides
in Wireline Communications Ali Sheikholeslami University of Toronto, Canada ali@ece.utoronto.ca

in Wireline Communications Ali Sheikholeslami University of Toronto, Canada ali@ece.utoronto.ca

Basics of Jitter in Wireline Communications Ali Sheikholeslami University of Toronto, Canada ali@ece.utoronto.ca sponsored by SSCS Distinguished Lecture Program August 23, 2019 San Diego, CA Ali Sheikholeslami Jitter in Wireline

1.1k views • 78 slides
Functions and Data Fitting COMPSCI 371D Machine Learning COMPSCI 371D Machine Learning

Functions and Data Fitting COMPSCI 371D Machine Learning COMPSCI 371D Machine Learning

Functions and Data Fitting COMPSCI 371D Machine Learning COMPSCI 371D Machine Learning Functions and Data Fitting 1 / 17 Outline 1 Functions 2 Features 3 Polynomial Fitting: Univariate Least Squares Fitting Choosing a Degree 4

397 views • 17 slides
Southern Methodist University February 13-15, 2017 Introduction W/Z@LHC Reweighting Conclusion

Southern Methodist University February 13-15, 2017 Introduction W/Z@LHC Reweighting Conclusion

Santa Fe Jets and Heavy Flavor Workshop Vector boson production in pPb and PbPb collisions at the LHC and its impact on nCTEQ15 PDFs arxiv: 1610.02925 Florian Lyonnet with A. Kusina, D.B. Clark, E. Godat, T. Jezo, K. Kovarik, F.I. Olness, I.

708 views • 41 slides
Linear Models for Classification II Henrik I Christensen Robotics & Intelligent Machines @ GT

Linear Models for Classification II Henrik I Christensen Robotics & Intelligent Machines @ GT

Introduction Generative Models Prob. Disc. Models Class Projects Summary Linear Models for Classification II Henrik I Christensen Robotics & Intelligent Machines @ GT Georgia Institute of Technology, Atlanta, GA 30332-0280

413 views • 25 slides
Model-Constructing Satisfiability Calculus Dejan Jovanovi c Clark Barrett Leonardo de Moura

Model-Constructing Satisfiability Calculus Dejan Jovanovi c Clark Barrett Leonardo de Moura

Model-Constructing Satisfiability Calculus Dejan Jovanovi c Clark Barrett Leonardo de Moura SRI International NYU Microsoft Research Satisfiability Modulo Theories and DPLL(T) Problem Check a given formula for satisfiability modulo the

889 views • 65 slides
Statistical Machine Learning Lecture 09: Classification Kristian Kersting TU Darmstadt Summer

Statistical Machine Learning Lecture 09: Classification Kristian Kersting TU Darmstadt Summer

Statistical Machine Learning Lecture 09: Classification Kristian Kersting TU Darmstadt Summer Term 2020 K. Kersting based on Slides from J. Peters Statistical Machine Learning Summer Term 2020 1 / 61 Todays Objectives Make you

1.21k views • 61 slides
Greedy algorithms: greed is good? Greedy algorithms Greed, for lack of a better word, Coin

Greedy algorithms: greed is good? Greedy algorithms Greed, for lack of a better word, Coin

Greedy algorithms: greed is good? Greedy algorithms Greed, for lack of a better word, Coin Changing, Interval Scheduling, Interval Partitioning is good. Greed is right. Greed works. Greed clarifies, cuts through, and captures, the Tyler Moore

70 views • 6 slides

More recommend