Decision Tree Mahdi Roozbahani Lecturer, Computational Science and - PowerPoint PPT Presentation

Mar 10, 2024 •175 likes •652 views

Class Website CX4242: Decision Tree Mahdi Roozbahani Lecturer, Computational Science and Engineering, Georgia Tech These slides are adopted from Polo, Andrew w. Moore, and Vivek Srikumar 2 1 Visual Introduction to Decision Tree

Class Website CX4242: Decision Tree Mahdi Roozbahani Lecturer, Computational Science and Engineering, Georgia Tech These slides are adopted from Polo, Andrew w. Moore, and Vivek Srikumar
𝑌 2 𝑌 1
Visual Introduction to Decision Tree Building a tree to distinguish homes in New York from homes in San Francisco 3
Decision Tree: Example (2) Will I play tennis today? 4
Decision trees (DT) Outlook? The classifier: f T (x) : majority class in the leaf in the tree T containing x Model parameters: The tree structure and size 5
Decision trees Pieces: 1. Find the best attribute to split on 2. Find the best split on the chosen attribute 3. Decide on when to stop splitting 6
Categorical or Discrete attributes Label
Attribute
Continuous attributes
Test data
Information Content Coin flip Entropy ~ Uncertainty Which coin will give us the purest information? Lower uncertainty, higher information gain
different
What will happen if a tree is too large? Overfitting High variance Instability in predicting test data
How to avoid overfitting? • Acquire more training data • Remove irrelevant attributes (manual process – not always possible) • Grow full tree, then post-prune • Ensemble learning
Reduced-Error Pruning Split data into training and validation sets Grow tree based on training set Do until further pruning is harmful: 1. Evaluate impact on validation set of pruning each possible node (plus those below it) 2. Greedily remove the node that most improves validation set accuracy
How to decide to remove it a node using pruning • Pruning of the decision tree is done by replacing a whole subtree by a leaf node. • The replacement takes place if a decision rule establishes that the expected error rate in the subtree is greater than in the single leaf.

Download Presentation

Download Policy: The content available on the website is offered to you 'AS IS' for your personal information and use only. It cannot be commercialized, licensed, or distributed on other websites without prior consent from the author. To download a presentation, simply click this link. If you encounter any difficulties during the download process, it's possible that the publisher has removed the file from their server.

Recommend

Decision Tree Decision Trees A decision tree is a decision support tool that uses a tree-like

Decision Tree Decision Trees A decision tree is a decision support tool that uses a tree-like graph or model of decisions and their possible MSE 2400 EaLiCaRA consequences, including chance event Dr. Tom Way outcomes, resource costs, and

568 views • 8 slides

1 Optimization in decision graphs Unfolding to decision tree Only option until Shachter

Decision graphs II Influence Diagrams Advanced Herd Management Anders Ringgaard Kristensen Slide 1 Outline Optimization methods Decision tree Strong junction tree Single Policy Updating Decision node ordering Advantages and

467 views • 10 slides

1 Optimization in decision graphs The repeated milk test problem Unfolding to decision tree The

Outline Decision trees Optimization methods Decision tree Decision graphs II Strong junction tree (Limited Memory) Influence Diagrams Single Policy Updating Markov property versus no forgetting Advanced Herd Management

512 views • 4 slides

A Brief History of Decision Tree Implementation MAX AUSTIN Overview Famous Decision Tree

A Brief History of Decision Tree Implementation MAX AUSTIN Overview Famous Decision Tree Algorithms Chi-squared Automatic Interaction Detector (CHAID) Classification and Regression Tree (CART) Iterative Dichotomiser 3 (ID3)

114 views • 9 slides

Decision Tree and Automata Learning Stefan Edelkamp 1 Overview - Decision tree representation

Decision Tree and Automata Learning Stefan Edelkamp 1 Overview - Decision tree representation - Top Down Induction, Attribute Selection - Entropy, Information gain - ID3 learning algorithm - Overfitting - Continous, multi-valued, costly and

718 views • 33 slides

Decision tree learning Aim: find a small tree consistent with the training examples Idea:

Decision tree learning Aim: find a small tree consistent with the training examples Idea: (recursively) choose most significant attribute as root of (sub)tree function DTL ( examples, attributes, default ) returns a decision tree if examples

261 views • 13 slides

Decision Tree Learning: Part 2 CS 760@UW-Madison Goals for the last lecture you should

Decision Tree Learning: Part 2 CS 760@UW-Madison Goals for the last lecture you should understand the following concepts the decision tree representation the standard top-down approach to learning a tree Occams razor

505 views • 38 slides

Decision-Tree for Classication MACH IN E LEARN IN G W ITH TREE-BAS ED MODELS IN P YTH ON

Decision-Tree for Classication MACH IN E LEARN IN G W ITH TREE-BAS ED MODELS IN P YTH ON Elie Kawerk Data Scientist Course Overview Chap 1 : Classication And Regression Tree (CART) Chap 2 : The Bias-Variance Tradeoff Chap 3 : Bagging

320 views • 29 slides

Assessing The Necessity Survey and Decision Tree Activities Conducted Decision tree created

Assessing The Necessity Survey and Decision Tree Activities Conducted Decision tree created December 2013. Survey closed March 5, 2014. 132 survey responses. 72 public health and 60 non-public health (NF, ALF, HCBS, other).

280 views • 17 slides

Learning Decision Trees Representation is a decision tree. Bias is towards simple decision

Learning Decision Trees Representation is a decision tree. Bias is towards simple decision trees. Search through the space of decision trees, from simple decision trees to more complex ones. Decision trees A decision tree is

484 views • 10 slides

Decision Tree R Greiner Cmput 466 / 551 Learning Decision Trees Def'n: Decision Trees

HTF: 9.2 B: 14.4 R N , C h a p t e r 1 8 1 8 . 3 Decision Tree R Greiner Cmput 466 / 551 Learning Decision Trees Def'n: Decision Trees Algorithm for Learning Decision Trees Entropy, Inductive Bias (Occam's

1.43k views • 87 slides

Decision Tree Algorithm Decision Tree Algorithm Week 4 1 Team Homework Assignment #5 Team

Decision Tree Algorithm Decision Tree Algorithm Week 4 1 Team Homework Assignment #5 Team Homework Assignment #5 Read pp. 105 117 of the text book. R d 105 117 f h b k Do Examples 3.1, 3.2, 3.3 and Exercise 3.4 (a). Prepare

385 views • 21 slides

Chapter 18 Learning from Observations Decision tree examples Additional source used in preparing

Chapter 18 Learning from Observations Decision tree examples Additional source used in preparing the slides: Jean-Claude Latombes CS121 slides: robotics.stanford.edu/~latombe/cs121 1 Decision Trees A decision tree allows a

608 views • 33 slides

Keep the Decision Tree and Estimate the Class Probabilities Using its Decision Boundary Isabelle

Keep the Decision Tree and Estimate the Class Probabilities Using its Decision Boundary Isabelle Alvarez (1 , 2) Stephan Bernard (2) Guillaume Deffuant (2) (1) LIP6, Paris VI University (2) Cemagref, LISC (2) Cemagref, LISC 4 place Jussieu

310 views • 6 slides

Decision Tree CE-717 : Machine Learning Sharif University of Technology M. Soleymani Fall 2019

Decision Tree CE-717 : Machine Learning Sharif University of Technology M. Soleymani Fall 2019 Decision tree } One of the most intuitive classifiers that is easy to understand and construct } However, it also works very (very) well }

453 views • 41 slides

Decision Tree Learning Mitchell, Chapter 3 CptS 570 Machine Learning School of EECS Washington

Decision Tree Learning Mitchell, Chapter 3 CptS 570 Machine Learning School of EECS Washington State University Outline Decision tree representation ID3 learning algorithm Entropy and information gain Overfitting

540 views • 30 slides

DecisionTrees MachineLearning10601 GeoffGordon,MiroslavDudk

DecisionTrees MachineLearning10601 GeoffGordon,MiroslavDudk ([[[partlybasedonslidesofCarlosGuestrinandAndrewMoore] hHp://www.cs.cmu.edu/~ggordon/10601/

589 views • 48 slides

Module 3 Lessons learned from mini project & Verify your results with structured testing

Module 3 Lessons learned from mini project & Verify your results with structured testing Lessons learned from mini project: Tops Things you did great: Friendly contact with your client Ask context of system. This will help to

301 views • 12 slides

Specifying Operations Specifying Operations Why operations are specified Algorithmic methods

Basi di Dati e Sistemi Informativi II Specifying Operations Specifying Operations Why operations are specified Algorithmic methods Non-algorithmic methods 2003 Giorgini Specifying Operations -- 1 Basi di Dati e Sistemi Informativi II The

439 views • 7 slides

Liberating distributed consensus Heidi Howard @ Cambridge University heidi.howard@cl.cam.ac.uk

Liberating distributed consensus Heidi Howard @ Cambridge University heidi.howard@cl.cam.ac.uk @heidiann360 heidihoward.co.uk Distributed Dream Performance - scalability, low latency, high throughput, low cost, energy e ffj ciency,

1.51k views • 116 slides

Lecture 4 Review from last lecture Nearest neighbor classifier A lazy learning

Oct - 6 - 2008 Lecture 4 Review from last lecture Nearest neighbor classifier A lazy learning algorithm Decision boundary can be obtained by the Voronoi diagram of the training set Complex, sensitive to label noise

444 views • 15 slides

Decision Tree Learning Based on Machine Learning, T. Mitchell, McGRAW Hill, 1997, ch. 3

0. Decision Tree Learning Based on Machine Learning, T. Mitchell, McGRAW Hill, 1997, ch. 3 Acknowledgement: The present slides are an adaptation of slides drawn by T. Mitchell 1. PLAN DT Learning: Basic Issues 1. Concept learning:

641 views • 52 slides

Learning Objectives At the end of the class you should be able to: show an example of

Learning Objectives At the end of the class you should be able to: show an example of decision-tree learning explain how to avoid overfitting in decision-tree learning explain the relationship between linear and logistic regression explain how

714 views • 40 slides

and Random Forests Pr. Fabien MOUTARDE Center for Robotics MINES ParisTech PSL Universit

Decision Trees and Random Forests Pr. Fabien MOUTARDE Center for Robotics MINES ParisTech PSL Universit Paris Fabien.Moutarde@mines-paristech.fr http://people.mines-paristech.fr/fabien.moutarde Decision Trees and Random Forests, Pr.

412 views • 11 slides