PDF Editor
PDF Converter
Image Converter
Video Converter
Audio Converter
File Compressor
lecture 5 classification and dimension reduction

Lecture 5: Classification and dimension reduction Felix Held, - PowerPoint PPT Presentation

Sep 24, 2022 β€’165 likes β€’426 views

Lecture 5: Classification and dimension reduction Felix Held, Mathematical Sciences MSA220/MVE440 Statistical Learning for Big Data 4th April 2019 Random Forests 1. Given a training sample with features, do for = 1, , on

  1. Lecture 5: Classification and dimension reduction Felix Held, Mathematical Sciences MSA220/MVE440 Statistical Learning for Big Data 4th April 2019

  2. Random Forests 1. Given a training sample with π‘ž features, do for 𝑐 = 1, … , 𝐢 on bootstrapped data. Note: Step 1.2.1 leads to less correlation between trees built Majority vote at 𝐲 across trees Classification: 𝐢 1 𝑠𝑐 (𝐲) = 𝑔 Λ† Regression: 2. For a new 𝐲 predict 1.2.3 Split the node 1.2.2 Find best splitting variable among these 𝑛 1.2.1 Randomly select 𝑛 variables from the π‘ž available π‘œ min replacement) 1.1 Draw a bootstrap sample of size π‘œ from training data (with 1/21 1.2 Grow a tree π‘ˆ 𝑐 until each node reaches minimal node size 𝐢 βˆ‘ 𝑐=1 π‘ˆ 𝑐 (𝐲)

  3. Comparison of RF, Bagging and CART 𝐲 ∼ 𝑂(𝟏, 𝚻), 𝐲 ∈ ℝ 5 , Toy example Training and test data were sampled from the true model. Results 2/21 𝜁 ∼ 𝑂(0, 1) 𝑧 = 𝑦 2 where 1 + 𝜁 𝚻 π‘šπ‘š = 1, 𝚻 π‘šπ‘™ = 0.98, π‘š β‰  𝑙 for RF, bagged CART and a single CART, using 𝑦 1 , … , 𝑦 5 as predictor variables. ( π‘œ 𝑒𝑠 = 50 , π‘œ 𝑒𝑓 = 100 ) ● Test error 2.1 ● ● 1.8 ● ● ● ● ● ● ● ● ● 1.5 ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● 0 100 200 300 Number of trees

  4. Variable importance set for that particular tree, since they were not used serves as an importance measure for variable π‘˜ 1 𝐹 (π‘˜) 1 calculate test error again 𝐹 (π‘˜) 1. Impurity index: Splitting on a feature leads to a reduction during training. Resulting in test error 𝐹 0 a chance of about 63% to be selected trees per feature gives a measure for variable importance of node impurity. Summing all improvements over all 3/21 2. Out-of-bag error β–Ά During bootstrapping for large enough π‘œ , each sample has β–Ά For bagging the remaining samples are out-of-bag . β–Ά These out-of-bag samples for tree π‘ˆ 𝑐 can be used as a test β–Ά Permute variable π‘˜ in the out-of-bag samples and β–Ά The increase in error βˆ’ 𝐹 0 β‰₯ 0

  5. RF applied to cardiovascular dataset Monica dataset ( http://thl.fi/monica , π‘œ = 6367 , π‘ž = 11 ) 4/21 number of cardiovascular risk factors (class ratio 1.25 alive : 1 dead) Predicting whether or not patients survive a 10 year period given a Error estimate Variable importance 0.25 ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● yronset ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● stroke Type ● ● Type Alive Acc. smstat Alive 0.20 ● Dead Acc. Dead ● Outβˆ’ofβˆ’bag error Mean Acc. sex ● OOB Mean Gini ● ● premi ● ● ● 0.15 ● ● ● ● ● ● ● ● ● ● ● hosp ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● highbp ● 0.10 ● hichol ● ● ● ● ● diabetes ● ● ● ● ● ● ● 0.05 ● angina ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● age ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● 0 50 100 150 200 0 1 2 3 Number of Trees Decrease

Recommend

Dimension Reduction and Nearest Neighbor Search Advanced Algorithms Nanjing University, Fall

Dimension Reduction and Nearest Neighbor Search Advanced Algorithms Nanjing University, Fall

Dimension Reduction and Nearest Neighbor Search Advanced Algorithms Nanjing University, Fall 2018 Dimension reduction: Why we care? High dimension data are common, yet working on them directly is expensive. Dimension reduction: Why we

1.61k views β€’ 116 slides
Dimension Reduction CSE 6242 / CX 4242 Thanks : Prof. Jaegul Choo , Dr. Ramakrishnan Kannan,

Dimension Reduction CSE 6242 / CX 4242 Thanks : Prof. Jaegul Choo , Dr. Ramakrishnan Kannan,

Dimension Reduction CSE 6242 / CX 4242 Thanks : Prof. Jaegul Choo , Dr. Ramakrishnan Kannan, Prof. Le Song What is Dimension Reduction? Data item index (n) low-dim Columns as data data items Dimension index (d) Dimension Reduction

972 views β€’ 44 slides
Linear Dimension Reduction (in L 2 ) Linear Dimension Reduction: R D R d Goal: Find a low-dim.

Linear Dimension Reduction (in L 2 ) Linear Dimension Reduction: R D R d Goal: Find a low-dim.

Linear Dimension Reduction (in L 2 ) Linear Dimension Reduction: R D R d Goal: Find a low-dim. linear map that preserves the relevant information Application dependent ie find a d x D matrix M Different definitions yield different

541 views β€’ 30 slides
VC-dimension and Erd os-P osa property Nicolas Bousquet LIRMM, University Montpellier II

VC-dimension and Erd os-P osa property Nicolas Bousquet LIRMM, University Montpellier II

VC-dimension VC-dimension for graphs Conclusion VC-dimension and Erd os-P osa property Nicolas Bousquet LIRMM, University Montpellier II Nicolas Bousquet VC-dimension and Erd os-P osa property VC-dimension Packing and

953 views β€’ 55 slides
Dimension Reduction CS 760@UW-Madison Goals for the lecture you should understand the following

Dimension Reduction CS 760@UW-Madison Goals for the lecture you should understand the following

Dimension Reduction CS 760@UW-Madison Goals for the lecture you should understand the following concepts dimension reduction principal component analysis: definition and formulation two interpretations strength and

925 views β€’ 27 slides
VC Dimension and classification John Duchi Prof. John Duchi Outline I Setting: classification

VC Dimension and classification John Duchi Prof. John Duchi Outline I Setting: classification

VC Dimension and classification John Duchi Prof. John Duchi Outline I Setting: classification problems II Finite hypothesis classes 1 Union bounds 2 Zero error case III Shatter coe ffi cients and Rademacher complexity IV VC Dimension Prof.

445 views β€’ 26 slides
The Human Dimension Sue Manns Regional Director Pegasus The Human Dimension The Human

The Human Dimension Sue Manns Regional Director Pegasus The Human Dimension The Human

27/02/2015 UK Onshore Oil and Gas: Policy, Planning and Future Developments The Human Dimension Sue Manns Regional Director Pegasus The Human Dimension The Human Dimension Understanding the psychology and physiology of change

115 views β€’ 10 slides
Packing Dimension Results for Anisotropic Gaussian Random Fields Dongsheng Wu Department of

Packing Dimension Results for Anisotropic Gaussian Random Fields Dongsheng Wu Department of

Introduction Packing Dimension and Packing Dimension Profile on ( R N , ) Packing Dimension Results for Anisotropic Gaussian Fields Packing Dimension Results for Anisotropic Gaussian Random Fields Dongsheng Wu Department of Mathematical

490 views β€’ 28 slides
The Metric Dimension Problem. J. D az Monash U., May 2018 The Metric Dimension problem

The Metric Dimension Problem. J. D az Monash U., May 2018 The Metric Dimension problem

The Metric Dimension Problem. J. D az Monash U., May 2018 The Metric Dimension problem Given G ( V , E ) its metric dimension, ( G ) is the cardinality of the smallest L V s.t. x , y V , z L with d G ( x , z ) = d

450 views β€’ 29 slides
Nonparametric Variable Selection via Sufficient Dimension Reduction Lexin Li Workshop on Current

Nonparametric Variable Selection via Sufficient Dimension Reduction Lexin Li Workshop on Current

Nonparametric Variable Selection via Sufficient Dimension Reduction Lexin Li Workshop on Current Trends and Challenges in Model Selection and Related Areas Vienna, Austria July 24, 2008 1 Dimension Reduction and Variable Selection Outline

411 views β€’ 25 slides
Extreme Value Theory and Dimension GARDES Inference on reduction for the study of hyperspectral

Extreme Value Theory and Dimension GARDES Inference on reduction for the study of hyperspectral

Extreme Value Theory and Dimension reduction for the study of hyperspectral images Laurent Extreme Value Theory and Dimension GARDES Inference on reduction for the study of hyperspectral images Weibull tail distributions Extreme

925 views β€’ 45 slides
Dimension Reduction CS 6242 Ramakrishnan Kannan Thanks : Prof. Jaegul Choo and Prof. Le

Dimension Reduction CS 6242 Ramakrishnan Kannan Thanks : Prof. Jaegul Choo and Prof. Le

Dimension Reduction CS 6242 Ramakrishnan Kannan Thanks : Prof. Jaegul Choo and Prof. Le Song What is Dimension Reduction? Data item index (n) low-dim Columns as data data items

701 views β€’ 44 slides
Dimension reduction numerical methods for Bermudan options Scott Sues Probability, Numerics, and

Dimension reduction numerical methods for Bermudan options Scott Sues Probability, Numerics, and

Dimension reduction numerical methods for Bermudan options Scott Sues Probability, Numerics, and Finance, Le Mans 29 Jun-1 Jul 2016 Outline 1. Dimension reduction approach 2. Fourier cosine method 3. Bermudan option pricing 1 / 25 Abstract

497 views β€’ 29 slides
Geometric perspectives for supervised dimension reduction A Tale of Two Manifolds S. Mukherjee,

Geometric perspectives for supervised dimension reduction A Tale of Two Manifolds S. Mukherjee,

Geometric perspectives for supervised dimension reduction Geometric perspectives for supervised dimension reduction A Tale of Two Manifolds S. Mukherjee, K. Mao, F. Liang, Q. Wu, D-X. Zhou, J. Guinney Department of Statistical Science

1.32k views β€’ 99 slides
Graph Classification Classification Outline Introduction, Overview Classification using

Graph Classification Classification Outline Introduction, Overview Classification using

Graph Classification Classification Outline Introduction, Overview Classification using Graphs Graph classification Direct Product Kernel Predictive Toxicology example dataset Vertex classification Laplacian Kernel

605 views β€’ 33 slides
Classification of Symmetry Classification of Symmetry Classification of Symmetry Classification

Classification of Symmetry Classification of Symmetry Classification of Symmetry Classification

Classification of Symmetry Classification of Symmetry Classification of Symmetry Classification of Symmetry Protected Topological Phases Protected Topological Phases Protected Topological Phases Protected Topological Phases in Interacting

492 views β€’ 32 slides
Stylization with a Purpose Stylization with a Purpose Stylization with a Purpose Stylization

Stylization with a Purpose Stylization with a Purpose Stylization with a Purpose Stylization

Stylization with a Purpose Stylization with a Purpose Stylization with a Purpose Stylization with a Purpose The Illustrative World The Illustrative World of Team Fortress 2 of Team Fortress 2 Jason Mitchell Jason Mitchell Jason Mitchell

1.15k views β€’ 82 slides
The Gas Network Control Problem and How to Approach It Felix Hennings Combinatorial Optimization

The Gas Network Control Problem and How to Approach It Felix Hennings Combinatorial Optimization

The Gas Network Control Problem and How to Approach It Felix Hennings Combinatorial Optimization @ Work 2020 The Gas Network Control Problem General description Optimization of short-term transient gas network control of large real-world

917 views β€’ 39 slides
Quality Energy Services Brochure Locations and Capabilities E-Line & Cased Hole Services

Quality Energy Services Brochure Locations and Capabilities E-Line & Cased Hole Services

Quality Energy Services Brochure Locations and Capabilities E-Line & Cased Hole Services Cased Hole/Electric Line Services Quality Energy Services provides Cased Hole Electric Line, Combination Slick line/EL and Memory Tool Logging services

939 views β€’ 58 slides
CPSC 875 CPSC 875 John D McGregor John D. McGregor ABS module a work in progress Purpose

CPSC 875 CPSC 875 John D McGregor John D. McGregor ABS module a work in progress Purpose

CPSC 875 CPSC 875 John D McGregor John D. McGregor ABS module a work in progress Purpose Purpose This is a unit that blends software architecture This is a unit that blends software architecture information with telematics information

655 views β€’ 40 slides
Effect of the PCSK9 Inhibitor Evolocumab on Cardiovascular Outcomes MS Sabatine, RP Giugliano,

Effect of the PCSK9 Inhibitor Evolocumab on Cardiovascular Outcomes MS Sabatine, RP Giugliano,

Effect of the PCSK9 Inhibitor Evolocumab on Cardiovascular Outcomes MS Sabatine, RP Giugliano, SD Wiviott, FJ Raal, CM Ballantyne, R Somaratne, J Legg, SM Wasserman, R Scott, MJ Koren, and EA Stein for the OSLER Investigators American College

226 views β€’ 18 slides
Data Warehousing & OLAP Motivation: Business Intelligence Customer information Product

Data Warehousing & OLAP Motivation: Business Intelligence Customer information Product

Data Warehousing & OLAP Motivation: Business Intelligence Customer information Product information (customer-id, gender, age, home- (Product-id, category, address, occupation, income, manufacturer, made-in, stock- family-size, )

464 views β€’ 27 slides
Therapy of Ph+ ALL elderly pa1ents Cris%na Papayannidis,

Therapy of Ph+ ALL elderly pa1ents Cris%na Papayannidis,

Therapy of Ph+ ALL elderly pa1ents Cris%na Papayannidis, MD, PhD DIMES University of Bologna Ph+ ALL in the elderly: background Philadelphia

566 views β€’ 32 slides
What if my panel is negative? How do I manage my patient? Robert Campbell, MD September 30, 2016

What if my panel is negative? How do I manage my patient? Robert Campbell, MD September 30, 2016

What if my panel is negative? How do I manage my patient? Robert Campbell, MD September 30, 2016 NO DISCLOSURES Childrens Healthcare of Atlanta | Sibley Heart Center Cardiology Negative LQTS gene panel report Childrens Healthcare of

586 views β€’ 25 slides

More recommend

Logo Sambuz

Unleash a World of Digital Possibilitiesβ€”Browse, Share, and Explore Content Without Boundaries

Useful Links

About Us Privacy Services FAQ's Blogs Contact

Newsletter

Stay Informed & Inspiredβ€”Subscribe for the Latest Updates, Tips, and Exclusive Content Directly to Your Inbox.

Mail Us

mail@sambuz.com

2025 SAMBUZ, All right reserved.