Applied Machine Learning in Biomedicine Enrico Grisan - PowerPoint PPT Presentation

Applied Machine Learning in Biomedicine Enrico Grisan enrico.grisan@dei.unipd.it

Course details Mon-Wed 10.30-12.00 Room 318 May 4 th through May 27 th Contact enrico.grisan@dei.unipd.it Exam: project assignment

Cancer detection

Face detection How would you detect a face? How does album software tag your frienss?

What do we do?

What do we do?

Speech recognition

Brain-coputer interface

Recommender systems Amazon, Netflix, Spotify tell you what you might like The Netflix Prize was an open competition: predict user ratings for films, based on previous ratings without any other information about the users or films, The grand prize of US$1,000,000 was given to the BellKor's Pragmatic Chaos team which bested Netflix's own algorithm for predicting ratings by 10.06%

The age of big data 10 9 messages/day CERN Collider Personal connectome 320x10 12 bytes/s 10 18 bytes/person “Every day, people create the equivalent of 2.5 quintillion bytes of data from sensors, mobile devices, online 30x10 6 messages/day transactions, and social networks; so much that 90 percent of the world's data has been generated in the past two years.. ” The Huffington Post: Arnal Dayaratna: IBM Releases Big Data

The role of machine learning Design and analyze algorithms that - improve their performance - at some task - with experience Data Knowledge Learning ( experience ) algorithm ( performance on task )

Imagenet challenge

Kaggle challenge 100 000 $ prize 35000 retinal images 4 DR classes ongoing!!!

Machine learning in biomedicine Usually extreme conditions: Very few samples (with respect to the problem) Very large amount of descriptors per sample Very large amount of noise/uncertainty

Categories – Supervised learning classification, regression – Unsupervised learning Density estimation, clustering, dimensionality reduction – Semisupervised learning – Active learning – Reinforcement learning – …

Supervised learning Feature space Target space Normal Metaplastic Gene expression Discrete labels Benign neoplastic Classification Malign neoplastic Demographic and Continuous labels CHD risk score Clinical data Regression

Roadmap Binary classification - Parametric and non-parametric prediction Other supervised settings Principles for learning

Oranges and Lemons

A two dimensional space

Stars and galaxies Minor elliptical axis (y) against Major elliptical axis (x) for stars (red) and galaxies (blue)

Coronoray Heart Disease Patients with (red) and without (blue) coronary heart disease in South Africa (Rousseauw et al, 1983)

Parametric model

Linear classifier

The weight vector

Geometric meaning

The weight vector % ww = Dx1 weights % Xstar = NxD test cases y_pred = sign(Xstar*ww); % Nx1

Learning the weights Rosenblatt ’s Perceptron Learning Perceptron criterion: Stochastic gradient descent:

Learning the weights % ww = Dx1 weights % xx = NxD test cases % yy = Nx1 targets (-1,+1) old_ww=[]; ww=zeros(D,1); while (~isequal(ww,old_ww)) old_ww=ww; for ct=1:N, pred=sign(xx(ct,:)*ww); ww=ww+(yy(ct)-pred)*xx(ct ,:)’; end; end;

Learning the weights

Implementing the bias

Output of the perceptron

Linear classifier revisited If not linearly separable must - extend model - add features

Nonlinear basis function

From model to no model Faith in previous knowledge Faith in the data Strong assumption on No assumption on the - data structure underlying structure - separating boundary shape Data tell me everything I need

K-nearest neighbours classifier Fix an Hodges 1951

Decision boundaries 1-nearest neighbour 15-nearest neighbour Linear classification classifier classifier

Brain MRI application MICCAI MS lesion challenge 2008 http://www.ia.unc.edu/MSseg/index.html

LANDSAT application

Identification via gait analysis Characterize each person by the way he moves: gait signature Nowlan 2009 Choi 2014

Parametric vs non-parametric • Starting assuming decision boundary is a plane • Non-parametric KNN has no fixed assumption: boundaries gets more complicated with more data • Non-parametric methods may need more data and can be computationally intensive

Batch supervised learning Given : example inputs and targets (training set) Task : predicting target for new inputs (test set) Examples: - classification (binary or multi-class) - regression - ordinal regression - Poisson regression - ranking …

Batch supervised learning • Many ways of mapping inputs outputs • How do we choose what to do? • How do we know if we are doing well?

Algorithm’s objective cost Formal objective for algorithms: - minimize a cost function - maximize an objective function Proving convergence: - does objective monotonically improve? Considering alternatives : - does another algorithm score better?

Loss function

Choosing a loss function • Motivated by the application – 0-1 error, achieving a tolerance, business cost • Computational convenience: – Differentiability, convexity • Beware of loss dominated by artifacts: – Outliers – Unbalanced classes