Visual Media Capture of shape and appearance Preservation of of - PowerPoint PPT Presentation

Visual Media Capture of shape and appearance Preservation of of real objects and people cultural artefacts 3D Broadcast production Sign language recognition Animation

Medical Imaging and Remote Sensing 3D liver reconstruction Seismic Vascular reconstruction Pipeline detection Microcalcification detection  3D MRI image analysis (brain tumour detection) Alzheimer’s condition diagnosis (PET brain  imaging)  2D-3D Elastic image matching

Robot Vision 3D object recognition from 2D views Target detection Visual surveillance Vision based navigation  Visual learning  Scene interpretation  Model selection  Control of perception

Multimedia Signal Processing and Interpretation Biometrics VOICE Image/Video Retrieval LIPS FACE Fusion

Ensemble MLP classifier Design Terry Windeatt University of Surrey, UK

Introduction  Ensembles – Multiple Classifier Systems (MCS)  Ensemble Multi-layer Perceptron Architecture  Tuning Base Classifiers using measures & OOB estimate  Multi-class ECOC using OOB  Feature Selection and Feature Ranking  Face Recognition

SINGLE CLASSIFIER APPROACH Goals  assign a pattern to one of several classes  find best possible feature set training set learning machine structure & parameters Task is especially difficult when  number of classes is high  classes highly overlapped in feature space  training samples are few and very noisy Learning is ill-posed problem & requires built-in assumptions

Multi-layer Perceptron (MLP) #hidden nodes varies complexity & #epochs varies degree of training Input layer Hidden layer Output layer Unstable Base Classifier from random starting weights

MCS Architecture  1 MLP Classifier 1  2 Combiner MLP Classifier 2 Bias/Variance 0/1 loss function more complex than  B regression MLP Classifier B -ensemble reduces variance Idea is to use multiple & tuning base classifier simple MLPs rather than reduces bias single complex MLP

Multiple Classifiers (MCS) MCS based upon: • finding classifiers that perform well but diversely • appropriate combining strategy Techniques: • Different types of classifiers • Different parameters same classifier • Different unstable base classifiers e.g MLP • Different Feature Sets e.g Random Subspace • Different Training Sets e.g. Bagging/Boosting • Different class labeling e.g. ECOC Measures of Diversity • Accuracy/Diversity Dilemma

Base Classifier Parameter Tuning  Importance of Parameter Tuning  Every researcher seems to get good results but how?  Need to measure sensitivity to parameters  Helps understand significance of results  Requires systematic change of parameters  How to set parameters?  Alternates to validation set or cross-validation techniques

Out-of-Bootstrap (OOB) Bootstrapping – Sample with Replacement Promotes diversity among classifiers OOB provides alternative to validation Base classifier OOB uses training patterns left out approx one third Ensemble OOB uses classifiers left out approx one third

BINARY-TO-BINARY MAPPING   X ( x , x , , x ) m m 1 m 2 mB  = f(X m ), where m = 1,…  . is number of patterns x mi and   {0,1} , i = 1 …B • 2-class problem • B parallel base classifiers • incompletely specified & noisy function

CLASS SEPARABILITY MEASURE 2-CLASS calculated over pairs of patterns chosen from different classes Example: 1 indicates correct classification class  1 0 0 1 1 1 0 0 1 1 0 class  2 1 1 1 0 1 0 0 1 1 0

CLASS SEPARABILITY MEASURE B  ~     ab a b a,b  {0,1} N ( ) pj pq qj  j 1     1 0 y , y , calculated over pairs of patterns p & q chosen from different classes    ~  ~ 11 00 N N   pq pq    q 1 q 1 p K K 1 2

PAIR-WISE DIVERSITY MEASURES Q Use counts between classifier pairs:       ab a b N a,b  {0,1} ij mi mj  1 m     1 0 y , y , Giving N 11 N 10 N 01 N 00  11 00 01 10 N N N N  Q j  i 11 00 01 10 N N N N

EXPERIMENTS 2-CLASS  100 single hidden-layer MLP base classifiers  Levenberg-Marquardt training, default parameters  Systematic variation of epochs and nodes  Different random starting weights + bootstrapping  Datasets random 20/80 train/test split (10 runs) – with added classification noise to encourage overfitting

DATASET #pat #class #con #dis cancer 699 2 0 9 card 690 2 6 9 credita 690 2 3 11 diabetes 768 2 8 0 heart 920 2 5 30 ion 351 2 31 3 vote 435 2 0 16

Figure : Mean test error rates, OOB estimates, measures  , Q for Diabetes 20/80 with [2,4,8,16] nodes

mean test error,  , Q over seven 20/80 two-class datasets using 8 hidden- node bootstrapped base classifiers for [0,20,40] % noise

MULTI-CLASS ECOC  Coding step: – Map training patterns into two super- classes according to 1’s and 0’s in ECOC matrix Z  Train base classifier on 2-class decompositions  Decoding step: – Assign test pattern according to minimum distance to row of ECOC matrix Z

MULTI-CLASS ECOC CODE MATRIX Example ECOC matrix: 0 1 1 1 ............. 6 classes 1 0 0 0 ............. 0 1 0 1 ............. 1 0 1 0 ............. 1 1 0 1 ............. 0 0 1 0 ............. each row is a code word each column defines two super-classes

Distance-based decoding rules (e.g. Hamming, L 1 ) Pattern Space ECOC Ensemble Target Classes MLP  1 10…1 MLP  2 10…1 01…0  3 11…1 MLP *** OOB uses only classifiers that are not used in training

Experiments Multi-class  200 base classifiers  Random ECOC matrices  20/80 train/test split repeated 10 times  Levenberg-Marquardt training algorithm

DATASET #pat #class #con #dis 366 dermatology 6 1 33 ecoli 336 8 5 2 glass 214 6 9 0 iris 150 3 4 0 segment 2310 7 19 0 soybean 683 19 0 35 thyroid 7200 3 6 15 vehicle 846 4 18 0 vowel 990 11 10 1 wave 5000 3 21 0 yeast 1484 10 7 1

Yeast 2/4/8/16 nodes 1-69 epochs

Feature Ranking  Intended for large number of features – small sample  One vs multi-dimensional  Context of MCS - base classifier vs combiner  Simple one-dim methods  Sophisticated multi-dim search methods Modulus of MLP weights – ‘product of weights’    1 2 w W W i ij j j W 1 is the first layer weight matrix and W 2 is the output weight vector

Recursive Feature Elimination Simple algorithm for eliminating irrelevant features and operates recursively as follows: 1) Rank the features according to a suitable feature ranking method 2) Identify and remove the r least ranked features If r>1 , usually desirable from an efficiency viewpoint, a feature subset ranking is obtained.

Mean test error rates, Bias, Variance for RFE MLP ensemble over seven 2-class Datasets 20/80, 10/90. 5/95 train/test split

Yeast RFE for [20/80 10/90 5/95]

Face Recognition - ORL Database  400 images of forty faces - 40 class identification problem  Variation in lighting,facial hair, pose ….  Controlled background with subjects upright frontal  No need for face detection so fair comparison  We use 40-dim PCA + 20-dim LDA  Random 50/50 train/test split  16 hidden node MLP L-M base classifiers (x200)  Expts repeated twenty times with 40 x 200 ECOC code matrix

ORL Database - Results Test error,  , Q for ORL 50/50 database using 16 hidden-node base classifiers for [0,20,40] % classification noise.

Facial Action Unit (FACS)  Difficult because depends on age, ethnicity, gender, and occlusions due to cosmetics, hair, glasses  FACS categorises deformation and motion into visual classes  Decouples interpretation from individual actions  Requires skilled practitioners  Small sample size problem – Large #features and small #training pats

Cohn-Kanade Database • frontal camera from 100 university students • contains posed (as opposed to the more difficult spontaneous) expression sequences • only the last image is au coded. • combinations of aus , in some cases non-additive • Upper face aus au1 au2 au4 au5 au6 au7

Design Decisions a) All image sequences of size 640 x 480 chosen from the database b) Last image in sequence (no neutral) giving 424 images, 115 containing au1 c) Full image resolution, no compression d) Manually located eye centres plus rotation/scaling into 2 common eye coordinates e) Window extracted of size 150 x 75 pixels centred on eye coordinates f) Forty Gabor filters [18], five special frequencies at five orientations with top 4 principle components for each Gabor filter, 160-dimensional feature vector g) Comparison of feature selection schemes described in Section 3 h) Comparison of MLP ensemble and Support Vector Classifier Random training/test split of 90/10 and 50/50 repeated twenty times and i) averaged

ECOC super-classes of action units and number of patterns ID sc1 sc2 sc3 sc4 sc5 sc6 superclass {} 1,2 1,2,5 4 6 1,4 #patterns 149 21 44 26 64 18 sc7 sc8 sc9 sc10 sc11 sc12 1,4,7 4,7 4,6,7 6,7 1 1,2,4 10 39 16 7 6 4