Supervised Ensembles of Prediction Methods for Subcellular - - PowerPoint PPT Presentation

▶
supervised ensembles of prediction methods for
SMART_READER_LITE
LIVE PREVIEW

Supervised Ensembles of Prediction Methods for Subcellular - - PowerPoint PPT Presentation

Dec 18, 2022 26 likes •251 views

LUDWIG- MAXIMILIANS- DEPARTMENT DATABASE UNIVERSITY INSTITUTE FOR SYSTEMS MUNICH INFORMATICS GROUP Supervised Ensembles of Prediction Methods for Subcellular Localization APBC 2008 Johannes Afalg, Jing Gong, Hans-Peter Kriegel,

slide-1
SLIDE 1

LUDWIG- MAXIMILIANS- UNIVERSITY MUNICH DATABASE SYSTEMS GROUP DEPARTMENT INSTITUTE FOR INFORMATICS

Supervised Ensembles of Prediction Methods for Subcellular Localization

APBC 2008

Johannes Aßfalg, Jing Gong, Hans-Peter Kriegel, Alexey Pryakhin, Tiandi Wei, Arthur Zimek Ludwig-Maximilians-Universität München Munich, Germany http://www.dbs.ifi.lmu.de {assfalg,gongj,kriegel,pryakhin,tiandi,zimek}@dbs.ifi.lmu.de

slide-2
SLIDE 2

DATABASE SYSTEMS GROUP

Outline

  • Background
  • Localization Prediction Methods
  • Ensemble Methods (Theory)
  • Supervised Ensemble Methods

– Ensemble using a Voting Schema – Ensemble based on Decision Tree

  • Data and Results
  • Conclusions

Assfalg et al.: Supervised Ensembles of Prediction Methods for Subcellular Localization (APBC 2008)

2

slide-3
SLIDE 3

DATABASE SYSTEMS GROUP

Background

  • cells are organized in

regions and compartments

  • different regions serve

different functionalities

  • certain functionalities are

performed by specific proteins

  • proteins are adapted to the

specific biophysical environment of its proper compartment

Assfalg et al.: Supervised Ensembles of Prediction Methods for Subcellular Localization (APBC 2008)

3

slide-4
SLIDE 4

DATABASE SYSTEMS GROUP

Background

  • proper function of a protein

requires correct localization

  • co-translational or post-

translational transport of proteins into specific subcellular localizations

  • highly regulated and complex

cellular process

Assfalg et al.: Supervised Ensembles of Prediction Methods for Subcellular Localization (APBC 2008)

4

slide-5
SLIDE 5

DATABASE SYSTEMS GROUP

Localization Prediction Methods: Basis for Predictions

  • adaptation of a protein to a

certain region is reflected in amino-acid composition (surface exposed to specific milieu)

  • transport and localization is

guided e.g. by peptide signals

  • homology of proteins

Assfalg et al.: Supervised Ensembles of Prediction Methods for Subcellular Localization (APBC 2008)

5

Nobel prize 1999 Günter Blobel “proteins have intrinsic signals that govern their transport and localization in the cell”

Prediction methods for subcellular localization are based on:

slide-6
SLIDE 6

DATABASE SYSTEMS GROUP

Localization Prediction Methods: Using Different Information

Assfalg et al.: Supervised Ensembles of Prediction Methods for Subcellular Localization (APBC 2008)

6

Category 1: methods based on amino acid composition Category 3: methods based on homology search Category 2: methods based on sorting signals Category 4: hybrid methods

slide-7
SLIDE 7

DATABASE SYSTEMS GROUP

Localization Prediction Methods: Different Computational Basis

  • naïve Bayes
  • Bayes networks
  • k-nearest neighbor methods
  • SVM
  • neural networks
  • rules

Assfalg et al.: Supervised Ensembles of Prediction Methods for Subcellular Localization (APBC 2008)

7

slide-8
SLIDE 8

DATABASE SYSTEMS GROUP

Localization Prediction Methods: Different Limitations of Methods

  • Localization coverage

– e.g. “SubLoc” predicts 4 localizations – “PLOC” predicts 12 localizations

  • Taxonomic coverage

– e.g. “HSLPred” predicts for human proteins – “PLOC” predicts for plant, animal and fungi proteins

  • Sequence coverage

– e.g. “ESLPred (2004)” and “SubLoc (2001)” used data set generated by another method “NNPSL” in 1998

Assfalg et al.: Supervised Ensembles of Prediction Methods for Subcellular Localization (APBC 2008)

8

slide-9
SLIDE 9

DATABASE SYSTEMS GROUP

Localization Prediction Methods: Different Limitations of Methods

  • different means to assess the accuracy in publications
  • inexact assignment of localizations for methods based on

sorting signals

– secretory pathway E.R. / Golgi / Lysosome / Extracellular

  • strong dependence on the quality of N-terminal sequence

assignment for methods based on sorting signals

  • strong dependence on the existence of homologous protein

for methods based on homology search

Assfalg et al.: Supervised Ensembles of Prediction Methods for Subcellular Localization (APBC 2008)

9

slide-10
SLIDE 10

DATABASE SYSTEMS GROUP

Ensemble Methods: Theory (unsupervised)

  • Ensemble methods combine several self-contained

classifiers to gain better accuracy.

  • Prerequisites to enhance accuracy by combination of base

classifiers:

– the single base classifier is “accurate” (i.e., better than random) – the base classifiers differ:

  • statistical variance (different prediction models perform equally well on

training data)

  • computational variance (using different heuristics to overcome

computational restrictions)

  • different bias

– effect: the base classifiers make different (uncorrelated) errors

Assfalg et al.: Supervised Ensembles of Prediction Methods for Subcellular Localization (APBC 2008)

10

slide-11
SLIDE 11

DATABASE SYSTEMS GROUP

Ensemble Methods: Theory (unsupervised)

  • ensemble of k hypotheses for dichotomous problem
  • error rate of each hypothesis is p < 0.5
  • ensemble is wrong if (and only if) more than members

are wrong

  • overall error rate of ensemble:

area under binomial distribution, where (i.e., at least k/2 hypotheses are wrong)

Assfalg et al.: Supervised Ensembles of Prediction Methods for Subcellular Localization (APBC 2008)

11

⎥ ⎥ ⎤ ⎢ ⎢ ⎡ ≥ 2 k k ⎥ ⎥ ⎤ ⎢ ⎢ ⎡ 2 k

slide-12
SLIDE 12

DATABASE SYSTEMS GROUP

  • example: single error rate p = 0.3 equally for each member

Ensemble Methods: Example

Assfalg et al.: Supervised Ensembles of Prediction Methods for Subcellular Localization (APBC 2008)

12

( )

i k i k k i

p p i k p k p

− ⎥ ⎥ ⎤ ⎢ ⎢ ⎡ =

− ⎟ ⎟ ⎠ ⎞ ⎜ ⎜ ⎝ ⎛ = ∑ 1 ) , (

2

slide-13
SLIDE 13

DATABASE SYSTEMS GROUP

Ensemble Methods: Selection of Base Methods

  • diversity of used information and computational methods

makes localization prediction methods ideal base classifiers for ensembles

  • prerequisites:

– comparison of methods with different coverage: derive reliability index – assess accuracy of methods by comparable statistics – choose representative methods for different categories and algorithmic approaches

Assfalg et al.: Supervised Ensembles of Prediction Methods for Subcellular Localization (APBC 2008)

13

slide-14
SLIDE 14

DATABASE SYSTEMS GROUP

Ensemble Methods: Selection of Base Methods

Assfalg et al.: Supervised Ensembles of Prediction Methods for Subcellular Localization (APBC 2008)

14

Category Method Foundation Algorithm 1 aa SVM dipeptide SVM n-peptide SVM 2 detecting sorting signals AA-index detecting sorting signals NN 3 BLAST against Swiss-Prot Naive Bayes 4 aa+signal+motif+structure k-NN aa+length+signal k-NN aa+signal+motif+structure SVM aa+di+properties+psi-BLAST SVM aa+di+gap+properties+psi-BLAST SVM

slide-15
SLIDE 15

DATABASE SYSTEMS GROUP

Ensemble Methods: Exclusion of Some Methods

Assfalg et al.: Supervised Ensembles of Prediction Methods for Subcellular Localization (APBC 2008)

15

Category Method Foundation Algorithm 1 aa SVM dipeptide SVM n-peptide SVM 2 detecting sorting signals AA-index detecting sorting signals NN 3 BLAST against Swiss-Prot Naive Bayes 4 aa+signal+motif+structure k-NN aa+length+signal k-NN aa+signal+motif+structure SVM aa+di+properties+psi-BLAST SVM aa+di+gap+properties+psi-BLAST SVM

too simple foundation, lower rank in preliminary tests

based on virtually all SWISSPROT entries that provide a localization extension WoLFPSORT is used

slide-16
SLIDE 16

DATABASE SYSTEMS GROUP

Ensemble Methods: From Unsupervised to Supervised

  • preliminary tests and evaluations: several prediction

methods unsuitable for unsupervised ensembles

  • problem:

– low accuracy for some localization classes – some errors may be correlated

  • approach: supervised ensembles based on prior knowledge
  • f the performance of the single methods

Method 1: voting scheme based on prior evaluation of base classifiers Method 2: decision tree learns reliability of the single methods for single predictions

Assfalg et al.: Supervised Ensembles of Prediction Methods for Subcellular Localization (APBC 2008)

16

slide-17
SLIDE 17

DATABASE SYSTEMS GROUP

Supervised Ensemble Method 1: Voting Schema

Assfalg et al.: Supervised Ensembles of Prediction Methods for Subcellular Localization (APBC 2008)

17

  • Each method gives its vote to one or several localizations

e.g. Golgi Golgi SP

  • Score calculation for each localization according to the

gained votes and the weight of each vote For a certain localization i: score scorei

i =

= ∑ ∑j

j=1 =1… …N N (Vote

(Votej

j * (

* (N N -

  • Rank

Rankj

j + 1))

+ 1))

N N : number of methods used by the ensemble method : number of methods used by the ensemble method Rank Rankj

j : rank of method

: rank of method j j during comparison during comparison Vote Votej

j = 1 if method

= 1 if method j j gives the vote to the localization gives the vote to the localization i i, otherwise , otherwise Vote Votej

j = 0.

= 0.

Vote Vote Vote Vote Vote

E.R. Golgi Lysosome Extracellular

slide-18
SLIDE 18

DATABASE SYSTEMS GROUP

Supervised Ensemble Method 2: Decision Tree

  • Decision Trees learn to map prediction vectors of the base

classifiers to a single prediction: (localization index)N → localization index

  • Example: decision tree for taxonomic group “plant” learns

rules like “If CELLO predicts class 6 and WoLFPSORT predicts class 4, then class 4 is correct.”

  • The prediction servers and the learned models are available
  • nline via

http://www.dbs.ifi.lmu.de/research/locpred/ensemble/

Assfalg et al.: Supervised Ensembles of Prediction Methods for Subcellular Localization (APBC 2008)

18

slide-19
SLIDE 19

DATABASE SYSTEMS GROUP

Data Preparation

Assfalg et al.: Supervised Ensembles of Prediction Methods for Subcellular Localization (APBC 2008)

19

non-standard aa characters length < 60 aa Sequence Retrieval System (SRS) All proteins with subcellular location annotation

  • ther locations multi-location

All proteins with single location as: Cytoplasm, Chloroplast, E.R., Golgi, Lysosome, Mitochondrion, Nucleus, Peroxisome, Extracellular, Vacuole Raw data set with 80,668 entries Fungi Plant Human Other Animal complement of 28 Golgi proteins

Release 53.0

(20,920) final data sets final data sets

  • exp. confirmed data

not confirmed data 34,261 46,407

slide-20
SLIDE 20

DATABASE SYSTEMS GROUP

Results: Accuracy

Assfalg et al.: Supervised Ensembles of Prediction Methods for Subcellular Localization (APBC 2008)

20

0,3 0,4 0,5 0,6 0,7 0,8 0,9 Plant Fungi Animal Human Total Accuracy DT-Ensemble Voting PLOC CELLO iPSORT Predotar WoLFPSORT MultiLoc ESLPred HSLPred

slide-21
SLIDE 21

DATABASE SYSTEMS GROUP

Results: Specificity

Assfalg et al.: Supervised Ensembles of Prediction Methods for Subcellular Localization (APBC 2008)

21

0,5 0,6 0,7 0,8 0,9 1 Plant Fungi Animal Human

  • Avg. Specificity

DT-Ensemble Voting PLOC CELLO iPSORT Predotar WoLFPSORT MultiLoc ESLPred HSLPred

slide-22
SLIDE 22

DATABASE SYSTEMS GROUP

Conclusions

  • Localization prediction methods use different kind of

information and different computational approaches.

  • Combination of several methods to an ensemble yields

considerably increased accuracy.

  • Methods are seemingly unsuitable for unsupervised

ensemble methods.

  • Two supervised ensemble methods:

– voting schema, based on prior knowledge (evaluation of single methods) – decision tree (trained to learn ideal combination of single methods for specific localization classes)

  • Decision tree models provide further insight in reliability of

single methods for specific localization classes.

Assfalg et al.: Supervised Ensembles of Prediction Methods for Subcellular Localization (APBC 2008)

22