An Ensemble of Epoch-wise Empirical Bayes for Few-Shot Learning - PowerPoint PPT Presentation

An Ensemble of Epoch-wise Empirical Bayes for Few-Shot Learning Yaoyao Liu Bernt Schiele Qianru Sun MPI Informatics MPI Informatics Singapore Management University

Research background Limitation : most algorithms are based on supervised learning , • so we need lots of labeled samples to train the model Medical images: expensive to label the data Mitosis detection (Image from Yao Lu) Yaoyao Liu | An Ensemble of Epoch-wise Empirical Bayes for Few-Shot Learning 2

Few-shot learning: learning with limited data Question: how to learn a model with limited labeled data? Task: few-shot image classification Seen classes Many-shot Unseen classes Few-shot (Images from Ravi and Larochelle) Yaoyao Liu | An Ensemble of Epoch-wise Empirical Bayes for Few-Shot Learning 3

Review: meta-learning Training tasks Seen classes Test task Meta-train Meta-test Unseen classes (Images from Ravi, Larochelle, and Zhou) Yaoyao Liu | An Ensemble of Epoch-wise Empirical Bayes for Few-Shot Learning 4

Existing methods vs. our E 3 BM SGD SGD E 3 BM SIB θ θ θ (d) E 3 BM (ours) (a) MAML [1] (b) MTL [2] (c) SIB [3] Existing methods : Our E 3 BM : • • A single base-learner An ensemble of multiple base-learners • • Arbitrary base-learning hyperparameters Task-specific base-learning hyperparameters - Unstable + Stable and robust Reference [1] Finn, Chelsea, Pieter Abbeel, and Sergey Levine. “Model-agnostic meta-learning for fast adaptation of deep networks.” ICML 2017; [2] Sun, Qianru, et al. “Meta-transfer learning for few-shot learning.” CVPR 2019; [3] Hu, Shell Xu, et al. “Empirical Bayes Transductive Meta-Learning with Synthetic Gradients.” ICLR 2020. Yaoyao Liu | An Ensemble of Epoch-wise Empirical Bayes for Few-Shot Learning 5

Existing method: MAML [1] Predictions from a single base-learner For one training task: ... ... Arbitrary base-learning hyperparameters meta update Epoch-wise base-learner Learning rate Combination weight Base-learner initializer Reference [1] Finn, Chelsea, Pieter Abbeel, and Sergey Levine. “Model-agnostic meta-learning for fast adaptation of deep networks.” ICML 2017. Yaoyao Liu | An Ensemble of Epoch-wise Empirical Bayes for Few-Shot Learning 6

Our method: E 3 BM framework For one training task: Predictions from multiple base-learners ... deploy ... ... ... ... meta update Hyperprior Learner ... deploy Task-specific base-learning hyperparameters Epoch-wise base-learner Learning rate Combination weight Base-learner initializer Yaoyao Liu | An Ensemble of Epoch-wise Empirical Bayes for Few-Shot Learning 7

Two options of hyperprior learner For the m -th base epoch: mean FC concat mean FC (a) Epoch-independent Yaoyao Liu | An Ensemble of Epoch-wise Empirical Bayes for Few-Shot Learning 8

Two options of hyperprior learner For the m -th base epoch: LSTM mean concat LSTM mean (b) Epoch-dependent Yaoyao Liu | An Ensemble of Epoch-wise Empirical Bayes for Few-Shot Learning 9

Boost the performance on THREE baselines The 5-class few-shot classification results (%). Yaoyao Liu | An Ensemble of Epoch-wise Empirical Bayes for Few-Shot Learning 10

Values of generated and Settings: MTL+E 3 BM, ResNet-25, #base-learners = 100 Yaoyao Liu | An Ensemble of Epoch-wise Empirical Bayes for Few-Shot Learning 11

Thank you! An Ensemble of Epoch-wise Empirical Bayes for Few-Shot Learning Webpage: https://e3bm.yyliu.net/ Code: https://gitlab.mpi-klsb.mpg.de/yaoyaoliu/e3bm Yaoyao Liu | An Ensemble of Epoch-wise Empirical Bayes for Few-Shot Learning 12