Efficient Graph-Based Active Learning with Probit Likelihood via - PowerPoint PPT Presentation

Efficient Graph-Based Active Learning with Probit Likelihood via Gaussian Approximations Kevin Miller, Hao Li, & Andrea Bertozzi University of California Los Angeles July 18, 2020

Active Learning Graph-Based SSL Graph-Based SSL Objective: 1 u ∗ = arg min � 2 � u , L τ u � + ℓ ( u j , y j ) =: arg min u ∈ R N J ℓ ( u ; y ) , (1) u ∈ R N j ∈L for different loss functions ℓ . Bayesian Probabilistic Perspective: P ( u | y ) ∝ exp( − J ℓ ( u ; y )) Most choices of ℓ lead to non-Gaussian posterior, P ( u | y ) Main Idea: Use Gaussian approximations of non-Gaussian posterior distributions to allow for more general uses of Gaussian-based acquisition functions in active learning. “Model Change” acquisition function Kevin Miller, Hao Li, & Andrea Bertozzi Probit AL in GBSSL July 18, 2020 2 / 5

Accuracy Results Checkerboard Results: 1 . 0 1 . 0 1 . 0 0 . 9 0 . 9 0 . 9 Accuracy 0 . 8 Accuracy 0 . 8 Accuracy 0 . 8 GR MC HF VOpt GR VOpt Probit MC 0 . 7 0 . 7 0 . 7 HF MBR GR MBR Probit VOpt HF SOpt GR SOpt Probit MBR 0 . 6 0 . 6 0 . 6 HF Random GR Random Probit Random HF Uncertainty GR Uncertainty Probit Uncertainty 0 . 5 0 . 5 0 . 5 0 50 100 150 200 0 50 100 150 200 0 50 100 150 200 Number of labeled points Number of labeled points Number of labeled points (a) HF (b) GR (c) Probit MNIST Results: 1 . 00 1 . 00 1 . 00 0 . 95 0 . 95 0 . 95 0 . 90 0 . 90 0 . 90 Accuracy Accuracy Accuracy 0 . 85 0 . 85 0 . 85 GR MC HF VOpt GR VOpt Probit MC 0 . 80 0 . 80 0 . 80 HF MBR GR MBR Probit VOpt 0 . 75 0 . 75 0 . 75 HF SOpt GR SOpt Probit MBR NA HF Random GR Random Probit Random 0 . 70 0 . 70 0 . 70 HF Uncertainty GR Uncertainty Probit Uncertainty 0 . 65 0 . 65 0 . 65 0 20 40 60 80 100 0 20 40 60 80 100 0 20 40 60 80 100 Number of labeled points Number of labeled points Number of labeled points (a) HF (b) GR (c) Probit Harmonic Functions (HF)1 Kevin Miller, Hao Li, & Andrea Bertozzi Probit AL in GBSSL July 18, 2020 3 / 5 1

Active Learning Choices - Checkerboard (a) HF-MBR (b) GR-MC (c) Probit-MC (d) HF-Vopt (e) GR-VOpt (f) Probit-Uncertainty Kevin Miller, Hao Li, & Andrea Bertozzi Figure 3: Acquisition function choices on the Checkerboard dataset. Yellow stars show Probit AL in GBSSL July 18, 2020 4 / 5

References I Ji, Ming and Jiawei Han. “A Variance Minimization Criterion to Active Learning on Graphs”. en. In: Artificial Intelligence and Statistics . ISSN: 1938-7228 Section: Machine Learning. Mar. 2012, pp. 556–564. url : http://proceedings.mlr.press/v22/ji12.html (visited on 06/11/2020). Ma, Yifei, Roman Garnett, and Jeff Schneider. “ Σ -Optimality for Active Learning on Gaussian Random Fields”. In: Advances in Neural Information Processing Systems 26 . Ed. by C. J. C. Burges et al. Curran Associates, Inc., 2013, pp. 2751–2759. url : http://papers.nips.cc/paper/4951-optimality-for-active- learning-on-gaussian-random-fields.pdf (visited on 06/11/2020). Zhu, Xiaojin, Zoubin Ghahramani, and John Lafferty. “Semi-supervised learning using Gaussian fields and harmonic functions”. In: Proceedings of the Twentieth International Conference on International Conference on Machine Learning . ICML’03. Washington, DC, USA: AAAI Press, Aug. 2003, pp. 912–919. isbn : 978-1-57735-189-4. (Visited on 06/11/2020). Zhu, Xiaojin, John Lafferty, and Zoubin Ghahramani. “Combining Active Learning and Semi-Supervised Learning Using Gaussian Fields and Harmonic Functions”. In: ICML 2003 workshop on The Continuum from Labeled to Unlabeled Data in Machine Learning and Data Mining . 2003, pp. 58–65. Kevin Miller, Hao Li, & Andrea Bertozzi Probit AL in GBSSL July 18, 2020 5 / 5