Efficient Graph-Based Active Learning with Probit Likelihood via - - PowerPoint PPT Presentation

▶

Mar 07, 2023 291 likes •352 views

Efficient Graph-Based Active Learning with Probit Likelihood via Gaussian Approximations Kevin Miller, Hao Li, & Andrea Bertozzi University of California Los Angeles July 18, 2020 Active Learning Graph-Based SSL Graph-Based SSL Objective:

SLIDE 1

Efficient Graph-Based Active Learning with Probit Likelihood via Gaussian Approximations

Kevin Miller, Hao Li, & Andrea Bertozzi

University of California Los Angeles

July 18, 2020

SLIDE 2

Active Learning Graph-Based SSL

Graph-Based SSL Objective: u∗ = arg min

u∈RN

1 2u, Lτu +

j∈L

ℓ(uj, yj) =: arg min

u∈RN Jℓ(u; y),

(1) for different loss functions ℓ. Bayesian Probabilistic Perspective: P(u|y) ∝ exp(−Jℓ(u; y)) Most choices of ℓ lead to non-Gaussian posterior, P(u|y) Main Idea: Use Gaussian approximations of non-Gaussian posterior distributions to allow for more general uses of Gaussian-based acquisition functions in active learning. “Model Change” acquisition function

Kevin Miller, Hao Li, & Andrea Bertozzi Probit AL in GBSSL July 18, 2020 2 / 5

SLIDE 3

Accuracy Results

Checkerboard Results:

50 100 150 200 Number of labeled points 0.5 0.6 0.7 0.8 0.9 1.0 Accuracy HF VOpt HF MBR HF SOpt HF Random HF Uncertainty

(a) HF

50 100 150 200 Number of labeled points 0.5 0.6 0.7 0.8 0.9 1.0 Accuracy GR MC GR VOpt GR MBR GR SOpt GR Random GR Uncertainty

(b) GR

50 100 150 200 Number of labeled points 0.5 0.6 0.7 0.8 0.9 1.0 Accuracy Probit MC Probit VOpt Probit MBR Probit Random Probit Uncertainty

MNIST Results:

20 40 60 80 100 Number of labeled points 0.65 0.70 0.75 0.80 0.85 0.90 0.95 1.00 Accuracy HF VOpt HF MBR HF SOpt HF Random HF Uncertainty

(a) HF

20 40 60 80 100 Number of labeled points 0.65 0.70 0.75 0.80 0.85 0.90 0.95 1.00 Accuracy GR MC GR VOpt GR MBR GR SOpt GR Random GR Uncertainty

(b) GR

20 40 60 80 100 Number of labeled points 0.65 0.70 0.75 0.80 0.85 0.90 0.95 1.00 Accuracy Probit MC Probit VOpt Probit MBR NA Probit Random Probit Uncertainty

SLIDE 4

Active Learning Choices - Checkerboard

(a) HF-MBR (b) GR-MC (c) Probit-MC (d) HF-Vopt (e) GR-VOpt (f) Probit-Uncertainty

Figure 3: Acquisition function choices on the Checkerboard dataset. Yellow stars show

Kevin Miller, Hao Li, & Andrea Bertozzi Probit AL in GBSSL July 18, 2020 4 / 5

SLIDE 5

References I

Ji, Ming and Jiawei Han. “A Variance Minimization Criterion to Active Learning on Graphs”. en. In: Artificial Intelligence and Statistics. ISSN: 1938-7228 Section: Machine Learning. Mar. 2012, pp. 556–564. url: http://proceedings.mlr.press/v22/ji12.html (visited on 06/11/2020). Ma, Yifei, Roman Garnett, and Jeff Schneider. “Σ-Optimality for Active Learning on Gaussian Random Fields”. In: Advances in Neural Information Processing Systems 26. Ed. by C. J. C. Burges et al. Curran Associates, Inc., 2013, pp. 2751–2759. url: http://papers.nips.cc/paper/4951-optimality-for-active- learning-on-gaussian-random-fields.pdf (visited on 06/11/2020). Zhu, Xiaojin, Zoubin Ghahramani, and John Lafferty. “Semi-supervised learning using Gaussian fields and harmonic functions”. In: Proceedings of the Twentieth International Conference on International Conference on Machine

Learning. ICML’03. Washington, DC, USA: AAAI Press, Aug. 2003, pp. 912–919. isbn: 978-1-57735-189-4.

(Visited on 06/11/2020). Zhu, Xiaojin, John Lafferty, and Zoubin Ghahramani. “Combining Active Learning and Semi-Supervised Learning Using Gaussian Fields and Harmonic Functions”. In: ICML 2003 workshop on The Continuum from Labeled to Unlabeled Data in Machine Learning and Data Mining. 2003, pp. 58–65.

Kevin Miller, Hao Li, & Andrea Bertozzi Probit AL in GBSSL July 18, 2020 5 / 5