Poster #20 Bayesian Nonparametric Federated Learning of Neural - - PowerPoint PPT Presentation

▶
poster 20 bayesian nonparametric federated learning of
SMART_READER_LITE
LIVE PREVIEW

Poster #20 Bayesian Nonparametric Federated Learning of Neural - - PowerPoint PPT Presentation

Jan 20, 2023 30 likes •123 views

IBM Research, MIT-IBM Watson AI Lab PFNM , Poster #20 ICML 2019 Poster #20 Bayesian Nonparametric Federated Learning of Neural Networks Mikhail Yurochkin Mayank Agarwal, Soumya Ghosh, Kristjan Greenewald, Nghia Hoang, Yasaman Khazaeni IBM

slide-1
SLIDE 1

IBM Research, MIT-IBM Watson AI Lab PFNM, Poster #20

1

Poster #20 Bayesian Nonparametric Federated Learning of Neural Networks

ICML 2019

IBM Research, MIT-IBM Watson AI Lab

Mikhail Yurochkin

Mayank Agarwal, Soumya Ghosh, Kristjan Greenewald, Nghia Hoang, Yasaman Khazaeni

June 12th

slide-2
SLIDE 2

IBM Research, MIT-IBM Watson AI Lab PFNM, Poster #20

Federated Learning

slide-3
SLIDE 3

IBM Research, MIT-IBM Watson AI Lab PFNM, Poster #20

Model fusion perspective

slide-4
SLIDE 4

IBM Research, MIT-IBM Watson AI Lab PFNM, Poster #20

Probabilistic Federated Neural Matching

slide-5
SLIDE 5

IBM Research, MIT-IBM Watson AI Lab PFNM, Poster #20

Simulated heterogeneous Federated Learning on MNIST

Client 1 Client 2

slide-6
SLIDE 6

IBM Research, MIT-IBM Watson AI Lab PFNM, Poster #20

Examples of first layer weights

Client 1 Client 2

Neuron 21 Neuron 36 Neuron 49 Neuron 8 Neuron 7 Neuron 12

slide-7
SLIDE 7

IBM Research, MIT-IBM Watson AI Lab PFNM, Poster #20

PFNM discovers correspondences among weights

Client 1 Neuron 12 Client 2 Neuron 8 Matched neuron 8 Client 1 Neuron 49 Client 2 Neuron 7 Matched neuron 33 Client 2 Neuron 36 Matched neuron 44 Client 1 Neuron 21 Matched neuron 58

slide-8
SLIDE 8

IBM Research, MIT-IBM Watson AI Lab PFNM, Poster #20

Summary

PFNM is a method for combining pre-trained fully-connected neural networks:

  • Can combine NNs trained on heterogeneous data without access to data
  • Can be further improved with few communication rounds (if data is available)
  • Outperforms Distributed SGD and Federated Averaging

Technical contributions:

  • Indian Buffet Process based model to govern correspondences between

weights of local neural networks. Applicable to multilayer networks

  • BNP allows for adaptive learning of global NN size
  • Fast MAP inference using iterative Hungarian algorithm
slide-9
SLIDE 9

IBM Research, MIT-IBM Watson AI Lab PFNM, Poster #20

9

THANK YOU | Please come to poster #2 #20