linearized two layers neural network in high dimensions
play

Linearized two-layers neural network in high dimensions Song Mei - PowerPoint PPT Presentation

Dec 09, 2022 •46 likes •436 views

Linearized two-layers neural network in high dimensions Song Mei Stanford University May 26, 2019 Joint work with Andrea Montanari, Theodor Misiakiewicz, Behrooz Ghorbani Song Mei (Stanford University) Linearized two layers neural network

  1. Linearized two-layers neural network in high dimensions Song Mei Stanford University May 26, 2019 Joint work with Andrea Montanari, Theodor Misiakiewicz, Behrooz Ghorbani Song Mei (Stanford University) Linearized two layers neural network May 26, 2019 1 / 22

  2. Song Mei (Stanford University) Linearized two layers neural network May 26, 2019 2 / 22

  3. ❘ ✭ Θ ✮ ❂ ♠✐♥ Θ E ❬ ❵ ✭ ②❀ W ✶ ✛ ✍ W ✷ ✍ ✛ ✍ ✁ ✁ ✁ ✍ W ❦ ✍ x ✮❪ ✿ Empirical surprise of neural network [Zhang et al. , 2016] ◮ Over-parameterized regime. ◮ Optimization surprise: efficiently fit all the data. ◮ Generalization surprise: generalize well. Song Mei (Stanford University) Linearized two layers neural network May 26, 2019 3 / 22

  4. Two-layers neural network ◆ ❫ ❳ ❢ ◆ ✭ x ❀ Θ ✮ ❂ ❛ ✐ ✛ ✭ ❤ w ✐ ❀ x ✐ ✮ ❀ Θ ❂ ✭ ❛ ✶ ❀ w ✶ ❀ ✿ ✿ ✿ ❀ ❛ ◆ ❀ w ◆ ✮ ✿ ✐ ❂✶ ◮ Feature x ✷ R ❞ . ◮ Bottom layer weights w ✐ ✷ R ❞ , ✐ ❂ ✶ ❀ ✷ ❀ ✿ ✿ ✿ ❀ ◆ . ◮ Top layer weights ❛ ✐ ✷ R , ✐ ❂ ✶ ❀ ✷ ❀ ✿ ✿ ✿ ❀ ◆ . ◮ Over-parametrization: ◆ large. Song Mei (Stanford University) Linearized two layers neural network May 26, 2019 4 / 22

  5. Two-layers neural network ◆ ❫ ❳ ❢ ◆ ✭ x ❀ Θ ✮ ❂ ❛ ✐ ✛ ✭ ❤ w ✐ ❀ x ✐ ✮ ❀ Θ ❂ ✭ ❛ ✶ ❀ w ✶ ❀ ✿ ✿ ✿ ❀ ❛ ◆ ❀ w ◆ ✮ ✿ ✐ ❂✶ ◮ Feature x ✷ R ❞ . ◮ Bottom layer weights w ✐ ✷ R ❞ , ✐ ❂ ✶ ❀ ✷ ❀ ✿ ✿ ✿ ❀ ◆ . ◮ Top layer weights ❛ ✐ ✷ R , ✐ ❂ ✶ ❀ ✷ ❀ ✿ ✿ ✿ ❀ ◆ . ◮ Over-parametrization: ◆ large. Song Mei (Stanford University) Linearized two layers neural network May 26, 2019 4 / 22

  6. Two-layers neural network ◆ ❫ ❳ ❢ ◆ ✭ x ❀ Θ ✮ ❂ ❛ ✐ ✛ ✭ ❤ w ✐ ❀ x ✐ ✮ ❀ Θ ❂ ✭ ❛ ✶ ❀ w ✶ ❀ ✿ ✿ ✿ ❀ ❛ ◆ ❀ w ◆ ✮ ✿ ✐ ❂✶ ◮ Feature x ✷ R ❞ . ◮ Bottom layer weights w ✐ ✷ R ❞ , ✐ ❂ ✶ ❀ ✷ ❀ ✿ ✿ ✿ ❀ ◆ . ◮ Top layer weights ❛ ✐ ✷ R , ✐ ❂ ✶ ❀ ✷ ❀ ✿ ✿ ✿ ❀ ◆ . ◮ Over-parametrization: ◆ large. Song Mei (Stanford University) Linearized two layers neural network May 26, 2019 4 / 22

  7. Two-layers neural network ◆ ❫ ❳ ❢ ◆ ✭ x ❀ Θ ✮ ❂ ❛ ✐ ✛ ✭ ❤ w ✐ ❀ x ✐ ✮ ❀ Θ ❂ ✭ ❛ ✶ ❀ w ✶ ❀ ✿ ✿ ✿ ❀ ❛ ◆ ❀ w ◆ ✮ ✿ ✐ ❂✶ ◮ Feature x ✷ R ❞ . ◮ Bottom layer weights w ✐ ✷ R ❞ , ✐ ❂ ✶ ❀ ✷ ❀ ✿ ✿ ✿ ❀ ◆ . ◮ Top layer weights ❛ ✐ ✷ R , ✐ ❂ ✶ ❀ ✷ ❀ ✿ ✿ ✿ ❀ ◆ . ◮ Over-parametrization: ◆ large. Song Mei (Stanford University) Linearized two layers neural network May 26, 2019 4 / 22

  8. Two-layers neural network ◆ ❫ ❳ ❢ ◆ ✭ x ❀ Θ ✮ ❂ ❛ ✐ ✛ ✭ ❤ w ✐ ❀ x ✐ ✮ ❀ Θ ❂ ✭ ❛ ✶ ❀ w ✶ ❀ ✿ ✿ ✿ ❀ ❛ ◆ ❀ w ◆ ✮ ✿ ✐ ❂✶ ◮ Feature x ✷ R ❞ . ◮ Bottom layer weights w ✐ ✷ R ❞ , ✐ ❂ ✶ ❀ ✷ ❀ ✿ ✿ ✿ ❀ ◆ . ◮ Top layer weights ❛ ✐ ✷ R , ✐ ❂ ✶ ❀ ✷ ❀ ✿ ✿ ✿ ❀ ◆ . ◮ Over-parametrization: ◆ large. Song Mei (Stanford University) Linearized two layers neural network May 26, 2019 4 / 22

  9. Two-layers neural network Input layer Hidden layer Output layer w 1 a 1 w 2 a 2 w 3 a 3 a 4 w 4 Song Mei (Stanford University) Linearized two layers neural network May 26, 2019 5 / 22

  10. Gradient flow with random initialization Empirical risk: ( ♥ : ★ data; ◆ : ★ neuron) ❘ ♥❀◆ ✭ Θ ✮ ❂ ❫ E x ❀♥ ❬✭ ② � ❫ ❢ ◆ ✭ x ❀ Θ ✮✮ ✷ ❪ Gradient flow, on empirical risk, with random initialization: ❴ Θ ✭ t ✮ ❂ �r ❘ ♥❀◆ ✭ Θ ✭ t ✮✮ ❀ ✭ ❛ ✐ ✭✵✮ ❀ w ✐ ✭✵✮✮ ✘ ✐✿✐✿❞✿ P ❛❀ w ✿ Song Mei (Stanford University) Linearized two layers neural network May 26, 2019 6 / 22

  11. ◆ ✢ ♥ ✶✰ ❝ t ✦✶ ❘ ♥❀◆ ✭ ❧✐♠ ✭ t ✮✮ ❂ ✵ ❀ ✵ Convergence guarantees Lemma (Global min. Not surprise. ) For ◆ ❃ ♥ , we have ✐♥❢ Θ ❘ ♥❀◆ ✭ Θ ✮ ❂ ✵ ✿ There are many global minimizer with empirical risk ✵ . Song Mei (Stanford University) Linearized two layers neural network May 26, 2019 7 / 22

  12. ◆ ✢ ♥ ✶✰ ❝ t ✦✶ ❘ ♥❀◆ ✭ ❧✐♠ ✭ t ✮✮ ❂ ✵ ❀ ✵ Convergence guarantees Lemma (Global min. Not surprise. ) For ◆ ❃ ♥ , we have ✐♥❢ Θ ❘ ♥❀◆ ✭ Θ ✮ ❂ ✵ ✿ There are many global minimizer with empirical risk ✵ . But there are also local minimizers with non-zero risk. Song Mei (Stanford University) Linearized two layers neural network May 26, 2019 7 / 22

  13. Convergence guarantees Lemma (Global min. Not surprise. ) For ◆ ❃ ♥ , we have ✐♥❢ Θ ❘ ♥❀◆ ✭ Θ ✮ ❂ ✵ ✿ There are many global minimizer with empirical risk ✵ . But there are also local minimizers with non-zero risk. Theorem (The optimization surprise. ) For ◆ ✢ ♥ ✶✰ ❝ , we have t ✦✶ ❘ ♥❀◆ ✭ Θ ✭ t ✮✮ ❂ ✵ ❀ ❧✐♠ i.e., training loss converges to ✵ . Song Mei (Stanford University) Linearized two layers neural network May 26, 2019 7 / 22

  14. Convergence guarantees Lemma (Global min. Not surprise. ) For ◆ ❃ ♥ , we have ✐♥❢ Θ ❘ ♥❀◆ ✭ Θ ✮ ❂ ✵ ✿ There are many global minimizer with empirical risk ✵ . But there are also local minimizers with non-zero risk. Theorem (The optimization surprise. ) For ◆ ✢ ♥ ✶✰ ❝ , we have t ✦✶ ❘ ♥❀◆ ✭ Θ ✭ t ✮✮ ❂ ✵ ❀ ❧✐♠ i.e., training loss converges to ✵ . Under what assumptions? Song Mei (Stanford University) Linearized two layers neural network May 26, 2019 7 / 22

  15. Three variants of the convergence theorem Gradient flow ( ♥ : ★ data; ◆ : ★ neuron): Θ ✭ t ✮ ❂ �r ❫ E x ❀♥ ❬✭ ② � ❫ ❴ ❢ ◆ ✭ x ❀ Θ ✭ t ✮✮✮ ✷ ❪ ❀ w ✐ ✭✵✮ ✘ ✐✿✐✿❞✿ ◆ ✭ 0 ❀ I ❞ ❂❞ ✮ ✿ Theorem: for ◆ large enough, we have t ✦✶ ❘ ♥❀◆ ✭ Θ ✭ t ✮✮ ❂ ✵ ✿ ❧✐♠ Random feature ( RF ) regime ◆ ❫ ❛ ✐ ✭✵✮ ✘ ✐✿✐✿❞✿ ◆ ✭✵ ❀ ✶ ❂◆ ✷ ✮ ✿ ❳ ❢ ◆ ✭ x ❀ Θ ✮ ❂ ❛ ✐ ✛ ✭ ❤ w ✐ ❀ x ✐ ✮ ❀ ✐ ❂✶ [Andoni et al. , 2014], [Danialy, 2017], [Yehudai and Shamir, 2019] ... Song Mei (Stanford University) Linearized two layers neural network May 26, 2019 8 / 22

  16. Three variants of the convergence theorem Gradient flow ( ♥ : ★ data; ◆ : ★ neuron): Θ ✭ t ✮ ❂ �r ❫ ❴ E x ❀♥ ❬✭ ② � ❫ ❢ ◆ ✭ x ❀ Θ ✭ t ✮✮✮ ✷ ❪ ❀ w ✐ ✭✵✮ ✘ ✐✿✐✿❞✿ ◆ ✭ 0 ❀ I ❞ ❂❞ ✮ ✿ Theorem: for ◆ large enough, we have t ✦✶ ❘ ♥❀◆ ✭ Θ ✭ t ✮✮ ❂ ✵ ✿ ❧✐♠ Neural tangent ( NT ) regime ◆ ✶ ❫ ❳ ♣ ❛ ✐ ✛ ✭ ❤ w ✐ ❀ x ✐ ✮ ❀ ❛ ✐ ✭✵✮ ✘ ✐✿✐✿❞✿ ◆ ✭✵ ❀ ✶✮ ✿ ❢ ◆ ✭ x ❀ Θ ✮ ❂ ◆ ✐ ❂✶ [Jacot et al. , 2018], [Du et al. , 2018], [Du et al. , 2018], [Allen-Zhu el al. , 2018], [Zou et al. , 2018]... Song Mei (Stanford University) Linearized two layers neural network May 26, 2019 8 / 22

  17. Three variants of the convergence theorem Gradient flow ( ♥ : ★ data; ◆ : ★ neuron): Θ ✭ t ✮ ❂ �r ❫ ❴ E x ❀♥ ❬✭ ② � ❫ ❢ ◆ ✭ x ❀ Θ ✭ t ✮✮✮ ✷ ❪ ❀ w ✐ ✭✵✮ ✘ ✐✿✐✿❞✿ ◆ ✭ 0 ❀ I ❞ ❂❞ ✮ ✿ Theorem: for ◆ large enough, we have * t ✦✶ ❘ ♥❀◆ ✭ Θ ✭ t ✮✮ ❂ ✵ ✿ ❧✐♠ Mean field ( MF ) regime ◆ ❢ ◆ ✭ x ❀ Θ ✮ ❂ ✶ ❫ ❳ ❛ ✐ ✛ ✭ ❤ w ✐ ❀ x ✐ ✮ ❀ ❛ ✐ ✭✵✮ ✘ ✐✿✐✿❞✿ ◆ ✭✵ ❀ ✶✮ ✿ ◆ ✐ ❂✶ [Mei et al. , 2018], [Rotskoff and Vanden-Eijden, 2018], [Chizat and Bach, 2018]... Song Mei (Stanford University) Linearized two layers neural network May 26, 2019 8 / 22

  18. Three variants of the convergence theorem Random feature ( RF ) regime ◆ ❫ ❛ ✐ ✘ ◆ ✭✵ ❀ ✶ ❂◆ ✷ ✮ ✿ ❳ ❢ ◆ ✭ x ❀ Θ ✮ ❂ ❛ ✐ ✛ ✭ ❤ w ✐ ❀ x ✐ ✮ ❀ ✐ ❂✶ Neural tangent ( NT ) regime ◆ ✶ ❫ ❳ ❢ ◆ ✭ x ❀ Θ ✮ ❂ ♣ ❛ ✐ ✛ ✭ ❤ w ✐ ❀ x ✐ ✮ ❀ ❛ ✐ ✘ ◆ ✭✵ ❀ ✶✮ ✿ ◆ ✐ ❂✶ Mean field ( MF ) regime ◆ ❢ ◆ ✭ x ❀ Θ ✮ ❂ ✶ ❫ ❳ ❛ ✐ ✛ ✭ ❤ w ✐ ❀ x ✐ ✮ ❀ ❛ ✐ ✘ ◆ ✭✵ ❀ ✶✮ ✿ ◆ ✐ ❂✶ Song Mei (Stanford University) Linearized two layers neural network May 26, 2019 9 / 22

  19. ... but different behavior of dynamics Random feature ( RF ) regime ◆ ❫ ❛ ✐ ✘ ◆ ✭✵ ❀ ✶ ❂◆ ✷ ✮ ✿ ❳ ❢ ◆ ✭ x ❀ Θ ✮ ❂ ❛ ✐ ✛ ✭ ❤ w ✐ ❀ x ✐ ✮ ❀ ✐ ❂✶ ◮ The limiting dynamics is linear (effectively only a is updated). ◮ Prediction function: kernel ridge regression with kernel ❦ RF ✭ x ❀ z ✮ ❂ ❫ E w ❀◆ ❬ ✛ ✭ ❤ w ❀ z ✐ ✮ ✛ ✭ ❤ w ❀ z ✐ ✮❪ ✿ [Andoni et al. , 2014], [Danialy, 2017], [Yehudai and Shamir, 2019] ... Song Mei (Stanford University) Linearized two layers neural network May 26, 2019 10 / 22

Download Presentation
Download Policy: The content available on the website is offered to you 'AS IS' for your personal information and use only. It cannot be commercialized, licensed, or distributed on other websites without prior consent from the author. To download a presentation, simply click this link. If you encounter any difficulties during the download process, it's possible that the publisher has removed the file from their server.

Recommend

Perceptrons 2-29-16 What is a neural network? activation connection functions A NN is a

Perceptrons 2-29-16 What is a neural network? activation connection functions A NN is a

Perceptrons 2-29-16 What is a neural network? activation connection functions A NN is a directed acyclic graph. weights Nodes are organized into layers. -0.5 Consecutive layers are fully connected. 1.5 0.2 Edges have

147 views • 13 slides
Cost function Machine Learning Neural Network (Classification) total no. of layers in network

Cost function Machine Learning Neural Network (Classification) total no. of layers in network

Neural Networks: Learning Cost function Machine Learning Neural Network (Classification) total no. of layers in network no. of units (not counting bias unit) in layer Layer 1 Layer 2 Layer 3 Layer 4 Multi-class classification (K classes)

711 views • 33 slides
Analyzing Backprop 3-4-16 Reading Quiz Q1: If a neural network has 3 layers with 10 input, 6

Analyzing Backprop 3-4-16 Reading Quiz Q1: If a neural network has 3 layers with 10 input, 6

Analyzing Backprop 3-4-16 Reading Quiz Q1: If a neural network has 3 layers with 10 input, 6 hidden, and 8 output units, what is the dimension of backpropagations local search space? a) 10 + 6 + 8 = 24 b) 10 + 6 * 8 = 58 c) 10 * 6 + 6 *

381 views • 12 slides
The committee machine: Computational to statistical gaps in learning a two-layers neural network

The committee machine: Computational to statistical gaps in learning a two-layers neural network

The committee machine: Computational to statistical gaps in learning a two-layers neural network Benjamin Aubin , Antoine Maillard, Jean Barbier Nicolas Macris, Florent Krzakala & Lenka Zdeborov Benjamin Aubin - Institut de Physique T

1.2k views • 32 slides
Generalization of linearized neural networks: staircase decay and double descent Song Mei UC

Generalization of linearized neural networks: staircase decay and double descent Song Mei UC

Generalization of linearized neural networks: staircase decay and double descent Song Mei UC Berkeley July 23, 2020 Department of Mathematics, HKUST Deep Learning Revolution Machine translation Autonomous Vehicle Robotics Healthcare

1.3k views • 72 slides
A Mean Field View of the Landscape of Two-Layers Neural Networks Song Mei Stanford University

A Mean Field View of the Landscape of Two-Layers Neural Networks Song Mei Stanford University

A Mean Field View of the Landscape of Two-Layers Neural Networks Song Mei Stanford University November 14, 2018 Joint work with Andrea Montanari and Phan-Minh Nguyen Song Mei (Stanford University) Mean Field Dynamics for Neural Network

807 views • 70 slides
Dense layers IN TRODUCTION TO TEN S ORF LOW IN P YTH ON Isaiah Hull Economist The linear

Dense layers IN TRODUCTION TO TEN S ORF LOW IN P YTH ON Isaiah Hull Economist The linear

Dense layers IN TRODUCTION TO TEN S ORF LOW IN P YTH ON Isaiah Hull Economist The linear regression model INTRODUCTION TO TENSORFLOW IN PYTHON What is a neural network? INTRODUCTION TO TENSORFLOW IN PYTHON What is a neural network? A

715 views • 42 slides
The One Hundred Layers Tiramisu: Fully Convolutional DenseNets for Semantic Segmentation Simon

The One Hundred Layers Tiramisu: Fully Convolutional DenseNets for Semantic Segmentation Simon

The One Hundred Layers Tiramisu: Fully Convolutional DenseNets for Semantic Segmentation Simon Jegou , Michal Drozdzal, David Vazquez, Adriana Romero Yoshua Bengio 1 Deep Neural Network use a cascade of multiple layers of units for

640 views • 29 slides
Neural Information Retrieval Wassila Lalouani 1 Plan Neural network architectures Neural

Neural Information Retrieval Wassila Lalouani 1 Plan Neural network architectures Neural

Neural Information Retrieval Wassila Lalouani 1 Plan Neural network architectures Neural IR tasks Neural IR architecture Feature Representations Neural IR query auto completion Neural IR query suggestion Neural IR document

1.48k views • 18 slides
Limitations of recurrent neural networks N ATURAL LAN GUAGE GEN ERATION IN P YTH ON Biswanath

Limitations of recurrent neural networks N ATURAL LAN GUAGE GEN ERATION IN P YTH ON Biswanath

Limitations of recurrent neural networks N ATURAL LAN GUAGE GEN ERATION IN P YTH ON Biswanath Halder Data Scientist Simple neural networks Nodes arranged in layers. Nodes in different layers connected by weights. Input to rst layer :

802 views • 29 slides
Convolutional Neural Networks in Speech Lecture 20 CS 753 Instructor: Preethi Jyothi

Convolutional Neural Networks in Speech Lecture 20 CS 753 Instructor: Preethi Jyothi

Convolutional Neural Networks in Speech Lecture 20 CS 753 Instructor: Preethi Jyothi Convolutional Neural Networks (CNNs) Fully connected (dense) layers have no awareness of spatial information Key concept behind convolutional layers is

983 views • 41 slides
Linearly Augmented Deep Neural Network Pegah Ghahremani, Johns Hopkins University, Jasha Droppo,

Linearly Augmented Deep Neural Network Pegah Ghahremani, Johns Hopkins University, Jasha Droppo,

Linearly Augmented Deep Neural Network Pegah Ghahremani, Johns Hopkins University, Jasha Droppo, Michael L. Seltzer, Microsoft Research Typical DNN Architecture Softmax L4 Layers are a composition of an affine and a non- linear function.

199 views • 19 slides
Deep Neural Network Mathematical Mysteries for High Dimensional Learning Stphane Mallat

Deep Neural Network Mathematical Mysteries for High Dimensional Learning Stphane Mallat

Deep Neural Network Mathematical Mysteries for High Dimensional Learning Stphane Mallat cole Normale Suprieure www.di.ens.fr/data High Dimensional Learning High-dimensional x = ( x (1) , ..., x ( d )) R d : Classification:

1.34k views • 87 slides
Learning From Data Lecture 20 Multilayer Perceptron Multiple layers Universal Approximation

Learning From Data Lecture 20 Multilayer Perceptron Multiple layers Universal Approximation

Learning From Data Lecture 20 Multilayer Perceptron Multiple layers Universal Approximation The Neural Network M. Magdon-Ismail CSCI 4100/6100 recap: Unsupervised Learning k -Means Clustering Gaussian Mixture Model P ( x ) x Hard

699 views • 18 slides
CS 440 Fall 2005 Name: ______________________________ Midterm Exam 1. a. Draw the layers of

CS 440 Fall 2005 Name: ______________________________ Midterm Exam 1. a. Draw the layers of

CS 440 Fall 2005 Name: ______________________________ Midterm Exam 1. a. Draw the layers of the ISO OSI network architecture. (5) Application Presentation Session Transport Network Data Link Physical b. Draw the layers of the ARPA

96 views • 6 slides
Artificial Neural Networks By: Kodi Neumiller Overview What is an artificial neural network

Artificial Neural Networks By: Kodi Neumiller Overview What is an artificial neural network

Artificial Neural Networks By: Kodi Neumiller Overview What is an artificial neural network Perceptron Feed forward Backpropagation Learning rate/momentum What is a neural network? Artificial Human Neurons Each

888 views • 12 slides
World's Fastest Machine Learning With GPUs http://github.com/h2oai/h2o4gpu Speaker: Jonathan C.

World's Fastest Machine Learning With GPUs http://github.com/h2oai/h2o4gpu Speaker: Jonathan C.

World's Fastest Machine Learning With GPUs http://github.com/h2oai/h2o4gpu Speaker: Jonathan C. McKinney Mateusz Erin Navdeep Rory Terry Karen Arno Jonathan S teve H2O4GPU TEAM Machine Learning Deep Learning c RIS

572 views • 33 slides
The effectiveness of fiscal policy The effectiveness of fiscal policy in Australia - - selected

The effectiveness of fiscal policy The effectiveness of fiscal policy in Australia - - selected

The effectiveness of fiscal policy The effectiveness of fiscal policy in Australia - - selected issues selected issues in Australia By Comley,B By Comley,B. . Anthony,S Anthony,S. and . and Ferguson,B Ferguson,B. . Outline Outline

264 views • 12 slides
Understanding EDA Revolving Loan Funds April 9, 2019 1 Agenda RLF PROGRAM OVERVIEW AND RISK

Understanding EDA Revolving Loan Funds April 9, 2019 1 Agenda RLF PROGRAM OVERVIEW AND RISK

Understanding EDA Revolving Loan Funds April 9, 2019 1 Agenda RLF PROGRAM OVERVIEW AND RISK ANALYSIS SYSTEM RISK ANALYSIS CASE STUDY CORRECTIVE ACTION PLAN EDA Revolving Loan Funds EDA RLFs established for lending Economic Adjustment

700 views • 34 slides
Projects Presentation Programs & Administration Committee Planning & Organization

Projects Presentation Programs & Administration Committee Planning & Organization

Projects Presentation Programs & Administration Committee Planning & Organization Committee and Recycling Board February 12, 2015 Projects Presentations Schedule A continuation of Januarys Past, Present, Future

818 views • 67 slides
More efficient representations of compounds for machine learning models Bing Huang and Anatole von

More efficient representations of compounds for machine learning models Bing Huang and Anatole von

More efficient representations of compounds for machine learning models Bing Huang and Anatole von Lilienfeld Institute of Physical Chemistry and National Centre for Computational Design and Discovery of Novel Materials (MARVEL) Department of

428 views • 40 slides
Estimation of the Kernel Mean Embedding (with uncertainty) Paul Rubenstein University of

Estimation of the Kernel Mean Embedding (with uncertainty) Paul Rubenstein University of

Estimation of the Kernel Mean Embedding (with uncertainty) Paul Rubenstein University of Cambridge Max-Planck Institute for Intelligent Systems, Tbingen 20th January 2016 RKHS theory A function k : X X R is a kernel if given x 1 ,

582 views • 11 slides
Parallel and Distributed Training of Neural Networks via Successive Convex Approximation Authors

Parallel and Distributed Training of Neural Networks via Successive Convex Approximation Authors

2016 IEEE International Workshop on Machine Learning for Signal Processing (MLSP16) Parallel and Distributed Training of Neural Networks via Successive Convex Approximation Authors : Paolo Di Lorenzo and Simone Scardapane Contents

686 views • 26 slides
Neuromorphic Computing with Reservoir Neural Networks on Memristive Hardware Aaron Stockdill

Neuromorphic Computing with Reservoir Neural Networks on Memristive Hardware Aaron Stockdill

Neuromorphic Computing with Reservoir Neural Networks on Memristive Hardware Aaron Stockdill September 2016 Neuromorphic Computing with Reservoir Neural Networks on Memristive Hardware Aaron Stockdill September 2016 Neuromorphic Computing

657 views • 26 slides

More recommend