Anatomy and Interpretability of Neural Networks Leon Yin ~ Data - PowerPoint PPT Presentation

Jan 06, 2023 •204 likes •403 views

Anatomy and Interpretability of Neural Networks Leon Yin ~ Data Scientist | Research Engineer SMaPP and CDS PRG 2017-11-15 Todays talking points: How do Neural Networks work? How can we see what theyre learning? Discussion about

Anatomy and Interpretability of Neural Networks Leon Yin ~ Data Scientist | Research Engineer SMaPP and CDS PRG 2017-11-15
Today’s talking points: How do Neural Networks work? How can we see what they’re learning? Discussion about training data and policy.
First of all All models are wrong, but some are useful!
Neural Networks: Transforms one dataset (D) into another dataset (D’). The D’ is optimized for discrimination.
Basic Functions 1. Matrix multiplication 2. Thresholding
Matrix Multiplication Input gets multiplied by N randomly initialized weight s , Where N is equal to the number of nodes (neurons) in the next layer.
Convolutions Kernel or Filter https://nbviewer.jupyter.org/github/yinleon/interpreting_nerual_networks/blob/master/null_features/neural_netw ork_basics.ipynb
Thresholding
Thresholding or Activation Functions Rectified Linear Units (ReLU) remove negative values.
Downsampling Use pooling function either Max, Avg, Sum Also for simplification and amplification
Recap: Matrix multiplication creates new features. Thresholding and downsampling simplify the math and amplify signals. This is repeated and combined to identify patterns with increasing complexity.
Feature Visualization https://distill.pub/2017/feature-visualization/
Let’s Look at Logits: https://nbviewer.jupyter.org/github/yinleon/interpreting_nerual_networks/blob/mast er/null_features/model_conv_feature_evaluation.ipynb
What about Text?
Bias on Yelp Different tasks have the same outcomes: Mexican food is associated with negative reviews and negative connotations!
Training Data We build infrastructure around availability What are we feeding models? Cool paper about reducing training data gender bias: https://homes.cs.washington.edu/~my89/publications/bias.pdf
Looking for Context NLP community standardizing metadata RE: origin, app and audience.
Thoughts about Interpretability?
Thanks! @leonyin

Download Presentation

Download Policy: The content available on the website is offered to you 'AS IS' for your personal information and use only. It cannot be commercialized, licensed, or distributed on other websites without prior consent from the author. To download a presentation, simply click this link. If you encounter any difficulties during the download process, it's possible that the publisher has removed the file from their server.

Recommend

Interpretability and Visualization of Deep Neural Networks Au Aude Oliva MI MIT Convoluti

Interpretability and Visualization of Deep Neural Networks Au Aude Oliva MI MIT Convoluti tional Neura ral Netw twork rks convolution max-pooling normalization full connected Alexnet Ea Each ch laye yer learns prog ogressive

356 views • 34 slides

Interpretability in Convolutional Neural Networks for Building Damage Classification in Satellite

Interpretability in Convolutional Neural Networks for Building Damage Classification in Satellite Imagery NeurIPS 2020 Workshop Tackling Climate Change with Machine Learning Thomas Y. Chen Computer Vision, Satellite Imagery, and Building

401 views • 16 slides

Intro to Artificial Neural Networks Oscar Maas @oscmansan Outline 1. Perceptrons 2.

Intro to Artificial Neural Networks Oscar Maas @oscmansan Outline 1. Perceptrons 2. Sigmoid Neurons 3. Linear vs Nonlinear Classification 4. Neural Networks Anatomy 5. Training and loss a. Reducing loss b. Gradient Descent c.

832 views • 21 slides

Outline Evolution of neurocomputing Artificial neural networks Feed forward

Introduction to Neural Networks and Deep Learning Ling Guan References: S. Haykin, Neural Networks , 2 nd Edition, New Jersey: Prentice Hall, 1. 2004. C. M. Bishop, Neural Networks for Pattern Re cognition, New York: 2. Oxford University

451 views • 17 slides

Neural Networks 1. Introduction Fall 2017 Neural Networks are taking over! Neural networks

Neural Networks 1. Introduction Fall 2017 Neural Networks are taking over! Neural networks have become one of the major thrust areas recently in various pattern recognition, prediction, and analysis problems In many problems they have

1.17k views • 91 slides

Neural Networks 0. Logistics Spring 2019 1 Neural Networks are taking over! Neural networks

Neural Networks 0. Logistics Spring 2019 1 Neural Networks are taking over! Neural networks have become one of the major thrust areas recently in various pattern recognition, prediction, and analysis problems In many problems they have

852 views • 33 slides

CONVOLUTIONAL AND RECURRENT NEURAL NETWORKS Neural networks Fully connected networks

CONVOLUTIONAL AND RECURRENT NEURAL NETWORKS Neural networks Fully connected networks Neuron Non-linearity Softmax layer DNN training Loss function and regularization SGD and backprop Learning rate Overfitting

1.8k views • 86 slides

Neural Networks 1. Introduction Spring 2020 1 Neural Networks are taking over! Neural

Neural Networks 1. Introduction Spring 2020 1 Neural Networks are taking over! Neural networks have become one of the major thrust areas recently in various pattern recognition, prediction, and analysis problems In many problems they

1.63k views • 119 slides

Neural Networks 1. Introduction Spring 2019 1 Neural Networks are taking over! Neural

Neural Networks 1. Introduction Spring 2019 1 Neural Networks are taking over! Neural networks have become one of the major thrust areas recently in various pattern recognition, prediction, and analysis problems In many problems they

1.57k views • 124 slides

Relaxation and Hopfield Networks Neural Networks Neural Networks - Hopfield 1 Bibliography

Relaxation and Hopfield Networks Neural Networks Neural Networks - Hopfield 1 Bibliography Hopfield, J. J., "Neural networks and physical systems with emergent collective computational abilities," Proceedings of the National Academy

367 views • 19 slides

Model Interpretation Danish Pruthi April 28, 2020 Why interpretability? Task:

CS 11-747 Neural Networks for NLP Model Interpretation Danish Pruthi April 28, 2020 Why interpretability? Task: predict probability of death for patients with pneumonia Why : so that high-risk patients can be admitted, low risk

826 views • 29 slides

Neural Networks and their Application to Go Neural Networks Learning Blackjack Theory Training

Neural Networks and their Application to Go A. Bausch Neural Networks and their Application to Go Neural Networks Learning Blackjack Theory Training neural networks Problems AlphaGo Anne-Marie Bausch The Game of Go Policy Network

280 views • 24 slides

(Very) Brief Introduction to Neural Networks IITP-03 Algorithms for NLP 1 / 31 Learning

(Very) Brief Introduction to Neural Networks IITP-03 Algorithms for NLP 1 / 31 Learning objectives What are neural networks? What are deep neural networks? How do we train neural networks? What variants of neural network

331 views • 31 slides

How Neural Networks (NN) Biological Neuron: A . . . Can (Hopefully) Learn Artificial Neural . .

Need for Machine . . . Neural Networks . . . How to Speed Up . . . Neural Networks: A . . . How Neural Networks (NN) Biological Neuron: A . . . Can (Hopefully) Learn Artificial Neural . . . Resulting Algorithm Faster by Taking Into How to

464 views • 17 slides

Outline Why model neural networks? Modeling Neural Networks A brief look at the neuron.

Outline Why model neural networks? Modeling Neural Networks A brief look at the neuron. A look at some current works. Adding an evolutionary strategy. Paul Nuytten CPSC 607 Why Model Neural Networks? The Neuron The nervous

697 views • 4 slides

Introduction to Deep Neural Networks 0. Logistics Spring 2020 1 Neural Networks are taking

Introduction to Deep Neural Networks 0. Logistics Spring 2020 1 Neural Networks are taking over! Neural networks have become one of the major thrust areas recently in various pattern recognition, prediction, and analysis problems In

987 views • 40 slides

Puzzling Neutron: A Window to Dark Matter? A Detective Story in three parts A Detective Story

Puzzling Neutron: A Window to Dark Matter? Puzzling Neutron: A Window to Dark Matter? A Detective Story in three parts A Detective Story in three parts Zurab Berezhiani Summary Preliminaries Chapter I: Into Zurab Berezhiani the

649 views • 51 slides

Analysis and Computation for Analysis and Computation for Nonlinear Eigenvalue Eigenvalue

Analysis and Computation for Analysis and Computation for Nonlinear Eigenvalue Eigenvalue Problems Problems Nonlinear Under Constraints Under Constraints Fong Yin Lim Fong Yin Lim Department of Mathematics Department of Mathematics and

587 views • 46 slides

Exploring Social Tagging Graph for Web Object Classification Zhijun Yin, Rui Li, Qiaozhu Mei,

Data e Web Mining AA 2009/2010 Exploring Social Tagging Graph for Web Object Classification Zhijun Yin, Rui Li, Qiaozhu Mei, Jiawei Han Giulia Mialich 825102 Luca Rossi 825038 Stats recently announced 300

733 views • 54 slides

Multichannel Variable-Size Convolution for Sentence Classification - WenPeng Yin - Hinrich

Multichannel Variable-Size Convolution for Sentence Classification - WenPeng Yin - Hinrich Schutze K.Vinay Sameer Raja IIT Kanpur INTRODUCTION Enhance word vector representations by combining various word embedding methods trained on

513 views • 13 slides

Numerical analysis and random matrix theory Tom Trogdon ttrogdon@math.uci.edu UC Irvine

Numerical analysis and random matrix theory Tom Trogdon ttrogdon@math.uci.edu UC Irvine Acknowledgements This is joint work with: Percy Deift Govind Menon Sheehan Olver Raj Rao Numerical analysis and random matrix theory

836 views • 40 slides

DS504/CS586: Big Data Analytics Data Pre-processing and Cleaning Prof. Yanhua Li Time: 6:00pm

Welcome to DS504/CS586: Big Data Analytics Data Pre-processing and Cleaning Prof. Yanhua Li Time: 6:00pm 8:50pm R Location: AK 233 Spring 2018 Merged CS586 and DS504 Graded one review Examples of Reviews/ Critiques Random selection.

405 views • 28 slides

YIN Moot Court: Blossom Online Dating Friday 20 th April 2017 09:00 to 10:30 Introduction Sanjay

YIN Moot Court: Blossom Online Dating Friday 20 th April 2017 09:00 to 10:30 Introduction Sanjay Iyer (Hong Kong/Singapore) Young IFA Global Committee Chair What is YIN? YIN = Young IFA Network Setup over 10 years ago to encourage the

873 views • 55 slides

An optimistic view on inflaton hunt Fumi Takahashi (Tohoku) Mar. 7 2019@Tohoku University Based

An optimistic view on inflaton hunt Fumi Takahashi (Tohoku) Mar. 7 2019@Tohoku University Based on Daido, FT, Yin, 1702.03284, 1710.11107, FT and Yin, 1903.00462 Revealing the history of the universe with underground particle and nuclear

988 views • 56 slides