SLIDE 1
Neural Networks
2/20
Introduction to Artificial Intelligence Neural Networks - Deep - - PowerPoint PPT Presentation
Introduction to Artificial Intelligence Neural Networks - Deep - - PowerPoint PPT Presentation
Introduction to Artificial Intelligence Neural Networks - Deep Learning for NLP Janyl Jumadinova November 21, 2016 Neural Networks 2/20 Neural Networks 3/20 Neural Networks Neural computing requires a number of neurons , to be connected
SLIDE 2
SLIDE 3
Neural Networks
3/20
SLIDE 4
Neural Networks
Neural computing requires a number of neurons, to be connected together into a neural network. Neurons are arranged in layers.
4/20
SLIDE 5
Activation Functions
◮ The activation function is generally non-linear. ◮ Linear functions are limited because the output is simply
proportional to the input.
5/20
SLIDE 6
Activation Functions
6/20
SLIDE 7
Network structures
Feed-forward networks:
◮ Single-layer perceptrons ◮ Multi-layer perceptrons 7/20
SLIDE 8
Feed-forward example
8/20
SLIDE 9
Single-layer Perceptrons
Output units all operate separately – no shared weights. Adjusting weights moves the location, orientation, and steepness of cliff.
9/20
SLIDE 10
Multi-layer Perceptrons
◮ Layers are usually fully connected. ◮ Numbers of hidden units typically chosen by hand. 10/20
SLIDE 11
A neural network for learning word vector
◮ Idea: A word and its context is a posiGve training sample ◮ A random word in that same context gives a negative training
sample:
11/20
SLIDE 12
A neural network for learning word vector
12/20
SLIDE 13
A neural network for learning word vector
These are the word features we want to learn .
13/20
SLIDE 14
A neural network for learning word vector
14/20
SLIDE 15
Deep Learning
◮ Most current machine learning works well because of
human-designed representations and input features .
◮ Machine learning becomes just optimizing weights to best make
a final prediction.
15/20
SLIDE 16
Deep Learning
◮ Most current machine learning works well because of
human-designed representations and input features .
◮ Machine learning becomes just optimizing weights to best make
a final prediction.
◮ Deep learning algorithms attempt to learn multiple levels of
representation of increasing complexity/abstraction.
15/20
SLIDE 17
A Deep Architecture
16/20
SLIDE 18
The Need for Distributed Representations
Current NLP systems are incredibly fragile because of their atomic symbol representations
17/20
SLIDE 19
Handling the recursivity of human language
18/20
SLIDE 20
Recursive Deep Learning: Building on Word Vector Space Models
19/20
SLIDE 21
How should we map phrases into a vector space?
20/20