SLIDE 1
Competitive Learning Neural Networks Neural Networks - Competitive - - PDF document
Competitive Learning Neural Networks Neural Networks - Competitive - - PDF document
Competitive Learning Neural Networks Neural Networks - Competitive 1 Bibliography Rumelhart, D. E. and McClelland, J. L., Parallel Distributed Processing , MIT Press, 1986. - Chapter 5, pp. 151-193. Kohonen, T., Self-Organization and
SLIDE 2
SLIDE 3
Spontaneous Learning Unsupervised Learning No Teacher The system must come up with a spontaneous but reasonable scheme of categorizing patterns Like-to-Like
Neural Networks - Competitive 3
SLIDE 4
Example ART II Classifications
Neural Networks - Competitive 4
SLIDE 5
Supervised and Unsupervised have very different goals Categorization vs Decision Systems Different Target Applications
Environment Motor Units Decision System Categorization System
Neural Networks - Competitive 5
SLIDE 6
Competitive Learning Most common scheme for spontaneous learning Relatively simple and intuitive Weight vectors a prototypes assume real weights
Z X1 X2 X3 Xn w2 w1 w3 wn Net
Net most active for pattern similar to weights
Neural Networks - Competitive 6
SLIDE 7
Standard Cluster Diagram Localist Model
Neural Networks - Competitive 7
SLIDE 8
2 prototype example (Lateral Inhibition, Winner take all)
1 2
X Y
1 2
Desired Goal
Neural Networks - Competitive 8
SLIDE 9
How do we reach it from an initial state
X Y
1 2 1 2
Neural Networks - Competitive 9
SLIDE 10
Simple Competitive Learning Algorithm Binary Inputs Top nodes winner take all Only winning unit has weights adjusted Each unit as fixed weight ∑1, weight is shifted during learning ∆wij = ⎩ ⎪ ⎨ ⎪ ⎧ 0 if unit j loses else g( si n - wij) where n is the number of active si Weight is shifted such that weight vector better matches the current winning input
Neural Networks - Competitive 10
SLIDE 11
Extended models Arbitrary inputs and weights can use a distance metric rather the net
Neural Networks - Competitive 11
SLIDE 12
Simple Unsupervised Learning Model With Distance Metric
Initialize n nodes in the attribute dimension space (could be very small n and be constructive) Until Convergence (∆d very small) Input new x Choose node i closest to x (Argmini (D(ni,x)) Optional: Add new node at x (how to decide?) Move ni slightly closer to x (∆di = di + cxi) (d is a node dimension and c is a learning rate) Optional: Prune nodes (how to decide?)
Neural Networks - Competitive 12
SLIDE 13
Dynamic Node Growth
Y X Y
1 2
What will happen here vigilance metric for node growth non-global vigilance noisy patterns
Neural Networks - Competitive 13
SLIDE 14
Supervised learning with competitive scheme Simply assign output value to each prototype Basically, multiple prototypes can have the same value
X Y
1 3 4 2
Neural Networks - Competitive 14
SLIDE 15
Multi-layer net using competitive learning
1 2 3 4 A B
Neural Networks - Competitive 15
SLIDE 16
RCE Learning (Restricted Coulomb Energy)
Neural Networks - Competitive 16
SLIDE 17
ART (Adaptive Resonance Theory) Spontaneous Competitive Learner Dynamic Node Growth Global Vigilance
Neural Networks - Competitive 17
SLIDE 18