SLIDE 1 Applying Category Theory to Improve the Performance of a Neural Architecture
Michael J. Healy Richard D. Olinger Robert J. Young Thomas P. Caudell University of New Mexico Kurt W. Larson Sandia National Laboratories
This work was supported in part by Sandia National Laboratories, Albuquerque, New Mexico, under contract no. 238984. Sandia is a multiprogram laboratory operated by Sandia Corporation, a Lockheed Martin Company, for the United States Department
- f Energy's National Nuclear Security Administration under
Contract DE-AC04-94AL85000.
SLIDE 2
P1 P2 P3 m T T’
Semantic Representation
Functor M Concept category Neural category Neural network M (m) M (T) M (T’)
SLIDE 3 Mod(m) P1 P2 P3 m T T’ Mod(T’) Mod(T) M(m) M(T) M(T’)
Model-Space Morphisms ==> Reciprocal Connections
Functor M Functor Mod
Instances
Instances
SLIDE 4 Colimits Express Specialization - Limits Express Abstraction
T1 T2 T3 T4 T5
Least specialization
T3 T1 T2 T5
Maximally specific abstraction
SLIDE 5
Classifying Pixels by Spectral Similarity
Multispectral camera data . . . Intensities for 10 optical bands Data for pixel i ( = one input pattern) Neural network classifier Pixel class (color) Colored pixel i Multispectral image
…
SLIDE 6
Stack Interval Network
− + StimVal − − − − − + + + − − − − + + StimLB0 StimUBΝ−1 Positive stack nodes Complement stack nodes 2 Ν−1 Ν−1 Ν + +
SLIDE 7
Stack Interval Patterns Represent Real Intervals
0 < v <= 1 Width 1 unit Positive stack Complement 0 < v <= 2 Width 2 units Intersection of stack patterns (in template patterns) 1 < v <= 2 Width 1 unit Positive stack Complement
SLIDE 8 ART-1 with Stack Interval Inputs
F1 F2 F0
. . .
GC − Band 1 Band 2 + + − V b1 b1
c
b2 b2
c
Template pattern Composite input pattern (two stack Intervals) +
SLIDE 9 ART-1 + F1 Colimits, Limits
F1 F2 F0 − + + + − V − + F+
1
. . . . . . . . .
S L F+
−1
SLIDE 10
Panchromatic Image - 1 m Resolution
SLIDE 11
Multispectral Image - Generic ART-1
ρ = 0.795 Template density ordering
SLIDE 12
Multispectral Image - ART-1 with Limits
ρ = 0.55 F-1 tol = 0.55 Template density ordering
SLIDE 13
SLIDE 14 References
- M. J. Healy, R. D. Olinger, R. J. Young, T. P. Caudell,
and K. W. Larson, “Applying Category Theory to Improve the Performance of a Neural Architecture” (under review).
- M. J. Healy and T. P. Caudell (2006) “Ontologies and Worlds
in Category Theory: Implications for Neural Systems”, Axiomathes, 16 (1), pp. 165-214.
- M. J. Healy and T, P. Caudell (2004) “Neural Networks,
Knowledge, and Cognition: A Mathematical Semantic Model Based upon Category Theory”, UNM Technical Report EECE-TR-04-020, University of New Mexico, Albuquerque, NM, USA .
SLIDE 15
Template Patterns
Band 1 Band 2 Template 1 Template 2
SLIDE 16
Stack Numeral Quanta
v = 0 3 < v <= 4 0 < v <= 1 0 <= v <= 1 0 < v <= 2 2 < v <= 4 Width 0 units Width 1 unit Width 1 unit Width 1 unit Width 2 units Width 2 units
. . . . . . . . . . . . . . . . . .
SLIDE 17 Neural Network Research Objective:
Associate an Evolving Knowledge Structure with Neural Structure and Activity
Concept hierarchy Environment Neural network Sensors and actuators Learning and representation Modality-specific input streams Event stream Motor functions
SLIDE 18
Limits Express Abstraction
T3 T1 T2 T5 … maximally specific abstraction
SLIDE 19 Colimits Express Specialization
T1 T2 T3 T4 T5
… least specialization
