Applying Category Theory to Improve the Performance of a Neural - PowerPoint PPT Presentation

Applying Category Theory to Improve the Performance of a Neural Architecture Michael J. Healy Richard D. Olinger Robert J. Young Thomas P. Caudell University of New Mexico Kurt W. Larson Sandia National Laboratories This work was supported in part by Sandia National Laboratories, Albuquerque, New Mexico, under contract no. 238984. Sandia is a multiprogram laboratory operated by Sandia Corporation, a Lockheed Martin Company, for the United States Department of Energy's National Nuclear Security Administration under Contract DE-AC04-94AL85000.

Semantic Representation T’ P 3 M (T’) Functor M P 2 m M (m) M (T) P 1 Concept Neural Neural T category category network

Model-Space Morphisms ==> Reciprocal Connections T’ Mod(T’) Instances P 3 of M(T’) Functor M P 2 m M(m) Mod(m) Functor Mod M(T) Instances of P 1 T Mod(T)

Colimits Express Specialization - Limits Express Abstraction Least specialization T 5 T 3 T 4 T 3 T 2 T 1 T 2 T 1 T 5 Maximally specific abstraction

Classifying Pixels by Spectral Similarity Neural network classifier Pixel class Intensities for (color) 10 optical bands … Colored pixel i . Data for pixel i . ( = one input pattern) . Multispectral Multispectral camera data image

Stack Interval Network Positive stack nodes Complement stack nodes − − − − − − − − − − StimLB 0 StimUB Ν−1 + + + 0 Ν−1 Ν 2 Ν−1 + + + + + StimVal

Stack Interval Patterns Represent Real Intervals Positive stack Complement Width 1 unit 0 < v <= 1 Positive stack Complement Width 1 unit 1 < v <= 2 Intersection of stack patterns (in template patterns) Width 2 units 0 < v <= 2

ART-1 with Stack Interval Inputs F 2 . . . Template pattern + − GC + F 1 V − Composite input pattern + (two stack Band 1 Band 2 Intervals) b 1 b 1 c b 2 b 2 c F 0

ART-1 + F 1 Colimits, Limits F 2 . . . + L − F + 1 V − + F 1 + S + − F + −1 . . . . . . F 0

Panchromatic Image - 1 m Resolution

Multispectral Image - Generic ART-1 ρ = 0.795 Template density ordering

Multispectral Image - ART-1 with Limits ρ = 0.55 F -1 tol = 0.55 Template density ordering

References M. J. Healy, R. D. Olinger, R. J. Young, T. P. Caudell, and K. W. Larson, “Applying Category Theory to Improve the Performance of a Neural Architecture” (under review). M. J. Healy and T. P. Caudell (2006) “Ontologies and Worlds in Category Theory: Implications for Neural Systems”, Axiomathes , 16 (1), pp. 165-214. M. J. Healy and T, P. Caudell (2004) “Neural Networks, Knowledge, and Cognition: A Mathematical Semantic Model Based upon Category Theory”, UNM Technical Report EECE-TR-04-020 , University of New Mexico, Albuquerque, NM, USA .

Template Patterns Template 1 Band Template 2 2 Band 1

Stack Numeral Quanta Width 0 v = 0 units Width 1 unit 0 < v <= 1 . . . . . . . . . Width 1 unit 3 < v <= 4 Width 1 unit 0 <= v <= 1 Width 2 units 0 < v <= 2 . . . . . . . . . Width 2 units 2 < v <= 4

Neural Network Research Objective: Associate an Evolving Knowledge Structure with Neural Structure and Activity Concept hierarchy Learning and representation Modality-specific input streams Neural network Motor functions Sensors and actuators Event stream Environment

Limits Express Abstraction T 3 T 1 T 2 T 5 … maximally specific abstraction

Colimits Express Specialization T 5 … least specialization T 4 T 3 T 2 T 1