SLIDE 1
Machine Learning in Conceptual Spaces / Lucas Bechberger 2
Machine Learning in Conceptual Spaces Two Learning Processes Lucas - - PowerPoint PPT Presentation
Machine Learning in Conceptual Spaces Two Learning Processes Lucas - - PowerPoint PPT Presentation
Machine Learning in Conceptual Spaces Two Learning Processes Lucas Bechberger https://www.lucas-bechberger.de Conceptual Spaces x :apple ( x ) red ( x ) Symbolic Layer Formal Logics Geometric ? Conceptual Layer Representation
SLIDE 2
SLIDE 3
Machine Learning in Conceptual Spaces / Lucas Bechberger 3
My PhD Project / Outline
Symbolic Layer Subsymbolic Layer Conceptual Layer
Manually define regions Manually define dimensions
3.) Learning Concepts 2.) Learning Dimensions 1.) Mathematical Formalization
SLIDE 4
Machine Learning in Conceptual Spaces / Lucas Bechberger 4
My PhD Project / Outline
Symbolic Layer Subsymbolic Layer Conceptual Layer
Manually define regions Manually define dimensions
3.) Learning Concepts 2.) Learning Dimensions 1.) Mathematical Formalization
SLIDE 5
Machine Learning in Conceptual Spaces / Lucas Bechberger 5
Learning Dimensions
- There are (at least) three approaches:
- Handcrafting
- Multidimensional Scaling
- Artificial Neural Networks
- Bonus: A Hybrid Approach
SLIDE 6
Machine Learning in Conceptual Spaces / Lucas Bechberger 6
Learning Dimensions
- There are (at least) three approaches:
- Handcrafting
- Multidimensional Scaling
- Artificial Neural Networks
- Bonus: A Hybrid Approach
SLIDE 7
Machine Learning in Conceptual Spaces / Lucas Bechberger 7
Learning Dimensions: MDS
Psychological grounding Dealing with unseen inputs 1) Psychological experiment 2) Average across participants 3) Multidimensional Scaling
space # dimensions matrix similarity judgments
SLIDE 8
Machine Learning in Conceptual Spaces / Lucas Bechberger 8
Learning Dimensions
- There are (at least) three approaches:
- Handcrafting
- Multidimensional Scaling
- Artificial Neural Networks
- Bonus: A Hybrid Approach
SLIDE 9
Machine Learning in Conceptual Spaces / Lucas Bechberger 9
Learning Dimensions: ANNs
- Autoencoder (e.g., β-VAE): compress and reconstruct input
- Hidden neurons = dimensions in our conceptual space
- utput
hidden representation input
24 75 02 53 42 91 22 76 03 50
Higgins, I.; Matthey, L.; Pal, A.; Burgess, C.; Glorot, X.; Botvinick, M.; Mohamed, S. & Lerchner, A. β-VAE: Learning Basic Visual Concepts with a Constrained Variational Framework, ICLR 2017
Dealing with unseen inputs Psychological grounding
SLIDE 10
Machine Learning in Conceptual Spaces / Lucas Bechberger 10
Learning Dimensions: ANNs
- Centered, unrotated rectangles
- Differing only with respect to width and height
- Use InfoGAN to learn interpretable dimensions
Chen, X.; Duan, Y.; Houthooft, R.; Schulman, J.; Sutskever, I. & Abbeel, P. InfoGAN: Interpretable Representation Learning by Information Maximizing Generative Adversarial Nets Advances in Neural Information Processing Systems, 2016
SLIDE 11
Machine Learning in Conceptual Spaces / Lucas Bechberger 11
Learning Dimensions
- There are (at least) three approaches:
- Handcrafting
- Multidimensional Scaling
- Artificial Neural Networks
- Bonus: A Hybrid Approach
SLIDE 12
Machine Learning in Conceptual Spaces / Lucas Bechberger 12
Learning Dimensions: Hybrid
Psychological Experiment
MDS
ANN
dog cat . . .
Psychological grounding Dealing with unseen inputs
Bechberger, L. & Kypridemou, E. Mapping Images to Psychological Similarity Spaces Using Neural Networks. AIC 2018
SLIDE 13
Machine Learning in Conceptual Spaces / Lucas Bechberger 13
My PhD Project / Outline
Symbolic Layer Subsymbolic Layer Conceptual Layer
Manually define regions Manually define dimensions
3.) Learning Concepts 2.) Learning Dimensions 1.) Mathematical Formalization
SLIDE 14
Machine Learning in Conceptual Spaces / Lucas Bechberger 14
Learning Concepts
Machine Learning Engineer Cognitive Science Researcher
Give me a big data set of labeled examples! I’ll train a neural network for a bunch of epochs to find a nice decision boundary. It’s just a standard ML problem! Wait a second, that’s cognitively implausible! In real life, we have more unlabeled than labeled examples. Plus: Humans don’t learn via batch processing. That’s too complicated for now.
SLIDE 15
Machine Learning in Conceptual Spaces / Lucas Bechberger 15
Learning Concepts: LTN
- Fuzzy Logic
- Degree of membership between 0 and 1
- One can generalize logical operators:
- apple AND red = min(apple, red)
- We can express rules over these fuzzy sets
apple: 1.0 red: 0.9 round: 0.7 banana: 0.0
Symbolic Subsymbolic Conceptual
SLIDE 16
Machine Learning in Conceptual Spaces / Lucas Bechberger 16
Learning Concepts: LTN
- Use neural networks to
learn membership functions
- Constraints:
- Labels
- Rules
- Tune NN weights such that
all constraints are fulfilled
apple red sweet 0.99 0.75 0.31
Apple AND red IMPLIES sweet: 0.31
SLIDE 17
Machine Learning in Conceptual Spaces / Lucas Bechberger 17
Learning Concepts: LTN
- Conceptual space of movies from Derrac and Schockaert
- Extracted conceptual space from movie reviews
- 15.000 data points, labeled with one or more of 23 genres
- Use LTN to learn genres in that space
- Compare to kNN with respect to classification performance
- Compare to simple counting with respect to rule extraction
- Long run: align LTN with conceptual spaces theory
- Convexity, domain structure, ...
Joaquín Derrac and Steven Schockaert. Inducing semantic relations from conceptual spaces: a data-driven approach to commonsense reasoning, Artificial Intelligence, vol. 228, pages 66-94, 2015
SLIDE 18
Machine Learning in Conceptual Spaces / Lucas Bechberger 18
My PhD Project / Outline
Symbolic Layer Subsymbolic Layer Conceptual Layer
Manually define regions Manually define dimensions
3.) Learning Concepts 2.) Learning Dimensions 1.) Mathematical Formalization
SLIDE 19