Cognitive Modeling Symbolic School Lecture 2: Approaches Symbolic - - PowerPoint PPT Presentation

Approaches to Cognitive Modeling Symbolic Models Connectionist Models Hybrid Models Cognitive Architectures

Cognitive Modeling

Lecture 2: Approaches Frank Keller

School of Informatics University of Edinburgh keller@inf.ed.ac.uk

January 31, 2005

Frank Keller Cognitive Modeling 1 Approaches to Cognitive Modeling Symbolic Models Connectionist Models Hybrid Models Cognitive Architectures

1

Approaches to Cognitive Modeling What makes a good model? Information Processing Connectionist School Symbolic School

2

Symbolic Models Symbolic Representations Production Systems

3

Connectionist Models Parallel Distributed Processing Feature Based Representations Learning, Generalization, Degradation

4

Hybrid Models

5

Cognitive Architectures Reading: Cooper (2002: Ch. 1)

Frank Keller Cognitive Modeling 2 Approaches to Cognitive Modeling Symbolic Models Connectionist Models Hybrid Models Cognitive Architectures What makes a good model? Information Processing Connectionist School Symbolic School

A Cognitive Model of a Task

Example: a teacher trying to diagnose the problems a student has with learning subtraction. Model may consist of a computer program that: takes some representation of the stimulus (the arithmetic test items) as input; produces a prediction of student s answer as output; perhaps also describes the difference between this model and that expected if the student were able to perform the task correctly.

Frank Keller Cognitive Modeling 3 Approaches to Cognitive Modeling Symbolic Models Connectionist Models Hybrid Models Cognitive Architectures What makes a good model? Information Processing Connectionist School Symbolic School

What makes a good model?

A good model has two critical properties:

1 it is complete – it does not abstract properties that are

important;

2 it is faithful – it does not introduce confounding details during

abstraction.

Frank Keller Cognitive Modeling 4

Approaches to Cognitive Modeling Symbolic Models Connectionist Models Hybrid Models Cognitive Architectures What makes a good model? Information Processing Connectionist School Symbolic School

Rise of Cognitive Modeling

Ebbinghaus: empirical/cognitive psychologist, learned lists of nonsense words to study memory (1885). Mid 20th century, it was demonstrated that for higher mental processes (e.g., language, Chomsky vs. Skinner debate): stimulus-response links cannot explain range of behavior; stimulus-response links mediated by internal mental states; internal mental states essential for causal explanations of cognitive processes. The mind as an information processor, cognition as information processing.

Frank Keller Cognitive Modeling 5 Approaches to Cognitive Modeling Symbolic Models Connectionist Models Hybrid Models Cognitive Architectures What makes a good model? Information Processing Connectionist School Symbolic School

Information Processing

Sensory processes act as input devices: information from the environment converted into internal representation. Mental processes manipulate and transform these representations, triggering responses via output processes. Major changes in second half of the 20th century:

1 computer simulation techniques used to explore theories of

cognitive processing and evaluate competing theories of empirical phenomena;

2 brain imaging techniques developed to localize cognitive

processing and relate functioning of the mind to that of the brain – now primary focus of Cognitive Neuroscience.

Frank Keller Cognitive Modeling 6 Approaches to Cognitive Modeling Symbolic Models Connectionist Models Hybrid Models Cognitive Architectures What makes a good model? Information Processing Connectionist School Symbolic School

Behavior, Process, and Theory

Classical relation between behavior, cognitive process underlying the behavior, and theory of the process:

Cognitive Process Behavior Generates Explains Describes Theory

Frank Keller Cognitive Modeling 7 Approaches to Cognitive Modeling Symbolic Models Connectionist Models Hybrid Models Cognitive Architectures What makes a good model? Information Processing Connectionist School Symbolic School

Modeling as an Added Dimension

Cognitive Process Model Behavior Generates Explains Describes Theory Generates Simulates Implements

Frank Keller Cognitive Modeling 8

Approaches to Cognitive Modeling Symbolic Models Connectionist Models Hybrid Models Cognitive Architectures What makes a good model? Information Processing Connectionist School Symbolic School

Modeling and Cognitive Neuropsychology

Cognitive Neuropsychology is concerned with different patterns of behavior following neurological damage, and using such patterns to inform normal cognitive functioning. A Cognitive Model of normal functioning can be damaged or lesioned in a principled way. We compare behavior of the damaged model with that of neurological patients, to account for both normal and impaired performance. Cognitive neuropsychology can also provide data against which models can be tested.

Frank Keller Cognitive Modeling 9 Approaches to Cognitive Modeling Symbolic Models Connectionist Models Hybrid Models Cognitive Architectures What makes a good model? Information Processing Connectionist School Symbolic School

Connectionist School

Two main schools of thought in cognitive modeling: connectionist and symbolic. The connectionist school: assumes that properties of the neural tissue that implement information processing in the mind are critical; builds models of many simple interacting units functioning in parallel; typically regards these models as analogues of neurons or neural cell assemblies.

Frank Keller Cognitive Modeling 10 Approaches to Cognitive Modeling Symbolic Models Connectionist Models Hybrid Models Cognitive Architectures What makes a good model? Information Processing Connectionist School Symbolic School

Symbolic School

The symbolic school: describes information processing in terms of the manipulation

• f symbol representations;

views the neural substrate as an implementation medium of secondary importance. Share: idea that the functioning of the mind is computational and can be simulated by machine. Differ in: their approaches; assumptions about mental representations; views on relation between a cognitive model and the brain.

Frank Keller Cognitive Modeling 11 Approaches to Cognitive Modeling Symbolic Models Connectionist Models Hybrid Models Cognitive Architectures What makes a good model? Information Processing Connectionist School Symbolic School

Other Approaches

Hybrid approaches: try to combine the strengths (and avoid the weaknesses) of both connectionist and symbolic approaches. Architectural approach: hypothesized organization of complete set of information processing structures that comprise the mind/brain; use this as a guide to developing models, e.g., ACT-R and Cogent.

Frank Keller Cognitive Modeling 12

Approaches to Cognitive Modeling Symbolic Models Connectionist Models Hybrid Models Cognitive Architectures Symbolic Representations Production Systems

Symbolic Propositional Representations

Conjunctions of propositions concerning objects, their properties, and relations between them. Example: the red pyramid is on the blue cube. pyramid(p) & red(p) & cube(c) & blue(c) & on(p, c) Propositions may be true or false, depending on the state of the

• bjects to which they apply.

Frank Keller Cognitive Modeling 13 Approaches to Cognitive Modeling Symbolic Models Connectionist Models Hybrid Models Cognitive Architectures Symbolic Representations Production Systems

Properties of Symbolic Representations

A representation is systematic if: it consists of a number of parts and replacing some part with other parts of the same kind is also a meaningful representation. A representation is compositional if: it consists of a number of parts and the meaning of the whole is a function of the meaning of the parts (Fodor and Pylyshyn 1988). Representations that are systematic and compositional may be manipulated by rules dependent only on the form of the representation, not its meaning.

Frank Keller Cognitive Modeling 14 Approaches to Cognitive Modeling Symbolic Models Connectionist Models Hybrid Models Cognitive Architectures Symbolic Representations Production Systems

Symbolic Representations

Provide a general means of representing information. Are supplemented by symbol manipulation rules that operate on the representations to transform them, or build new ones. Symbolic programming languages commonly used for developing cognitive models: Lisp (early 60s, John McCarthy & co. at MIT): based in part

• n work by Newell, Shaw and Simon at CMU;

Prolog (early 70s, Alain Colmeraur & co., France). Problem: general purpose programming languages too unconstrained for cognitive modeling. Alternative: production systems.

Frank Keller Cognitive Modeling 15 Approaches to Cognitive Modeling Symbolic Models Connectionist Models Hybrid Models Cognitive Architectures Symbolic Representations Production Systems

Production Systems

Provide a general, yet constrained, framework for symbolic models: Propositional store: propositions referred to as Working memory elements (WME) (analogue of working or short term memory). Rule database: inference rules (productions) correspond to long term (general and task specific) knowledge (analogue of long term memory). Recognize phase: inference rule selected; act phase: selected rule applied; conflict resolution procedure for selection between rules.

Frank Keller Cognitive Modeling 16

Approaches to Cognitive Modeling Symbolic Models Connectionist Models Hybrid Models Cognitive Architectures Parallel Distributed Processing Feature Based Representations Learning, Generalization, Degradation

Parallel Distributed Processing

Assumptions underlying connectionism (or neural networks, or parallel distributed processing): Brain consists of many billions of neurons. Each acts as simple computing device which receives electrical impulses from other neurons. If sum of impulses is sufficiently great, neuron generates its

• wn impulse and transmits it to other neurons.

Individual neurons operate in parallel; computation is distributed across many interconnected neurons. This approach requires an new type of representation: feature-based representations (instead of symbolic representations).

Frank Keller Cognitive Modeling 17 Approaches to Cognitive Modeling Symbolic Models Connectionist Models Hybrid Models Cognitive Architectures Parallel Distributed Processing Feature Based Representations Learning, Generalization, Degradation

Feature Based Representations

Featural representations of some animals: is mammal can fly has fur has long tail is vegetarian Person 1 Cat 1 1 1 Dog 1 1 1 Bat 1 1 Bird 1 1 Mouse 1 1 1 1

Frank Keller Cognitive Modeling 18 Approaches to Cognitive Modeling Symbolic Models Connectionist Models Hybrid Models Cognitive Architectures Parallel Distributed Processing Feature Based Representations Learning, Generalization, Degradation

Feature Based Representations

Different patterns of activation represent different animals, or categories, as a vector of features: person corresponds to vector (1, 0, 0, 0, 0); cat corresponds to vector (1, 0, 1, 1, 0) [cannot distinguish between dogs and cats]. Feature based representations can represent instances or classes of

• bjects, but lack the expressive power of symbolic propositional

representations [representation of relations through indirect means]. Connectionist models focus on aspect of cognition that do not require representation of relational information.

Frank Keller Cognitive Modeling 19 Approaches to Cognitive Modeling Symbolic Models Connectionist Models Hybrid Models Cognitive Architectures Parallel Distributed Processing Feature Based Representations Learning, Generalization, Degradation

Simple Model in Category Learning

A B large small black white If weights from large to A, and from small to B, are near 1 and

• ther weights near 0,

then presenting the feature vector (1, 0, 1, 0) [a large black object] would cause node A to become active.

Frank Keller Cognitive Modeling 20

Approaches to Cognitive Modeling Symbolic Models Connectionist Models Hybrid Models Cognitive Architectures Parallel Distributed Processing Feature Based Representations Learning, Generalization, Degradation

Learning, Generalization, Degradation

Learning: Initially set weights to random values: if given input fails to generate correct output, connection weights can be adjusted; results in network generating more accurate categorizations when input later repeated. Generalization: train subjects on exemplars: present new exemplars sharing some of features; generalize from previous categories new items [networks can exhibit performance similar to humans on this]. Degradation: networks can show effects of input degradation (suboptimal viewing, noise) similar to those in human subjects.

Frank Keller Cognitive Modeling 21 Approaches to Cognitive Modeling Symbolic Models Connectionist Models Hybrid Models Cognitive Architectures

Physically Hybrid Models

Physically hybrid models have separate symbolic and connectionist subsystems: symbolic models best in high level domains (e.g., reasoning, problem solving); connectionist models best for low level domains (e.g., perception). Decompose tasks into high and low level sub-processes. Example: Sun’s (1994) model of common-sense reasoning: symbolic for reasoning rules: all men are mortal; sub-symbolic for elements in rules: concept of Socrates and category of men.

Frank Keller Cognitive Modeling 22 Approaches to Cognitive Modeling Symbolic Models Connectionist Models Hybrid Models Cognitive Architectures

Non-physically Hybrid Systems

Non-physically hybrid models consists of a single system that functions in both symbolic and connectionist terms. Example: connectionist production system of Touretzky and Hinton (1988). Showed how connectionist system can be used to implement structures and processes of a typical symbolic production system: implements working memory, production memory, and symbolic rules containing variables.

Frank Keller Cognitive Modeling 23 Approaches to Cognitive Modeling Symbolic Models Connectionist Models Hybrid Models Cognitive Architectures

Cognitive Architectures

Cognitive architectures try to encompass the organization of a complete set of information processing structures that comprise the mind/brain. Model everything from low-level functions (e.g., perception) to high-level functions (e.g., student learning). Examples: ACT-R: lecture 3; Cogent: lectures 4–8.

Frank Keller Cognitive Modeling 24

Approaches to Cognitive Modeling Symbolic Models Connectionist Models Hybrid Models Cognitive Architectures

References

Cooper, Richard P. 2002. Modelling High-Level Cognitive Processes. Lawrence Erlbaum Associates, Mahwah, NJ. Fodor, Jerry A. and Zenon Pylyshyn. 1988. Connectionism and cognitive architecture: A critical analysis. Cognition 28(1):3–71. Sun, Ron. 1994. Integrating Rules and Connectionism for Robust Common Sense

• Reasoning. John Wiley and Sons, New York.

Touretzky, David S. and Geoffrey E. Hinton. 1988. A distributed connectionist production system. Cognitive Science 12(3):423–466.

Frank Keller Cognitive Modeling 25