The New Artificial Intelligence Keith L. Downing The Norwegian - - PowerPoint PPT Presentation

The New Artificial Intelligence

Keith L. Downing

The Norwegian University of Science and Technology (NTNU) Trondheim, Norway keithd@idi.ntnu.no

January 4, 2014

Keith L. Downing The New Artificial Intelligence

Overview

1

What is AI?

2

What is The New AI?

3

Emergence

4

Artificial Neural Networks and Evolutionary Computation (very briefly)

5

Evolving Neural Networks

6

Fun examples of evolving neural networks

Keith L. Downing The New Artificial Intelligence

Defining AI

Artificial Intelligence

Made by man rather than

• ccurring

in nature Ability to acquire and apply knowledge State or fact of knowing Possessing knowledge, intelligence

• r understanding

Knowing thoroughly

Keith L. Downing The New Artificial Intelligence

Practical Definintions of AI

Elaine Rich - author of one of the first popular AI textbooks AI is the study of how to make computer do things at which, at the moment, people are better. Jim Hendler - well-known AI researcher AI is what computers can’t do yet.

Unsolved AI Problems Recently Solved AI Problems Solved Computer-Science Problems

What has AI done for us lately?? All I see are unsolved problems.

Keith L. Downing The New Artificial Intelligence

A Brief History of Artificial Intelligence

Early (1955 - 1980) focus (and success) on tasks that humans find difficult: chess, geometry, physics... Later (1985- 2010) focus on easy human tasks, which are hard for computers. In the 1980’s, it became clear that computers lack common sense, and it’s not easy to give it to them in the same way that we give them high-level, expert knowledge of a specific domain. In the 1990’s, Situated and Embodied AI (SEAI) recognized as a promising low road to intelligence. Computers will only acquire common sense about the world by experiencing it and having to survive in it.

Keith L. Downing The New Artificial Intelligence

The Physical Symbol System Hypothesis

Newell and Simon, 1976 A physical symbol system (PSS) is a machine that produces a series of symbol structures over time. A PSS has the necessary and sufficient means for general intelligence action. Result of PSS Research LOTS of impressive AI reasoning systems, with many more to come. Erroneous view of the human mind as a PSS running atop computer-like hardware. PSSH is still important for AI engineering, but much less so for AI-inspired cognitive science.

Keith L. Downing The New Artificial Intelligence

Robot (Hans Moravec)

Comparative Evolution Living Organisms Computers Sense & Act 10,000,000 years 25 years Reason 100,000 years 40 years Calculate 1,000 years 60 years Key Implication You cannot produce general intelligence in a vacuum. If we are to create computers with human intelligence, then these systems should have a solid sensorimotor base upon which higher cognitive functioning can be built (or evolved).

Keith L. Downing The New Artificial Intelligence

Cognitive Incrementalism

Mindware (pg. 135), Andy Clark, 2001 This is the idea that you do indeed get full-blown human cognition by gradually adding bells and whistles to basic (embodied and embedded) strategies of relating to the present at hand. Cornerstone belief of The New AI. I am, therefore I think. The New AI ≈ Situated and Embodied AI (SEAI) ≈ Bio-Inspired AI (Bio-AI)

Keith L. Downing The New Artificial Intelligence

Artificial Life

Properties of ALife Systems: Synthetic: Bottom-up, multiple interacting agents. Self-Organizing: Global structure emerges from local interactions. Self-Regulating: Distributed (non-global) control (self-maintaining, autopoietic) Adaptive Learning and/or evolving. Complex: On the edge of chaos; dissipative.

Keith L. Downing The New Artificial Intelligence

Stigmergy: Emergent Structure from Indirect Signals.

Positive Feedback: Pheremone concentration in middle gets higher and higher as more dirt balls are added. Pheremones from the termites rub

• ff on the dirt balls.

Keith L. Downing The New Artificial Intelligence

Mistaken Genius

In emergent systems, intelligence is often in the eye of the

• bserver (who sees the global pattern), but not in the brain
• f the agent, which only understands local interactions.

Unfortunately, given a desired global pattern, it is very hard to reverse engineer the necessary set of local interactions. Evolution is very helpful here. Thus, the rules themselves emerge from an evolutionary process. SEAI emphasizes the evolutionary emergence of both agent structure (i.e., the body) and functionality (e.g. the neural network that controls the agent). Termite example from: Turtles, Termites and Traffic Jams: Explorations in Massively Parallel Microworlds, Resnick, 1994.

Keith L. Downing The New Artificial Intelligence

Herbert Simon’s Ant

Complexity is in the environment, not the ant’s neural controller. Key caveat: Simon includes the ant’s memory in the environment. The Sciences of the Artificial, Herbert Simon, 1996.

Keith L. Downing The New Artificial Intelligence

Neural Networks

Utility for SEAI Simple, homogeneous substrate Same, basic, neural signals carry information of perceptual, cognitive and motor nature. No need for special representations for each aspect of intelligence. Artificial neural networks (ANNs) are easy to implement (but hard to analyze). ANNs are easy to modify via learning. They are relatively unbiased, so many types of concepts can be learned, depending upon the learning context. ANNs are easy to evolve in an evolutionary algorithm (EA).

Keith L. Downing The New Artificial Intelligence

The ANN Abstraction

Human Brains 1011 neurons 1014 connections between them (a.k.a. synapses), many modifiable Complex physical and chemical activity to transmit ONE signal along ONE connection. Artificial Neural Networks (ANNs) N = 101 −104 nodes Max N2 connections All physics and chemistry represented by a few parameters associated with nodes and arcs.

Keith L. Downing The New Artificial Intelligence

Basic ANN Structural Abstraction

Soma

Axons

AP Soma Soma Soma Soma

Dendrites

AP

Synapses

Soma Node Node Node Node Node Node

w w w w w w w

Abstraction

Keith L. Downing The New Artificial Intelligence

Darwinian Evolution

3 Pillars Variation - Create phenotypic diversity upon which selection can work. Selection - Survival of the fittest. Inheritance - Children retain many of the parents’ phenotypic traits.

Keith L. Downing The New Artificial Intelligence

Evolutionary Algorithms

!"#$%& '()*+,

• "#$%&

.+/*"

• "#$%&
• 01%2"
• "#$%&
• 01%2"

!"#$%& 301)0")42 52(%1)"026% 7%8%*4$9%2" :)"2%&& ;%&")2< .+/*", =%*%6")42

• 01%2",

=%*%6")42 !"#$% '4$#)2< >%$14+/6")42? >%649@)20")42,A B/"0")42 .<%CD0&%+ :)*"%1)2< =%*%6")42 Keith L. Downing The New Artificial Intelligence

Training Artificial Neural Networks

Encoder Decoder E = r3 - r* r* r3 d3 Training/Test Cases: {(d1, r1) (d2, r2) (d3, r3)....} dE/dW Training Test Cases

Neural Net

N times, with learning 1 time, without learning

Keith L. Downing The New Artificial Intelligence

Backpropagation

Advantages

Powerful tool for learning complex input-output mappings in diverse problem domains. Relatively simple algorithm with solid mathematical foundation.

Drawbacks

Requires a known, correct output for each input → impractical for training autonomous systems. Requires many training rounds, often hundreds or thousands. Can easily get stuck in local error minima during gradient descent. Recurrent networks are a problem. Biologically unrealistic

Keith L. Downing The New Artificial Intelligence

Evolving Artificial Neural Networks (EANNs)

Encoder Decoder Test Cases

Neural Net

1 time, without learning Total Error Fitness

0 0 1 0 0 1 0 0 0 1 0 0 0 1 1 1 1 0 0 0 1 1 0 0 0 Genome (Direct Encoding)

Keith L. Downing The New Artificial Intelligence

Deep Biological Inspiration

Learning Devp Genotype Fitness Function Classification Entropy Effort Topological Degree Input Layer Kohonen Layer

Exploring interactions between evolution, development and learning in the emergence of intelligent neural networks.

Keith L. Downing The New Artificial Intelligence

Standard EANNs: Pros & Cons

Advantages

No training (learning) needed. Works with or without explicit test cases and explicit target outputs → useful in supervised and unsupervised learning scenarios. For fitness assessment, total error is easily replaced with other performance measures. Recurrent networks are no additional work. Better at avoiding local error minima due to parallel nature of evolutionary search.

Drawbacks

Requires a whole population of weight vectors. Scales poorly: large networks → large genotype weight vectors → large search space. General problem with direct-encoded EAs. No more biologically realistic than backpropagation, since animal genomes do not encode all synaptic strengths.

Keith L. Downing The New Artificial Intelligence

Swarm Intelligence

11 00 10 10 01

Genome Clone Simulate Annular Sorting Fitness

Vegard Hartmann (2005) & Andre Heie Vik (2005)

Keith L. Downing The New Artificial Intelligence

Karl Sims (1994)

(Loading Sims Creatures)

Keith L. Downing The New Artificial Intelligence