The New Artificial Intelligence Keith L. Downing The Norwegian - PowerPoint PPT Presentation

The New Artificial Intelligence Keith L. Downing The Norwegian University of Science and Technology (NTNU) Trondheim, Norway keithd@idi.ntnu.no January 4, 2014 Keith L. Downing The New Artificial Intelligence

Overview What is AI? 1 What is The New AI? 2 Emergence 3 Artificial Neural Networks and Evolutionary Computation 4 (very briefly) Evolving Neural Networks 5 Fun examples of evolving neural networks 6 Keith L. Downing The New Artificial Intelligence

Defining AI Artificial Intelligence Made by man Ability to acquire rather than and apply occurring knowledge in nature Knowing State or thoroughly fact of knowing Possessing knowledge, intelligence or understanding Keith L. Downing The New Artificial Intelligence

Practical Definintions of AI Elaine Rich - author of one of the first popular AI textbooks AI is the study of how to make computer do things at which, at the moment, people are better. Jim Hendler - well-known AI researcher AI is what computers can’t do yet. What has AI done for us lately?? All I see are unsolved problems. Unsolved AI Recently Problems Solved AI Problems Solved Computer-Science Problems Keith L. Downing The New Artificial Intelligence

A Brief History of Artificial Intelligence Early (1955 - 1980) focus (and success) on tasks that humans find difficult: chess, geometry, physics... Later (1985- 2010) focus on easy human tasks, which are hard for computers. In the 1980’s, it became clear that computers lack common sense, and it’s not easy to give it to them in the same way that we give them high-level, expert knowledge of a specific domain. In the 1990’s, Situated and Embodied AI (SEAI) recognized as a promising low road to intelligence. Computers will only acquire common sense about the world by experiencing it and having to survive in it. Keith L. Downing The New Artificial Intelligence

The Physical Symbol System Hypothesis Newell and Simon, 1976 A physical symbol system (PSS) is a machine that produces a series of symbol structures over time. A PSS has the necessary and sufficient means for general intelligence action. Result of PSS Research LOTS of impressive AI reasoning systems, with many more to come. Erroneous view of the human mind as a PSS running atop computer-like hardware. PSSH is still important for AI engineering, but much less so for AI-inspired cognitive science. Keith L. Downing The New Artificial Intelligence

Robot (Hans Moravec) Comparative Evolution Living Organisms Computers Sense & Act 10,000,000 years 25 years Reason 100,000 years 40 years Calculate 1,000 years 60 years Key Implication You cannot produce general intelligence in a vacuum. If we are to create computers with human intelligence, then these systems should have a solid sensorimotor base upon which higher cognitive functioning can be built (or evolved). Keith L. Downing The New Artificial Intelligence

Cognitive Incrementalism Mindware (pg. 135), Andy Clark, 2001 This is the idea that you do indeed get full-blown human cognition by gradually adding bells and whistles to basic (embodied and embedded) strategies of relating to the present at hand. Cornerstone belief of The New AI. I am, therefore I think. The New AI ≈ Situated and Embodied AI (SEAI) ≈ Bio-Inspired AI (Bio-AI) Keith L. Downing The New Artificial Intelligence

Artificial Life Properties of ALife Systems: Synthetic: Bottom-up, multiple interacting agents. Self-Organizing: Global structure emerges from local interactions. Self-Regulating: Distributed (non-global) control (self-maintaining, autopoietic) Adaptive Learning and/or evolving. Complex: On the edge of chaos; dissipative. Keith L. Downing The New Artificial Intelligence

Stigmergy: Emergent Structure from Indirect Signals. Pheremones from the termites rub off on the dirt balls. Positive Feedback: Pheremone concentration in middle gets higher and higher as more dirt balls are added. Keith L. Downing The New Artificial Intelligence

Mistaken Genius In emergent systems, intelligence is often in the eye of the observer (who sees the global pattern), but not in the brain of the agent, which only understands local interactions. Unfortunately, given a desired global pattern, it is very hard to reverse engineer the necessary set of local interactions. Evolution is very helpful here. Thus, the rules themselves emerge from an evolutionary process. SEAI emphasizes the evolutionary emergence of both agent structure (i.e., the body) and functionality (e.g. the neural network that controls the agent). Termite example from: Turtles, Termites and Traffic Jams: Explorations in Massively Parallel Microworlds , Resnick, 1994. Keith L. Downing The New Artificial Intelligence

Herbert Simon’s Ant Complexity is in the environment, not the ant’s neural controller. Key caveat: Simon includes the ant’s memory in the environment . The Sciences of the Artificial , Herbert Simon, 1996. Keith L. Downing The New Artificial Intelligence

Neural Networks Utility for SEAI Simple, homogeneous substrate Same, basic, neural signals carry information of perceptual, cognitive and motor nature. No need for special representations for each aspect of intelligence. Artificial neural networks (ANNs) are easy to implement (but hard to analyze). ANNs are easy to modify via learning. They are relatively unbiased, so many types of concepts can be learned, depending upon the learning context. ANNs are easy to evolve in an evolutionary algorithm (EA). Keith L. Downing The New Artificial Intelligence

The ANN Abstraction Human Brains 10 11 neurons 10 14 connections between them (a.k.a. synapses), many modifiable Complex physical and chemical activity to transmit ONE signal along ONE connection. Artificial Neural Networks (ANNs) N = 10 1 − 10 4 nodes Max N 2 connections All physics and chemistry represented by a few parameters associated with nodes and arcs. Keith L. Downing The New Artificial Intelligence

Basic ANN Structural Abstraction Soma Soma Soma Dendrites Synapses Axons Soma AP AP Soma Abstraction Soma w w Node Node Node w w w Node w Node w Node Keith L. Downing The New Artificial Intelligence

Darwinian Evolution 3 Pillars Variation - Create phenotypic diversity upon which selection can work. Selection - Survival of the fittest. Inheritance - Children retain many of the parents’ phenotypic traits. Keith L. Downing The New Artificial Intelligence

Evolutionary Algorithms .<%CD0&%+ :)*"%1)2< =%*%6")42 :)"2%&& =%*%6")42 ;%&")2< .+/*", -01%2", '()*+, .+/*" -01%2" -"#$%& -"#$%& -"#$%& =%*%6")42 52(%1)"026% !"#$% 7%8%*4$9%2" '4$#)2< 301)0")42 >%$14+/6")42? -01%2" !"#$%& >%649@)20")42,A !"#$%& B/"0")42 Keith L. Downing The New Artificial Intelligence

Training Artificial Neural Networks Training/Test Cases: {(d1, r1) (d2, r2) (d3, r3)....} d3 r3 Encoder Decoder r* E = r3 - r* dE/dW Cases N times, with learning Training Neural Net Test 1 time, without learning Keith L. Downing The New Artificial Intelligence

Backpropagation Advantages Powerful tool for learning complex input-output mappings in diverse problem domains. Relatively simple algorithm with solid mathematical foundation. Drawbacks Requires a known, correct output for each input → impractical for training autonomous systems. Requires many training rounds, often hundreds or thousands. Can easily get stuck in local error minima during gradient descent. Recurrent networks are a problem. Biologically unrealistic Keith L. Downing The New Artificial Intelligence

Evolving Artificial Neural Networks (EANNs) Genome (Direct Encoding) 0 0 1 0 0 1 0 0 0 1 0 0 0 1 1 1 1 0 0 0 1 1 0 0 0 Encoder Decoder Cases Fitness Test Total Neural Error Net 1 time, without learning Keith L. Downing The New Artificial Intelligence

Deep Biological Inspiration Genotype Input Layer Devp Learning Kohonen Layer Effort Classification Topological Entropy Degree Fitness Function Exploring interactions between evolution, development and learning in the emergence of intelligent neural networks. Keith L. Downing The New Artificial Intelligence

Standard EANNs: Pros & Cons Advantages No training (learning) needed. Works with or without explicit test cases and explicit target outputs → useful in supervised and unsupervised learning scenarios. For fitness assessment, total error is easily replaced with other performance measures. Recurrent networks are no additional work. Better at avoiding local error minima due to parallel nature of evolutionary search. Drawbacks Requires a whole population of weight vectors. Scales poorly : large networks → large genotype weight vectors → large search space. General problem with direct-encoded EAs . No more biologically realistic than backpropagation, since animal genomes do not encode all synaptic strengths. Keith L. Downing The New Artificial Intelligence

Swarm Intelligence Genome 10 10 00 11 01 Fitness Annular Sorting Clone Simulate Vegard Hartmann (2005) & Andre Heie Vik (2005) Keith L. Downing The New Artificial Intelligence

Karl Sims (1994) (Loading Sims Creatures) Keith L. Downing The New Artificial Intelligence