4. Learning in MAS 4.1. What is learning? One definition: Bower, - PowerPoint PPT Presentation

4. Learning in MAS 4.1. What is learning? One definition: Bower, Hilgard: Theories of learning, Prentice-Hall, 1975: Learning refers to the change in a subject’s behavior to a given situation brought about by his repeated experiences in that situation, provided that the behavior change cannot be explained on the basis of native response tendencies, maturation, or temporary states of the subject (e.g. fatigue, drugs, etc.) Multi-Agent Systems Jörg Denzinger

What is learning (II) An AI definition: P. Langley: Elements of Machine Learning, Morgan Kaufmann, 1996: Learning is the improvement of performance in some environment through acquisition of knowledge resulting from experience in that environment. Or (my definition): Learning encompasses all self modifications of a (combined) system that allow an improved future system behavior. Multi-Agent Systems Jörg Denzinger

Consequences for our agent definition n For an agent Ag we can have f Ag (s) at time t 1 ≠ f Ag (s) at time t 2 n For a not-learning agent Ag , its learning variant Ag L and a sequence of situations s 1 ,…,s i ,..., we have there is an i such that Ag (s j ) = Ag L (s j ) for j < i and Ag (s i ) ≠ Ag L (s i ) n Predicting what a learning agent is doing is very difficult (see Asimovian Agents) Multi-Agent Systems Jörg Denzinger

A basic learning model Langley (1996) performance element knowledge environment base learner Multi-Agent Systems Jörg Denzinger

Agents and the basic learning model n Online Learning performance element knowledge L environment base learner n Offline Learning performance L element knowledge environment base learner Multi-Agent Systems Jörg Denzinger

Learning in AI: Characterizing Learning (I) n Learn activity and frequency: always learning vs learn phases The later allows for offline learning! n Feedback: l Unsupervised learning (self-organized) ab ! (ab)* abab ? Multi-Agent Systems Jörg Denzinger

Learning in AI: Characterizing Learning (II) n Feedback: l Supervised learning n Teacher specifies desired system behavior ababac ! ababac n Teacher specifies quality of the performed system behavior ababac 42 Multi-Agent Systems Jörg Denzinger

Learning in AI: Characterizing Learning (III) n Learn methodology l Learning by heart acbcac, acbcac, ... acbcac l Learning by instruction and advice ababa_ ? b Multi-Agent Systems Jörg Denzinger

Learning in AI: Characterizing Learning (IV) n Learn methodology l Learning from examples and by experiences Red! l Learning by using analogy ! ? Multi-Agent Systems Jörg Denzinger

Learning in AI: Characterizing Learning (IV) n Learn methodology l Learning by discovery F = G ⋅ m 1 ⋅ m 2 / r 2 Multi-Agent Systems Jörg Denzinger

Learning in AI: Characterizing Learning (V) n Most common forms of inferences Deduction: P(a); " x.(P(x) Æ Q(x)) fi Q(a) Induction: P(0); P(1) Ÿ ... Ÿ P(n) fi " x.P(x) Abduction: P(a); " x.(Q(x) Æ P(x)) fi Q(a) Multi-Agent Systems Jörg Denzinger

Learning in AI: Research Areas (I) n Neural Learning l Simple interacting processing units l Parallel, distributed, subsymbolic n Centralized Cognitive Learning l Single, complex systems l Knowledge-based, symbol oriented l Especially: n Rule-based Learning n Inductive Learning n Case-based Reasoning Multi-Agent Systems Jörg Denzinger

Learning in AI: Research Areas (II) n Distributed Cognitive Learning l Interacting complex systems l Knowledge-based, symbol oriented l See later n Evolutionary Learning l Populations of (simple or complex) systems l Numerical and using random influences l Genetic Algorithms, classifier systems, etc. Multi-Agent Systems Jörg Denzinger

Learning in AI: Basic Problems (I) n Similarity measures for l Analogical reasoning l Case-based reasoning l Situations l Examples n Generalizations for l Decision trees l Classification tasks Multi-Agent Systems Jörg Denzinger

Learning in AI: Basic Problems (II) n Credit assignment for feedback: l How good was an action? l What did an action in an action sequence contribute to the success of the sequence? l Which parts of Dat should be changed/ remain unchanged due to feedback? n Exploitation vs Exploration When do I exploit knowledge I have already learned and when should I try to make additional experiences by trying out new things? Multi-Agent Systems Jörg Denzinger