Pavlovian, Skinner and other Intelligence. Desription Learning - PowerPoint PPT Presentation

Abstract Pavlovian, Skinner and other Intelligence. Desription Learning Behaviourism reign behaviourists contribution to AI Biological learning AI and animal learning Witold KOSI ´ Propositions for solution NSKI Dominika ZACZEK-CHRZANOWSKA Neural network . . . Polish–Japanese Institute of Information Technology, Research Center Conclusions Polsko–Japo´ nska Wy˙ zsza Szko� la Technik Komputerowych ul. Koszykowa 86 , 02-008 Warszawa Home Page wkos@pjwstk.edu.pl ˜ mado@pjwstk.edu.pl Title Page ◭◭ ◮◮ PJWSTK Avatar ◭ ◮ Page 1 of 12 Go Back Full Screen Close Quit

1. Abstract A version of the definition of intelligent behaviour will be supplied in the context of real and artificial systems. Short presentation of principles of learning, starting with Pavlo- Abstract Intelligence. Desription vian’s conditioned response (reflex) through reinforced re- Learning sponse and operant conditioning of Thorndike and Skinner Behaviourism reign Biological learning will be given. Some tools of artificial intelligence that act AI and animal learning according to those principle will be presented. An attempt Propositions for solution Neural network . . . will be made to show how some simple rules for behaviour Conclusions modifications can lead to a complex intelligent behaviour. Home Page Contents Title Page • Pavlov -a Nobel price winner Pavlov[1906] ◭◭ ◮◮ • Intelligence ◭ ◮ • Learning: visions of main behaviourists Page 2 of 12 • Biological learning –resume Go Back • AI and animal learning Full Screen • Critics and propositions for solution Close • Conclusions Quit

2. Intelligence. Desription Abstract Intelligence. Desription 1. Abilities to: Learning Behaviourism reign • reasoning Biological learning AI and animal learning • imagination Propositions for solution Neural network . . . • insight Conclusions • judgement ⇒ Binet test, IQ test Home Page 2. Three fundamental cognitive processes: Title Page • abstraction ◭◭ ◮◮ • learning ◭ ◮ Page 3 of 12 • dealing with novelty . Go Back 3. Ability to profit from experiments ⇒ ability to behave adaptively, to function successfully within particular envi- Full Screen ronments. Close Quit

3. Learning Abstract Intelligence. Desription Learning Adaptive changes of behaviour = learning Behaviourism reign Biological learning AI and animal learning Propositions for solution Behaviour is considered intelligent when it can be seen to Neural network . . . be adaptive. Conclusions Home Page Critics: there are many behavioural changes that one would Title Page like to call learning although they are not at all adaptive. ◭◭ ◮◮ ◭ ◮ We call behaviour intelligence only when we see how that behaviour is adaptive. Page 4 of 12 ⇒ intelligence is in the eyes of the observer [Brooks 1991] Go Back Principle of learning Full Screen Close Quit

4. Biological (animal) learning • Stimuli–response associations make the behaviour unsta- Abstract ble (depends on the place, initial position A 0 ). Intelligence. Desription Learning • Stimulus–approach associations make the behaviour sta- Behaviourism reign Biological learning ble (the goal stimulus S 0 can be approached from many AI and animal learning directions, places A i , i = 1 , 2 , ... ) Propositions for solution Neural network . . . • Place–approach associations make the behaviour stable Conclusions (more advanced learning: it requires the ability to use a Home Page configuration of stimuli to identify a place to approach). Title Page • Response chain makes the behaviour unstable (more than ◭◭ ◮◮ a simple S − R , single stimulus S 0 triggers a whole se- ◭ ◮ quence of responses: R 0 , R 1 , R 2 , ... ). Page 5 of 12 • Stimulus–approach chain makes the behaviour stable (se- quence of stimuli: S 0 , S 1 , S 2 , ... are approached in order). Go Back • Place–approach association can be also linked in chains Full Screen (the same stimuli ( landmarks ) can be used many times Close to locate different places). Quit

• S − R − S ′ associations, where S and S ′ are situations, Abstract one can make inference that if it is in S and performs re- Intelligence. Desription sponse R it will end up in S ′ (it is a form of expectation Learning Behaviourism reign learning and correspond to Tolman’s postulate about an- Biological learning imals’ learning). AI and animal learning Propositions for solution • S − R − S ∗ associations, where a stimulus S is followed Neural network . . . by a response R with a reinforcement, stimulus, S ∗ ( S ∗ Conclusions gets more intense as a goal is approched). Home Page • S − S ′ associations are examples of the classical con- Title Page ditioning of Pavlov’s dog ( in which the bell has been ◭◭ ◮◮ associated with food which in turn activates salivation) ◭ ◮ =stimulus–substitution theory of conditioning ⇒ Hebb’s Page 6 of 12 theory. Go Back Balkenius [1994] Skinner, 1999 , Sahrkey and Ziemke[1999] Full Screen Close Quit

5. AI and animal learning How to match relevant animal learning theories with AI re- Abstract search: Intelligence. Desription Learning • Rule based systems are very often similar to S-R associ- Behaviourism reign ations. Biological learning AI and animal learning Example: look-up tables (LUT) in both AI and control. Propositions for solution (Notice some inputs may be not stored) Neural network . . . Conclusions • Samuel’s checkers program , 1959, is similar to reinforce- ment learning described by a set of S − R − S ∗ associ- Home Page ations. Title Page ◭◭ ◮◮ • Most AI planning systems, based on acquisition of knowl- edge about the world, make use of representations similar ◭ ◮ to S − R − S ′ associations of the form: Page 7 of 12 precondition × action → outcome Go Back Full Screen where the outcome of one rule is the precondition for the next. Planning is a search for a sequence of rules that Close leads from the start to the goal. Quit

Recent critics: • Classical AI systems lack generalization capabilities: Abstract Intelligence. Desription complete systems cannot be made from studies of iso- Learning lated modulus. Behaviourism reign Biological learning • Classical AI systems lack robustness and cannot perform AI and animal learning Propositions for solution in real time, and run on sequential machines. Neural network . . . Conclusions • Classical AI systems are goal based and organized hier- archically; their processing is done centrally. Home Page Title Page • Real world differs from virtual ones: it has its own dy- namics. Virtual world has states with complete informa- ◭◭ ◮◮ tion on them, they are static. ◭ ◮ • The frame problem appears, i.e. how models of parts of Page 8 of 12 the real world can be kept in tune with the real world as Go Back it is changing, and how to determine which changes in the world are relevant to a given situation without having Full Screen to test all possible changes. Close Quit

6. Suggested solutions • Complete system (agent) must possess the architecture: Abstract 1. with direct coupling of perception to action, Intelligence. Desription Learning 2. with dynamic interaction with the environment, Behaviourism reign Biological learning 3. with intrinsic mechanisms to cope with resource lim- AI and animal learning itations and incomplete knowledge, Propositions for solution Neural network . . . 4. with decentralized processing, Conclusions • Complete system has to be the autonomous agent (self- Home Page sufficient agent, equipped with the appropriate learning Title Page mechanism, with its own history, adaptive), ◭◭ ◮◮ • Complete system has to be the situated agent (it ac- ◭ ◮ quires information about its environment only through its sensors and interacts with the world on its own), Page 9 of 12 Go Back • Complete system has to be embodied ( it must inter- act with its environment, is continuously subjected to Full Screen physical forces, to energy dissipation, to damage, to any Close influence in the environment). Quit

Abstract Intelligence. Desription Learning 7. Neural network architecture Behaviourism reign Biological learning embedded in robot AI and animal learning Propositions for solution Neural network . . . Robot Conclusions Home Page Example: Learning rule of Hebb to modify the weights w ij of an artificial neural network (ANN) Title Page ◭◭ ◮◮ w ij ( t + 1) = w ij ( t ) + ηo j a i ◭ ◮ where η is the so-called learning rate, o j the output of the Page 10 of 12 node (PE) j , and a i the activation of the node i . Go Back Full Screen Close Quit