04/12/2012 Outline Special Topics in AI: Intelligent Agents and Multi-Agent Systems • Mobile robots and uncertainty • Localization for mobile robots – State estimation based on Bayesian filters Probabilistic approaches to Robotics • Motion planning (State Estimation and Motion Planning) – Markov Decision Processes for path planning • Acknowledgment: material based on slides from Alessandro Farinelli – Russel and Norvig; Artificial Intelligence: a Modern Approach – Thrun, Burgard, Fox; Probabilistic Robotics Mobile robots Sensors 1
04/12/2012 Uncertainty Probability Probabilistic assertions summarize effects of open = open a door • laziness (enumeration of all relevant facts), Will open actually open the door ? • ignorance (lack of relevant facts) Problems: Subjective or Bayesian probability: • 1) partial observability and noisy sensors • Probabilities relate propositions to one's own state of • 2) uncertainty in action outcomes knowledge • 3) immense complexity of modelling and predicting – P(open|I am in front of the door) = 0.6 environment – P(open|I am in front of the door; door is not locked) = 0.8 Simple Example of State Estimation Causal vs. Diagnostic Reasoning Suppose a robot obtains measurement z P(open|z) is diagnostic What is P(open|z)? P(z|open) is causal Often causal knowledge is easier to obtain Bayes rule allows us to use causal knowledge: P ( z | open ) P ( open ) = P ( open | z ) P ( z ) count frequencies! 2
04/12/2012 Example Combining Evidence P(z|open) = 0.6 P(z| ¬ open) = 0.3 Suppose our robot obtains another observation z2. P(open) = P( ¬ open) = 0.5 How can we integrate this new information? P ( z | open ) P ( open ) = P ( open | z ) P ( z | open ) p ( open ) + P ( z | ¬ open ) p ( ¬ open ) More generally, how can we estimate P(x| z1...zn )? ⋅ 0 . 6 0 . 5 2 = = = P ( open | z ) 0 . 67 ⋅ + ⋅ 0 . 6 0 . 5 0 . 3 0 . 5 3 z raises the probability that the door is open. Recursive Bayesian Updating Example: Second Measurement � � P ( z | x , z , , z ) P ( x | z , , z ) n 1 n − 1 1 n − 1 � = P ( x | z , , z ) P(z2|open) = 0.5 P(z2| ¬ open) = 0.6 1 n � P ( z | z , , z ) − n 1 n 1 P(open|z1)=2/3 Markov assumption : z n independent of z 1 ,...,z n-1 if we know x P ( z | open ) P ( open | z ) 2 1 P ( open | z , z ) = 2 1 P ( z | open ) P ( open | z ) + P ( z | ¬ open ) P ( ¬ open | z ) 2 1 2 1 � P ( z | x ) P ( x | z , , z ) − n 1 n 1 � = P ( x | z , , z ) 1 2 1 n ⋅ � P ( z | z , , z ) 5 n 1 n − 1 2 3 = = = 0 . 625 1 2 3 1 8 � ⋅ + ⋅ = η P ( z | x ) P ( x | z , , z ) − n 1 n 1 2 3 5 3 ∏ = η P ( z | x ) P ( x ) i 1 ... n z 2 lowers the probability that the door is open. i = 1 ... n 3
04/12/2012 Actions Typical Actions Often the world is dynamic The robot moves – actions carried out by the robot, The robot moves objects – actions carried out by other agents, People move around the robot – time passing by Actions are never carried out with absolute certainty. How can we incorporate such actions? In contrast to measurements, actions generally increase the uncertainty. Modeling Actions Example: Closing the door To incorporate the outcome of an action u into the current “belief”, we use conditional pdf P(x|u,x’) This term specifies the pdf that executing u changes the state from x’ to x. 15 4
04/12/2012 State Transitions Integrating the Outcome of Actions • P(x|u,x’) for u = “close door”: Continuous case: � 0.9 = P ( x | u ) P ( x | u , x ' ) P ( x ' ) dx ' 0.1 open closed 1 Discrete case: 0 � = P ( x | u ) P ( x | u , x ' ) P ( x ' ) • If the door is open, the action “close door” succeeds in 90% of all cases. Example: The Resulting Belief Bayes Filters: Framework � = P ( closed | u ) P ( closed | u , x ' ) P ( x ' ) • Given: = P ( closed | u , open ) P ( open ) – Stream of observations z and action data u: + P ( closed | u , closed ) P ( closed ) = � d { u , z , u , z } t 1 1 t t 9 5 1 3 15 – Sensor model P(z|x) = ∗ + ∗ = 10 8 1 8 16 – Action model P(x|u,x’) � = P ( open | u ) P ( open | u , x ' ) P ( x ' ) – Prior probability of the system state P(x) • Compute: = P ( open | u , open ) P ( open ) – Estimate of the state X of a dynamical system + P ( open | u , closed ) P ( closed ) – The posterior of the state is also called Belief: 1 5 0 3 1 = ∗ + ∗ = = � Bel ( x ) P ( x | u , z , u , z ) 10 8 1 8 16 t t 1 1 t t = − 1 P ( closed | u ) 5
04/12/2012 Markov Assumption Bayes Filters z = observation u = action = � Bel ( x ) P ( x | u , z , u , z ) t t 1 1 t t x = state � � = η P ( z | x , u , z , , u ) P ( x | u , z , , u ) Bayes t t 1 1 t t 1 1 t � = η P ( z | x ) P ( x | u , z , , u ) Markov t t t 1 1 t � � = η P ( z | x ) P ( x | u , z , , u , x ) Total prob. t t t 1 1 t t − 1 = p ( z | x , z , u ) p ( z | x ) t 0 : t 1 : t 1 : t t t � P ( x | u , z , , u ) dx t − 1 1 1 t t − 1 = p ( x | x , z , u ) p ( x | x , u ) t 1 : t − 1 1 : t 1 : t t t − 1 t � = η � P ( z | x ) P ( x | u , x ) P ( x | u , z , , u ) dx Markov − − − Underlying Assumptions t t t t t 1 t 1 1 1 t t 1 � = η � • Static world (no one else changes the world) P ( z | x ) P ( x | u , x ) P ( x | u , z , , z ) dx Markov − − − − t t t t t 1 t 1 1 1 t 1 t 1 • Independent noise (over time) � = η ( | ) ( | , ) ( ) P z x P x u x Bel x dx t t t t t − 1 t − 1 t − 1 • Perfect model, no approximation errors Bayes Filter Algorithm Bayes Filters are Familiar! 1. Algorithm Bayes_filter ( Bel(x),d ): 2. η = 0 � = η Bel ( x ) P ( z | x ) P ( x | u , x ) Bel ( x ) dx 3. If d is a perceptual data item z then t t t t t t − 1 t − 1 t − 1 4. For all x do 5. = Bel ' ( x ) P ( z | x ) Bel ( x ) Kalman filters 6. η = η + Bel ' x ( ) 7. For all x do Particle filters 8. − 1 = η Bel ' ( x ) Bel ' ( x ) Hidden Markov models 9. Else if d is an action data item u then Dynamic Bayesian networks 10. For all x do � Partially Observable Markov Decision Processes 11. Bel ' ( x ) = P ( x | u , x ' ) Bel ( x ' ) dx ' (POMDPs) 12. Return Bel’(x) � = η Bel ( x ) P ( z | x ) P ( x | u , x ) Bel ( x ) dx − − − t t t t t t 1 t 1 t 1 6
04/12/2012 Bayesian filters for localization Kalman Filter for Localization How do I know whether I am in front of the door ? robot Localization as a state estimation process (filtering) landmark Gaussian pdf for belief • Pros: closed form representation, very fast update • Cons: Works only for linear action and sensor models (can use EKF to overcome this) State update Sensor Reading Works well only for unimodal beliefs Particle filters Particle Filters: prior Particles to represent the belief Pros: no assumption on belief, action and sensor models Cons: update can be computationally demanding Robot position (a) 7
04/12/2012 Particle Filters: bimodal belief Particle Filters: Unimodal beliefs Robot position Robot position (c) (b) Mapping and SLAM SLAM in action Localization: given map and observations, update pose Courtesy of Sebastian Thrun and estimation Dirk Haehnel ( link for the video) Mapping: given pose and observation, update map SLAM: given observations, update map and pose New observations increase uncertainty Loop closures reduce uncertainty 8
04/12/2012 Motion Planning for Mobile Robots Configuration Space Planning Convert free configuration space in finite state space Plan for motion in free configuration space (not workspace) workspace configuration space Cell decomposition Skeletonization (PRM) Planning the motion Markov Decision Process Given finite state space representing free configuration • Mathematical model to plan sequences of actions in space face of uncertainty Find a sequence of states from start to goal Several approaches: Rapidly-exploring Random Trees (RRT) Potential Fields Markov Decision Processes (i.e. building a navigation function) 9
04/12/2012 Example MDP Solving MDPs Risk and Reward Utility of State Sequences 10
04/12/2012 Utility of States Utilities contd. Dynamic Programming: The Value Iteration algorithm Bellman equation (4,3) 1 (3,3) 0.8 (1,1) Utility estimates (3,1) 0.6 0.4 (4,1) 0.2 0 -0.2 0 5 10 15 20 25 30 Number of iterations 11
04/12/2012 Policy Iteration MDP for robot navigation Partial Observability Solving POMDPs 12
Recommend
More recommend
