Course Overview Lecture 7 Introduction Uncertain knowledge and - PowerPoint PPT Presentation

Course Overview Lecture 7 ✔ Introduction Uncertain knowledge and Logical Agents Reasoning ✔ Artificial Intelligence Inference in First Order Logic ✔ Intelligent Agents Probability and Bayesian approach ✔ Search Bayesian Networks ✔ Uninformed Search Hidden Markov Chains Marco Chiarandini ✔ Heuristic Search Kalman Filters ✔ Adversarial Search Learning Deptartment of Mathematics & Computer Science ✔ Minimax search University of Southern Denmark Decision Trees ✔ Alpha-beta pruning Maximum Likelihood Knowledge representation and EM Algorithm Reasoning Learning Bayesian Networks Neural Networks ✔ Propositional logic Support vector machines ✔ First order logic Inference Slides by Stuart Russell and Peter Norvig 2 Summary Outline First-order logic: ♦ Reducing first-order inference to propositional inference – objects and relations are semantic primitives ♦ Unification – syntax: constants, functions, predicates, equality, quantifiers ♦ Generalized Modus Ponens Increased expressive power: sufficient to define wumpus world ♦ Forward and backward chaining Situation calculus: ♦ Logic programming – conventions for describing actions and change in FOL ♦ Resolution – can formulate planning as inference on a situation calculus KB 3 4

A brief history of reasoning Definitions For a predicate calculus expression X and an interpretation I : 450b.c. Stoics propositional logic, inference (maybe) If X has a value of T under I and a particular variable assignment, then 322b.c. Aristotle “syllogisms” (inference rules), quantifiers I is said to satisfy X . 1565 Cardano probability theory (propositional logic + uncertainty) 1847 Boole propositional logic (again) If I satisfies X for all variable assignments, then I is a model of X 1879 Frege first-order logic 1922 Wittgenstein proof by truth tables X is satisfiable if and only if there exist an interpretation and variable 1930 Gödel ∃ complete algorithm for FOL assignment that satisfy it; otherwise, it is unsatisfiable 1930 Herbrand complete algorithm for FOL (reduce to propositional) 1931 Gödel ¬∃ complete algorithm for arithmetic If a set of expressions is not satisfiable, it is said to be inconsistent 1960 Davis/Putnam “practical” algorithm for propositional logic 1965 Robinson “practical” algorithm for FOL—resolution If X has a value T for all possible interpretations, X is said to be valid. Eg.: ( p ( X ) ∧ ¬ p ( X )) while ∃ X ( P ( X ) ∨ ¬ p ( X )) 5 6 Rules of Inference for Propositions Definition A Proof Procedure is a combination of an inference rule and an algorithm for Rule of inference Name Rule of inference Name applying that rule to a set of logical expressions to generate new sentences. p Eg: Resolution inference rule. p → q Modus Ponens p Addition q p ∨ q ∴ ∴ ¬ q Definition p → q Modus tollens p ∨ q Simplification ¬ p p A predicate calculus expression X logically follows from a set S of predicate ∴ ∴ p → q p calculus expressions if every interpretation and variable assignment that q → r Hypothetical syllogism q Conjunction satisfies S also satisfies X . p → r p ∨ q ∴ ∴ An inference rule is sound if every predicate calculus expression produced by p ∨ q p ∨ q ¬ p Disjunctive syllogism ¬ p ∨ r Resolution the rule from a set S of predicate calculus expressions also logically follows q ∨ r q from S . ∴ ∴ An inference rule is complete if, given a set S of predicate calculus expressions, the rule can infer every expression that logically follows from S . 7 8

Rules of Inference for Quantified Statements Universal instantiation (UI) Every instantiation of a universally quantified sentence α is entailed by it: Rule of inference Name ∀ x P ( x ) Universal instantiation ∀ v α P ( c ) ∴ Subst ( { v/c } , α ) ∴ P ( c ) for an arbitrary c Universal generalization for any variable v and ground term c . (Note, here we used prolog notation.) ∀ xP ( x ) ∴ E.g., ∀ x King ( x ) ∧ Greedy ( x ) = ⇒ Evil ( x ) yields ∃ x P ( x ) Existential instantiation King ( John ) ∧ Greedy ( John ) = ⇒ Evil ( John ) P ( c ) for some element c ∴ King ( Richard ) ∧ Greedy ( Richard ) = ⇒ Evil ( Richard ) King ( Father ( John )) ∧ Greedy ( Father ( John )) = ⇒ Evil ( Father ( John )) P ( c ) for some element c Existential generalization . . ∃ xP ( x ) . ∴ 9 10 Existential instantiation (EI) Existential instantiation contd. For any sentence α , variable v , and constant symbol k that does not appear elsewhere in the knowledge base : ∃ v α UI can be applied several times to add new sentences; Subst ( { v/k } , α ) ∴ the new KB is logically equivalent to the old E.g., ∃ x Crown ( x ) ∧ OnHead ( x, John ) yields EI can be applied once to replace the existential sentence; the new KB is not equivalent to the old, Crown ( C 1 ) ∧ OnHead ( C 1 , John ) but is satisfiable iff the old KB was satisfiable provided C 1 is a new constant symbol, called a Skolem constant Another example: from ∃ x d ( x y ) /dy = x y we obtain d ( e y ) /dy = e y provided e is a new constant symbol 11 12

Reduction to propositional inference Reduction to propositional inference (contd.) Suppose the KB contains just the following: Claim: a ground sentence is entailed by new KB iff entailed by original KB ∀ x King ( x ) ∧ Greedy ( x ) = ⇒ Evil ( x ) King ( John ) Claim: every FOL KB can be propositionalized so as to preserve Greedy ( John ) entailment Brother ( Richard, John ) Idea: propositionalize KB and query, apply resolution, return result Problem: with variables and function symbols, there are infinitely many Instantiating the universal sentence in all possible ways, we have ground terms, King ( John ) ∧ Greedy ( John ) = ⇒ Evil ( John ) e.g., Father ( Father ( Father ( John ))) King ( Richard ) ∧ Greedy ( Richard ) = ⇒ Evil ( Richard ) Theorem: Herbrand (1930). If a sentence α is entailed by an FOL KB, King ( John ) it is entailed by a finite subset of the propositional KB Greedy ( John ) Idea: For n = 0 to ∞ do Brother ( Richard, John ) create a propositional KB by instantiating with depth- n terms The new KB is propositionalized: proposition symbols are see if α is entailed by this KB Problem: works if α is entailed, loops if α is not entailed King ( John ) , Greedy ( John ) , Evil ( John ) , King ( Richard ) , etc. Theorem: Turing (1936), Church (1936), entailment in FOL is and can therefore be solved by the methods seen with propositional logic semidecidable 13 14 Problems with propositionalization Unification We can get the inference immediately if we can find a substitution σ such that King ( x ) and Greedy ( x ) match King ( John ) and Greedy ( y ) Propositionalization seems to generate lots of irrelevant sentences. E.g., from σ = { x/John, y/John } works ∀ x King ( x ) ∧ Greedy ( x ) = ⇒ Evil ( x ) Unify ( α, β ) = σ if ασ = βσ King ( John ) ∀ y Greedy ( y ) p q σ Brother ( Richard, John ) Knows ( John, x ) Knows ( John, Jane ) { x/Jane } Knows ( John, x ) Knows ( y, OJ ) { x/OJ, y/John } it seems obvious that Evil ( John ) , but propositionalization produces lots of Knows ( John, x ) Knows ( y, Mother ( y )) { y/John, x/Mother ( John ) } facts such as Greedy ( Richard ) that are irrelevant Knows ( John, x ) Knows ( x, OJ ) fail With p k -ary predicates and n constants, there are p · n k instantiations With function symbols, it gets much much worse! Standardizing apart: rename variables to eliminate name overlap, e.g., Knows ( z, OJ ) 15 16

Generalized Modus Ponens (GMP) Soundness of GMP Any inference in FOL has to use unification Here is an inference rule with the use of unification Need to show that p 1 ′ , p 2 ′ , . . . , p n ′ ( p 1 ∧ p 2 ∧ . . . ∧ p n ⇒ q ) where p i ′ σ = p i σ for all i p 1 ′ , . . . , p n ′ , ( p 1 ∧ . . . ∧ p n ⇒ q ) | = qσ qσ ∴ provided that p i ′ σ = p i σ for all i p 1 ′ is King ( John ) Lemma: For any definite clause p , we have p | = pσ by UI p 1 is King ( x ) p 2 ′ is Greedy ( y ) 1. ( p 1 ∧ . . . ∧ p n ⇒ q ) | = ( p 1 ∧ . . . ∧ p n ⇒ q ) σ = ( p 1 σ ∧ . . . ∧ p n σ ⇒ qσ ) p 2 is Greedy ( x ) 2. p 1 ′ , . . . , p n ′ | = p 1 ′ ∧ . . . ∧ p n ′ | = p 1 ′ σ ∧ . . . ∧ p n ′ σ σ is { x/John, y/John } q is Evil ( x ) 3. From 1 and 2, qσ follows by ordinary Modus Ponens qσ is Evil ( John ) GMP used with KB of definite clauses ( exactly one positive literal) All variables assumed universally quantified 17 18 Unification Unification Unification: search substitution that match two expressions constants (ground instances) cannot be substituted Unifiers must be as general as possible otherwise eliminate possibility for future solutions: only variables can be substituted Eg: p ( X ) , p ( Y ) and { X/fred, Y/fred } cannot substitute x by p ( x ) � creates infinite regression Definition occur check If µ is any unifier of expressions E and σ is a most general unifier then for µ applied to E there exists µ ′ such that Eσ = Eσµ ′ where Eµ and Eσµ ′ is a variable can be substituted with another variable the composition of unifiers. future substitutions must be consistent (substitution sequence) mgu is unique (except for relabelling) Composition of substitutions: { Y/X, Z/W } ; { X/V } ; { V/a, W/f ( b )) } 19 20