Markov Logic Networks Matt Richardson and Pedro Domingos (2006), - - PDF document

▶
markov logic networks
SMART_READER_LITE
LIVE PREVIEW

Markov Logic Networks Matt Richardson and Pedro Domingos (2006), - - PDF document

Oct 22, 2023 180 likes •344 views

Markov Logic Networks Matt Richardson and Pedro Domingos (2006), Markov Logic Networks, Machine Learning , 62, 107-136, 2006. CS 786 University of Waterloo Lecture 23: July 19, 2012 Outline Markov Logic Networks Alchemy 2 CS786

slide-1
SLIDE 1

1

Markov Logic Networks

Matt Richardson and Pedro Domingos (2006), Markov Logic Networks, Machine Learning, 62, 107-136, 2006. CS 786 University of Waterloo Lecture 23: July 19, 2012

CS786 Lecture Slides (c) 2012 P. Poupart

2

Outline

  • Markov Logic Networks
  • Alchemy
slide-2
SLIDE 2

2

CS786 Lecture Slides (c) 2012 P. Poupart

3

Markov Logic Networks

  • Bayesian networks and Markov networks:

– Model uncertainty – But propositional representation (e.g., we need one variable per object in the world)

  • First-order logic:

– First-order representation (e.g., quantifiers allow us to reason about several objects simultaneously) – But we can’t deal with uncertainty

  • Markov logic networks: combine Markov

networks and first-order logic

CS786 Lecture Slides (c) 2012 P. Poupart

4

Markov Logic

  • A logical KB is a set of hard constraints
  • n the set of possible worlds
  • Let’s make them soft constraints:

when a world violates a formula, it becomes less probable, not impossible

  • Give each formula a weight:

(higher weight  stronger constraint) P(world)  e  weights of formulas it satisfies

slide-3
SLIDE 3

3

CS786 Lecture Slides (c) 2012 P. Poupart

5

Markov Logic: Definition

  • A Markov Logic Network (MLN) is a set of

pairs (F, w) where

– F is a formula in first-order logic – w is a real number

  • Together with a set of constants,

it defines a Markov network with

– One node for each grounding of each predicate in the MLN – One feature for each grounding of each formula F in the MLN, with the corresponding weight w

CS786 Lecture Slides (c) 2012 P. Poupart

6

Example: Friends & Smokers

habits. smoking similar have Friends cancer. causes Smoking

slide-4
SLIDE 4

4

CS786 Lecture Slides (c) 2012 P. Poupart

7

Example: Friends & Smokers

 

) ( ) ( ) , ( , ) ( ) ( y Smokes x Smokes y x Friends y x x Cancer x Smokes x     

CS786 Lecture Slides (c) 2012 P. Poupart

8

Example: Friends & Smokers

 

) ( ) ( ) , ( , ) ( ) ( y Smokes x Smokes y x Friends y x x Cancer x Smokes x      1 . 1 5 . 1

slide-5
SLIDE 5

5

CS786 Lecture Slides (c) 2012 P. Poupart

9

Example: Friends & Smokers

 

) ( ) ( ) , ( , ) ( ) ( y Smokes x Smokes y x Friends y x x Cancer x Smokes x      1 . 1 5 . 1 Two constants: Anna (A) and Bob (B)

CS786 Lecture Slides (c) 2012 P. Poupart

10

Example: Friends & Smokers

 

) ( ) ( ) , ( , ) ( ) ( y Smokes x Smokes y x Friends y x x Cancer x Smokes x      1 . 1 5 . 1

Cancer(A) Smokes(A) Smokes(B) Cancer(B)

Two constants: Anna (A) and Bob (B)

slide-6
SLIDE 6

6

CS786 Lecture Slides (c) 2012 P. Poupart

11

Example: Friends & Smokers

 

) ( ) ( ) , ( , ) ( ) ( y Smokes x Smokes y x Friends y x x Cancer x Smokes x      1 . 1 5 . 1

Cancer(A) Smokes(A) Friends(A,A) Friends(B,A) Smokes(B) Friends(A,B) Cancer(B) Friends(B,B)

Two constants: Anna (A) and Bob (B)

CS786 Lecture Slides (c) 2012 P. Poupart

12

Example: Friends & Smokers

 

) ( ) ( ) , ( , ) ( ) ( y Smokes x Smokes y x Friends y x x Cancer x Smokes x      1 . 1 5 . 1

Cancer(A) Smokes(A) Friends(A,A) Friends(B,A) Smokes(B) Friends(A,B) Cancer(B) Friends(B,B)

Two constants: Anna (A) and Bob (B)

slide-7
SLIDE 7

7

CS786 Lecture Slides (c) 2012 P. Poupart

13

Example: Friends & Smokers

 

) ( ) ( ) , ( , ) ( ) ( y Smokes x Smokes y x Friends y x x Cancer x Smokes x      1 . 1 5 . 1

Cancer(A) Smokes(A) Friends(A,A) Friends(B,A) Smokes(B) Friends(A,B) Cancer(B) Friends(B,B)

Two constants: Anna (A) and Bob (B)

CS786 Lecture Slides (c) 2012 P. Poupart

14

Markov Logic Networks

  • MLN is template for ground Markov nets
  • Probability of a world x:
  • Typed variables and constants greatly reduce

size of ground Markov net

Weight of formula i

  • No. of true groundings of formula i in x

      

i i i

x n w Z x P ) ( exp 1 ) (

slide-8
SLIDE 8

8

CS786 Lecture Slides (c) 2012 P. Poupart

15

Alchemy

  • Open Source AI package
  • http://alchemy.cs.washington.edu
  • Implementation of Markov logic networks
  • Problem specified in two files:

– File1.mln (Markov logic network) – File2.db (database / data set)

  • Learn weights and structure of MLN
  • Inference queries

CS786 Lecture Slides (c) 2012 P. Poupart

16

Markov Logic Encoding

  • File.mln
  • Two parts:

– Declaration

  • Domain of each variable
  • Predicates

– Formula

  • Pairs of weights with logical formula
slide-9
SLIDE 9

9

CS786 Lecture Slides (c) 2012 P. Poupart

17

Markov Logic Encoding

  • Example declaration

– Domain of each variable

  • person = {Anna, Bob}

– Predicates:

  • Friends(person,person)
  • Smokes(person)
  • Cancer(person)
  • Example formula

– 8 Smokes(x) => Cancer(x) – 5 Friends(x,y) => (Smokes(x)<=>Smokes(y))

NB: by default, formulas are universally quantified in Alchemy

CS786 Lecture Slides (c) 2012 P. Poupart

18

Dataset

  • File.db
  • List of facts (ground atoms)
  • Example:

– Friends(Anna,Bob) – Smokes(Anna) – Cancer(Bob)

slide-10
SLIDE 10

10

CS786 Lecture Slides (c) 2012 P. Poupart

19

Syntax

  • Logical connective:

– ! (not), ^ (and), v (or), => (implies), <=> (iff)

  • Quantifiers:

– forall (), exist () – By default unquantified variables are universally quantified in Alchemy

  • Operator precedence:

– ! > ^ > v > => > <=> > forall = exist

CS786 Lecture Slides (c) 2012 P. Poupart

20

Syntax

  • Short hand for predicates

– ! operator: indicates that the preceding variable has exactly one true grounding – Ex: HasPosition(x,y!): for each grounding of x, exactly one grounding of y satisfies HasPosition

  • Short hand for multiple weights

– + operator: indicates that a different weight should be learned for each grounding of the following variable – Ex: outcome(throw,+face): a different weight is learned for each grounding of face

slide-11
SLIDE 11

11

CS786 Lecture Slides (c) 2012 P. Poupart

21

Multinomial Distribution

Example: Throwing dice Types: throw = { 1, … , 20 } face = { 1, … , 6 } Predicate: Outcome(throw,face) Formulas: Outcome(t,f) ^ f!=f’ => !Outcome(t,f’). Exist f Outcome(t,f). Too cumbersome!

CS786 Lecture Slides (c) 2012 P. Poupart

22

Multinomial Distrib.: ! Notation

Example: Throwing dice Types: throw = { 1, … , 20 } face = { 1, … , 6 } Predicate: Outcome(throw,face!) Formulas: Semantics: Arguments without “!” determine args with “!”. Only one face possible for each throw.

slide-12
SLIDE 12

12

CS786 Lecture Slides (c) 2012 P. Poupart

23

Multinomial Distrib.: + Notation

Example: Throwing biased dice Types: throw = { 1, … , 20 } face = { 1, … , 6 } Predicate: Outcome(throw,face!) Formulas: Outcome(t,+f) Semantics: Learn weight for each grounding of args with “+”.

CS786 Lecture Slides (c) 2012 P. Poupart

24

Text Classification

page = { 1, … , n } word = { … } topic = { … } Topic(page,topic!) HasWord(page,word) Links(page,page) HasWord(p,+w) => Topic(p,+t) Topic(p,t) ^ Links(p,p') => Topic(p',t)

slide-13
SLIDE 13

13

CS486/686 Lecture Slides (c) 2012 P. Poupart

25

Information Retrieval

InQuery(word) HasWord(page,word) Relevant(page) Links(page,page) InQuery(+w) ^ HasWord(p,+w) => Relevant(p) Relevant(p) ^ Links(p,p’) => Relevant(p’)

  • Cf. L. Page, S. Brin, R. Motwani & T. Winograd, “The PageRank Citation

Ranking: Bringing Order to the Web,” Tech. Rept., Stanford University, 1998.

CS486/686 Lecture Slides (c) 2012 P. Poupart

26

Problem: Given database, find duplicate records

HasToken(token,field,record) SameField(field,record,record) SameRecord(record,record) HasToken(+t,+f,r) ^ HasToken(+t,+f,r’) => SameField(f,r,r’) SameField(+f,r,r’) => SameRecord(r,r’) SameRecord(r,r’) ^ SameRecord(r’,r”) => SameRecord(r,r”)

  • Cf. A. McCallum & B. Wellner, “Conditional Models of Identity Uncertainty

with Application to Noun Coreference,” in Adv. NIPS 17, 2005.

Record deduplication

slide-14
SLIDE 14

14

CS486/686 Lecture Slides (c) 2012 P. Poupart

27

Can also resolve fields: HasToken(token,field,record) SameField(field,record,record) SameRecord(record,record) HasToken(+t,+f,r) ^ HasToken(+t,+f,r’) => SameField(f,r,r’) SameField(+f,r,r’) <=> SameRecord(r,r’) SameRecord(r,r’) ^ SameRecord(r’,r”) => SameRecord(r,r”) SameField(f,r,r’) ^ SameField(f,r’,r”) => SameField(f,r,r”)

More: P. Singla & P. Domingos, “Entity Resolution with Markov Logic”, in Proc. ICDM-2006.

Record resolution

CS486/686 Lecture Slides (c) 2012 P. Poupart

28

Information Extraction

  • Problem: Extract database from text or

semi-structured sources

  • Example: Extract database of publications

from citation list(s) (the “CiteSeer problem”)

  • Two steps:

– Segmentation: Use HMM to assign tokens to fields – Record resolution: Use logistic regression and transitivity

slide-15
SLIDE 15

15

CS486/686 Lecture Slides (c) 2012 P. Poupart

29 Token(token, position, citation) InField(position, field, citation) SameField(field, citation, citation) SameCit(citation, citation) Token(+t,i,c) => InField(i,+f,c) InField(i,+f,c) <=> InField(i+1,+f,c) f != f’ => (!InField(i,+f,c) v !InField(i,+f’,c)) Token(+t,i,c) ^ InField(i,+f,c) ^ Token(+t,i’,c’) ^ InField(i’,+f,c’) => SameField(+f,c,c’) SameField(+f,c,c’) <=> SameCit(c,c’) SameField(f,c,c’) ^ SameField(f,c’,c”) => SameField(f,c,c”) SameCit(c,c’) ^ SameCit(c’,c”) => SameCit(c,c”)

Information Extraction

CS486/686 Lecture Slides (c) 2012 P. Poupart

30 Token(token, position, citation) InField(position, field, citation) SameField(field, citation, citation) SameCit(citation, citation) Token(+t,i,c) => InField(i,+f,c) InField(i,+f,c) ^ !Token(“.”,i,c) <=> InField(i+1,+f,c) f != f’ => (!InField(i,+f,c) v !InField(i,+f’,c)) Token(+t,i,c) ^ InField(i,+f,c) ^ Token(+t,i’,c’) ^ InField(i’,+f,c’) => SameField(+f,c,c’) SameField(+f,c,c’) <=> SameCit(c,c’) SameField(f,c,c’) ^ SameField(f,c’,c”) => SameField(f,c,c”) SameCit(c,c’) ^ SameCit(c’,c”) => SameCit(c,c”) More: H. Poon & P. Domingos, “Joint Inference in Information Extraction”, in Proc. AAAI-2007.

Information Extraction