A Short Introduction to Probabilistic Soft Logic Angelika Kimmig, - - PowerPoint PPT Presentation
A Short Introduction to Probabilistic Soft Logic Angelika Kimmig, Stephen H. Bach, Matthias Broecheler, Bert Huang and Lise Getoor NIPS Workshop on Probabilistic Programming 2012 http://psl.umiacs.umd.edu 1 Probabilistic Soft Logic (PSL)
Angelika Kimmig, Stephen H. Bach, Matthias Broecheler, Bert Huang and Lise Getoor NIPS Workshop on Probabilistic Programming 2012 http://psl.umiacs.umd.edu
1
2
[Broecheler et al, UAI 10]
Declarative language to specify graphical models
2
[Broecheler et al, UAI 10]
Declarative language to specify graphical models
2
[Broecheler et al, UAI 10]
Declarative language to specify graphical models
2
[Broecheler et al, UAI 10]
Declarative language to specify graphical models
2
[Broecheler et al, UAI 10]
Declarative language to specify graphical models
2
[Broecheler et al, UAI 10]
Declarative language to specify graphical models
Efficient MPE inference: continuous convex optimization
2
new approach NIPS 12
[Broecheler et al, UAI 10]
3
4 1
Organization Employees Developer Staff Customers Service & Products Hardware IT Services Sales Person Software
work for provides buys helps interacts sells to developsCompany Employee Technician Accountant Customer Products & Services Hardware Consulting Sales Software Dev
works for develop buys helps interacts with sells 4 1
Organization Employees Developer Staff Customers Service & Products Hardware IT Services Sales Person Software
work for provides buys helps interacts sells to developsCompany Employee Technician Accountant Customer Products & Services Hardware Consulting Sales Software Dev
works for develop buys helps interacts with sellssimilar names
4 1
Organization Employees Developer Staff Customers Service & Products Hardware IT Services Sales Person Software
work for provides buys helps interacts sells to developsCompany Employee Technician Accountant Customer Products & Services Hardware Consulting Sales Software Dev
works for develop buys helps interacts with sellssimilar names similar ranges
4 1
Organization Employees Developer Staff Customers Service & Products Hardware IT Services Sales Person Software
work for provides buys helps interacts sells to developsCompany Employee Technician Accountant Customer Products & Services Hardware Consulting Sales Software Dev
works for develop buys helps interacts with sellssimilar names similar ranges similar subconcepts
5
ann bob dan carla emma fred
5
0.9 0.7 0.8 0.5 0.2 0.8 0.9 0.5 0.4
5
ann bob dan carla emma fred
5
0.9 0.7 0.8 0.5 0.2 0.8 0.9 0.5 0.4
trusts(X,Y) ∧ trusts(Y,Z) → trusts(X,Z)
5
ann bob dan carla emma fred
5
0.9 0.7 0.8 0.5 0.2 0.8 0.9 0.5 0.4
trusts(X,Y) ∧ trusts(Y,Z) → trusts(X,Z)
✔
5
ann bob dan carla emma fred
5
0.9 0.7 0.8 0.5 0.2 0.8 0.9 0.5 0.4
trusts(X,Y) ∧ trusts(Y,Z) → trusts(X,Z)
✗
5
spouse spouse colleague colleague spouse friend friend friend friend ann bob dan carla emma fred
6
5
spouse spouse colleague colleague spouse friend friend friend friend ann bob dan carla emma fred
6
5
spouse spouse colleague colleague spouse friend friend friend friend ann bob dan carla emma fred
6
? ? ?
friend(carla,emma)=0.9 friend(bob,dan)=0.4 spouse(ann,bob)=1.0 prefers(ann, )=0.8 ...
7
c
b spouse
spouse colleague colleague spouse friend friend
friend
friend
?
a
d
e
f
? ?
0.3: lives(A,S) ∧ majority(S,P) → prefers(A,P) 0.8: spouse(B,A) ∧ prefers(B,P) → prefers(A,P) 0.1: similarAge(B,A) ∧ prefers(B,P) → prefers(A,P) 0.4: prefers(A,P) → prefersAvg({A.friend},P) partial-functional: prefers
friend(carla,emma)=0.9 friend(bob,dan)=0.4 spouse(ann,bob)=1.0 prefers(ann, )=0.8 ...
7
c
b spouse
spouse colleague colleague spouse friend friend
friend
friend
?
a
d
e
f
? ?
partial-functional: prefers
8
c
b spouse
spouse colleague colleague spouse friend friend
friend
friend
?
a
d
e
f
? ?
partial-functional: prefers
prefers(A, )+prefers(A, ) ≤ 1.0
8
c
b spouse
spouse colleague colleague spouse friend friend
friend
friend
?
a
d
e
f
? ?
0.3: lives(A,S) ∧ majority(S,P) → prefers(A,P)
9
c
b spouse
spouse colleague colleague spouse friend friend
friend
friend
?
a
d
e
f
? ?
0.8: spouse(B,A) ∧ prefers(B,P) → prefers(A,P)
10
c
b spouse
spouse colleague colleague spouse friend friend
friend
friend
?
a
d
e
f
? ?
0.1: similarAge(B,A) ∧ prefers(B,P) → prefers(A,P)
11
c
b spouse
spouse colleague colleague spouse friend friend
friend
friend
?
a
d
e
f
? ?
Similarity function with range [0,1]
0.4: prefers(A,P) → prefersAvg({A.friend},P)
12
c
b spouse
spouse colleague colleague spouse friend friend
friend
friend
?
a
d
e
f
? ?
0.4: prefers(A,P) → prefersAvg({A.friend},P)
12
c
b spouse
spouse colleague colleague spouse friend friend
friend
friend
?
a
d
e
f
? ?
all X such that friend(A,X)
0.4: prefers(A,P) → prefersAvg({A.friend},P)
12
c
b spouse
spouse colleague colleague spouse friend friend
friend
friend
?
a
d
e
f
? ?
all X such that friend(A,X) truth value ≔ average truth value of prefers(X,P)
13
14
f(I) = 1
Z exp
@− X
r∈P
X
g∈G(r)
wr(dg(I))k 1 A
Interpretation
14
f(I) = 1
Z exp
@− X
r∈P
X
g∈G(r)
wr(dg(I))k 1 A
Set of rule groundings Interpretation
14
f(I) = 1
Z exp
@− X
r∈P
X
g∈G(r)
wr(dg(I))k 1 A
Set of rule groundings Rule’s weight Interpretation
14
f(I) = 1
Z exp
@− X
r∈P
X
g∈G(r)
wr(dg(I))k 1 A
Set of rule groundings Rule’s weight Interpretation Ground rule’s distance from satisfaction given I
14
f(I) = 1
Z exp
@− X
r∈P
X
g∈G(r)
wr(dg(I))k 1 A
Set of rule groundings Rule’s weight Interpretation Ground rule’s distance from satisfaction given I ∈{1,2}
14
f(I) = 1
Z exp
@− X
r∈P
X
g∈G(r)
wr(dg(I))k 1 A
Set of rule groundings Rule’s weight Interpretation Ground rule’s distance from satisfaction given I ∈{1,2}
14
Normalization constant
Z = Z
J∈I
exp @− X
r∈P
X
g∈G(r)
wr(dg(J))k 1 A
f(I) = 1
Z exp
@− X
r∈P
X
g∈G(r)
wr(dg(I))k 1 A
15
dr(I) = max{0, I(body) − I(head)}
body → head satisfied ⇔ truth value of body ≤ truth value of head
15
dr(I) = max{0, I(body) − I(head)}
body → head satisfied ⇔ truth value of body ≤ truth value of head
I(v1 ∧ v2) = max{0, I(v1) + I(v2) − 1} I(v1 ∨ v2) = min{I(v1) + I(v2), 1} I(¬l1) = 1 − I(v1)
Lukasiewicz t-norm
15
dr(I) = max{0, I(body) − I(head)}
similarAge(bob,ann) ∧ prefers(bob, ) → prefers(ann, )
16
0.8 0.5 0.5
similarAge(bob,ann) ∧ prefers(bob, ) → prefers(ann, )
16
I(v1 ∧ v2) = max{0, I(v1) + I(v2) − 1} dr(I) = max{0, I(body) − I(head)}
0.8 0.5 0.5
similarAge(bob,ann) ∧ prefers(bob, ) → prefers(ann, )
16
max{0, 0.8+0.5−1} ≤ 0.5 d=0.0
I(v1 ∧ v2) = max{0, I(v1) + I(v2) − 1} dr(I) = max{0, I(body) − I(head)}
0.8 0.5 0.5
similarAge(bob,ann) ∧ prefers(bob, ) → prefers(ann, )
16
max{0, 0.8+0.5−1} ≤ 0.5 max{0, 0.8+0.5−1} ≤ 0.2 d=0.0 d=0.1
I(v1 ∧ v2) = max{0, I(v1) + I(v2) − 1} dr(I) = max{0, I(body) − I(head)}
0.8 0.5 0.2
similarAge(bob,ann) ∧ prefers(bob, ) → prefers(ann, )
16
max{0, 0.8+0.5−1} ≤ 0.5 max{0, 0.8+0.5−1} ≤ 0.2 max{0, 0.8+0.9−1} ≤ 0.2 d=0.0 d=0.1 d=0.5
I(v1 ∧ v2) = max{0, I(v1) + I(v2) − 1} dr(I) = max{0, I(body) − I(head)}
0.8 0.2 0.9
17
prefers(bob, ) ≥ prefers(bob, ) ?
17
prefers(bob, ) ≥ prefers(bob, ) ?
P(prefers(bob, ) ≥ 0.8) ?
17
prefers(bob, ) ≥ prefers(bob, ) ?
P(prefers(bob, ) ≥ 0.8) ?
17
1 1 x1 x3 x2 1 most likely interpretations given x1=0
18
d1(I) = max{0, I(x1) − I(x2)} d2(I) = max{0, I(x2) − I(x3)} I(x1) + I(x3) ≤ 1
for subproblems
19
20
r1(V') r2(V'') r3(V''') v1 v2 v3 v4
[Bach et al, NIPS 12]
20
r1(V') r2(V'') r3(V''') v1 v2 v3 v4
[Bach et al, NIPS 12]
20
r1(V') r2(V'') r3(V''') v1 v2 v3 v4
rules with local copies of random variables
[Bach et al, NIPS 12]
20
r1(V') r2(V'') r3(V''') v1 v2 v3 v4
values & agreement with
per rule rules with local copies of random variables
[Bach et al, NIPS 12]
20
r1(V') r2(V'') r3(V''') v1 v2 v3 v4
values & agreement with
per rule update variables to average
rules with local copies of random variables
[Bach et al, NIPS 12]
20
r1(V') r2(V'') r3(V''') v1 v2 v3 v4
values & agreement with
per rule update variables to average
rules with local copies of random variables
[Bach et al, NIPS 12]
20
r1(V') r2(V'') r3(V''') v1 v2 v3 v4
values & agreement with
per rule update variables to average
rules with local copies of random variables
[Bach et al, NIPS 12]
new: fast solutions
1 1 x1 x3 x2 1
| | 1 f
P(0.4 ≤ x2 ≤ 0.6)
21
d1(I) = max{0, I(x1) − I(x2)} d2(I) = max{0, I(x2) − I(x3)} I(x1) + I(x3) ≤ 1
xi Histogram sampling using hit-and-run Monte Carlo scheme
22
[Broecheler and Getoor, NIPS 10]
p
23
p d
23
p d
23
p d q
23
p d q
23
Declarative language to specify graphical models
Inference
24
http://psl.umiacs.umd.edu
25