Approximated Probabilistic Answer Set Programming Eduardo Menezes - - PowerPoint PPT Presentation

▶

Oct 04, 2022 320 likes •812 views

Motivation ASP PSAT Probabilistic Answer Set Programming Resolution Methods Approximations Future work Approximated Probabilistic Answer Set Programming Eduardo Menezes de Morais Department of Computer Science Universidade de S ao

SLIDE 1

Motivation ASP PSAT Probabilistic Answer Set Programming Resolution Methods Approximations Future work

Approximated Probabilistic Answer Set Programming

Eduardo Menezes de Morais

Department of Computer Science Universidade de S˜ ao Paulo S˜ ao Paulo, Brazil

2014

Eduardo Menezes de Morais Approximated Probabilistic Answer Set Programming

SLIDE 2

Motivation ASP PSAT Probabilistic Answer Set Programming Resolution Methods Approximations Future work

Toy Example

1 2 3 4 5 6 Limit the percentage of time the edge (1, 3) is used to 50% and the edge (3, 4) to 40%

Eduardo Menezes de Morais Approximated Probabilistic Answer Set Programming

SLIDE 3

Motivation ASP PSAT Probabilistic Answer Set Programming Resolution Methods Approximations Future work

Outline

1

Motivation

2

Answer Set Programming

3

Probabilistic Satisfiability

4

Probabilistic Answer Set Programming

5

Resolution Methods

6

Approximations

7

Future work

Eduardo Menezes de Morais Approximated Probabilistic Answer Set Programming

SLIDE 4

Motivation ASP PSAT Probabilistic Answer Set Programming Resolution Methods Approximations Future work

Next Topic

1

Motivation

2

Answer Set Programming

3

Probabilistic Satisfiability History Definition Applications

4

Probabilistic Answer Set Programming Definition

5

Resolution Methods

6

Approximations

7

Future work

Eduardo Menezes de Morais Approximated Probabilistic Answer Set Programming

SLIDE 5

Motivation ASP PSAT Probabilistic Answer Set Programming Resolution Methods Approximations Future work

What is ASP?

Non-monotonic, declarative programming paradigm for hard combinatorial problems A ASP Program is a set of rules h ← L1, . . . , Lm, not Lm+1, . . . , not Ln The symbol not represents default negation or negation as a failure to prove Programs may have variables and functions, but must be grounded before solving

Eduardo Menezes de Morais Approximated Probabilistic Answer Set Programming

SLIDE 6

Motivation ASP PSAT Probabilistic Answer Set Programming Resolution Methods Approximations Future work

Constraints and Weight Rules

Other types of rules: Restrictions: Rules without heads ← L1, . . . , Lm, not Lm+1, . . . , not Ln Weight rules: Rules made from weight constraints C0 ← C1, . . . , Cn Weight Constraints: L ≤ {h1 = w1, . . . , not hn = wn} ≤ U. Weight rules make ASP ΣP

2 -complete.

Eduardo Menezes de Morais Approximated Probabilistic Answer Set Programming

SLIDE 7

Motivation ASP PSAT Probabilistic Answer Set Programming Resolution Methods Approximations Future work

Answer Sets

For programs without default negation, the Answer Set is the minimal model that satisfies all rules For programs with default negation, maybe there is not a unique minimal model We must first assume a set a literals and them verify if this set is a minimal model of the resulting rules

Eduardo Menezes de Morais Approximated Probabilistic Answer Set Programming

SLIDE 8

Motivation ASP PSAT Probabilistic Answer Set Programming Resolution Methods Approximations Future work

Answer Sets

Definition Let M be a finite set of atoms of P, the program PM, obtained from P by removing: all the rules that have a literal A in their negative body if A ∈ M; the negative body of the remaining rules is called reduction of P by M. Definition Let PM be the reduction of the program P by M, M is an Answer Set of P if the minimal model of PM is M.

Eduardo Menezes de Morais Approximated Probabilistic Answer Set Programming

SLIDE 9

Motivation ASP PSAT Probabilistic Answer Set Programming Resolution Methods Approximations Future work

Toy Example

1 2 3 4 5 6 An ASP Program that finds all the paths from vertex 1 to n.

1 ≤ {visited(X, Y ) = 1 for each edge(X, Y )} ≤ 1 ← vertex(X), pathTo(X). pathTo(1). pathTo(Y ) ← pathTo(X), visited(X, Y ), edge(X, Y ). ← not pathTo(n).

Eduardo Menezes de Morais Approximated Probabilistic Answer Set Programming

SLIDE 10

Motivation ASP PSAT Probabilistic Answer Set Programming Resolution Methods Approximations Future work History Definition Applications

Next Topic

1

Motivation

2

Answer Set Programming

3

Probabilistic Satisfiability History Definition Applications

4

Probabilistic Answer Set Programming Definition

5

Resolution Methods

6

Approximations

7

Future work

Eduardo Menezes de Morais Approximated Probabilistic Answer Set Programming

SLIDE 11

Motivation ASP PSAT Probabilistic Answer Set Programming Resolution Methods Approximations Future work History Definition Applications

A Brief History of PSAT

Probabilistic logic was proposed in On the Laws of Thought [Boole 1854] Classical probability and classical logic No assumption of a priori statistical independence Rediscovered several times since Boole

De Finetti [1937, 1974], Good [1950], Smith [1961] Studied by Hailperin [1965] Nilsson [1986] (re)introduces PSAT to AI Papadimitriou et al [1988]: NP-complete Many other works; see Hansen & Jaumard [2000]

Eduardo Menezes de Morais Approximated Probabilistic Answer Set Programming

SLIDE 12

Motivation ASP PSAT Probabilistic Answer Set Programming Resolution Methods Approximations Future work History Definition Applications

The Setting: the language

Logical variables or atoms: P = {x1, . . . , xn} Connectives: ∧, ∨, ¬, →, ↔. Formulas (L) are inductively composed form atoms using connectives Formulas can be brought to clausal form, but need not be.

Eduardo Menezes de Morais Approximated Probabilistic Answer Set Programming

SLIDE 13

Motivation ASP PSAT Probabilistic Answer Set Programming Resolution Methods Approximations Future work History Definition Applications

Semantics

Propositional valuation v : P → {0, 1} Generalized for any propositional formula (clausal or not) v : L → {0, 1}

Eduardo Menezes de Morais Approximated Probabilistic Answer Set Programming

SLIDE 14

Motivation ASP PSAT Probabilistic Answer Set Programming Resolution Methods Approximations Future work History Definition Applications

Semantics

Propositional valuation v : P → {0, 1} Generalized for any propositional formula (clausal or not) v : L → {0, 1} A probability distribution over propositional valuations π : V → [0, 1]

2n

π(vi) = 1

Eduardo Menezes de Morais Approximated Probabilistic Answer Set Programming

SLIDE 15

Motivation ASP PSAT Probabilistic Answer Set Programming Resolution Methods Approximations Future work History Definition Applications

Semantics

Propositional valuation v : P → {0, 1} Generalized for any propositional formula (clausal or not) v : L → {0, 1} A probability distribution over propositional valuations π : V → [0, 1]

2n

π(vi) = 1 Probability of a formula α according to π Pπ(α) =

{π(vi)|vi(α) = 1}

Eduardo Menezes de Morais Approximated Probabilistic Answer Set Programming

SLIDE 16

Motivation ASP PSAT Probabilistic Answer Set Programming Resolution Methods Approximations Future work History Definition Applications

The PSAT Problem

Consider k formulas α1, . . . , αk defined on n atoms {x1, . . . , xn} A PSAT problem Σ is a set of k restrictions Σ = {P(αi) = pi|1 ≤ i ≤ k} Probabilistic Satisfiability: are these restrictions consistent?

Eduardo Menezes de Morais Approximated Probabilistic Answer Set Programming

SLIDE 17

Motivation ASP PSAT Probabilistic Answer Set Programming Resolution Methods Approximations Future work History Definition Applications

The PSAT Problem

Consider k formulas α1, . . . , αk defined on n atoms {x1, . . . , xn} A PSAT problem Σ is a set of k restrictions Σ = {P(αi) = pi|1 ≤ i ≤ k} Probabilistic Satisfiability: are these restrictions consistent? Given Σ = {P(φi) = pi|φi ∈ LPL, 1 ≤ i ≤ q}. Is there a π such that Pπ(φi) = pi, for 1 ≤ i ≤ q?

Eduardo Menezes de Morais Approximated Probabilistic Answer Set Programming

SLIDE 18

Motivation ASP PSAT Probabilistic Answer Set Programming Resolution Methods Approximations Future work History Definition Applications

A PSAT example

Is the Hypothesis Consistent with the Data

The problem: how to fit precise theories with an imprecise world? Doctor investigating disease D Examine role of genes G1, G2, G3 Hypothesis At least two genes have to be present for D to develop Data Gene occurrence in D-patients G1 60% G2 60% G3 60%

Eduardo Menezes de Morais Approximated Probabilistic Answer Set Programming

SLIDE 19

Motivation ASP PSAT Probabilistic Answer Set Programming Resolution Methods Approximations Future work History Definition Applications

A PSAT example

Σ = P(a ∨ b) = P(a ∨ c) = P(b ∨ c) = 1 P(a) = P(b) = P(c) = 0.6

Eduardo Menezes de Morais

Approximated Probabilistic Answer Set Programming

SLIDE 20

Motivation ASP PSAT Probabilistic Answer Set Programming Resolution Methods Approximations Future work History Definition Applications

A PSAT example

Σ = P(a ∨ b) = P(a ∨ c) = P(b ∨ c) = 1 P(a) = P(b) = P(c) = 0.6

v1 = {a = b = c = 1}, v2 = {a = b = 1; c = 0},

v3 = {a = c = 1; b = 0} and v4 = {a = 0; b = c = 1}.

Eduardo Menezes de Morais Approximated Probabilistic Answer Set Programming

SLIDE 21

Motivation ASP PSAT Probabilistic Answer Set Programming Resolution Methods Approximations Future work History Definition Applications

The PSAT Problem: An Algebraic Formalization

Vector of probabilities p of dimension k × 1 (given) Consider a “large” matrix Ak×2n = [aij] (computed) aij = vj(αi) ∈ {0, 1} PSAT: decide if there is vector π of dimension 2n ×1 such that Aπ = p πi = 1 π ≥ π: probability distribution of exponential size

Eduardo Menezes de Morais Approximated Probabilistic Answer Set Programming

SLIDE 22

Motivation ASP PSAT Probabilistic Answer Set Programming Resolution Methods Approximations Future work History Definition Applications

PSAT is NP-complete

PSAT is NP-complete: [Georgakopoulos & Kavvadias & Papadimitriou 1988] A PSAT problem has a solution, then there is a solution π with at most k + 1 elements πi > 0 Carath´ eodory’s Lemma So PSAT has a polynomial size witness      1 · · · 1

0/1

· · ·

0/1

. . . ... . . .

0/1

· · ·

0/1

     ·      π1 π2 . . . πk+1      =      1 p1 . . . pk     

Eduardo Menezes de Morais Approximated Probabilistic Answer Set Programming

SLIDE 23

Motivation ASP PSAT Probabilistic Answer Set Programming Resolution Methods Approximations Future work History Definition Applications

Possible Applications

PSAT has many potential applications

Computer models of biological processes Machine learning Fault tolerance/detection Software design and analysis Economics, econometrics, etc.

Already successfully applied in Materials Science [Finger 2013]

Eduardo Menezes de Morais Approximated Probabilistic Answer Set Programming

SLIDE 24

Motivation ASP PSAT Probabilistic Answer Set Programming Resolution Methods Approximations Future work Definition

Next Topic

1

Motivation

2

Answer Set Programming

3

Probabilistic Satisfiability History Definition Applications

4

Probabilistic Answer Set Programming Definition

5

Resolution Methods

6

Approximations

7

Future work

Eduardo Menezes de Morais Approximated Probabilistic Answer Set Programming

SLIDE 25

Motivation ASP PSAT Probabilistic Answer Set Programming Resolution Methods Approximations Future work Definition

ASP is more expressive than SAT We want to extend ASP with probabilistic logic Facilitate writing complex models with PSAT

Eduardo Menezes de Morais Approximated Probabilistic Answer Set Programming

SLIDE 26

Motivation ASP PSAT Probabilistic Answer Set Programming Resolution Methods Approximations Future work Definition

Definition

A PASP instance is a grounded ASP Program and a set of probabilities over the atoms The set of probabilities P = {P(ai) = pi|1 ≤ i ≤ k} is satisfied by S if there is a probability distribution π over all subsets where pi =

{π(vl)|ai ∈ vl and vl is Answer Set of S}

Eduardo Menezes de Morais Approximated Probabilistic Answer Set Programming

SLIDE 27

Motivation ASP PSAT Probabilistic Answer Set Programming Resolution Methods Approximations Future work Definition

Definition

Like the PSAT, we can write a PASP problem in matrix form Ak×2HBS = [aij], such that aij = 1 iff the j-th atom subset contains the i-th atom criterion for deciding the satisfiability of a PASP instance becomes: Aπ = p πi = 1 π ≥

Eduardo Menezes de Morais Approximated Probabilistic Answer Set Programming

SLIDE 28

Motivation ASP PSAT Probabilistic Answer Set Programming Resolution Methods Approximations Future work Definition

Definition

Proposition If there exists an solution for a PASP instance, there is a solution with at most k + 1 nonzero elements in π. This Proposition follows directly from Carath´ eodory’s Lemma. PASP is ΣP

2 -complete (due to weight rules)

Eduardo Menezes de Morais Approximated Probabilistic Answer Set Programming

SLIDE 29

Motivation ASP PSAT Probabilistic Answer Set Programming Resolution Methods Approximations Future work Definition

Toy Example

1 2 3 4 5 6 Limit the percentage of time the edge (1, 3) is used to 50% and the edge (3, 4) to 40% P(visited(1, 3)) = 0.5, P(visited(3, 4)) = 0.4 Satisfiable with π = [0.5, 0.1, 0.4]

Eduardo Menezes de Morais Approximated Probabilistic Answer Set Programming

SLIDE 30

Motivation ASP PSAT Probabilistic Answer Set Programming Resolution Methods Approximations Future work

Next Topic

1

Motivation

2

Answer Set Programming

3

Probabilistic Satisfiability History Definition Applications

4

Probabilistic Answer Set Programming Definition

5

Resolution Methods

6

Approximations

7

Future work

Eduardo Menezes de Morais Approximated Probabilistic Answer Set Programming

SLIDE 31

Motivation ASP PSAT Probabilistic Answer Set Programming Resolution Methods Approximations Future work

Solving via Linear Programming

We decide the consistency by finding a solution to the matrix equation This can be seen as a Linear Programming problem, with the Simplex algorithm We initialize the algorithm using artificial variables The equation with the first base is:      1 1 . . . 1 1 . . . 1 . . . . . . ... . . . . . . 1      ·      π0 π1 . . . πk      =      1 p1 . . . pk      The cost function is the number of columns that do not correspond to an Answer Set

Eduardo Menezes de Morais Approximated Probabilistic Answer Set Programming

SLIDE 32

Motivation ASP PSAT Probabilistic Answer Set Programming Resolution Methods Approximations Future work

Column generation

We must generate new columns to reduce the cost function Use an ASP Solver to generate new Answer Sets with the added restriction that it’s reduced cost is negative c′

BB−1Aj > 0

Be careful since ASP is non-monotonic, and you can’t guarantee that by adding new rules you will have the same Answer Set But you can if the only rules added are restriction

Eduardo Menezes de Morais Approximated Probabilistic Answer Set Programming

SLIDE 33

Motivation ASP PSAT Probabilistic Answer Set Programming Resolution Methods Approximations Future work

Inequalities as SAT instances

It’s possible to express an binary linear inequality as SAT formulas [J.P. Warners, 1998] The coefficients must be integer and are represented in binary form With that we obtain a CNF formula (a11 ∨ · · · ∨ an1) ∧ · · · ∧ (a1m ∨ · · · ∨ ajm) And express it in the program as the restrictions: ← not a11, . . . , not an1. . . . ← not a1m, . . . , not ajm.

Eduardo Menezes de Morais Approximated Probabilistic Answer Set Programming

SLIDE 34

Motivation ASP PSAT Probabilistic Answer Set Programming Resolution Methods Approximations Future work

Weight rules

Append the rule: ← {l1 = −c′

BB−1 1 , . . . , lk = −c′ BB−1 k } ≤ −c′ BB−1

(possible scaled to only allow integers)

Eduardo Menezes de Morais Approximated Probabilistic Answer Set Programming

SLIDE 35

Motivation ASP PSAT Probabilistic Answer Set Programming Resolution Methods Approximations Future work

Implementation

Implementation available at git://gitorious.org/pasp/pasp-asp.git

Eduardo Menezes de Morais Approximated Probabilistic Answer Set Programming

SLIDE 36

Motivation ASP PSAT Probabilistic Answer Set Programming Resolution Methods Approximations Future work

Next Topic

1

Motivation

2

Answer Set Programming

3

Probabilistic Satisfiability History Definition Applications

4

Probabilistic Answer Set Programming Definition

5

Resolution Methods

6

Approximations

7

Future work

Eduardo Menezes de Morais Approximated Probabilistic Answer Set Programming

SLIDE 37

Motivation ASP PSAT Probabilistic Answer Set Programming Resolution Methods Approximations Future work

Problems

Trying to use PASP on a POS-Tagger revealed some limitations...

Eduardo Menezes de Morais Approximated Probabilistic Answer Set Programming

SLIDE 38

Motivation ASP PSAT Probabilistic Answer Set Programming Resolution Methods Approximations Future work

Approximations

1 2 3 4 5 6 Limit the percentage of time the edge (1, 3) is used to 20% and the edges (3, 4) and (4, 5) to 40% Unsatisfiable

Eduardo Menezes de Morais Approximated Probabilistic Answer Set Programming

SLIDE 39

Motivation ASP PSAT Probabilistic Answer Set Programming Resolution Methods Approximations Future work

Approximations

1 2 3 4 5 6 Limit the percentage of time the edge (1, 3) is used to 20% and the edges (3, 4) and (4, 5) to 40% Unsatisfiable Can we find a π that best approximate to these probabilities?

Eduardo Menezes de Morais Approximated Probabilistic Answer Set Programming

SLIDE 40

Motivation ASP PSAT Probabilistic Answer Set Programming Resolution Methods Approximations Future work

Error

Definition Given a probability distribution π, we define the error of π as E(π) =

i∈P
pi −
{π(vl)|ai ∈ vl and vl is Answer Set of S}
If E(π) = 0, it is satisfiable.

Eduardo Menezes de Morais Approximated Probabilistic Answer Set Programming

SLIDE 41

Motivation ASP PSAT Probabilistic Answer Set Programming Resolution Methods Approximations Future work

We need a different algorithm

1 2 3 4 5 6 Limit the percentage of time the edge (1, 3) is used to 20% and the edges (3, 4) and (4, 5) to 40% π(I3) = 0.4, π(Ii) = 0 ∀i = 3: E(π) = 0.2

Eduardo Menezes de Morais Approximated Probabilistic Answer Set Programming

SLIDE 42

Motivation ASP PSAT Probabilistic Answer Set Programming Resolution Methods Approximations Future work

We need a different algorithm

1 2 3 4 5 6 Limit the percentage of time the edge (1, 3) is used to 20% and the edges (3, 4) and (4, 5) to 40% π(I3) = 0.4, π(Ii) = 0 ∀i = 3: E(π) = 0.2 The best solution the algorithm we saw can reach has error 0.4

Eduardo Menezes de Morais Approximated Probabilistic Answer Set Programming

SLIDE 43

Motivation ASP PSAT Probabilistic Answer Set Programming Resolution Methods Approximations Future work

Slack variables

A viable solution will never have a greater probability to an atom It may assign (non-zero) probabilities to non-Answer-Set sets These can be thought as positive slack variables To have a viable solution with greater probability, we need negative slack variables

Eduardo Menezes de Morais Approximated Probabilistic Answer Set Programming

SLIDE 44

Motivation ASP PSAT Probabilistic Answer Set Programming Resolution Methods Approximations Future work

Approximations

minimize cTπ [A|I| − I] π = p π ≥ πi = 1 πi > 0 for a column i in the Identity Matrix is summed or subtracted to the probability of the i-th atom. It is part of the error of this atom.

Eduardo Menezes de Morais Approximated Probabilistic Answer Set Programming

SLIDE 45

Motivation ASP PSAT Probabilistic Answer Set Programming Resolution Methods Approximations Future work

Approximations

minimize cTπ [A|I| − I] π = p π ≥ πi = 1 πi > 0 for a column i in the Identity Matrix is summed or subtracted to the probability of the i-th atom. It is part of the error of this atom. The result is not a distribution over the subsets of HBS, but such a distribution can be obtained.

Eduardo Menezes de Morais Approximated Probabilistic Answer Set Programming

SLIDE 46

Motivation ASP PSAT Probabilistic Answer Set Programming Resolution Methods Approximations Future work

Next Topic

1

Motivation

2

Answer Set Programming

3

Probabilistic Satisfiability History Definition Applications

4

Probabilistic Answer Set Programming Definition

5

Resolution Methods

6

Approximations

7

Future work

Eduardo Menezes de Morais Approximated Probabilistic Answer Set Programming

SLIDE 47

Motivation ASP PSAT Probabilistic Answer Set Programming Resolution Methods Approximations Future work

Future work

Implementation and optimization Part-of-speech tagger

Eduardo Menezes de Morais Approximated Probabilistic Answer Set Programming

SLIDE 48

Motivation ASP PSAT Probabilistic Answer Set Programming Resolution Methods Approximations Future work

Thank you!

Eduardo Menezes de Morais Approximated Probabilistic Answer Set Programming