Decidable Classes of Datalog Programs with Arithmetic Mark Kaminski - - PowerPoint PPT Presentation

Decidable Classes of Datalog Programs with Arithmetic

Mark Kaminski

joint work with
 Bernardo Cuenca Grau, Egor Kostylev, Boris Motik, and Ian Horrocks

Department of Computer Science, University of Oxford

Metafinite 2017

Data Analytics

• identifying patterns or trends in raw data:


market predictions, spot production bottlenecks, …

• gaining importance in research and business
• major challenge: heterogeneous data
• collected from different sources
• no uniform data format

State of the Art

• custom-made imperative data processing code

State of the Art

• custom-made imperative data processing code
• labour-intensive
• requires deep technical understanding
• error-prone

Declarative Analytics

• describe what to compute rather than how
• delegate low-level details to the query engine
• improve speed and cost of code development

Alvaro et al. 2010, Markl 2014, Seo et al. 2015, Shkapsky et al. 2016

Declarative Analytics

• describe what to compute rather than how
• delegate low-level details to the query engine
• improve speed and cost of code development
• query language: recursive rules + arithmetic


Loo et al. 2009, Alvaro et al. 2010, Eisner & Filardo 2011, Chin et al. 2015, 
 Seo et al. 2015, Wang et al. 2015, Shkapsky et al. 2016 Alvaro et al. 2010, Markl 2014, Seo et al. 2015, Shkapsky et al. 2016

Challenges

• datalog + arithmetic undecidable see Dantsin et al. 2011
• no universally agreed-on semantics for aggregation
• proposals in the literature suffer from
• high complexity / undecidability


Van Gelder 1993, Ross & Sagiv 1997, Greco 1999, Mazuran et al. 2013

• limited expressivity Mumick et al. 1990,


Consens & Mendelzon 1993, Greco 1999, Faber et al. 2011

• unnatural syntactic restrictions Ross & Sagiv 1997

Our Goal

unifying formal foundation for declarative analytics

• generalise existing approaches
• natural syntax and semantics
• sufficient expressive power
• theoretically understood computational properties
• amenable to efficient implementation

Overview

• datalogℤ
• decidability
• non-monotonic extension
• metafinite model theory?

Datalogℤ

• positive datalog extended with integer arithmetic
• example rule

A(x ) ∧ B(x,y,m ) ∧ C(y,z,n ) ∧ (m +1 ≤ 2 ·n) → D(y,z,m +n)

Datalogℤ

• positive datalog extended with integer arithmetic
• example rule

A(x ) ∧ B(x,y,m ) ∧ C(y,z,n ) ∧ (m +1 ≤ 2 ·n) → D(y,z,m +n)

• rdinary


datalog
 atoms

Datalogℤ

• positive datalog extended with integer arithmetic
• example rule

A(x ) ∧ B(x,y,m ) ∧ C(y,z,n ) ∧ (m +1 ≤ 2 ·n) → D(y,z,m +n) numeric
 atoms

Datalogℤ

• positive datalog extended with integer arithmetic
• example rule

A(x ) ∧ B(x,y,m ) ∧ C(y,z,n ) ∧ (m +1 ≤ 2 ·n) → D(y,z,m +n)

• ne numeric argument


per atom m n m +n

Datalogℤ

• positive datalog extended with integer arithmetic
• example rule

A(x ) ∧ B(x,y,m ) ∧ C(y,z,n ) ∧ (m +1 ≤ 2 ·n) → D(y,z,m +n) comparison
 atoms

Datalogℤ

• positive datalog extended with integer arithmetic
• example rule

A(x ) ∧ B(x,y,m ) ∧ C(y,z,n ) ∧ (m +1 ≤ 2 ·n) → D(y,z,m +n)

• P ⊧ A(a) if ∀I: I ⊧ P implies I ⊧ A(a)

Datalogℤ

• positive datalog extended with integer arithmetic
• example rule

A(x ) ∧ B(x,y,m ) ∧ C(y,z,n ) ∧ (m +1 ≤ 2 ·n) → D(y,z,m +n)

• P ⊧ A(a) if ∀I: I ⊧ P implies I ⊧ A(a)

two-sorted
 FO interpretation
 with integers

Datalogℤ

• positive datalog extended with integer arithmetic
• example rule

A(x ) ∧ B(x,y,m ) ∧ C(y,z,n ) ∧ (m +1 ≤ 2 ·n) → D(y,z,m +n)

• P ⊧ A(a) if ∀I: I ⊧ P implies I ⊧ A(a)
• P ⊧ A(a) iff A(a)∈TP(∅) TP immed. cons. operator

∞

Datalogℤ

• positive datalog extended with integer arithmetic
• example rule

A(x ) ∧ B(x,y,m ) ∧ C(y,z,n ) ∧ (m +1 ≤ 2 ·n) → D(y,z,m +n)

• P ⊧ A(a) if ∀I: I ⊧ P implies I ⊧ A(a)
• P ⊧ A(a) iff A(a)∈TP(∅) TP immed. cons. operator
• undecidable even when + is the only operator

∞

Limit Predicates

• keep only the minimal/maximal numeric value
• restrict interpretations to satisfy

A(x,m ) ∧ (m ≤ n ) → A(x,n ) for A a min predicate B(x,m ) ∧ (n ≤ m ) → B(x,n ) for B a max predicate

Limit Predicates

• keep only the minimal/maximal numeric value
• restrict interpretations to satisfy

A(x,m ) ∧ (m ≤ n ) → A(x,n ) for A a min predicate B(x,m ) ∧ (n ≤ m ) → B(x,n ) for B a max predicate

• limit datalogℤ: all numeric predicates in rule heads

limit predicates

Example

cheapest route from London to Reykjavík?

flight(x,y,c) → route(x,y,c)
 route(x,z,c1) ∧ flight(z,y,c2) → route(x,y,c1+c2)

route a min predicate 
 
 
 


flight London Hamburg 100 Hamburg Reykjavík 150 London Reykjavík 300

Example

cheapest route from London to Reykjavík?

flight(x,y,c) → route(x,y,c)
 route(x,z,c1) ∧ flight(z,y,c2) → route(x,y,c1+c2)

route a min predicate 
 
 
 


flight London Hamburg 100 Hamburg Reykjavík 150 London Reykjavík 300 route London Reykjavík 250 London Reykjavík 300 … … …

Example

cheapest route from London to Reykjavík?

flight(x,y,c) → route(x,y,c)
 route(x,z,c1) ∧ flight(z,y,c2) → route(x,y,c1+c2)

route a min predicate 
 
 
 


flight London Hamburg 100 Hamburg Reykjavík 150 London Reykjavík 300 route London Reykjavík 250 London Reykjavík 300 … … …

Pseudo-Interpretations

• Herbrand interpretations J
• for each min/max predicate A and constants a


store only the minimal/maximal k ∈ ℤ s.t. J ⊧ A(a,k )

Pseudo-Interpretations

• Herbrand interpretations J
• for each min/max predicate A and constants a


store only the minimal/maximal k ∈ ℤ s.t. J ⊧ A(a,k )

• each limit datalogℤ program P


has a pseudo-model J with |J | ≤ |P |

Limit Linearity

• limit datalogℤ undecidable: consider P as follows


→ A(0)
 A(x1) ∧ … ∧ A(xn) ∧ p(x1,…,xn)=0 → B P ⊧ B iff p(x1,…,xn)=0 has non-negative integer solution

Limit Linearity

• limit datalogℤ undecidable: consider P as follows


→ A(0)
 A(x1) ∧ … ∧ A(xn) ∧ p(x1,…,xn)=0 → B P ⊧ B iff p(x1,…,xn)=0 has non-negative integer solution

• limit linearity:


disallow multiplication between limit variables

Limit Linearity

• limit datalogℤ undecidable: consider P as follows


→ A(0)
 A(x1) ∧ … ∧ A(xn) ∧ p(x1,…,xn)=0 → B P ⊧ B iff p(x1,…,xn)=0 has non-negative integer solution

• limit linearity:


disallow multiplication between limit variables A(x ) ∧ B(y ) → C(x ·y ) not limit linear

Limit Linearity

• limit datalogℤ undecidable: consider P as follows


→ A(0)
 A(x1) ∧ … ∧ A(xn) ∧ p(x1,…,xn)=0 → B P ⊧ B iff p(x1,…,xn)=0 has non-negative integer solution

• limit linearity:


disallow multiplication between limit variables A(x ) ∧ B(y ) → C(x ·y ) not limit linear A(x ) ∧ B(y ) → C(x ·y ) limit linear

Limit-Linear Datalogℤ

• fact entailment coNEXPTIME-complete


and coNP-complete in data complexity

• upper bounds (data complexity)
• fact entailment reducible to Presburger validity

A(x ) → B(x +1) ↝ ∀x.defA ∧ (x ≤valA) → defB ∧ (x +1≤valB)

• magnitude of integers in countermodels


exponentially bounded using Chistikov & Haase 2016

• NP guess-and-check procedure for non-entailment

Limit-Linear Datalogℤ

• lower bounds: reduction from square tiling

Square Tiling input: finite set T of tiles horizontal compatibility relation H⊆T⨯T vertical compatibility relation V⊆T⨯T number N problem: is there a function N⨯N → T
 satisfying H and V (tiling)?

Limit-Linear Datalogℤ

• lower bounds: reduction from square tiling

Limit-Linear Datalogℤ

• lower bounds: reduction from square tiling
• interpret each N2⎡log2|T|⎤bit number n


as a candidate tiling; initialise n with 0 · -

Limit-Linear Datalogℤ

• lower bounds: reduction from square tiling
• interpret each N2⎡log2|T|⎤bit number n


as a candidate tiling; initialise n with 0

• if n not a tiling, increase n

· -

Limit-Linear Datalogℤ

• lower bounds: reduction from square tiling
• interpret each N2⎡log2|T|⎤bit number n


as a candidate tiling; initialise n with 0

• if n not a tiling, increase n
• if n > 2N ⎡log |T|⎤-1, return ‘noSolution’

· -

2· 2

Limit-Linear Datalogℤ

• lower bounds: reduction from square tiling
• interpret each N2⎡log2|T|⎤bit number n


as a candidate tiling; initialise n with 0

• if n not a tiling, increase n
• if n > 2N ⎡log |T|⎤-1, return ‘noSolution’
• P ⊧ noSolution iff no tiling exists

· -

2· 2

Accessing Limit Values

• want to check if max. value of A < max. value of B

Accessing Limit Values

• want to check if max. value of A < max. value of B
• A(x ) ∧ B(y ) ∧ (x <y ) → A_lessthan_B

Accessing Limit Values

• want to check if max. value of A < max. value of B
• A(x ) ∧ B(y ) ∧ (x <y ) → A_lessthan_B
• does the rule apply to ? yes!

I = { A(1) B(0) } A(0) B(-1) A(-1) B(-2) ⋮ ⋮

Accessing Limit Values

• want to check if max. value of A < max. value of B
• A(x ) ∧ B(y ) ∧ (x <y ) → A_lessthan_B
• does the rule apply to ? yes!

I = { A(1) B(0) } A(0) B(-1) A(-1) B(-2) ⋮ ⋮

Accessing Limit Values

• want to check if max. value of A < max. value of B
• A(x ) ∧ B(y ) ∧ (x <y ) → A_lessthan_B
• does the rule apply to ? yes!
• want to restrict rule application to min./max. values

I = { A(1) B(0) } A(0) B(-1) A(-1) B(-2) ⋮ ⋮

LUB Operator

• ⎡A(a,k)⎤ satisfied in I iff k ∈ ℤ maximal s.t. A(a,k) ∈ I

LUB Operator

• ⎡A(a,k)⎤ satisfied in I iff k ∈ ℤ maximal s.t. A(a,k) ∈ I
• ⎡A(x )⎤ ∧ ⎡B(y )⎤ ∧ (x <y ) → A_lessthan_B


does not apply to

I = { A(1) B(0) } A(0) B(-1) A(-1) B(-2) ⋮ ⋮

LUB Operator

• ⎡A(a,k)⎤ satisfied in I iff k ∈ ℤ maximal s.t. A(a,k) ∈ I
• ⎡A(x )⎤ ∧ ⎡B(y )⎤ ∧ (x <y ) → A_lessthan_B


does not apply to

• can simulate negation as failure

→ B(0)
 A → B(1) simulates not A → C
 ⎡B(0)⎤→ C

I = { A(1) B(0) } A(0) B(-1) A(-1) B(-2) ⋮ ⋮

Stratified-Linear Datalogℤ

• stratify application of⎡⎤

Stratified-Linear Datalogℤ

• stratify application of⎡⎤
• recall: A(x ) ∧ B(y ) → C(x ·y ) not limit linear


if A,B both max predicates

Stratified-Linear Datalogℤ

• stratify application of⎡⎤
• recall: A(x ) ∧ B(y ) → C(x ·y ) not limit linear


if A,B both max predicates

• A(x ) ∧⎡B(y )⎤→ C(x ·y ) stratified-linear if B <s C

Complexity

fact entailment for stratified-linear datalogℤ
 Δ2 -complete and Δ2-complete in data complexity

• upper bounds: stratified-linear programs

deterministic compositions of limit-linear programs

EXP P

Complexity

fact entailment for stratified-linear datalogℤ
 Δ2 -complete and Δ2-complete in data complexity

• upper bounds: stratified-linear programs

deterministic compositions of limit-linear programs

• lower bounds:
• show that stratified-linear datalogℤ captures Δ2


by simulating a DTM with an NP oracle

• Gottlob et al. 1999: if a logic L captures a

complexity class C in the polynomial hierarchy, then L is Exp(C )-hard in combined complexity

EXP P P

Metafinite Model Theory

• stratified-linear datalogℤ captures Δ2

• n ordered datasets containing no numeric facts

P

Metafinite Model Theory

• stratified-linear datalogℤ captures Δ2

• n ordered datasets containing no numeric facts
• what about datasets with numeric facts,


i.e., metafinite structures?

• does stratified-linear datalogℤ capture Δ2?
• does limit-linear datalogℤ capture coNP?

P P

Metafinite Model Theory

• stratified-linear datalogℤ captures Δ2

• n ordered datasets containing no numeric facts
• what about datasets with numeric facts,


i.e., metafinite structures?

• does stratified-linear datalogℤ capture Δ2?
• does limit-linear datalogℤ capture coNP?
• what about datalogℝ?

P P

Alternative Semantics

• interpret limit atoms by their maximal value


rather than a half-open interval following Ross & Sagiv. 1997
 ≈ put⎡⎤around every limit atom

• decidability without limit-linearity
• data complexity of monotone datalogℤ


in PPosSLP=BP(Pℝ) as defined in Allender et al. 2009

Thank you!