What do Shannon-type Inequalities, Submodular Width, and - - PowerPoint PPT Presentation

1/22

What do Shannon-type Inequalities, Submodular Width, and Disjunctive Datalog have to do with one another?

Mahmoud Abo Khamis1 Hung Q. Ngo1 Dan Suciu1,2

1LogicBlox Inc. 2University of Washington

PODS 2017

2/22

Contributions

◮ A Join Algorithm

2/22

Contributions

◮ A Join Algorithm

◮ first to meet the Submodular Width bound!

2/22

Contributions

◮ A Join Algorithm

◮ first to meet the Submodular Width bound! ◮ works for and relies on Disjunctive Datalog.

2/22

Contributions

◮ A Join Algorithm

◮ first to meet the Submodular Width bound! ◮ works for and relies on Disjunctive Datalog. ◮ fully utilizes Functional DEPs and Degree Bounds.

2/22

Contributions

◮ A Join Algorithm

◮ first to meet the Submodular Width bound! ◮ works for and relies on Disjunctive Datalog. ◮ fully utilizes Functional DEPs and Degree Bounds.

◮ A Unified Framework for Join Bounds

2/22

Contributions

◮ A Join Algorithm

◮ first to meet the Submodular Width bound! ◮ works for and relies on Disjunctive Datalog. ◮ fully utilizes Functional DEPs and Degree Bounds.

◮ A Unified Framework for Join Bounds

◮ subsumes most known bounds.

2/22

Contributions

◮ A Join Algorithm

◮ first to meet the Submodular Width bound! ◮ works for and relies on Disjunctive Datalog. ◮ fully utilizes Functional DEPs and Degree Bounds.

◮ A Unified Framework for Join Bounds

◮ subsumes most known bounds. ◮ extends them to Functional DEPs and Degree Bounds.

2/22

Contributions

◮ A Join Algorithm

◮ first to meet the Submodular Width bound! ◮ works for and relies on Disjunctive Datalog. ◮ fully utilizes Functional DEPs and Degree Bounds.

◮ A Unified Framework for Join Bounds

◮ subsumes most known bounds. ◮ extends them to Functional DEPs and Degree Bounds.

◮ Results on Shannon-type Inequalities

3/22

Table of Contents

Size Bounds for Full Conjunctive Queries Size Bounds for Disjunctive Datalog Algorithms for Disjunctive Datalog Algorithms for Conjunctive Queries

4/22

Table of Contents

Size Bounds for Full Conjunctive Queries Size Bounds for Disjunctive Datalog Algorithms for Disjunctive Datalog Algorithms for Conjunctive Queries

5/22

Size Bounds for Full Conjunctive Queries

Q(A1, A2, A3, A4) :- R12(A1, A2), R23(A2, A3), R34(A3, A4), R41(A4, A1). A1 A3 A2 A4 R12 R23 R34 R41

5/22

Size Bounds for Full Conjunctive Queries

Q(A1, A2, A3, A4) :- R12(A1, A2), R23(A2, A3), R34(A3, A4), R41(A4, A1). A1 A3 A2 A4 R12 R23 R34 R41

A1 A2 a 1 b 1 b 2 A2 A3 1 c 1 d 2 c A3 A4 c 3 d 4 d 5 A4 A1 3 b 4 a 4 b

5/22

Size Bounds for Full Conjunctive Queries

Q(A1, A2, A3, A4) :- R12(A1, A2), R23(A2, A3), R34(A3, A4), R41(A4, A1). A1 A3 A2 A4 R12 R23 R34 R41

A1 A2 A3 A4 a 1 d 4 b 1 c 3 b 1 d 4 b 2 c 3 A1 A2 a 1 b 1 b 2 A2 A3 1 c 1 d 2 c A3 A4 c 3 d 4 d 5 A4 A1 3 b 4 a 4 b

5/22

Size Bounds for Full Conjunctive Queries

Q(A1, A2, A3, A4) :- R12(A1, A2), R23(A2, A3), R34(A3, A4), R41(A4, A1). A1 A3 A2 A4 R12 R23 R34 R41

A1 A2 A3 A4 a 1 d 4 1/4 b 1 c 3 1/4 b 1 d 4 1/4 b 2 c 3 1/4 A1 A2 a 1 b 1 b 2 A2 A3 1 c 1 d 2 c A3 A4 c 3 d 4 d 5 A4 A1 3 b 4 a 4 b

5/22

Size Bounds for Full Conjunctive Queries

Q(A1, A2, A3, A4) :- R12(A1, A2), R23(A2, A3), R34(A3, A4), R41(A4, A1). A1 A3 A2 A4 R12 R23 R34 R41

A1 A2 A3 A4 a 1 d 4 1/4 b 1 c 3 1/4 b 1 d 4 1/4 b 2 c 3 1/4 A1 A2 a 1 1/4 b 1 2/4 b 2 1/4 A2 A3 1 c 1/4 1 d 2/4 2 c 1/4 A3 A4 c 3 2/4 d 4 2/4 d 5 A4 A1 3 b 2/4 4 a 1/4 4 b 1/4

5/22

Size Bounds for Full Conjunctive Queries

Q(A1, A2, A3, A4) :- R12(A1, A2), R23(A2, A3), R34(A3, A4), R41(A4, A1). A1 A3 A2 A4 R12 R23 R34 R41

A1 A2 A3 A4 a 1 d 4 1/4 b 1 c 3 1/4 b 1 d 4 1/4 b 2 c 3 1/4 h(A1A2A3A4) = log |Q| A1 A2 a 1 1/4 b 1 2/4 b 2 1/4 A2 A3 1 c 1/4 1 d 2/4 2 c 1/4 A3 A4 c 3 2/4 d 4 2/4 d 5 A4 A1 3 b 2/4 4 a 1/4 4 b 1/4

5/22

Size Bounds for Full Conjunctive Queries

Q(A1, A2, A3, A4) :- R12(A1, A2), R23(A2, A3), R34(A3, A4), R41(A4, A1). A1 A3 A2 A4 R12 R23 R34 R41

A1 A2 A3 A4 a 1 d 4 1/4 b 1 c 3 1/4 b 1 d 4 1/4 b 2 c 3 1/4 h(A1A2A3A4) = log |Q| A1 A2 a 1 1/4 b 1 2/4 b 2 1/4 A2 A3 1 c 1/4 1 d 2/4 2 c 1/4 A3 A4 c 3 2/4 d 4 2/4 d 5 A4 A1 3 b 2/4 4 a 1/4 4 b 1/4 h(A1A2) ≤ log |R12|, h(A2A3) ≤ log |R23|, h(A3A4) ≤ log |R34|, ...

5/22

Size Bounds for Full Conjunctive Queries

Q(A1, A2, A3, A4) :- R12(A1, A2), R23(A2, A3), R34(A3, A4), R41(A4, A1). A1 A3 A2 A4 R12 R23 R34 R41

A1 A2 A3 A4 a 1 d 4 1/4 b 1 c 3 1/4 b 1 d 4 1/4 b 2 c 3 1/4 h(A1A2A3A4) = log |Q| A1 A2 a 1 1/4 b 1 2/4 b 2 1/4 A2 A3 1 c 1/4 1 d 2/4 2 c 1/4 A3 A4 c 3 2/4 d 4 2/4 d 5 A4 A1 3 b 2/4 4 a 1/4 4 b 1/4 h(A1A2) ≤ log |R12|, h(A2A3) ≤ log |R23|, h(A3A4) ≤ log |R34|, ... h(A2|A1 = ‘a’) ≤ log

• σA1=‘a’R12
• ,

h(A2|A1 = ‘b’) ≤ log

• σA1=‘b’R12
• ,

...

5/22

Size Bounds for Full Conjunctive Queries

Q(A1, A2, A3, A4) :- R12(A1, A2), R23(A2, A3), R34(A3, A4), R41(A4, A1). A1 A3 A2 A4 R12 R23 R34 R41

A1 A2 A3 A4 a 1 d 4 1/4 b 1 c 3 1/4 b 1 d 4 1/4 b 2 c 3 1/4 h(A1A2A3A4) = log |Q| A1 A2 a 1 1/4 b 1 2/4 b 2 1/4 A2 A3 1 c 1/4 1 d 2/4 2 c 1/4 A3 A4 c 3 2/4 d 4 2/4 d 5 A4 A1 3 b 2/4 4 a 1/4 4 b 1/4 h(A1A2) ≤ log |R12|, h(A2A3) ≤ log |R23|, h(A3A4) ≤ log |R34|, ... h(A2|A1 = ‘a’) ≤ log

• σA1=‘a’R12
• ,

h(A2|A1 = ‘b’) ≤ log

• σA1=‘b’R12
• ,

... h(A2|A1) ≤ log max

x

• σA1=xR12

5/22

Size Bounds for Full Conjunctive Queries

Q(A1, A2, A3, A4) :- R12(A1, A2), R23(A2, A3), R34(A3, A4), R41(A4, A1). A1 A3 A2 A4 R12 R23 R34 R41

A1 A2 A3 A4 a 1 d 4 1/4 b 1 c 3 1/4 b 1 d 4 1/4 b 2 c 3 1/4 h(A1A2A3A4) = log |Q| A1 A2 a 1 1/4 b 1 2/4 b 2 1/4 A2 A3 1 c 1/4 1 d 2/4 2 c 1/4 A3 A4 c 3 2/4 d 4 2/4 d 5 A4 A1 3 b 2/4 4 a 1/4 4 b 1/4 h(A1A2) ≤ log |R12|, h(A2A3) ≤ log |R23|, h(A3A4) ≤ log |R34|, ... h(A2|A1 = ‘a’) ≤ log

• σA1=‘a’R12
• ,

h(A2|A1 = ‘b’) ≤ log

• σA1=‘b’R12
• ,

... h(A2|A1) ≤ log max

x

• σA1=xR12
• degR12(A2|A1)

6/22

Size Bounds: Input

◮ A full conjunctive query

Q(A[n]) :-

• F∈E

RF (AF )

6/22

Size Bounds: Input

◮ A full conjunctive query

Q(A[n]) :-

• F∈E

RF (AF )

◮ A[n] = {A1, . . . , An} is the full set of attributes

6/22

Size Bounds: Input

◮ A full conjunctive query

Q(A[n]) :-

• F∈E

RF (AF )

◮ A[n] = {A1, . . . , An} is the full set of attributes ◮ AF = {Af | f ∈ F ⊆ [n]} is a subset

6/22

Size Bounds: Input

◮ A full conjunctive query

Q(A[n]) :-

• F∈E

RF (AF )

◮ A[n] = {A1, . . . , An} is the full set of attributes ◮ AF = {Af | f ∈ F ⊆ [n]} is a subset

◮ Degree Constraints (DC):

degF (AY |AX) ≤ NY |X, X ⊂ Y ⊆ F ∈ E

6/22

Size Bounds: Input

◮ A full conjunctive query

Q(A[n]) :-

• F∈E

RF (AF )

◮ A[n] = {A1, . . . , An} is the full set of attributes ◮ AF = {Af | f ∈ F ⊆ [n]} is a subset

◮ Degree Constraints (DC):

degF (AY |AX) ≤ NY |X, X ⊂ Y ⊆ F ∈ E

◮ Cardinality Constraints (CC):

|RF | ≤ NF |∅

6/22

Size Bounds: Input

◮ A full conjunctive query

Q(A[n]) :-

• F∈E

RF (AF )

◮ A[n] = {A1, . . . , An} is the full set of attributes ◮ AF = {Af | f ∈ F ⊆ [n]} is a subset

◮ Degree Constraints (DC):

degF (AY |AX) ≤ NY |X, X ⊂ Y ⊆ F ∈ E

◮ Cardinality Constraints (CC):

|RF | ≤ NF |∅

◮ Functional Dependencies (FD):

AX → AY

6/22

Size Bounds: Input

◮ A full conjunctive query

Q(A[n]) :-

• F∈E

RF (AF )

◮ A[n] = {A1, . . . , An} is the full set of attributes ◮ AF = {Af | f ∈ F ⊆ [n]} is a subset

◮ Degree Constraints (DC):

degF (AY |AX) ≤ NY |X, X ⊂ Y ⊆ F ∈ E

◮ Cardinality Constraints (CC):

|RF | ≤ NF |∅

◮ Functional Dependencies (FD):

AX → AY

6/22

Size Bounds: Input

◮ A full conjunctive query

Q(A[n]) :-

• F∈E

RF (AF )

◮ A[n] = {A1, . . . , An} is the full set of attributes ◮ AF = {Af | f ∈ F ⊆ [n]} is a subset

◮ Degree Constraints (DC):

degF (AY |AX) ≤ NY |X, X ⊂ Y ⊆ F ∈ E

◮ Cardinality Constraints (CC):

|RF | ≤ NF |∅

◮ Functional Dependencies (FD):

AX → AY

Bound Idea

6/22

Size Bounds: Input

◮ A full conjunctive query

Q(A[n]) :-

• F∈E

RF (AF )

◮ A[n] = {A1, . . . , An} is the full set of attributes ◮ AF = {Af | f ∈ F ⊆ [n]} is a subset

◮ Degree Constraints (DC):

degF (AY |AX) ≤ NY |X, X ⊂ Y ⊆ F ∈ E

◮ Cardinality Constraints (CC):

|RF | ≤ NF |∅

◮ Functional Dependencies (FD):

AX → AY

Bound Idea log |Q| ≤ maximum h(A1, . . . , An)

6/22

Size Bounds: Input

◮ A full conjunctive query

Q(A[n]) :-

• F∈E

RF (AF )

◮ A[n] = {A1, . . . , An} is the full set of attributes ◮ AF = {Af | f ∈ F ⊆ [n]} is a subset

◮ Degree Constraints (DC):

degF (AY |AX) ≤ NY |X, X ⊂ Y ⊆ F ∈ E

◮ Cardinality Constraints (CC):

|RF | ≤ NF |∅

◮ Functional Dependencies (FD):

AX → AY

Bound Idea log |Q| ≤ maximum h(A1, . . . , An)

• ver all

entropies h

6/22

Size Bounds: Input

◮ A full conjunctive query

Q(A[n]) :-

• F∈E

RF (AF )

◮ A[n] = {A1, . . . , An} is the full set of attributes ◮ AF = {Af | f ∈ F ⊆ [n]} is a subset

◮ Degree Constraints (DC):

degF (AY |AX) ≤ NY |X, X ⊂ Y ⊆ F ∈ E

◮ Cardinality Constraints (CC):

|RF | ≤ NF |∅

◮ Functional Dependencies (FD):

AX → AY

Bound Idea log |Q| ≤ maximum h(A1, . . . , An)

• ver all

entropies h such that h satisfies degree constraints of Q

7/22

Size Bounds: Preliminaries

◮ HDC is the set of functions h : 2[n] → R+ satisfying the

degree constraints HDC def =

• h

| h(Y |X) ≤ log NY |X, ∀(X, Y, NY |X)

7/22

Size Bounds: Preliminaries

◮ HDC is the set of functions h : 2[n] → R+ satisfying the

degree constraints HDC def =

• h

| h(Y |X) ≤ log NY |X, ∀(X, Y, NY |X)

• ◮ Γ∗

n is the set of entropic functions

7/22

Size Bounds: Preliminaries

◮ HDC is the set of functions h : 2[n] → R+ satisfying the

degree constraints HDC def =

• h

| h(Y |X) ≤ log NY |X, ∀(X, Y, NY |X)

• ◮ Γ∗

n is the set of entropic functions ◮ Γ ∗ n is the topological closure of Γ∗ n

7/22

Size Bounds: Preliminaries

◮ HDC is the set of functions h : 2[n] → R+ satisfying the

degree constraints HDC def =

• h

| h(Y |X) ≤ log NY |X, ∀(X, Y, NY |X)

• ◮ Γ∗

n is the set of entropic functions ◮ Γ ∗ n is the topological closure of Γ∗ n ◮ Γn is the set of polymatroids, i.e. functions h : 2[n] → R+

satisfying

h(X ∪ Y ) + h(X ∩ Y ) ≤ h(X) + h(Y ), X, Y ⊆ [n] (submodularity) h(X) ≤ h(Y ), X ⊆ Y ⊆ [n] (monotonicity) h(∅) = 0 (strictness)

7/22

Size Bounds: Preliminaries

◮ HDC is the set of functions h : 2[n] → R+ satisfying the

degree constraints HDC def =

• h

| h(Y |X) ≤ log NY |X, ∀(X, Y, NY |X)

• ◮ Γ∗

n is the set of entropic functions ◮ Γ ∗ n is the topological closure of Γ∗ n ◮ Γn is the set of polymatroids, i.e. functions h : 2[n] → R+

satisfying

h(X ∪ Y ) + h(X ∩ Y ) ≤ h(X) + h(Y ), X, Y ⊆ [n] (submodularity) h(X) ≤ h(Y ), X ⊆ Y ⊆ [n] (monotonicity) h(∅) = 0 (strictness)

Γ∗

n

• entropic functions

⊂ Γ

∗ n

• topological closure of Γ∗

n

⊂ Γn

• polymatroids

8/22

Size Bounds for Full Conjunctive Queries

Γ∗

n

• entropic functions

⊂ Γ

∗ n

• topological closure of Γ∗

n

⊂ Γn

• polymatroids

8/22

Size Bounds for Full Conjunctive Queries

Γ∗

n

• entropic functions

⊂ Γ

∗ n

• topological closure of Γ∗

n

⊂ Γn

• polymatroids

Bound Entropic Bound Polymatroid Bound Definition

log |Q| ≤ max

h∈Γ∗

n∩HDC h([n])

log |Q| ≤ max

h∈Γn∩HDC h([n])

8/22

Size Bounds for Full Conjunctive Queries

Γ∗

n

• entropic functions

⊂ Γ

∗ n

• topological closure of Γ∗

n

⊂ Γn

• polymatroids

Bound Entropic Bound Polymatroid Bound Definition

log |Q| ≤ max

h∈Γ∗

n∩HDC h([n])

log |Q| ≤ max

h∈Γn∩HDC h([n])

CC only AGM bound (Tight) AGM bound (Tight)

[Atserias et al. FOCS’08] [Atserias et al. FOCS’08]

8/22

Size Bounds for Full Conjunctive Queries

Γ∗

n

• entropic functions

⊂ Γ

∗ n

• topological closure of Γ∗

n

⊂ Γn

• polymatroids

Bound Entropic Bound Polymatroid Bound Definition

log |Q| ≤ max

h∈Γ∗

n∩HDC h([n])

log |Q| ≤ max

h∈Γn∩HDC h([n])

CC only AGM bound (Tight) AGM bound (Tight)

[Atserias et al. FOCS’08] [Atserias et al. FOCS’08]

CC + FD only Entropic Bound for FD Polymatroid Bound for FD

[Gottlob et al. JACM’12] [Gottlob et al. JACM’12]

(Tight [Gogacz et al. ICDT’17]) (Not tight

[Our work] )

8/22

Size Bounds for Full Conjunctive Queries

Γ∗

n

• entropic functions

⊂ Γ

∗ n

• topological closure of Γ∗

n

⊂ Γn

• polymatroids

Bound Entropic Bound Polymatroid Bound Definition

log |Q| ≤ max

h∈Γ∗

n∩HDC h([n])

log |Q| ≤ max

h∈Γn∩HDC h([n])

CC only AGM bound (Tight) AGM bound (Tight)

[Atserias et al. FOCS’08] [Atserias et al. FOCS’08]

CC + FD only Entropic Bound for FD Polymatroid Bound for FD

[Gottlob et al. JACM’12] [Gottlob et al. JACM’12]

(Tight [Gogacz et al. ICDT’17]) (Not tight

[Our work] )

DC Entropic Bound for DC Polymatroid Bound for DC (Tight

[Our work] )

(Not tight

[Our work] )

9/22

Table of Contents

Size Bounds for Full Conjunctive Queries Size Bounds for Disjunctive Datalog Algorithms for Disjunctive Datalog Algorithms for Conjunctive Queries

10/22

Disjunctive Datalog

P :

• B∈B

TB(AB) :-

• F∈E

RF (AF )

10/22

Disjunctive Datalog

P :

• B∈B

TB(AB) :-

• F∈E

RF (AF )

A1 A3 A2 A4 R12 R23 R34 R41

P : T123(A1, A2, A3) ∨ T234(A2, A3, A4) :- R12(A1, A2), R23(A2, A3), R34(A3, A4), R41(A4, A1).

10/22

Disjunctive Datalog

P :

• B∈B

TB(AB) :-

• F∈E

RF (AF )

A1 A3 A2 A4 R12 R23 R34 R41

P : T123(A1, A2, A3) ∨ T234(A2, A3, A4) :- R12(A1, A2), R23(A2, A3), R34(A3, A4), R41(A4, A1).

10/22

Disjunctive Datalog

P :

• B∈B

TB(AB) :-

• F∈E

RF (AF )

A1 A3 A2 A4 R12 R23 R34 R41

P : T123(A1, A2, A3) ∨ T234(A2, A3, A4) :- R12(A1, A2), R23(A2, A3), R34(A3, A4), R41(A4, A1).

10/22

Disjunctive Datalog

P :

• B∈B

TB(AB) :-

• F∈E

RF (AF )

A1 A3 A2 A4 R12 R23 R34 R41

P : T123(A1, A2, A3) ∨ T234(A2, A3, A4) :- R12(A1, A2), R23(A2, A3), R34(A3, A4), R41(A4, A1).

A1 A2 a 1 b 1 b 2 A2 A3 1 c 1 d 2 c A3 A4 c 3 d 4 d 5 A4 A1 3 b 4 a 4 b

10/22

Disjunctive Datalog

P :

• B∈B

TB(AB) :-

• F∈E

RF (AF )

A1 A3 A2 A4 R12 R23 R34 R41

P : T123(A1, A2, A3) ∨ T234(A2, A3, A4) :- R12(A1, A2), R23(A2, A3), R34(A3, A4), R41(A4, A1).

A1 A2 a 1 b 1 b 2 A2 A3 1 c 1 d 2 c A3 A4 c 3 d 4 d 5 A4 A1 3 b 4 a 4 b A1 A2 A3 A4 a 1 d 4 b 1 c 3 b 1 d 4 b 2 c 3

10/22

Disjunctive Datalog

P :

• B∈B

TB(AB) :-

• F∈E

RF (AF )

A1 A3 A2 A4 R12 R23 R34 R41

P : T123(A1, A2, A3) ∨ T234(A2, A3, A4) :- R12(A1, A2), R23(A2, A3), R34(A3, A4), R41(A4, A1).

A1 A2 a 1 b 1 b 2 A2 A3 1 c 1 d 2 c A3 A4 c 3 d 4 d 5 A4 A1 3 b 4 a 4 b A1 A2 A3 A4 a 1 d 4 b 1 c 3 b 1 d 4 b 2 c 3 A1 A2 A3 b 1 c b 2 c A2 A3 A4 1 d 4 2 c 3 2 d 4

10/22

Disjunctive Datalog

P :

• B∈B

TB(AB) :-

• F∈E

RF (AF )

A1 A3 A2 A4 R12 R23 R34 R41

P : T123(A1, A2, A3) ∨ T234(A2, A3, A4) :- R12(A1, A2), R23(A2, A3), R34(A3, A4), R41(A4, A1).

A1 A2 a 1 b 1 b 2 A2 A3 1 c 1 d 2 c A3 A4 c 3 d 4 d 5 A4 A1 3 b 4 a 4 b A1 A2 A3 A4 a 1 d 4 b 1 c 3 b 1 d 4 b 2 c 3 A1 A2 A3 b 1 c b 2 c A2 A3 A4 1 d 4 2 c 3 2 d 4

10/22

Disjunctive Datalog

P :

• B∈B

TB(AB) :-

• F∈E

RF (AF )

A1 A3 A2 A4 R12 R23 R34 R41

P : T123(A1, A2, A3) ∨ T234(A2, A3, A4) :- R12(A1, A2), R23(A2, A3), R34(A3, A4), R41(A4, A1).

A1 A2 a 1 b 1 b 2 A2 A3 1 c 1 d 2 c A3 A4 c 3 d 4 d 5 A4 A1 3 b 4 a 4 b A1 A2 A3 A4 a 1 d 4 b 1 c 3 b 1 d 4 b 2 c 3 A1 A2 A3 b 1 c b 2 c A2 A3 A4 1 d 4 2 c 3 2 d 4

Model size is max(|T123|, |T234|) = 3

10/22

Disjunctive Datalog

P :

• B∈B

TB(AB) :-

• F∈E

RF (AF )

A1 A3 A2 A4 R12 R23 R34 R41

P : T123(A1, A2, A3) ∨ T234(A2, A3, A4) :- R12(A1, A2), R23(A2, A3), R34(A3, A4), R41(A4, A1).

A1 A2 a 1 b 1 b 2 A2 A3 1 c 1 d 2 c A3 A4 c 3 d 4 d 5 A4 A1 3 b 4 a 4 b A1 A2 A3 A4 a 1 d 4 b 1 c 3 b 1 d 4 b 2 c 3 A1 A2 A3 b 1 c b 2 c A2 A3 A4 1 d 4 2 c 3 2 d 4

Output size is the minimum over all models

10/22

Disjunctive Datalog

P :

• B∈B

TB(AB) :-

• F∈E

RF (AF )

A1 A3 A2 A4 R12 R23 R34 R41

P : T123(A1, A2, A3) ∨ T234(A2, A3, A4) :- R12(A1, A2), R23(A2, A3), R34(A3, A4), R41(A4, A1).

A1 A2 a 1 b 1 b 2 A2 A3 1 c 1 d 2 c A3 A4 c 3 d 4 d 5 A4 A1 3 b 4 a 4 b A1 A2 A3 A4 a 1 d 4 b 1 c 3 b 1 d 4 b 2 c 3 A1 A2 A3 b 1 c b 2 c A2 A3 A4 1 d 4

A minimum-sized model of size 2

10/22

Disjunctive Datalog

P :

• B∈B

TB(AB) :-

• F∈E

RF (AF )

A1 A3 A2 A4 R12 R23 R34 R41

P : T123(A1, A2, A3) ∨ T234(A2, A3, A4) :- R12(A1, A2), R23(A2, A3), R34(A3, A4), R41(A4, A1).

A1 A2 a 1 b 1 b 2 A2 A3 1 c 1 d 2 c A3 A4 c 3 d 4 d 5 A4 A1 3 b 4 a 4 b A1 A2 A3 A4 a 1 d 4 b 1 c 3 b 1 d 4 b 2 c 3 A1 A2 A3 b 1 c b 2 c A2 A3 A4 1 d 4

A minimum-sized model of size 2 ⇒ Output size is 2

11/22

Disjunctive Datalog: Output Size

P :

• B∈B

TB(AB) :-

• F∈E

RF (AF ) |P(D)| def = min

T:T| =P max B∈B |TB|

11/22

Disjunctive Datalog: Output Size

P :

• B∈B

TB(AB) :-

• F∈E

RF (AF ) |P(D)| def = min

T:T| =P max B∈B |TB|

A1 A3 A2 A4 R12 R23 R34 R41

11/22

Disjunctive Datalog: Output Size

P :

• B∈B

TB(AB) :-

• F∈E

RF (AF ) |P(D)| def = min

T:T| =P max B∈B |TB|

A1 A3 A2 A4 R12 R23 R34 R41

D : |R12| ≤ N, |R23| ≤ N, |R34| ≤ N, |R41| ≤ N.

11/22

Disjunctive Datalog: Output Size

P :

• B∈B

TB(AB) :-

• F∈E

RF (AF ) |P(D)| def = min

T:T| =P max B∈B |TB|

A1 A3 A2 A4 R12 R23 R34 R41

D : |R12| ≤ N, |R23| ≤ N, |R34| ≤ N, |R41| ≤ N. ◮ P : T123(A1, A2, A3) ∨ T234(A2, A3, A4) :- R12(A1, A2), R23(A2, A3), R34(A3, A4), R41(A4, A1).

11/22

Disjunctive Datalog: Output Size

P :

• B∈B

TB(AB) :-

• F∈E

RF (AF ) |P(D)| def = min

T:T| =P max B∈B |TB|

A1 A3 A2 A4 R12 R23 R34 R41

D : |R12| ≤ N, |R23| ≤ N, |R34| ≤ N, |R41| ≤ N. ◮ P : T123(A1, A2, A3) ∨ T234(A2, A3, A4) :- R12(A1, A2), R23(A2, A3), R34(A3, A4), R41(A4, A1). |P(D)| ≤ N 3/2 , for all D

11/22

Disjunctive Datalog: Output Size

P :

• B∈B

TB(AB) :-

• F∈E

RF (AF ) |P(D)| def = min

T:T| =P max B∈B |TB|

A1 A3 A2 A4 R12 R23 R34 R41

D : |R12| ≤ N, |R23| ≤ N, |R34| ≤ N, |R41| ≤ N. ◮ P : T123(A1, A2, A3) ∨ T234(A2, A3, A4) :- R12(A1, A2), R23(A2, A3), R34(A3, A4), R41(A4, A1). |P(D)| ≤ N 3/2 , for all D ◮ P ′ : T123(A1, A2, A3) :- R12(A1, A2), R23(A2, A3), R34(A3, A4), R41(A4, A1).

11/22

Disjunctive Datalog: Output Size

P :

• B∈B

TB(AB) :-

• F∈E

RF (AF ) |P(D)| def = min

T:T| =P max B∈B |TB|

A1 A3 A2 A4 R12 R23 R34 R41

D : |R12| ≤ N, |R23| ≤ N, |R34| ≤ N, |R41| ≤ N. ◮ P : T123(A1, A2, A3) ∨ T234(A2, A3, A4) :- R12(A1, A2), R23(A2, A3), R34(A3, A4), R41(A4, A1). |P(D)| ≤ N 3/2 , for all D ◮ P ′ : T123(A1, A2, A3) :- R12(A1, A2), R23(A2, A3), R34(A3, A4), R41(A4, A1). |P ′(D)| = N 2 , for some D

12/22

Disjunctive Datalog: Size Bounds

P :

• B∈B

TB(AB) :-

• F ∈E

RF (AF ) |P(D)|

def

= min

T:T| =P max B∈B|TB|

12/22

Disjunctive Datalog: Size Bounds

P :

• B∈B

TB(AB) :-

• F ∈E

RF (AF ) |P(D)|

def

= min

T:T| =P max B∈B|TB|

Recall that:

◮ HDC is the set of functions h : 2[n] → R+ satisfying the degree

constraints

◮

Γ∗

n

• entropic functions

⊂ Γ

∗ n

• topological closure of Γ∗

n

⊂ Γn

• polymatroids

12/22

Disjunctive Datalog: Size Bounds

P :

• B∈B

TB(AB) :-

• F ∈E

RF (AF ) |P(D)|

def

= min

T:T| =P max B∈B|TB|

Recall that:

◮ HDC is the set of functions h : 2[n] → R+ satisfying the degree

constraints

◮

Γ∗

n

• entropic functions

⊂ Γ

∗ n

• topological closure of Γ∗

n

⊂ Γn

• polymatroids

Theorem

log |P(D)|

12/22

Disjunctive Datalog: Size Bounds

P :

• B∈B

TB(AB) :-

• F ∈E

RF (AF ) |P(D)|

def

= min

T:T| =P max B∈B|TB|

Recall that:

◮ HDC is the set of functions h : 2[n] → R+ satisfying the degree

constraints

◮

Γ∗

n

• entropic functions

⊂ Γ

∗ n

• topological closure of Γ∗

n

⊂ Γn

• polymatroids

Theorem

log |P(D)| ≤ max

h∈Γ

∗ n∩HDC

min

B∈B h(B)

• entropic bound

12/22

Disjunctive Datalog: Size Bounds

P :

• B∈B

TB(AB) :-

• F ∈E

RF (AF ) |P(D)|

def

= min

T:T| =P max B∈B|TB|

Recall that:

◮ HDC is the set of functions h : 2[n] → R+ satisfying the degree

constraints

◮

Γ∗

n

• entropic functions

⊂ Γ

∗ n

• topological closure of Γ∗

n

⊂ Γn

• polymatroids

Theorem

log |P(D)| ≤ max

h∈Γ

∗ n∩HDC

min

B∈B h(B)

• entropic bound

≤ max

h∈Γn∩HDC

min

B∈B h(B)

• polymatroid bound

12/22

Disjunctive Datalog: Size Bounds

P :

• B∈B

TB(AB) :-

• F ∈E

RF (AF ) |P(D)|

def

= min

T:T| =P max B∈B|TB|

Recall that:

◮ HDC is the set of functions h : 2[n] → R+ satisfying the degree

constraints

◮

Γ∗

n

• entropic functions

⊂ Γ

∗ n

• topological closure of Γ∗

n

⊂ Γn

• polymatroids

Theorem

log |P(D)| ≤ max

h∈Γ

∗ n∩HDC

min

B∈B h(B)

• entropic bound

(asymptotically tight!)

≤ max

h∈Γn∩HDC

min

B∈B h(B)

• polymatroid bound

13/22

Table of Contents

Size Bounds for Full Conjunctive Queries Size Bounds for Disjunctive Datalog Algorithms for Disjunctive Datalog Algorithms for Conjunctive Queries

14/22

PANDA (Proof-Assisted eNtropic Degree-Aware)

◮ An algorithm for disjunctive datalog

14/22

PANDA (Proof-Assisted eNtropic Degree-Aware)

◮ An algorithm for disjunctive datalog

◮ computes a model

14/22

PANDA (Proof-Assisted eNtropic Degree-Aware)

◮ An algorithm for disjunctive datalog

◮ computes a model ◮ within the polymatroid bound:

14/22

PANDA (Proof-Assisted eNtropic Degree-Aware)

◮ An algorithm for disjunctive datalog

◮ computes a model ◮ within the polymatroid bound: ◮ the worst-case size of the minimum model.

14/22

PANDA (Proof-Assisted eNtropic Degree-Aware)

◮ An algorithm for disjunctive datalog

◮ computes a model ◮ within the polymatroid bound: ◮ the worst-case size of the minimum model.

◮ Outline

14/22

PANDA (Proof-Assisted eNtropic Degree-Aware)

◮ An algorithm for disjunctive datalog

◮ computes a model ◮ within the polymatroid bound: ◮ the worst-case size of the minimum model.

◮ Outline

◮ Construct a Proof Sequence for the bound.

14/22

PANDA (Proof-Assisted eNtropic Degree-Aware)

◮ An algorithm for disjunctive datalog

◮ computes a model ◮ within the polymatroid bound: ◮ the worst-case size of the minimum model.

◮ Outline

◮ Construct a Proof Sequence for the bound. ◮ Interpret each proof step as an algorithmic step.

15/22

PANDA

◮ Polymatroid bound:

max

h∈Γn∩HDC min B∈B h(B)

15/22

PANDA

◮ Polymatroid bound:

max

h∈Γn∩HDC min B∈B h(B) ◮

max

h∈Γn∩HDC min B∈Bh(B) =

max

h∈Γn∩HDC

• B∈B

λBh(B)

15/22

PANDA

◮ Polymatroid bound:

max

h∈Γn∩HDC min B∈B h(B) ◮

max

h∈Γn∩HDC min B∈Bh(B) =

max

h∈Γn∩HDC

• B∈B

λBh(B)

◮

• B∈B

λB · h(B) ≤

• (X,Y,NY |X)

δY |X · h(Y |X) ≤

log NY |X

15/22

PANDA

◮ Polymatroid bound:

max

h∈Γn∩HDC min B∈B h(B) ◮

max

h∈Γn∩HDC min B∈Bh(B) =

max

h∈Γn∩HDC

• B∈B

λBh(B)

◮

• B∈B

λB · h(B) ≤

• (X,Y,NY |X)

δY |X · h(Y |X) ≤

log NY |X ◮ Proof Sequence

Given X ⊆ Y : h(X) + h(Y |X) → h(Y ) h(Y ) → h(X) + h(Y |X) h(Y ) → h(X) h(Y |X) → h(Y ∪ Z|X ∪ Z)

15/22

PANDA

◮ Polymatroid bound:

max

h∈Γn∩HDC min B∈B h(B) ◮

max

h∈Γn∩HDC min B∈Bh(B) =

max

h∈Γn∩HDC

• B∈B

λBh(B)

◮

• B∈B

λB · h(B) ≤

• (X,Y,NY |X)

δY |X · h(Y |X) ≤

log NY |X ◮ Proof Sequence

Given X ⊆ Y : h(X) + h(Y |X) → h(Y ) (join) h(Y ) → h(X) + h(Y |X) (data partition) h(Y ) → h(X) (projection) h(Y |X) → h(Y ∪ Z|X ∪ Z) (nothing)

◮ Algorithmic Sequence

15/22

PANDA

◮ Polymatroid bound:

max

h∈Γn∩HDC min B∈B h(B) ◮

max

h∈Γn∩HDC min B∈Bh(B) =

max

h∈Γn∩HDC

• B∈B

λBh(B)

◮

• B∈B

λB · h(B) ≤

• (X,Y,NY |X)

δY |X · h(Y |X) ≤

log NY |X ◮ Proof Sequence

Given X ⊆ Y : h(X) + h(Y |X) → h(Y ) (join) h(Y ) → h(X) + h(Y |X) (data partition) h(Y ) → h(X) (projection) h(Y |X) → h(Y ∪ Z|X ∪ Z) (nothing)

◮ Algorithmic Sequence

Theorem

PANDA solves any disjunctive datalog rule P in time within the polymatroid bound of P.

16/22

PANDA: Example

P : T123(A1, A2, A3) ∨ T234(A2, A3, A4) :- R12(A1, A2), R23(A2, A3), R34(A3, A4), R41(A4, A1).

A1 A3 A2 A4 R12 R23 R34 R41

16/22

PANDA: Example

P : T123(A1, A2, A3) ∨ T234(A2, A3, A4) :- R12(A1, A2), R23(A2, A3), R34(A3, A4), R41(A4, A1).

A1 A3 A2 A4 R12 R23 R34 R41

|R12|, |R23|, |R34|, |R41| ≤ N

16/22

PANDA: Example

P : T123(A1, A2, A3) ∨ T234(A2, A3, A4) :- R12(A1, A2), R23(A2, A3), R34(A3, A4), R41(A4, A1).

A1 A3 A2 A4 R12 R23 R34 R41

|R12|, |R23|, |R34|, |R41| ≤ N ⇒ |P| ≤ N 3/2

16/22

PANDA: Example

P : T123(A1, A2, A3) ∨ T234(A2, A3, A4) :- R12(A1, A2), R23(A2, A3), R34(A3, A4), R41(A4, A1).

A1 A3 A2 A4 R12 R23 R34 R41

|R12|, |R23|, |R34|, |R41| ≤ N ⇒ |P| ≤ N 3/2 log |P| ≤ min(h(A1A2A3), h(A2A3A4))

16/22

PANDA: Example

P : T123(A1, A2, A3) ∨ T234(A2, A3, A4) :- R12(A1, A2), R23(A2, A3), R34(A3, A4), R41(A4, A1).

A1 A3 A2 A4 R12 R23 R34 R41

|R12|, |R23|, |R34|, |R41| ≤ N ⇒ |P| ≤ N 3/2 log |P| ≤ min(h(A1A2A3), h(A2A3A4)) ≤ 1 2

• h(A1A2A3) + h(A2A3A4)

16/22

PANDA: Example

P : T123(A1, A2, A3) ∨ T234(A2, A3, A4) :- R12(A1, A2), R23(A2, A3), R34(A3, A4), R41(A4, A1).

A1 A3 A2 A4 R12 R23 R34 R41

|R12|, |R23|, |R34|, |R41| ≤ N ⇒ |P| ≤ N 3/2 log |P| ≤ min(h(A1A2A3), h(A2A3A4)) ≤ 1 2

• h(A1A2A3) + h(A2A3A4)
• ≤

1 2

• h(A1A2) + h(A2A3) + h(A3A4)

16/22

PANDA: Example

P : T123(A1, A2, A3) ∨ T234(A2, A3, A4) :- R12(A1, A2), R23(A2, A3), R34(A3, A4), R41(A4, A1).

A1 A3 A2 A4 R12 R23 R34 R41

|R12|, |R23|, |R34|, |R41| ≤ N ⇒ |P| ≤ N 3/2 log |P| ≤ min(h(A1A2A3), h(A2A3A4)) ≤ 1 2

• h(A1A2A3) + h(A2A3A4)
• ≤

1 2

• h(A1A2) + h(A2A3) + h(A3A4)
• ≤

3 2 log N

16/22

PANDA: Example

P : T123(A1, A2, A3) ∨ T234(A2, A3, A4) :- R12(A1, A2), R23(A2, A3), R34(A3, A4), R41(A4, A1).

A1 A3 A2 A4 R12 R23 R34 R41

|R12|, |R23|, |R34|, |R41| ≤ N ⇒ |P| ≤ N 3/2 log |P| ≤ min(h(A1A2A3), h(A2A3A4)) ≤ 1 2

• h(A1A2A3) + h(A2A3A4)
• ≤

1 2

• h(A1A2) + h(A2A3) + h(A3A4)
• ≤

3 2 log N

h(A1A2) + h(A2A3) + h(A3A4)

16/22

PANDA: Example

P : T123(A1, A2, A3) ∨ T234(A2, A3, A4) :- R12(A1, A2), R23(A2, A3), R34(A3, A4), R41(A4, A1).

A1 A3 A2 A4 R12 R23 R34 R41

|R12|, |R23|, |R34|, |R41| ≤ N ⇒ |P| ≤ N 3/2 log |P| ≤ min(h(A1A2A3), h(A2A3A4)) ≤ 1 2

• h(A1A2A3) + h(A2A3A4)
• ≤

1 2

• h(A1A2) + h(A2A3) + h(A3A4)
• ≤

3 2 log N

h(A1A2) + h(A2A3) + h(A3A4) →

• h(A3A4) → h(A4|A3) + h(A3)

16/22

PANDA: Example

P : T123(A1, A2, A3) ∨ T234(A2, A3, A4) :- R12(A1, A2), R23(A2, A3), R34(A3, A4), R41(A4, A1).

A1 A3 A2 A4 R12 R23 R34 R41

|R12|, |R23|, |R34|, |R41| ≤ N ⇒ |P| ≤ N 3/2 log |P| ≤ min(h(A1A2A3), h(A2A3A4)) ≤ 1 2

• h(A1A2A3) + h(A2A3A4)
• ≤

1 2

• h(A1A2) + h(A2A3) + h(A3A4)
• ≤

3 2 log N

h(A1A2) + h(A2A3) + h(A3A4) →

• h(A3A4) → h(A4|A3) + h(A3)
• h(A1A2) + h(A2A3) + h(A4|A3) + h(A3)

16/22

PANDA: Example

P : T123(A1, A2, A3) ∨ T234(A2, A3, A4) :- R12(A1, A2), R23(A2, A3), R34(A3, A4), R41(A4, A1).

A1 A3 A2 A4 R12 R23 R34 R41

|R12|, |R23|, |R34|, |R41| ≤ N ⇒ |P| ≤ N 3/2 log |P| ≤ min(h(A1A2A3), h(A2A3A4)) ≤ 1 2

• h(A1A2A3) + h(A2A3A4)
• ≤

1 2

• h(A1A2) + h(A2A3) + h(A3A4)
• ≤

3 2 log N

h(A1A2) + h(A2A3) + h(A3A4) →

• h(A3A4) → h(A4|A3) + h(A3)
• h(A1A2) + h(A2A3) + h(A4|A3) + h(A3)

→

• h(A4|A3) → h(A4|A2A3)

16/22

PANDA: Example

P : T123(A1, A2, A3) ∨ T234(A2, A3, A4) :- R12(A1, A2), R23(A2, A3), R34(A3, A4), R41(A4, A1).

A1 A3 A2 A4 R12 R23 R34 R41

|R12|, |R23|, |R34|, |R41| ≤ N ⇒ |P| ≤ N 3/2 log |P| ≤ min(h(A1A2A3), h(A2A3A4)) ≤ 1 2

• h(A1A2A3) + h(A2A3A4)
• ≤

1 2

• h(A1A2) + h(A2A3) + h(A3A4)
• ≤

3 2 log N

h(A1A2) + h(A2A3) + h(A3A4) →

• h(A3A4) → h(A4|A3) + h(A3)
• h(A1A2) + h(A2A3) + h(A4|A3) + h(A3)

→

• h(A4|A3) → h(A4|A2A3)
• h(A1A2) + h(A2A3) + h(A4|A2A3) + h(A3)

16/22

PANDA: Example

P : T123(A1, A2, A3) ∨ T234(A2, A3, A4) :- R12(A1, A2), R23(A2, A3), R34(A3, A4), R41(A4, A1).

A1 A3 A2 A4 R12 R23 R34 R41

|R12|, |R23|, |R34|, |R41| ≤ N ⇒ |P| ≤ N 3/2 log |P| ≤ min(h(A1A2A3), h(A2A3A4)) ≤ 1 2

• h(A1A2A3) + h(A2A3A4)
• ≤

1 2

• h(A1A2) + h(A2A3) + h(A3A4)
• ≤

3 2 log N

h(A1A2) + h(A2A3) + h(A3A4) →

• h(A3A4) → h(A4|A3) + h(A3)
• h(A1A2) + h(A2A3) + h(A4|A3) + h(A3)

→

• h(A4|A3) → h(A4|A2A3)
• h(A1A2) + h(A2A3) + h(A4|A2A3) + h(A3)

→

• h(A2A3) + h(A4|A2A3) → h(A2A3A4)

16/22

PANDA: Example

P : T123(A1, A2, A3) ∨ T234(A2, A3, A4) :- R12(A1, A2), R23(A2, A3), R34(A3, A4), R41(A4, A1).

A1 A3 A2 A4 R12 R23 R34 R41

|R12|, |R23|, |R34|, |R41| ≤ N ⇒ |P| ≤ N 3/2 log |P| ≤ min(h(A1A2A3), h(A2A3A4)) ≤ 1 2

• h(A1A2A3) + h(A2A3A4)
• ≤

1 2

• h(A1A2) + h(A2A3) + h(A3A4)
• ≤

3 2 log N

h(A1A2) + h(A2A3) + h(A3A4) →

• h(A3A4) → h(A4|A3) + h(A3)
• h(A1A2) + h(A2A3) + h(A4|A3) + h(A3)

→

• h(A4|A3) → h(A4|A2A3)
• h(A1A2) + h(A2A3) + h(A4|A2A3) + h(A3)

→

• h(A2A3) + h(A4|A2A3) → h(A2A3A4)
• h(A1A2) + h(A2A3A4) + h(A3)

16/22

PANDA: Example

P : T123(A1, A2, A3) ∨ T234(A2, A3, A4) :- R12(A1, A2), R23(A2, A3), R34(A3, A4), R41(A4, A1).

A1 A3 A2 A4 R12 R23 R34 R41

|R12|, |R23|, |R34|, |R41| ≤ N ⇒ |P| ≤ N 3/2 log |P| ≤ min(h(A1A2A3), h(A2A3A4)) ≤ 1 2

• h(A1A2A3) + h(A2A3A4)
• ≤

1 2

• h(A1A2) + h(A2A3) + h(A3A4)
• ≤

3 2 log N

h(A1A2) + h(A2A3) + h(A3A4) →

• h(A3A4) → h(A4|A3) + h(A3)
• h(A1A2) + h(A2A3) + h(A4|A3) + h(A3)

→

• h(A4|A3) → h(A4|A2A3)
• h(A1A2) + h(A2A3) + h(A4|A2A3) + h(A3)

→

• h(A2A3) + h(A4|A2A3) → h(A2A3A4)
• h(A1A2) + h(A2A3A4) + h(A3)

→

• h(A1A2) → h(A1A2|A3)

16/22

PANDA: Example

P : T123(A1, A2, A3) ∨ T234(A2, A3, A4) :- R12(A1, A2), R23(A2, A3), R34(A3, A4), R41(A4, A1).

A1 A3 A2 A4 R12 R23 R34 R41

|R12|, |R23|, |R34|, |R41| ≤ N ⇒ |P| ≤ N 3/2 log |P| ≤ min(h(A1A2A3), h(A2A3A4)) ≤ 1 2

• h(A1A2A3) + h(A2A3A4)
• ≤

1 2

• h(A1A2) + h(A2A3) + h(A3A4)
• ≤

3 2 log N

h(A1A2) + h(A2A3) + h(A3A4) →

• h(A3A4) → h(A4|A3) + h(A3)
• h(A1A2) + h(A2A3) + h(A4|A3) + h(A3)

→

• h(A4|A3) → h(A4|A2A3)
• h(A1A2) + h(A2A3) + h(A4|A2A3) + h(A3)

→

• h(A2A3) + h(A4|A2A3) → h(A2A3A4)
• h(A1A2) + h(A2A3A4) + h(A3)

→

• h(A1A2) → h(A1A2|A3)
• h(A1A2|A3) + h(A2A3A4) + h(A3)

16/22

PANDA: Example

P : T123(A1, A2, A3) ∨ T234(A2, A3, A4) :- R12(A1, A2), R23(A2, A3), R34(A3, A4), R41(A4, A1).

A1 A3 A2 A4 R12 R23 R34 R41

|R12|, |R23|, |R34|, |R41| ≤ N ⇒ |P| ≤ N 3/2 log |P| ≤ min(h(A1A2A3), h(A2A3A4)) ≤ 1 2

• h(A1A2A3) + h(A2A3A4)
• ≤

1 2

• h(A1A2) + h(A2A3) + h(A3A4)
• ≤

3 2 log N

h(A1A2) + h(A2A3) + h(A3A4) →

• h(A3A4) → h(A4|A3) + h(A3)
• h(A1A2) + h(A2A3) + h(A4|A3) + h(A3)

→

• h(A4|A3) → h(A4|A2A3)
• h(A1A2) + h(A2A3) + h(A4|A2A3) + h(A3)

→

• h(A2A3) + h(A4|A2A3) → h(A2A3A4)
• h(A1A2) + h(A2A3A4) + h(A3)

→

• h(A1A2) → h(A1A2|A3)
• h(A1A2|A3) + h(A2A3A4) + h(A3)

→

• h(A1A2|A3) + h(A3) → h(A1A2A3)

16/22

PANDA: Example

P : T123(A1, A2, A3) ∨ T234(A2, A3, A4) :- R12(A1, A2), R23(A2, A3), R34(A3, A4), R41(A4, A1).

A1 A3 A2 A4 R12 R23 R34 R41

|R12|, |R23|, |R34|, |R41| ≤ N ⇒ |P| ≤ N 3/2 log |P| ≤ min(h(A1A2A3), h(A2A3A4)) ≤ 1 2

• h(A1A2A3) + h(A2A3A4)
• ≤

1 2

• h(A1A2) + h(A2A3) + h(A3A4)
• ≤

3 2 log N

h(A1A2) + h(A2A3) + h(A3A4) →

• h(A3A4) → h(A4|A3) + h(A3)
• h(A1A2) + h(A2A3) + h(A4|A3) + h(A3)

→

• h(A4|A3) → h(A4|A2A3)
• h(A1A2) + h(A2A3) + h(A4|A2A3) + h(A3)

→

• h(A2A3) + h(A4|A2A3) → h(A2A3A4)
• h(A1A2) + h(A2A3A4) + h(A3)

→

• h(A1A2) → h(A1A2|A3)
• h(A1A2|A3) + h(A2A3A4) + h(A3)

→

• h(A1A2|A3) + h(A3) → h(A1A2A3)
• h(A1A2A3) + h(A2A3A4)

17/22

PANDA: Example

P : T123(A1, A2, A3) ∨ T234(A2, A3, A4) :- R12(A1, A2), R23(A2, A3), R34(A3, A4), R41(A4, A1).

A1 A3 A2 A4 R12 R23 R34 R41

|R12|, |R23|, |R34|, |R41| ≤ N ⇒ |P| ≤ N 3/2

h(A3A4) → h(A4|A3) + h(A3) h(A4|A3) → h(A4|A2A3) h(A2A3) + h(A4|A2A3) → h(A2A3A4) h(A1A2) → h(A1A2|A3) h(A1A2|A3) + h(A3) → h(A1A2A3)

17/22

PANDA: Example

P : T123(A1, A2, A3) ∨ T234(A2, A3, A4) :- R12(A1, A2), R23(A2, A3), R34(A3, A4), R41(A4, A1).

A1 A3 A2 A4 R12 R23 R34 R41

|R12|, |R23|, |R34|, |R41| ≤ N ⇒ |P| ≤ N 3/2

h(A3A4) → h(A4|A3) + h(A3) h(A4|A3) → h(A4|A2A3) h(A2A3) + h(A4|A2A3) → h(A2A3A4) h(A1A2) → h(A1A2|A3) h(A1A2|A3) + h(A3) → h(A1A2A3) R34(A3, A4) → R(l)

34 (A3, A4), R(h) 3

(A3)

17/22

PANDA: Example

P : T123(A1, A2, A3) ∨ T234(A2, A3, A4) :- R12(A1, A2), R23(A2, A3), R34(A3, A4), R41(A4, A1).

A1 A3 A2 A4 R12 R23 R34 R41

|R12|, |R23|, |R34|, |R41| ≤ N ⇒ |P| ≤ N 3/2

h(A3A4) → h(A4|A3) + h(A3) h(A4|A3) → h(A4|A2A3) h(A2A3) + h(A4|A2A3) → h(A2A3A4) h(A1A2) → h(A1A2|A3) h(A1A2|A3) + h(A3) → h(A1A2A3) R34(A3, A4) → R(l)

34 (A3, A4), R(h) 3

(A3) R(l)

34 (A3, A4) → R(l) 34 (A3, A4)

17/22

PANDA: Example

P : T123(A1, A2, A3) ∨ T234(A2, A3, A4) :- R12(A1, A2), R23(A2, A3), R34(A3, A4), R41(A4, A1).

A1 A3 A2 A4 R12 R23 R34 R41

|R12|, |R23|, |R34|, |R41| ≤ N ⇒ |P| ≤ N 3/2

h(A3A4) → h(A4|A3) + h(A3) h(A4|A3) → h(A4|A2A3) h(A2A3) + h(A4|A2A3) → h(A2A3A4) h(A1A2) → h(A1A2|A3) h(A1A2|A3) + h(A3) → h(A1A2A3) R34(A3, A4) → R(l)

34 (A3, A4), R(h) 3

(A3) R(l)

34 (A3, A4) → R(l) 34 (A3, A4)

R23(A2, A3) ✶ R(l)

34 (A3, A4) → T234(A2, A3, A4)

17/22

PANDA: Example

P : T123(A1, A2, A3) ∨ T234(A2, A3, A4) :- R12(A1, A2), R23(A2, A3), R34(A3, A4), R41(A4, A1).

A1 A3 A2 A4 R12 R23 R34 R41

|R12|, |R23|, |R34|, |R41| ≤ N ⇒ |P| ≤ N 3/2

h(A3A4) → h(A4|A3) + h(A3) h(A4|A3) → h(A4|A2A3) h(A2A3) + h(A4|A2A3) → h(A2A3A4) h(A1A2) → h(A1A2|A3) h(A1A2|A3) + h(A3) → h(A1A2A3) R34(A3, A4) → R(l)

34 (A3, A4), R(h) 3

(A3) R(l)

34 (A3, A4) → R(l) 34 (A3, A4)

R23(A2, A3) ✶ R(l)

34 (A3, A4) → T234(A2, A3, A4)

R12(A1, A2) → R12(A1, A2)

17/22

PANDA: Example

P : T123(A1, A2, A3) ∨ T234(A2, A3, A4) :- R12(A1, A2), R23(A2, A3), R34(A3, A4), R41(A4, A1).

A1 A3 A2 A4 R12 R23 R34 R41

|R12|, |R23|, |R34|, |R41| ≤ N ⇒ |P| ≤ N 3/2

h(A3A4) → h(A4|A3) + h(A3) h(A4|A3) → h(A4|A2A3) h(A2A3) + h(A4|A2A3) → h(A2A3A4) h(A1A2) → h(A1A2|A3) h(A1A2|A3) + h(A3) → h(A1A2A3) R34(A3, A4) → R(l)

34 (A3, A4), R(h) 3

(A3) R(l)

34 (A3, A4) → R(l) 34 (A3, A4)

R23(A2, A3) ✶ R(l)

34 (A3, A4) → T234(A2, A3, A4)

R12(A1, A2) → R12(A1, A2) R12(A1, A2) ✶ R(h)

3

(A3) → T123(A1, A2, A3)

18/22

Table of Contents

Size Bounds for Full Conjunctive Queries Size Bounds for Disjunctive Datalog Algorithms for Disjunctive Datalog Algorithms for Conjunctive Queries

19/22

Beyond Worst-case Optimality

◮ Output-sensitive algorithms

19/22

Beyond Worst-case Optimality

◮ Output-sensitive algorithms

˜ O

• N d

+ |output|

19/22

Beyond Worst-case Optimality

◮ Output-sensitive algorithms

˜ O

• N d

+ |output|

• Intrinsic Cost

19/22

Beyond Worst-case Optimality

◮ Output-sensitive algorithms

˜ O

• N d

+ |output|

• Intrinsic Cost

Output Cost

19/22

Beyond Worst-case Optimality

◮ Output-sensitive algorithms

˜ O

• N d

+ |output|

• Intrinsic Cost

Output Cost

◮ Submodular width as a candidate for d

19/22

Beyond Worst-case Optimality

◮ Output-sensitive algorithms

˜ O

• N d

+ |output|

• Intrinsic Cost

Output Cost

◮ Submodular width as a candidate for d

[Marx JACM’13]

19/22

Beyond Worst-case Optimality

◮ Output-sensitive algorithms

˜ O

• N d

+ |output|

• Intrinsic Cost

Output Cost

◮ Submodular width as a candidate for d

[Marx JACM’13] FPT ⇔ Bounded subw(Q)

19/22

Beyond Worst-case Optimality

◮ Output-sensitive algorithms

˜ O

• N d

+ |output|

• Intrinsic Cost

Output Cost

◮ Submodular width as a candidate for d

[Marx JACM’13] FPT ⇔ Bounded subw(Q) Boolean Q ⇒ ˜ O

• Nsubw(Q) ×c

19/22

Beyond Worst-case Optimality

◮ Output-sensitive algorithms

˜ O

• N d

+ |output|

• Intrinsic Cost

Output Cost

◮ Submodular width as a candidate for d

[Marx JACM’13] FPT ⇔ Bounded subw(Q) Boolean Q ⇒ ˜ O

• Nsubw(Q) ×c

◮ Our goals

19/22

Beyond Worst-case Optimality

◮ Output-sensitive algorithms

˜ O

• N d

+ |output|

• Intrinsic Cost

Output Cost

◮ Submodular width as a candidate for d

[Marx JACM’13] FPT ⇔ Bounded subw(Q) Boolean Q ⇒ ˜ O

• Nsubw(Q) ×c

◮ Our goals

Any Q ⇒ ˜ O

• N da- subw(Q) ×1 + |output|

20/22

Submodular Width

fhtw(Q)

def

= min

(T,χ)

max

t∈V (T)

ρ∗(χ(t))

20/22

Submodular Width

fhtw(Q)

def

= min

(T,χ)

max

t∈V (T)

ρ∗(χ(t)) = min

(T,χ)

max

t∈V (T)

max

h∈ED∩Γnh(χ(t))

20/22

Submodular Width

fhtw(Q)

def

= min

(T,χ)

max

t∈V (T)

ρ∗(χ(t)) = min

(T,χ)

max

t∈V (T)

max

h∈ED∩Γnh(χ(t))

= min

(T,χ)

max

h∈ED∩Γn

max

t∈V (T)

h(χ(t))

20/22

Submodular Width

fhtw(Q)

def

= min

(T,χ)

max

t∈V (T)

ρ∗(χ(t)) = min

(T,χ)

max

t∈V (T)

max

h∈ED∩Γnh(χ(t))

= min

(T,χ)

max

h∈ED∩Γn

max

t∈V (T)

h(χ(t)) subw(Q)

def

= max

h∈ED∩Γn

min

(T,χ)

max

t∈V (T)

h(χ(t))

20/22

Submodular Width

fhtw(Q)

def

= min

(T,χ)

max

t∈V (T)

ρ∗(χ(t)) = min

(T,χ)

max

t∈V (T)

max

h∈ED∩Γnh(χ(t))

= min

(T,χ)

max

h∈ED∩Γn

max

t∈V (T)

h(χ(t)) subw(Q)

def

= max

h∈ED∩Γn

min

(T,χ)

max

t∈V (T)

h(χ(t)) subw(Q) ≤ fhtw(Q)

21/22

Submodular Width: Example

fhtw(Q)

def

= min

(T,χ) max t∈V (T )ρ∗(χ(t)),

subw(Q)

def

= max

h∈ED∩Γn min (T,χ) max t∈V (T )h(χ(t))

Q(A1, A2, A3, A4) :- R12(A1, A2), R23(A2, A3), R34(A3, A4), R41(A4, A1).

A1 A3 A2 A4 R12 R23 R34 R41

21/22

Submodular Width: Example

fhtw(Q)

def

= min

(T,χ) max t∈V (T )ρ∗(χ(t)),

subw(Q)

def

= max

h∈ED∩Γn min (T,χ) max t∈V (T )h(χ(t))

Q(A1, A2, A3, A4) :- R12(A1, A2), R23(A2, A3), R34(A3, A4), R41(A4, A1).

A1 A3 A2 A4 R12 R23 R34 R41 A1 A2 A3 A1 A4 A3

21/22

Submodular Width: Example

fhtw(Q)

def

= min

(T,χ) max t∈V (T )ρ∗(χ(t)),

subw(Q)

def

= max

h∈ED∩Γn min (T,χ) max t∈V (T )h(χ(t))

Q(A1, A2, A3, A4) :- R12(A1, A2), R23(A2, A3), R34(A3, A4), R41(A4, A1).

A1 A3 A2 A4 R12 R23 R34 R41 A1 A2 A3 A1 A4 A3 A2 A3 A4 A2 A1 A4

21/22

Submodular Width: Example

fhtw(Q)

def

= min

(T,χ) max t∈V (T )ρ∗(χ(t)),

subw(Q)

def

= max

h∈ED∩Γn min (T,χ) max t∈V (T )h(χ(t))

Q(A1, A2, A3, A4) :- R12(A1, A2), R23(A2, A3), R34(A3, A4), R41(A4, A1).

fhtw(Q) = 2 A1 A3 A2 A4 R12 R23 R34 R41 A1 A2 A3 A1 A4 A3 A2 A3 A4 A2 A1 A4

21/22

Submodular Width: Example

fhtw(Q)

def

= min

(T,χ) max t∈V (T )ρ∗(χ(t)),

subw(Q)

def

= max

h∈ED∩Γn min (T,χ) max t∈V (T )h(χ(t))

Q(A1, A2, A3, A4) :- R12(A1, A2), R23(A2, A3), R34(A3, A4), R41(A4, A1).

fhtw(Q) = 2 subw(Q) = 3/2 A1 A3 A2 A4 R12 R23 R34 R41 A1 A2 A3 A1 A4 A3 A2 A3 A4 A2 A1 A4

21/22

Submodular Width: Example

fhtw(Q)

def

= min

(T,χ) max t∈V (T )ρ∗(χ(t)),

subw(Q)

def

= max

h∈ED∩Γn min (T,χ) max t∈V (T )h(χ(t))

Q(A1, A2, A3, A4) :- R12(A1, A2), R23(A2, A3), R34(A3, A4), R41(A4, A1).

fhtw(Q) = 2 subw(Q) = 3/2 min( max(h(A1A2A3), h(A3A4A1)), max(h(A4A1A2), h(A2A3A4))) ≤ 3/2 A1 A3 A2 A4 R12 R23 R34 R41 A1 A2 A3 A1 A4 A3 A2 A3 A4 A2 A1 A4

21/22

Submodular Width: Example

fhtw(Q)

def

= min

(T,χ) max t∈V (T )ρ∗(χ(t)),

subw(Q)

def

= max

h∈ED∩Γn min (T,χ) max t∈V (T )h(χ(t))

Q(A1, A2, A3, A4) :- R12(A1, A2), R23(A2, A3), R34(A3, A4), R41(A4, A1).

fhtw(Q) = 2 subw(Q) = 3/2 min( max(h(A1A2A3), h(A3A4A1)), max(h(A4A1A2), h(A2A3A4))) ≤ 3/2 min(h(A1A2A3), h(A4A1A2)) ≤ 3/2 min(h(A1A2A3), h(A2A3A4)) ≤ 3/2 min(h(A3A4A1), h(A4A1A2)) ≤ 3/2 min(h(A3A4A1), h(A2A3A4)) ≤ 3/2 A1 A3 A2 A4 R12 R23 R34 R41 A1 A2 A3 A1 A4 A3 A2 A3 A4 A2 A1 A4

21/22

Submodular Width: Example

fhtw(Q)

def

= min

(T,χ) max t∈V (T )ρ∗(χ(t)),

subw(Q)

def

= max

h∈ED∩Γn min (T,χ) max t∈V (T )h(χ(t))

Q(A1, A2, A3, A4) :- R12(A1, A2), R23(A2, A3), R34(A3, A4), R41(A4, A1).

T123(A1, A2, A3) ∨ T412(A4, A1, A2) :- R12(A1, A2), R23(A2, A3), R34(A3, A4), R41(A4, A1). T123(A1, A2, A3) ∨ T234(A2, A3, A4) :- R12(A1, A2), R23(A2, A3), R34(A3, A4), R41(A4, A1). T341(A3, A4, A1) ∨ T412(A4, A1, A2) :- R12(A1, A2), R23(A2, A3), R34(A3, A4), R41(A4, A1). T341(A3, A4, A1) ∨ T234(A2, A3, A4) :- R12(A1, A2), R23(A2, A3), R34(A3, A4), R41(A4, A1). A1 A3 A2 A4 R12 R23 R34 R41 A1 A2 A3 A1 A4 A3 A2 A3 A4 A2 A1 A4

22/22

Summary of Bounds

X Y Z Γ

∗ n

Γn SAn HDC HCC ED · log N VD · log N

LogSizeBoundX∩Y (Q)

log2 VB(Q) log2 VB(Q) log2 VB(Q) ρ(Q) · log2 N ρ∗(Q) · log2 N ρ∗(Q) · log2 N ρ(Q, (NF)F∈E) log2 AGM(Q) log2 AGM(Q) DAPB(Q) DAEB(Q)

MinimaxwidthX∩Y (Q)

tw(Q) + 1 tw(Q) + 1 tw(Q) + 1 ghtw(Q) fhtw(Q) fhtw(Q) da-fhtw(Q) eda-fhtw(Q)

MaximinwidthX∩Y (Q)

tw(Q) + 1 tw(Q) + 1 tw(Q) + 1 ghtw(Q) subw(Q) da-subw(Q) eda-subw(Q)