[PPT] - XPath Evaluation in Linear Time Mikoaj Bojaczyk, Pawe Parys Warsaw PowerPoint Presentation

SLIDE 1

XPath Evaluation in Linear Time

Mikołaj Bojańczyk, Paweł Parys Warsaw University

SLIDE 2

find the nodes in an XML document d that satisfy an XPath unary query q.

We consider a fragment of XPath called FOXPath.

Previous algorithms:

– exponential in the document size – quadratic in the document size (Benedikt, Koch)

We give two algorithms:

– linear in the document size: O( 2|q|·|d| ) – good combined complexity: O(|q|·|d|·log(|d|))

Goal:

SLIDE 3

XML Document

SLIDE 4

XML Document

document node, i.e. opening tag attribute name attribute name

SLIDE 5

XML Document

document node, i.e. opening tag attribute name attribute name

SLIDE 6

XML Document XPath query: “select teams that share a player with another team”

document node, i.e. opening tag attribute name attribute name

SLIDE 7

XML Document XPath query: “select teams that share a player with another team”

document node, i.e. opening tag attribute name attribute name

SLIDE 8

XML Document XPath query: “select teams that share a player with another team”

document node, i.e. opening tag attribute name attribute name

SLIDE 9

XML Document XPath query: “select teams that share a player with another team”

document node, i.e. opening tag attribute name attribute name

SLIDE 10

XML Document XPath query: “select teams that share a player with another team”

document node, i.e. opening tag attribute name attribute name

SLIDE 11

XML Document XPath query: “select teams that share a player with another team”

document node, i.e. opening tag attribute name attribute name

SLIDE 12

XML Document XPath query: “select teams that share a player with another team”

document node, i.e. opening tag attribute name attribute name

SLIDE 13

XML Document XPath query: “select teams that share a player with another team”

document node, i.e. opening tag attribute name attribute name

SLIDE 14

XML Document XPath query: “select teams that share a player with another team”

child[player]@name = sibling[team]/child[player]@name document node, i.e. opening tag attribute name attribute name

SLIDE 15

FOXPath Programs - select node pairs.

child, parent, next-sibling, prev-sibling, descendant, etc.
any regular expression on programs is a program, e.g. child*
if t is a test, then [t] is a program that selects (x,x) if node x satisfies t

Tests - select single nodes.

any tag name a is a test that selects nodes with this tag.
boolean operations: or, and, not
if p,q are programs, and a,b attribute names, then p@a=q@b and p@a ≠ p@b are tests.

SLIDE 16

FOXPath Programs - select node pairs.

child, parent, next-sibling, prev-sibling, descendant, etc.
any regular expression on programs is a program, e.g. child*
if t is a test, then [t] is a program that selects (x,x) if node x satisfies t

Tests - select single nodes.

any tag name a is a test that selects nodes with this tag.
boolean operations: or, and, not
if p,q are programs, and a,b attribute names, then p@a=q@b and p@a ≠ p@b are tests.

x

A node x is selected by p@a=q@b if

SLIDE 17

FOXPath Programs - select node pairs.

child, parent, next-sibling, prev-sibling, descendant, etc.
any regular expression on programs is a program, e.g. child*
if t is a test, then [t] is a program that selects (x,x) if node x satisfies t

Tests - select single nodes.

any tag name a is a test that selects nodes with this tag.
boolean operations: or, and, not
if p,q are programs, and a,b attribute names, then p@a=q@b and p@a ≠ p@b are tests.

x

A node x is selected by p@a=q@b if there are some nodes y and z such that

SLIDE 18

FOXPath Programs - select node pairs.

child, parent, next-sibling, prev-sibling, descendant, etc.
any regular expression on programs is a program, e.g. child*
if t is a test, then [t] is a program that selects (x,x) if node x satisfies t

Tests - select single nodes.

any tag name a is a test that selects nodes with this tag.
boolean operations: or, and, not
if p,q are programs, and a,b attribute names, then p@a=q@b and p@a ≠ p@b are tests.

x p y

A node x is selected by p@a=q@b if the pair (x,y) is selected by p. there are some nodes y and z such that

SLIDE 19

FOXPath Programs - select node pairs.

child, parent, next-sibling, prev-sibling, descendant, etc.
any regular expression on programs is a program, e.g. child*
if t is a test, then [t] is a program that selects (x,x) if node x satisfies t

Tests - select single nodes.

any tag name a is a test that selects nodes with this tag.
boolean operations: or, and, not
if p,q are programs, and a,b attribute names, then p@a=q@b and p@a ≠ p@b are tests.

x p y q z

A node x is selected by p@a=q@b if the pair (x,y) is selected by p. the pair (x,z) is selected by q. there are some nodes y and z such that

SLIDE 20

FOXPath Programs - select node pairs.

child, parent, next-sibling, prev-sibling, descendant, etc.
any regular expression on programs is a program, e.g. child*
if t is a test, then [t] is a program that selects (x,x) if node x satisfies t

Tests - select single nodes.

any tag name a is a test that selects nodes with this tag.
boolean operations: or, and, not
if p,q are programs, and a,b attribute names, then p@a=q@b and p@a ≠ p@b are tests.

x p y q z

A node x is selected by p@a=q@b if the pair (x,y) is selected by p. the pair (x,z) is selected by q. the attribute values y@a and z@b are the same. there are some nodes y and z such that

SLIDE 21

FOXPath Programs - select node pairs.

child, parent, next-sibling, prev-sibling, descendant, etc.
any regular expression on programs is a program, e.g. child*
if t is a test, then [t] is a program that selects (x,x) if node x satisfies t

Tests - select single nodes.

any tag name a is a test that selects nodes with this tag.
boolean operations: or, and, not
if p,q are programs, and a,b attribute names, then p@a=q@b and p@a ≠ p@b are tests.

x p y q z

A node x is selected by p@a=q@b if the pair (x,y) is selected by p. the pair (x,z) is selected by q. the attribute values y@a and z@b are the same. there are some nodes y and z such that

m. Let t be an FOXPath test and d an XML document. e set of nodes in d selected by t can be computed in time O(|d|2|t|) as well as in time O(|d|log(|d|)|t|2)

SLIDE 22

find nodes that satisfy p@a = q@a

1. decompose trees into classes

(class = set of nodes with same value of atribute a)

High level overview

SLIDE 23

find nodes that satisfy p@a = q@a

1. decompose trees into classes

(class = set of nodes with same value of atribute a)

High level overview

SLIDE 24

find nodes that satisfy p@a = q@a

1. decompose trees into classes

(class = set of nodes with same value of atribute a)

High level overview

SLIDE 25

find nodes that satisfy p@a = q@a

1. decompose trees into classes

(class = set of nodes with same value of atribute a)

High level overview

SLIDE 26

find nodes that satisfy p@a = q@a

1. decompose trees into classes

(class = set of nodes with same value of atribute a)

High level overview

SLIDE 27

find nodes that satisfy p@a = q@a

1. decompose trees into classes

(class = set of nodes with same value of atribute a)

High level overview

SLIDE 28

find nodes that satisfy p@a = q@a

1. decompose trees into classes

(class = set of nodes with same value of atribute a)

High level overview

2. for each class, find nodes that are witnessed by

that class

SLIDE 29

find nodes that satisfy p@a = q@a

1. decompose trees into classes

(class = set of nodes with same value of atribute a)

High level overview

2. for each class, find nodes that are witnessed by

that class

SLIDE 30

find nodes that satisfy p@a = q@a

1. decompose trees into classes

(class = set of nodes with same value of atribute a)

High level overview

2. for each class, find nodes that are witnessed by

that class p@a q@a

SLIDE 31

find nodes that satisfy p@a = q@a

1. decompose trees into classes

(class = set of nodes with same value of atribute a)

High level overview

2. for each class, find nodes that are witnessed by

that class p@a q@a

SLIDE 32

find nodes that satisfy p@a = q@a

1. decompose trees into classes

(class = set of nodes with same value of atribute a)

High level overview

2. for each class, find nodes that are witnessed by

that class p@a q@a p@a q@a

SLIDE 33

find nodes that satisfy p@a = q@a

1. decompose trees into classes

(class = set of nodes with same value of atribute a)

High level overview

2. for each class, find nodes that are witnessed by

that class p@a q@a p@a q@a

Goal: avoid repetition

– do a constant number of operations per node – or at least logarithmic

SLIDE 34

find nodes that satisfy p@a = q@a

1. decompose trees into classes

(class = set of nodes with same value of atribute a)

High level overview

2. for each class, find nodes that are witnessed by

that class p@a q@a p@a q@a

Goal: avoid repetition

– do a constant number of operations per node – or at least logarithmic

Using Simon decompositions, a fancy algebraic result

SLIDE 35

What is the Simon decomposition?

SLIDE 36

L a regular word language. Do a linear time precomputation on w=a1a2...an For any infix, membership ai...aj L can be computed in time log n

SLIDE 37

L a regular word language. Do a linear time precomputation on w=a1a2...an For any infix, membership ai...aj L can be computed in time log n

A an automaton recognizing L, and Q its state space. Each word u induces a tranformation on states .

SLIDE 38

a b b a a b b a

L a regular word language. Do a linear time precomputation on w=a1a2...an For any infix, membership ai...aj L can be computed in time log n

A an automaton recognizing L, and Q its state space. Each word u induces a tranformation on states .

SLIDE 39

a b b a a b b a

L a regular word language. Do a linear time precomputation on w=a1a2...an For any infix, membership ai...aj L can be computed in time log n

½

A an automaton recognizing L, and Q its state space. Each word u induces a tranformation on states .

SLIDE 40

a b b a a b b a

L a regular word language. Do a linear time precomputation on w=a1a2...an For any infix, membership ai...aj L can be computed in time log n

½ ¼

A an automaton recognizing L, and Q its state space. Each word u induces a tranformation on states .

SLIDE 41

a b b a a b b a

L a regular word language. Do a linear time precomputation on w=a1a2...an For any infix, membership ai...aj L can be computed in time log n

½ ¼ ⅛

A an automaton recognizing L, and Q its state space. Each word u induces a tranformation on states .

SLIDE 42

a b b a a b b a

L a regular word language. Do a linear time precomputation on w=a1a2...an For any infix, membership ai...aj L can be computed in time log n

½ ¼ ⅛

A an automaton recognizing L, and Q its state space. Each word u induces a tranformation on states .

SLIDE 43

a b b a a b b a

L a regular word language. Do a linear time precomputation on w=a1a2...an For any infix, membership ai...aj L can be computed in time log n

½ ¼ ⅛

A an automaton recognizing L, and Q its state space. Each word u induces a tranformation on states .

SLIDE 44

a b b a a b b a

L a regular word language. Do a linear time precomputation on w=a1a2...an For any infix, membership ai...aj L can be computed in time log n

½ ¼ ⅛

A an automaton recognizing L, and Q its state space. Each word u induces a tranformation on states .

SLIDE 45

a b b a a b b a

L a regular word language. Do a linear time precomputation on w=a1a2...an For any infix, membership ai...aj L can be computed in time log n

½ ¼ ⅛

A an automaton recognizing L, and Q its state space. Each word u induces a tranformation on states .

j i

SLIDE 46

a b b a a b b a

L a regular word language. Do a linear time precomputation on w=a1a2...an For any infix, membership ai...aj L can be computed in time log n

½ ¼ ⅛

A an automaton recognizing L, and Q its state space. Each word u induces a tranformation on states .

j i

SLIDE 47

a b b a a b b a

L a regular word language. Do a linear time precomputation on w=a1a2...an For any infix, membership ai...aj L can be computed in time log n

½ ¼ ⅛

A an automaton recognizing L, and Q its state space. Each word u induces a tranformation on states .

j i Big news: Simon decomposition does this with constant depth!

SLIDE 48

Back to XPath evaluation...

SLIDE 49

To simplify, consider a special case of XPath:

SLIDE 50

abaabaabbabababbababbabbababbababbabbbababbbabbababbabbbabaababbabba

To simplify, consider a special case of XPath:

words not trees

SLIDE 51

abaabaabbabababbababbabbababbababbabbbababbbabbababbabbbabaababbabba

To simplify, consider a special case of XPath:

words not trees
a test p@a =q@a

SLIDE 52

abaabaabbabababbababbabbababbababbabbbababbbabbababbabbbabaababbabba

To simplify, consider a special case of XPath:

words not trees
a test p@a =q@a

SLIDE 53

abaabaabbabababbababbabbababbababbabbbababbbabbababbabbbabaababbabba

To simplify, consider a special case of XPath:

words not trees
a test p@a =q@a

p q

SLIDE 54

abaabaabbabababbababbabbababbababbabbbababbbabbababbabbbabaababbabba

To simplify, consider a special case of XPath:

words not trees
a test p@a =q@a
each attribute value appears exactly twice

p q

SLIDE 55

abaabaabbabababbababbabbababbababbabbbababbbabbababbabbbabaababbabba

To simplify, consider a special case of XPath:

words not trees
a test p@a =q@a
each attribute value appears exactly twice

p q

SLIDE 56

abaabaabbabababbababbabbababbababbabbbababbbabbababbabbbabaababbabba

To simplify, consider a special case of XPath:

words not trees
a test p@a =q@a
programs p,q have no nested tests, except label tests
each attribute value appears exactly twice

p q

SLIDE 57

abaabaabbabababbababbabbababbababbabbbababbbabbababbabbbabaababbabba

To simplify, consider a special case of XPath:

words not trees
a test p@a =q@a
programs p,q have no nested tests, except label tests
p only goes left, q only goes right
each attribute value appears exactly twice

p q

SLIDE 58

abaabaabbabababbababbabbababbababbabbbababbbabbababbabbbabaababbabba

To simplify, consider a special case of XPath:

words not trees
a test p@a =q@a
programs p,q have no nested tests, except label tests
p only goes left, q only goes right
each attribute value appears exactly twice

p q

Problem. Fix a set of tag names and regular word languages
Input. matching
Output. Set of nodes x with

a1 · · · an ∈ Σ∗ E ⊆ {1, . . . , n}2 Σ L, K ⊆ Σ∗ L K x ∈ E

SLIDE 59

abaabaabbabababbababbabbababbababbabbbababbbabbababbabbbabaababbabba

Problem. Fix a set of tag names and regular word languages
Input. matching
Output. Set of nodes x with

a1 · · · an ∈ Σ∗ E ⊆ {1, . . . , n}2 Σ L, K ⊆ Σ∗

SLIDE 60

abaabaabbabababbababbabbababbababbabbbababbbabbababbabbbabaababbabba

Problem. Fix a set of tag names and regular word languages
Input. matching
Output. Set of nodes x with

a1 · · · an ∈ Σ∗ E ⊆ {1, . . . , n}2 Σ L, K ⊆ Σ∗ Naive algorithm. For every match (y, z) ∈ E ∈ E y z

SLIDE 61

abaabaabbabababbababbabbababbababbabbbababbbabbababbabbbabaababbabba

Problem. Fix a set of tag names and regular word languages
Input. matching
Output. Set of nodes x with

a1 · · · an ∈ Σ∗ E ⊆ {1, . . . , n}2 Σ L, K ⊆ Σ∗ Naive algorithm. For every match (y, z) ∈ E ∈ E y z Find nodes x such that w[y..x] ∈ L w[x..z] ∈ K

SLIDE 62

abaabaabbabababbababbabbababbababbabbbababbbabbababbabbbabaababbabba

Problem. Fix a set of tag names and regular word languages
Input. matching
Output. Set of nodes x with

a1 · · · an ∈ Σ∗ E ⊆ {1, . . . , n}2 Σ L, K ⊆ Σ∗ Naive algorithm. For every match (y, z) ∈ E ∈ E y z x x x x x x x Find nodes x such that w[y..x] ∈ L w[x..z] ∈ K

SLIDE 63

abaabaabbabababbababbabbababbababbabbbababbbabbababbabbbabaababbabba

Problem. Fix a set of tag names and regular word languages
Input. matching
Output. Set of nodes x with

a1 · · · an ∈ Σ∗ E ⊆ {1, . . . , n}2 Σ L, K ⊆ Σ∗ Naive algorithm. For every match (y, z) ∈ E ∈ E y z x x x x x x x Find nodes x such that w[y..x] ∈ L w[x..z] ∈ K time O(n2)

SLIDE 64

abaabaabbabababbababbabbababbababbabbbababbbabbababbabbbabaababbabba

Problem. Fix a set of tag names and regular word languages
Input. matching
Output. Set of nodes x with

a1 · · · an ∈ Σ∗ E ⊆ {1, . . . , n}2 Σ L, K ⊆ Σ∗ Naive algorithm. For every match (y, z) ∈ E ∈ E y z x x x x x x x Find nodes x such that w[y..x] ∈ L w[x..z] ∈ K Divide and conquer dynamic algorithm. Solves the problem in time O(n log(n)) time O(n2)

SLIDE 65

abaabaabbabababbababbabbababbababbabbbababbbabbababbabbbabaababbabba

Problem. Fix a set of tag names and regular word languages
Input. matching
Output. Set of nodes x with

a1 · · · an ∈ Σ∗ E ⊆ {1, . . . , n}2 Σ L, K ⊆ Σ∗ Naive algorithm. For every match (y, z) ∈ E ∈ E y z x x x x x x x Find nodes x such that w[y..x] ∈ L w[x..z] ∈ K Divide and conquer dynamic algorithm. Solves the problem in time O(n log(n)) An algorithm that uses the Simon decomposition Solves the problem in time O(n) time O(n2)

SLIDE 66

abaabaabbabababbababbabbababbababbabbbababbbabbababbabbbabaababbabba

Problem. Fix a set of tag names and regular word languages
Input. matching
Output. Set of nodes x with

a1 · · · an ∈ Σ∗ E ⊆ {1, . . . , n}2 Σ L, K ⊆ Σ∗ Divide and conquer dynamic algorithm.

SLIDE 67

abaabaabbabababbababbabbababbababbabbbababbbabbababbabbbabaababbabba

Problem. Fix a set of tag names and regular word languages
Input. matching
Output. Set of nodes x with

a1 · · · an ∈ Σ∗ E ⊆ {1, . . . , n}2 Σ L, K ⊆ Σ∗ Divide and conquer dynamic algorithm.

SLIDE 68

abaabaabbabababbababbabbababbababbabbbababbbabbababbabbbabaababbabba

Problem. Fix a set of tag names and regular word languages
Input. matching
Output. Set of nodes x with

a1 · · · an ∈ Σ∗ E ⊆ {1, . . . , n}2 Σ L, K ⊆ Σ∗ ∈ E y z For every match (y, z) ∈ E Find nodes x such that w[y..x] ∈ L w[x..z] ∈ K Divide and conquer dynamic algorithm.

SLIDE 69

abaabaabbabababbababbabbababbababbabbbababbbabbababbabbbabaababbabba

Problem. Fix a set of tag names and regular word languages
Input. matching
Output. Set of nodes x with

a1 · · · an ∈ Σ∗ E ⊆ {1, . . . , n}2 Σ L, K ⊆ Σ∗ ∈ E y z For every match (y, z) ∈ E Find nodes x such that w[y..x] ∈ L w[x..z] ∈ K Divide and conquer dynamic algorithm. but only do logarithmically many operations each time

SLIDE 70

abaabaabbabababbababbabbababbababbabbbababbbabbababbabbbabaababbabba

Problem. Fix a set of tag names and regular word languages
Input. matching
Output. Set of nodes x with

a1 · · · an ∈ Σ∗ E ⊆ {1, . . . , n}2 Σ L, K ⊆ Σ∗ ∈ E y z For every match (y, z) ∈ E Find nodes x such that w[y..x] ∈ L w[x..z] ∈ K Divide and conquer dynamic algorithm. but only do logarithmically many operations each time

SLIDE 71

abaabaabbabababbababbabbababbababbabbbababbbabbababbabbbabaababbabba

Problem. Fix a set of tag names and regular word languages
Input. matching
Output. Set of nodes x with

a1 · · · an ∈ Σ∗ E ⊆ {1, . . . , n}2 Σ L, K ⊆ Σ∗ ∈ E y z For every match (y, z) ∈ E Find nodes x such that w[y..x] ∈ L w[x..z] ∈ K Divide and conquer dynamic algorithm. but only do logarithmically many operations each time

SLIDE 72

abaabaabbabababbababbabbababbababbabbbababbbabbababbabbbabaababbabba

Problem. Fix a set of tag names and regular word languages
Input. matching
Output. Set of nodes x with

a1 · · · an ∈ Σ∗ E ⊆ {1, . . . , n}2 Σ L, K ⊆ Σ∗ ∈ E y z For every match (y, z) ∈ E Find nodes x such that w[y..x] ∈ L w[x..z] ∈ K Divide and conquer dynamic algorithm. but only do logarithmically many operations each time

SLIDE 73

abaabaabbabababbababbabbababbababbabbbababbbabbababbabbbabaababbabba

Problem. Fix a set of tag names and regular word languages
Input. matching
Output. Set of nodes x with

a1 · · · an ∈ Σ∗ E ⊆ {1, . . . , n}2 Σ L, K ⊆ Σ∗ ∈ E y z For every match (y, z) ∈ E Find nodes x such that w[y..x] ∈ L w[x..z] ∈ K Divide and conquer dynamic algorithm. but only do logarithmically many operations each time

SLIDE 74

Summary

SLIDE 75

Summary – We evaluate XPath queries with linear time data complexity, improving previous quadratic algorithms.

(the constant is exponential in the query, because we use semigroup theory)

SLIDE 76

Summary – We evaluate XPath queries with linear time data complexity, improving previous quadratic algorithms.

(the constant is exponential in the query, because we use semigroup theory)

– Works for both unary and binary queries

SLIDE 77

Summary – We evaluate XPath queries with linear time data complexity, improving previous quadratic algorithms.

(the constant is exponential in the query, because we use semigroup theory)

– Works for both unary and binary queries – Semigroups a good tool for efficient query evaluation

SLIDE 78

Summary – We evaluate XPath queries with linear time data complexity, improving previous quadratic algorithms.

(the constant is exponential in the query, because we use semigroup theory)

– Works for both unary and binary queries Future work – Semigroups a good tool for efficient query evaluation

SLIDE 79

Summary – We evaluate XPath queries with linear time data complexity, improving previous quadratic algorithms.

(the constant is exponential in the query, because we use semigroup theory)

– Works for both unary and binary queries – Preliminary results indicate that semigroups can be avoided, and the constant becomes polynomial in the query. Future work – Semigroups a good tool for efficient query evaluation

SLIDE 80

Summary – We evaluate XPath queries with linear time data complexity, improving previous quadratic algorithms.

(the constant is exponential in the query, because we use semigroup theory)

– Works for both unary and binary queries – Preliminary results indicate that semigroups can be avoided, and the constant becomes polynomial in the query. – We want to investigate more of XPath, and other languages Future work – Semigroups a good tool for efficient query evaluation

SLIDE 81

Let A be an automaton with state space Q Two rules for splitting words.

SLIDE 82

Let A be an automaton with state space Q Two rules for splitting words. Simon eorem. For fixed A, there is a splitting depth K, such that every word can be split in depth K down to single letters.

SLIDE 83

Let A be an automaton with state space Q Two rules for splitting words. Simon eorem. For fixed A, there is a splitting depth K, such that every word can be split in depth K down to single letters. abaabbbababbbabba bbabbbabbbabbaba Rule 1.

split into two parts

SLIDE 84

Let A be an automaton with state space Q Two rules for splitting words. Simon eorem. For fixed A, there is a splitting depth K, such that every word can be split in depth K down to single letters. abaabbbababbbabba bbabbbabbbabbaba Rule 1.

split into two parts

Rule 2.

split into many parts, each with the same transformation

abaab bbababb babba bba bbbabb babba ba

SLIDE 85

Let A be an automaton with state space Q Two rules for splitting words. Simon eorem. For fixed A, there is a splitting depth K, such that every word can be split in depth K down to single letters. abaabbbababbbabba bbabbbabbbabbaba Rule 1.

split into two parts

Rule 2.

split into many parts, each with the same transformation

abaab bbababb babba bba bbbabb babba ba