[PPT] - Compressed Membership for NFA (DFA) with Compressed Labels is in NP PowerPoint Presentation

SLIDE 1

Compressed Membership for NFA (DFA) with Compressed Labels is in NP (P)

Artur Jeż University of Wrocław

Artur Jeż Compressed membership for NFA 1 / 17

SLIDE 2

What this talk is about

Fully compressed membership problem for automata

Artur Jeż Compressed membership for NFA 2 / 17

SLIDE 3

What this talk is about

Fully compressed membership problem for automata no automata in this talk

Artur Jeż Compressed membership for NFA 2 / 17

SLIDE 4

What this talk is about

Fully compressed membership problem for automata no automata in this talk SPLs and a technique for them

Artur Jeż Compressed membership for NFA 2 / 17

SLIDE 5

What this talk is about

Fully compressed membership problem for automata no automata in this talk SPLs and a technique for them more general (word equations)

Artur Jeż Compressed membership for NFA 2 / 17

SLIDE 6

What this talk is about

Fully compressed membership problem for automata no automata in this talk SPLs and a technique for them more general (word equations)

Results

Artur Jeż Compressed membership for NFA 2 / 17

SLIDE 7

What this talk is about

Fully compressed membership problem for automata no automata in this talk SPLs and a technique for them more general (word equations)

Results

Fully compressed membership problem for NFA (in NP)

Artur Jeż Compressed membership for NFA 2 / 17

SLIDE 8

What this talk is about

Fully compressed membership problem for automata no automata in this talk SPLs and a technique for them more general (word equations)

Results

Fully compressed membership problem for NFA (in NP) Fully compressed membership problem for DFA (in P)

Artur Jeż Compressed membership for NFA 2 / 17

SLIDE 9

What this talk is about

Fully compressed membership problem for automata no automata in this talk SPLs and a technique for them more general (word equations)

Results

Fully compressed membership problem for NFA (in NP) Fully compressed membership problem for DFA (in P) (SLP) fully compressed pattern matching (in O(n2))

Artur Jeż Compressed membership for NFA 2 / 17

SLIDE 10

What this talk is about

Fully compressed membership problem for automata no automata in this talk SPLs and a technique for them more general (word equations)

Results

Fully compressed membership problem for NFA (in NP) Fully compressed membership problem for DFA (in P) (SLP) fully compressed pattern matching (in O(n2)) word equations: simple, unified proof for everything that is known

Artur Jeż Compressed membership for NFA 2 / 17

SLIDE 11

Straight Line Programms SLPs

Definition (Straight Line Programms (SLP))

Context free grammar defining a single word. (Chomsky normal form).

Artur Jeż Compressed membership for NFA 3 / 17

SLIDE 12

Straight Line Programms SLPs

Definition (Straight Line Programms (SLP))

Context free grammar defining a single word. (Chomsky normal form). Up to exponential compression.

Artur Jeż Compressed membership for NFA 3 / 17

SLIDE 13

Straight Line Programms SLPs

Definition (Straight Line Programms (SLP))

Context free grammar defining a single word. (Chomsky normal form). Up to exponential compression.

SLPs as a compression model

application (LZ, logarithmic transformation) theory (formal languages) preserves/captures word properties

Artur Jeż Compressed membership for NFA 3 / 17

SLIDE 14

Straight Line Programms SLPs

Definition (Straight Line Programms (SLP))

Context free grammar defining a single word. (Chomsky normal form). Up to exponential compression.

SLPs as a compression model

application (LZ, logarithmic transformation) theory (formal languages) preserves/captures word properties Applied in many proofs and constructions.

Artur Jeż Compressed membership for NFA 3 / 17

SLIDE 15

Usage and work on SLP

Theory

word equations (Plandowski: satisfiability in PSPACE)

Artur Jeż Compressed membership for NFA 4 / 17

SLIDE 16

Usage and work on SLP

Theory

word equations (Plandowski: satisfiability in PSPACE)

LZW/LZ dealing algorithms

O(n log(N/n)) pattern matching for LZ compressed text O(n) pattern matching for fully LZW compressed text

Artur Jeż Compressed membership for NFA 4 / 17

SLIDE 17

Usage and work on SLP

Theory

word equations (Plandowski: satisfiability in PSPACE)

LZW/LZ dealing algorithms

O(n log(N/n)) pattern matching for LZ compressed text O(n) pattern matching for fully LZW compressed text

String algorithms

equality pattern matching

Artur Jeż Compressed membership for NFA 4 / 17

SLIDE 18

Usage and work on SLP

Theory

word equations (Plandowski: satisfiability in PSPACE)

LZW/LZ dealing algorithms

O(n log(N/n)) pattern matching for LZ compressed text O(n) pattern matching for fully LZW compressed text

String algorithms

equality pattern matching

Independent interest

indexing structure for SLP

Artur Jeż Compressed membership for NFA 4 / 17

SLIDE 19

Compressed membership

SLPs are used develop tools/gain understanding membership problem

Artur Jeż Compressed membership for NFA 5 / 17

SLIDE 20

Compressed membership

SLPs are used develop tools/gain understanding membership problem

Compressed membership [Plandowski & Rytter 1999]

In membership problems, words are given as SLPs.

Artur Jeż Compressed membership for NFA 5 / 17

SLIDE 21

Compressed membership

SLPs are used develop tools/gain understanding membership problem

Compressed membership [Plandowski & Rytter 1999]

In membership problems, words are given as SLPs.

Known results

RE, CFG, Conjunctive grammars. . .

Artur Jeż Compressed membership for NFA 5 / 17

SLIDE 22

Compressed membership

SLPs are used develop tools/gain understanding membership problem

Compressed membership [Plandowski & Rytter 1999]

In membership problems, words are given as SLPs.

Known results

RE, CFG, Conjunctive grammars. . .

Open questions

Compressed membership for NFA

Artur Jeż Compressed membership for NFA 5 / 17

SLIDE 23

Compressed membership for NFA

Input: SLP, NFA N Output: Yes/No

Artur Jeż Compressed membership for NFA 6 / 17

SLIDE 24

Compressed membership for NFA

Input: SLP, NFA N Output: Yes/No Simple dynamic algorithm: for Xi calculate {(p, q) | δ(p, val(Xi), q)}

Artur Jeż Compressed membership for NFA 6 / 17

SLIDE 25

Compressed membership for NFA

Input: SLP, NFA N Output: Yes/No Simple dynamic algorithm: for Xi calculate {(p, q) | δ(p, val(Xi), q)} Where is the hardness?

Artur Jeż Compressed membership for NFA 6 / 17

SLIDE 26

Compressed membership for NFA

Input: SLP, NFA N Output: Yes/No Simple dynamic algorithm: for Xi calculate {(p, q) | δ(p, val(Xi), q)} Where is the hardness? Compress N as well: allow transition by words.

Artur Jeż Compressed membership for NFA 6 / 17

SLIDE 27

Compressed membership for NFA

Input: SLP, NFA N Output: Yes/No Simple dynamic algorithm: for Xi calculate {(p, q) | δ(p, val(Xi), q)} Where is the hardness? Compress N as well: allow transition by words. Fully compressed NFA membership SLP for w NFA N, compressed transitions

Artur Jeż Compressed membership for NFA 6 / 17

SLIDE 28

Compressed membership for NFA

Input: SLP, NFA N Output: Yes/No Simple dynamic algorithm: for Xi calculate {(p, q) | δ(p, val(Xi), q)} Where is the hardness? Compress N as well: allow transition by words. Fully compressed NFA membership SLP for w NFA N, compressed transitions

a X Y b A X

Artur Jeż Compressed membership for NFA 6 / 17

SLIDE 29

Compressed membership for NFA: complexity

Complexity

NP-hardness (subsum), already for

◮ acyclic NFA ◮ unary alphabet

in PSPACE: enough to store positions inside decompressed words

Artur Jeż Compressed membership for NFA 7 / 17

SLIDE 30

Compressed membership for NFA: complexity

Complexity

NP-hardness (subsum), already for

◮ acyclic NFA ◮ unary alphabet

in PSPACE: enough to store positions inside decompressed words

Conjecture

In NP.

Partial results

Plandowski & Rytter (unary in NP) Lohrey & Mathissen (highly periodic in NP, highly aperiodic in P)

Artur Jeż Compressed membership for NFA 7 / 17

SLIDE 31

New results

Theorem

Fully compressed membership for NFA is in NP.

Theorem

Fully compressed membership for DFA is in P.

Artur Jeż Compressed membership for NFA 8 / 17

SLIDE 32

Idea: Recompression

Difficulty: the words are long. Shorten them.

Artur Jeż Compressed membership for NFA 9 / 17

SLIDE 33

Idea: Recompression

Difficulty: the words are long. Shorten them. a b c a a b

Artur Jeż Compressed membership for NFA 9 / 17

SLIDE 34

Idea: Recompression

Difficulty: the words are long. Shorten them. d c a d

Artur Jeż Compressed membership for NFA 9 / 17

SLIDE 35

Idea: Recompression

Difficulty: the words are long. Shorten them. d c a d

Deeper understanding

New production: d → ab. Building new SLP (recompression). SLP problems: hard, as SLP are different. Building canonical SLP for the instance.

Artur Jeż Compressed membership for NFA 9 / 17

SLIDE 36

Idea: Recompression

Difficulty: the words are long. Shorten them. d c a d

Deeper understanding

New production: d → ab. Building new SLP (recompression). SLP problems: hard, as SLP are different. Building canonical SLP for the instance. What to do with an? a a c a a a

Artur Jeż Compressed membership for NFA 9 / 17

SLIDE 37

Idea: Recompression

Difficulty: the words are long. Shorten them. d c a d

Deeper understanding

New production: d → ab. Building new SLP (recompression). SLP problems: hard, as SLP are different. Building canonical SLP for the instance. What to do with an? Replace each maximal an by a single symbol. a2 c a3

Artur Jeż Compressed membership for NFA 9 / 17

SLIDE 38

Idea: Recompression

Difficulty: the words are long. Shorten them. d c a d

Deeper understanding

New production: d → ab. Building new SLP (recompression). SLP problems: hard, as SLP are different. Building canonical SLP for the instance. What to do with an? Replace each maximal an by a single symbol. a2 c a3

Problems

Easy for text, what about grammar?

Artur Jeż Compressed membership for NFA 9 / 17

SLIDE 39

Local recompression

Re-compression

decompressed text: easy; size: large, compressed text: hard; size: small.

Artur Jeż Compressed membership for NFA 10 / 17

SLIDE 40

Local recompression

Re-compression

decompressed text: easy; size: large, compressed text: hard; size: small.

Local decompression

Decompress locally the SLP: X → uYvZ u, v: blocks of letters, linear size Y , Z: nonterminals recompression inside u, v

Artur Jeż Compressed membership for NFA 10 / 17

SLIDE 41

Outline

Outline of the algorithm

while | val(Xn) > n| do LΣ ← list of letters, LP ← list of pairs for ab ∈ LP do compress pair ab for a ∈ LΣ do compress a maximal blocks Decompress the word and solve the problem naively.

Artur Jeż Compressed membership for NFA 11 / 17

SLIDE 42

Outline

Outline of the algorithm

while | val(Xn) > n| do LΣ ← list of letters, LP ← list of pairs for ab ∈ LP do compress pair ab for a ∈ LΣ do compress a maximal blocks Decompress the word and solve the problem naively.

Theorem

There are O(log | val(Xn)|) iterations.

Proof.

Consider two consecutive letters ab. One of them is compressed. So word shortens by a constant factor.

Artur Jeż Compressed membership for NFA 11 / 17

SLIDE 43

What is hard, what is easy

What is hard to compress, what easy?

Artur Jeż Compressed membership for NFA 12 / 17

SLIDE 44

What is hard, what is easy

What is hard to compress, what easy?

Hard

a pair ab is crossing if Xi → uaXjvXk, where val(Xj) = b . . . a letter a has crossing appearances if aa is a crossing pair

Artur Jeż Compressed membership for NFA 12 / 17

SLIDE 45

What is hard, what is easy

What is hard to compress, what easy?

Hard

a pair ab is crossing if Xi → uaXjvXk, where val(Xj) = b . . . a letter a has crossing appearances if aa is a crossing pair

Easy

a pair ab is non-crossing otherwise a letter a has no crossing appearances otherwise

Artur Jeż Compressed membership for NFA 12 / 17

SLIDE 46

A little detailed outline

Detailed outline

while | val(Xn) > n| do while possible do for non-crossing pair ab in val(Xn) do compress ab for a: without crossing blocks do compress appearances of a

Artur Jeż Compressed membership for NFA 13 / 17

SLIDE 47

A little detailed outline

Detailed outline

while | val(Xn) > n| do while possible do for non-crossing pair ab in val(Xn) do compress ab for a: without crossing blocks do compress appearances of a L ← list of letters with crossing blocks P ← list of crossing pairs for each ab in P do compress ab for a ∈ L do compress appearances of a Decompress Xn and solve the problem naively.

Artur Jeż Compressed membership for NFA 13 / 17

SLIDE 48

Non-crossing pair compression

for each production Xi → uXjvXk do replace each ab in u, v by c

Artur Jeż Compressed membership for NFA 14 / 17

SLIDE 49

Non-crossing pair compression

for each production Xi → uXjvXk do replace each ab in u, v by c

Appearance compression for a without crossing blocks

compute the lengths ℓ1, . . . , ℓk of a’s maximal blocks for each aℓm do for each production Xi → uXjvXk do replace maximal aℓm in in u, v by aℓm

Artur Jeż Compressed membership for NFA 14 / 17

SLIDE 50

Non-crossing pair compression

for each production Xi → uXjvXk do replace each ab in u, v by c

Appearance compression for a without crossing blocks

compute the lengths ℓ1, . . . , ℓk of a’s maximal blocks for each aℓm do for each production Xi → uXjvXk do replace maximal aℓm in in u, v by aℓm

Lemma

It works.

Proof.

The pair is non-crossing: it always appears inside production.

Artur Jeż Compressed membership for NFA 14 / 17

SLIDE 51

Convert hard to easy

Convert crossing pairs to noncrossing and letters with crossing blocks to letters without crossing blocks (Sequentially).

Artur Jeż Compressed membership for NFA 15 / 17

SLIDE 52

Convert hard to easy

Convert crossing pairs to noncrossing and letters with crossing blocks to letters without crossing blocks (Sequentially). aXi and Xi begins with b Xjb and Xj ends with a

Artur Jeż Compressed membership for NFA 15 / 17

SLIDE 53

Convert hard to easy

Convert crossing pairs to noncrossing and letters with crossing blocks to letters without crossing blocks (Sequentially). aXi and Xi begins with b Xjb and Xj ends with a ‘pop’ first letter of Xi replace: val(Xi) = bu → val(Xi) = u grammar: remove leading b from rule for Xi, replace Xi by bXi

Artur Jeż Compressed membership for NFA 15 / 17

SLIDE 54

Convert hard to easy

Convert crossing pairs to noncrossing and letters with crossing blocks to letters without crossing blocks (Sequentially). aXi and Xi begins with b Xjb and Xj ends with a ‘pop’ first letter of Xi replace: val(Xi) = bu → val(Xi) = u grammar: remove leading b from rule for Xi, replace Xi by bXi ‘pop’ last letter of Xi

Artur Jeż Compressed membership for NFA 15 / 17

SLIDE 55

Convert hard to easy

Convert crossing pairs to noncrossing and letters with crossing blocks to letters without crossing blocks (Sequentially). aXi and Xi begins with b Xjb and Xj ends with a ‘pop’ first letter of Xi replace: val(Xi) = bu → val(Xi) = u grammar: remove leading b from rule for Xi, replace Xi by bXi ‘pop’ last letter of Xi

Lemma

After popping letters, ab is noncrossing.

Proof.

Easy, some simple cases.

Artur Jeż Compressed membership for NFA 15 / 17

SLIDE 56

Removing crossing blocks of a

aa is a crossing pair: pop a can be insufficient cut a-prefix or a-suffix Represent val(Xi) as aℓiwari, turn it into w.

Artur Jeż Compressed membership for NFA 16 / 17

SLIDE 57

Removing crossing blocks of a

aa is a crossing pair: pop a can be insufficient cut a-prefix or a-suffix Represent val(Xi) as aℓiwari, turn it into w.

Changing a letter a with crossing blocks to one without

for i = 1 . . n do let Xi → uXjvXk calculate the a-prefix aℓi and a-suffix ari, remove them replace Xi in rules bodies by aℓiXiari

Artur Jeż Compressed membership for NFA 16 / 17

SLIDE 58

Removing crossing blocks of a

aa is a crossing pair: pop a can be insufficient cut a-prefix or a-suffix Represent val(Xi) as aℓiwari, turn it into w.

Changing a letter a with crossing blocks to one without

for i = 1 . . n do let Xi → uXjvXk calculate the a-prefix aℓi and a-suffix ari, remove them replace Xi in rules bodies by aℓiXiari

Lemma

After the algorithm a has no crossing block.

Artur Jeż Compressed membership for NFA 16 / 17

SLIDE 59

Removing crossing blocks of a

aa is a crossing pair: pop a can be insufficient cut a-prefix or a-suffix Represent val(Xi) as aℓiwari, turn it into w.

Changing a letter a with crossing blocks to one without

for i = 1 . . n do let Xi → uXjvXk calculate the a-prefix aℓi and a-suffix ari, remove them replace Xi in rules bodies by aℓiXiari

Lemma

After the algorithm a has no crossing block. Represent aℓ succinctly, using O(log ℓ) bits.

Artur Jeż Compressed membership for NFA 16 / 17

SLIDE 60

Sizes and running time

Running time

All algorithms run in time poly(n, |G|, |Σ|).

Artur Jeż Compressed membership for NFA 17 / 17

SLIDE 61

Sizes and running time

Running time

All algorithms run in time poly(n, |G|, |Σ|).

Size of G

abbbcceaXjaddfeaaf Xk In each iteration

Artur Jeż Compressed membership for NFA 17 / 17

SLIDE 62

Sizes and running time

Running time

All algorithms run in time poly(n, |G|, |Σ|).

Size of G

abbbcceabhaXjabaddfeaaf cdaXk In each iteration O(n) new letters

Artur Jeż Compressed membership for NFA 17 / 17

SLIDE 63

Sizes and running time

Running time

All algorithms run in time poly(n, |G|, |Σ|).

Size of G

abbbcceabhaXjabaddfeaaf cdaXk In each iteration O(n) new letters shrinking by a constant factor

Artur Jeż Compressed membership for NFA 17 / 17

SLIDE 64

Sizes and running time

Running time

All algorithms run in time poly(n, |G|, |Σ|).

Size of G

uvbhaXjabxyzcdaXk In each iteration O(n) new letters shrinking by a constant factor

Artur Jeż Compressed membership for NFA 17 / 17

SLIDE 65

Sizes and running time

Running time

All algorithms run in time poly(n, |G|, |Σ|).

Size of G

uvbhaXjabxyzcdaXk In each iteration O(n) new letters shrinking by a constant factor

New letters (|Σ|)

noncrossing pairs, noncrossing blocks compression (shrinks |G|) letters with crossing blocks and crossing pairs: there are O(n) such letters and O(n2) pairs in val(Xn)

Artur Jeż Compressed membership for NFA 17 / 17

SLIDE 66

Modifications

compressed membership: how to modify automaton?

◮ for NFA: nondeterminism ◮ for DFA: deterministically Artur Jeż Compressed membership for NFA 18 / 17

SLIDE 67

Modifications

compressed membership: how to modify automaton?

◮ for NFA: nondeterminism ◮ for DFA: deterministically

compressed pattern matching

◮ better analysis ◮ careful implementation ◮ details (ends of the pattern) Artur Jeż Compressed membership for NFA 18 / 17

SLIDE 68

Modifications

compressed membership: how to modify automaton?

◮ for NFA: nondeterminism ◮ for DFA: deterministically

compressed pattern matching

◮ better analysis ◮ careful implementation ◮ details (ends of the pattern)

word equations: some further understanding

Artur Jeż Compressed membership for NFA 18 / 17

SLIDE 69

Modifications

compressed membership: how to modify automaton?

◮ for NFA: nondeterminism ◮ for DFA: deterministically

compressed pattern matching

◮ better analysis ◮ careful implementation ◮ details (ends of the pattern)

word equations: some further understanding

Questions

Any further results? How efficient for DFA? Are word equations in NP?

Artur Jeż Compressed membership for NFA 18 / 17