Derivation Beyond Regular Languages Albert-Ludwigs-Universit at - - PowerPoint PPT Presentation

Derivation Beyond Regular Languages

Albert-Ludwigs-Universit¨ at Freiburg

Peter Thiemann1

1University of Freiburg

14 Oct 2018

Plan

1 Derivatives for Regular Languages

Brzozowski Derivatives Antimirov’s Partial Derivatives Applications: Equivalence and Containment Computational Interpretation

2 Derivatives for Context-Free Languages

Pragmatic Parsing Deriving Automata Computational Interpretation

3 Derivation for ω-Regular Languages

Attempt #1: Brzozowski-style Derivatives Attempt #2: Extending Partial Derivatives

Thiemann Derivation Beyond Regular Languages 14 Oct 2018 2 / 77

Languages

Notation

Σ alphabet (a finite set) Σ∗ the set of (finite) words over Σ P(Σ∗) the set of languages over Σ a, b, c, . . . ∈ Σ u, v, w, . . . ∈ Σ∗; empty word ε, concatenation · L, U, V , . . . ∈ P(Σ∗)

Thiemann Derivation Beyond Regular Languages 14 Oct 2018 3 / 77

Operations on Languages

Product and Power

U · V = {u · v | u ∈ U, v ∈ V } U0 = {ε} Un+1 = U · Un

Kleene Closure

U∗ =

• n≥0

Un

Thiemann Derivation Beyond Regular Languages 14 Oct 2018 4 / 77

Finite Automata

A nondeterministic finite automaton (NFA) M = (Q, Σ, I, δ, F) consists of Q finite set of states Σ I ⊆ Q initial states δ : Q × Σ → PQ transition function F ⊆ Q accepting states M is deterministic (DFA) if |I| = 1 and, for all q, a, |δ(q, a)| = 1.

Thiemann Derivation Beyond Regular Languages 14 Oct 2018 5 / 77

Runs and Languages

Run

A run of M on a1 . . . an ∈ Σ∗ is a sequence q0, . . . , qn ∈ Q∗ s.t. q0 ∈ I, for all 0 ≤ i < n, qi+1 ∈ δ(q, ai). The run is accepting if qn ∈ F.

Language of Automaton M

L(M) = {w ∈ Σ∗ | M has an accepting run on w}

Thiemann Derivation Beyond Regular Languages 14 Oct 2018 6 / 77

Regular Languages

Regular Expressions

Syntax REΣ ∋ r, s ::= 0 | 1 | a | r + s | r.s | r∗ Semantics · : REΣ → P(Σ∗) 0 = ∅ 1 = {ε} a = {a} r + s = r ∪ s r.s = r · s r∗ = r∗

Thiemann Derivation Beyond Regular Languages 14 Oct 2018 7 / 77

From Regular Expressions to Finite Automata

Thompson’s construction

RE → NFA-ε → NFA → DFA three steps: one compositional, two global automata transformations key: transitive closure, powerset construction standard technique

Thiemann Derivation Beyond Regular Languages 14 Oct 2018 8 / 77

From Regular Expressions to Finite Automata

Thompson’s construction

RE → NFA-ε → NFA → DFA three steps: one compositional, two global automata transformations key: transitive closure, powerset construction standard technique

Brzozowski’s construction

RE → DFA

• ne compositional step: algebraic flavor

key: derivative of RE

Thiemann Derivation Beyond Regular Languages 14 Oct 2018 8 / 77

Plan

1 Derivatives for Regular Languages

Brzozowski Derivatives Antimirov’s Partial Derivatives Applications: Equivalence and Containment Computational Interpretation

2 Derivatives for Context-Free Languages

Pragmatic Parsing Deriving Automata Computational Interpretation

3 Derivation for ω-Regular Languages

Attempt #1: Brzozowski-style Derivatives Attempt #2: Extending Partial Derivatives

Thiemann Derivation Beyond Regular Languages 14 Oct 2018 9 / 77

Foundations of Derivation

Definition: Left quotient

U, V ⊆ Σ∗: U−1V = {w | ∃u ∈ U, uw ∈ V }

Thiemann Derivation Beyond Regular Languages 14 Oct 2018 10 / 77

Foundations of Derivation

Definition: Left quotient

U, V ⊆ Σ∗: U−1V = {w | ∃u ∈ U, uw ∈ V }

Proposition: Closure of regular languages under quotient

U, V ⊆ Σ∗ regular ⇒ U−1V regular

Thiemann Derivation Beyond Regular Languages 14 Oct 2018 10 / 77

Foundations of Derivation

Definition: Left quotient

U, V ⊆ Σ∗: U−1V = {w | ∃u ∈ U, uw ∈ V }

Proposition: Closure of regular languages under quotient

U, V ⊆ Σ∗ regular ⇒ U−1V regular

Corrolary: Closure of regular languages under derivation

Derivation by a ∈ Σ is the special case where U = {a} V ⊆ Σ∗ regular ⇒ a−1V regular

Thiemann Derivation Beyond Regular Languages 14 Oct 2018 10 / 77

Foundations of Derivation

Definition: Left quotient

U, V ⊆ Σ∗: U−1V = {w | ∃u ∈ U, uw ∈ V }

Proposition: Closure of regular languages under quotient

U, V ⊆ Σ∗ regular ⇒ U−1V regular

Corrolary: Closure of regular languages under derivation

Derivation by a ∈ Σ is the special case where U = {a} V ⊆ Σ∗ regular ⇒ a−1V regular

Proposition: Decidable Nullability

V ⊆ Σ∗ regular ⇒ ε ∈ V is decidable

Thiemann Derivation Beyond Regular Languages 14 Oct 2018 10 / 77

Brzozowski Derivatives

Derivatives of Regular Expressions

d : Σ × REΣ → REΣ such that L(dr(a)) = a−1L(r)

Thiemann Derivation Beyond Regular Languages 14 Oct 2018 11 / 77

Brzozowski Derivatives

Derivatives of Regular Expressions

d : Σ × REΣ → REΣ such that L(dr(a)) = a−1L(r)

Nullability

N : REΣ → REΣ such that L(N(r)) = {ε} ∩ L(r)

Thiemann Derivation Beyond Regular Languages 14 Oct 2018 11 / 77

Brzozowski Derivatives

Derivatives of Regular Expressions

d : Σ × REΣ → REΣ such that L(dr(a)) = a−1L(r)

Nullability

N : REΣ → REΣ such that L(N(r)) = {ε} ∩ L(r)

Smart Concatenation

r ⊙ s =            r = 0 or s = 0 r s = 1 s r = 1 r.s

• therwise

Thiemann Derivation Beyond Regular Languages 14 Oct 2018 11 / 77

Brzozowski Derivatives

Derivatives of Regular Expressions

d : Σ × REΣ → REΣ such that L(dr(a)) = a−1L(r)

Nullability

N : REΣ → REΣ such that L(N(r)) = {ε} ∩ L(r)

Smart Concatenation

r ⊙ s =            r = 0 or s = 0 r s = 1 s r = 1 r.s

• therwise

Smart Alternative

r ⊕ s =            r s = 0 s r = 0 r r = s r + s

• therwise

Thiemann Derivation Beyond Regular Languages 14 Oct 2018 11 / 77

Brzozowski Derivatives

Nullability and Derivative

N(0) = 0 d(a, 0) = 0 N(1) = 1 d(a, 1) = 0 N(a) = 0 d(a, b) =

• 1

a = b a = b N(r + s) = N(r) ⊕ N(s) d(a, r + s) = d(a, r) ⊕ d(a, s) N(r.s) = N(r) ⊙ N(s) d(a, r.s) = (d(a, r) ⊙ s) ⊕ (N(r) ⊙ d(a, s)) N(r ∗) = 1 d(a, r ∗) = d(a, r) ⊙ r ∗

Thiemann Derivation Beyond Regular Languages 14 Oct 2018 12 / 77

Brzozowski Derivatives

Iterated Derivative

dΣ∗ : REΣ → P(REΣ) the iterated derivative is defined inductively r ∈ dΣ∗(r) if s ∈ dΣ∗(r) and a ∈ Σ, then d(a, s) ∈ dΣ∗(r)

Thiemann Derivation Beyond Regular Languages 14 Oct 2018 13 / 77

Brzozowski’s Results

Representation theorem

L(r) = L(N(r)) ∪

• a∈Σ

{a} · L(da(r))

Thiemann Derivation Beyond Regular Languages 14 Oct 2018 14 / 77

Brzozowski’s Results

Representation theorem

L(r) = L(N(r)) ∪

• a∈Σ

{a} · L(da(r))

Finiteness theorem

For each r ∈ RΣ, the set dΣ∗(r)/≈ is finite. (≈ is a similarity relation on RΣ, i.e., associativity and idempotence of +)

Thiemann Derivation Beyond Regular Languages 14 Oct 2018 14 / 77

Brzozowski’s Results

Representation theorem

L(r) = L(N(r)) ∪

• a∈Σ

{a} · L(da(r))

Finiteness theorem

For each r ∈ RΣ, the set dΣ∗(r)/≈ is finite. (≈ is a similarity relation on RΣ, i.e., associativity and idempotence of +)

Theorem: RE → DFA

Let Q = dΣ∗(r)/≈ and F = {s ∈ Q | N(s) = 1}. Then A = (Q, Σ, d, {r}, F) is a DFA with L(A) = L(r).

Thiemann Derivation Beyond Regular Languages 14 Oct 2018 14 / 77

Plan

1 Derivatives for Regular Languages

Brzozowski Derivatives Antimirov’s Partial Derivatives Applications: Equivalence and Containment Computational Interpretation

2 Derivatives for Context-Free Languages

Pragmatic Parsing Deriving Automata Computational Interpretation

3 Derivation for ω-Regular Languages

Attempt #1: Brzozowski-style Derivatives Attempt #2: Extending Partial Derivatives

Thiemann Derivation Beyond Regular Languages 14 Oct 2018 15 / 77

Partial Derivatives

Due to Antimirov (1996) Direct translation RE → NFA Simpler proofs Introduces the important notion of linear factors

Thiemann Derivation Beyond Regular Languages 14 Oct 2018 16 / 77

Linear Factors

Definition

The set of linear factors of a regular expression is defined by a function lf : REΣ → P(Σ × REΣ) lf(0) = ∅ lf(1) = ∅ lf(a) = {a, 1} lf(r + s) = lf(r) ∪ lf(s) lf(r.s) = lf(r) ⊙ s ∪ lf(s) ⊙ N(r) lf(r∗) = lf(r) ⊙ r∗ where L ⊙ s = {a, r ⊙ s | a, r ∈ L, s = 0}

Thiemann Derivation Beyond Regular Languages 14 Oct 2018 17 / 77

Properties of Linear Factors

Definition

L(a, r) = {a} · L(r) L(L) =

• {{a} · L(r) | a, r ∈ L}

Thiemann Derivation Beyond Regular Languages 14 Oct 2018 18 / 77

Properties of Linear Factors

Definition

L(a, r) = {a} · L(r) L(L) =

• {{a} · L(r) | a, r ∈ L}

Lemma (Representation)

L(r) = L(N(r)) ∪ L(lf(r))

Thiemann Derivation Beyond Regular Languages 14 Oct 2018 18 / 77

Partial Derivatives

Definition

Let r ∈ REΣ, a ∈ Σ, w ∈ Σ∗. The partial derivative ∂a(r) ⊆ REΣ of r by a is defined by ∂a(r) = {r′ | a, r′ ∈ lf(r)} The partial derivative ∂w(r) ⊆ REΣ of r by w is defined inductively by ∂ε(r) = {r} ∂aw(r) = ∂w(∂a(r)) The iterated partial derivative ∂Σ∗(r) ⊆ REΣ is defined by ∂Σ∗(r) =

• {∂w(r) | w ∈ Σ∗}

Thiemann Derivation Beyond Regular Languages 14 Oct 2018 19 / 77

Properties of Linear Factors

Linear factors automatically perform the simplifications needed to

• btain finiteness of iterated derivatives.

Definition

Let S(r) ⊆ REΣ defined by s ∈ S(r) if s = s1 . . . sn such that each si is a subterm of r if 1 ≤ i < n, then si occurs strictly below si+1 in r (if n = 0 then s = 1)

Thiemann Derivation Beyond Regular Languages 14 Oct 2018 20 / 77

Properties of Linear Factors

Linear factors automatically perform the simplifications needed to

• btain finiteness of iterated derivatives.

Definition

Let S(r) ⊆ REΣ defined by s ∈ S(r) if s = s1 . . . sn such that each si is a subterm of r if 1 ≤ i < n, then si occurs strictly below si+1 in r (if n = 0 then s = 1)

Proposition

S(r) is finite

Thiemann Derivation Beyond Regular Languages 14 Oct 2018 20 / 77

Finiteness

Lemma

If a, s ∈ lf(r), then s ∈ S(r).

Thiemann Derivation Beyond Regular Languages 14 Oct 2018 21 / 77

Finiteness

Lemma

If a, s ∈ lf(r), then s ∈ S(r).

Lemma

If s ∈ S(r), then ∂a(s) ⊆ S(r)

Thiemann Derivation Beyond Regular Languages 14 Oct 2018 21 / 77

Finiteness

Lemma

If a, s ∈ lf(r), then s ∈ S(r).

Lemma

If s ∈ S(r), then ∂a(s) ⊆ S(r)

Finiteness Theorem for Partial Derivatives

∂Σ∗(r) ⊆ S(r) and thus finite

Thiemann Derivation Beyond Regular Languages 14 Oct 2018 21 / 77

Finiteness

Lemma

If a, s ∈ lf(r), then s ∈ S(r).

Lemma

If s ∈ S(r), then ∂a(s) ⊆ S(r)

Finiteness Theorem for Partial Derivatives

∂Σ∗(r) ⊆ S(r) and thus finite

Theorem: RE → NFA

Let Q = ∂Σ∗(r) and F = {s ∈ Q | N(s) = 1}. Then A = (Q, Σ, ∂, {r}, F) is a NFA with L(A) = L(r).

Thiemann Derivation Beyond Regular Languages 14 Oct 2018 21 / 77

Alternative Finiteness Proof (Broda et al 2011)

Definition (Approximate Iterated Derivatives)

Specification δ+ : REΣ → P(REΣ) δ+(0) = ∅ δ+(1) = ∅ δ+(a) = {1} δ+(r + s) = δ+(r) ∪ δ+(s) δ+(r.s) = δ+(r) ⊙ s ∪ δ+(s) δ+(r∗) = δ+(r) ⊙ r∗

Remark

An over approximation of the set of iterated derivatives!

Thiemann Derivation Beyond Regular Languages 14 Oct 2018 22 / 77

Alternative Finiteness Proof (Broda et al 2011)

Lemma

For all r, δ+(r) is finite

Thiemann Derivation Beyond Regular Languages 14 Oct 2018 23 / 77

Alternative Finiteness Proof (Broda et al 2011)

Lemma

For all r, δ+(r) is finite

Lemma

∂a(r) ⊆ δ+(r) s ∈ δ+(r) implies ∂a(s) ⊆ δ+(r)

Thiemann Derivation Beyond Regular Languages 14 Oct 2018 23 / 77

Alternative Finiteness Proof (Broda et al 2011)

Lemma

For all r, δ+(r) is finite

Lemma

∂a(r) ⊆ δ+(r) s ∈ δ+(r) implies ∂a(s) ⊆ δ+(r)

Finiteness Theorem for Partial Derivatives

∂Σ∗(r) ⊆ δ+(r) and thus finite

Thiemann Derivation Beyond Regular Languages 14 Oct 2018 23 / 77

Plan

1 Derivatives for Regular Languages

Brzozowski Derivatives Antimirov’s Partial Derivatives Applications: Equivalence and Containment Computational Interpretation

2 Derivatives for Context-Free Languages

Pragmatic Parsing Deriving Automata Computational Interpretation

3 Derivation for ω-Regular Languages

Attempt #1: Brzozowski-style Derivatives Attempt #2: Extending Partial Derivatives

Thiemann Derivation Beyond Regular Languages 14 Oct 2018 24 / 77

Applications

Definition: Regular Expression Equivalence and Containment

Let r, s ∈ REΣ Equivalence r ∼ s if L(r) = L(s) Containment r s if L(r) ⊆ L(s)

Thiemann Derivation Beyond Regular Languages 14 Oct 2018 25 / 77

Applications

Definition: Regular Expression Equivalence and Containment

Let r, s ∈ REΣ Equivalence r ∼ s if L(r) = L(s) Containment r s if L(r) ⊆ L(s)

Equivalence and Containment are Decidable

Standard proof for containment r s

Construct DFA for L(r) \ L(s) Check for emptiness

Equivalence and containment are equally powerful

r ∼ s iff r s and s r Observe that r s iff r + s ∼ s

Thiemann Derivation Beyond Regular Languages 14 Oct 2018 25 / 77

Derivative-based Equivalence Test

Coinductive Definition of Equivalence

N(r) = N(s) (∀a ∈ Σ) Σ∂a(r) ∼ Σ∂a(s) r ∼ s

Thiemann Derivation Beyond Regular Languages 14 Oct 2018 26 / 77

Derivative-based Equivalence Test

Coinductive Definition of Equivalence

N(r) = N(s) (∀a ∈ Σ) Σ∂a(r) ∼ Σ∂a(s) r ∼ s

Remarks

Establishes bisimulation between r and s Labeled transition system: r

a

→ Σ∂a(r) Analogous to equivalence of finite automata

Thiemann Derivation Beyond Regular Languages 14 Oct 2018 26 / 77

Implementation

Implementation

c o l l e c t R [ ] = True c o l l e c t R (( r , s ) : W) = i f ( r , s ) in R then c o l l e c t R W e l s e i f N( r ) == N( s ) then c o l l e c t (( r , s ) : R) (map (\ a −> (∂a r , ∂a s )) Σ ++ W) e l s e False equiv r s = c o l l e c t [ ] [ ( r , s ) ]

Thiemann Derivation Beyond Regular Languages 14 Oct 2018 27 / 77

Derivative-based Equivalence Test

Properties of collect

attempts to construct a bisimulation in R returns True iff r ∼ s (prove this!) termination guaranteed by finiteness of iterated derivatives straightforward extension with counterexample on failure

Thiemann Derivation Beyond Regular Languages 14 Oct 2018 28 / 77

Extension of collect with Counterexample

c o l l e c t R [ ] = Yes c o l l e c t R ((w, r , s ) : W) = i f ( r , s ) in R then c o l l e c t R W e l s e i f N( r ) == N( s ) then c o l l e c t (( r , s ) : R) (map (\ a −> ( a :w, ∂a r , ∂a s )) Σ ++ W) e l s e No w equiv r s = c o l l e c t [ ] [ ( ε , r , s ) ]

Thiemann Derivation Beyond Regular Languages 14 Oct 2018 29 / 77

Derivative-Based Containment

Coinductive Definition of Containment (cf. Antimirov)

N(r) ⇒ N(s) (∀a ∈ Σ) Σ∂a(r) Σ∂a(s) r s

Thiemann Derivation Beyond Regular Languages 14 Oct 2018 30 / 77

Derivative-Based Containment

Coinductive Definition of Containment (cf. Antimirov)

N(r) ⇒ N(s) (∀a ∈ Σ) Σ∂a(r) Σ∂a(s) r s

Remarks

Establishes a simulation between r and s Implementation very similar to equivalence Finiteness follows for the same reason

Thiemann Derivation Beyond Regular Languages 14 Oct 2018 30 / 77

Example

Show that 1 + a∗b a∗b∗

1 ⇒ 1 0 ⇒ 1 coind. a∗b a∗b∗ 1 ⇒ 1 1 b∗ a∗b a∗b∗ 1 ⇒ 1 1 b∗ 1 + a∗b a∗b∗

Thiemann Derivation Beyond Regular Languages 14 Oct 2018 31 / 77

Plan

1 Derivatives for Regular Languages

Brzozowski Derivatives Antimirov’s Partial Derivatives Applications: Equivalence and Containment Computational Interpretation

2 Derivatives for Context-Free Languages

Pragmatic Parsing Deriving Automata Computational Interpretation

3 Derivation for ω-Regular Languages

Attempt #1: Brzozowski-style Derivatives Attempt #2: Extending Partial Derivatives

Thiemann Derivation Beyond Regular Languages 14 Oct 2018 32 / 77

Regular Expressions Are Types

Parse Trees Are Values

⊢r Eps : 1 ⊢r Sym a : a ⊢r Empty : r∗ ⊢r p : r ⊢r q : s ⊢r Seq p q : r.s ⊢r p : r ⊢r q : r∗ ⊢r Cons p q : r∗ ⊢r p : r ⊢r Inl p : r + s ⊢r q : s ⊢r Inr q : r + s

Thiemann Derivation Beyond Regular Languages 14 Oct 2018 33 / 77

Regular Expressions Are Types

Parse Trees Are Values

⊢r Eps : 1 ⊢r Sym a : a ⊢r Empty : r∗ ⊢r p : r ⊢r q : s ⊢r Seq p q : r.s ⊢r p : r ⊢r q : r∗ ⊢r Cons p q : r∗ ⊢r p : r ⊢r Inl p : r + s ⊢r q : s ⊢r Inr q : r + s

Relations Between Types Yield Functions on Parse Trees

Equivalence Proof r ∼ s ⇒ Isomorphism r ↔ s Containment Proof r s ⇒ Embedding r → s

Thiemann Derivation Beyond Regular Languages 14 Oct 2018 33 / 77

Example (cont)

Embedding for 1 + a∗b a∗b∗

Inl Eps : 1 + a∗b → Seq Empty Empty : a∗b∗

Thiemann Derivation Beyond Regular Languages 14 Oct 2018 34 / 77

Example (cont)

Embedding for 1 + a∗b a∗b∗

Inl Eps : 1 + a∗b → Seq Empty Empty : a∗b∗ Inr (Seq Empty (Sym b)) → Seq Empty (Cons (Sym b) Empty)

Thiemann Derivation Beyond Regular Languages 14 Oct 2018 34 / 77

Example (cont)

Embedding for 1 + a∗b a∗b∗

Inl Eps : 1 + a∗b → Seq Empty Empty : a∗b∗ Inr (Seq Empty (Sym b)) → Seq Empty (Cons (Sym b) Empty) Inr (Seq (Cons (Sym a) Empty) (Sym b)) → Seq (Cons (Sym a) Empty) (Cons (Sym b) Empty)

Thiemann Derivation Beyond Regular Languages 14 Oct 2018 34 / 77

Example (cont)

Embedding for 1 + a∗b a∗b∗

Inl Eps : 1 + a∗b → Seq Empty Empty : a∗b∗ Inr (Seq Empty (Sym b)) → Seq Empty (Cons (Sym b) Empty) Inr (Seq (Cons (Sym a) Empty) (Sym b)) → Seq (Cons (Sym a) Empty) (Cons (Sym b) Empty)

Goal

Derive functions between parse tree that witness containment

1 Embedding and Projection from Derivation 2 Coinductive Embedding

Thiemann Derivation Beyond Regular Languages 14 Oct 2018 34 / 77

Step 1: Embedding and Projection from Derivation

Derivation of r by a yields a sublanguage of r: {a} · d(a, r) ⊆ r

Thiemann Derivation Beyond Regular Languages 14 Oct 2018 35 / 77

Step 1: Embedding and Projection from Derivation

Derivation of r by a yields a sublanguage of r: {a} · d(a, r) ⊆ r Hence, there is an embedding and a projection a.d(a, r) → r r → a.d(a, r)

Thiemann Derivation Beyond Regular Languages 14 Oct 2018 35 / 77

Step 1: Embedding and Projection from Derivation

Derivation of r by a yields a sublanguage of r: {a} · d(a, r) ⊆ r Hence, there is an embedding and a projection a.d(a, r) → r r → a.d(a, r) Reformulate derivation to construct “embedding” and “projection” functions ⊢a t : d(a, r) → r ⊢−

a t : r → d(a, r)

(leaving the a argument implicit)

Thiemann Derivation Beyond Regular Languages 14 Oct 2018 35 / 77

Auxiliary: Parse Tree for Empty Word

Every nullable expression has a parse tree for the empty word mkE(·) : {r | N(r)} → r mkE(1) = Eps N(r) mkE(r + s) = Inl mkE(r) N(s) mkE(r + s) = Inr mkE(s) mkE(r.s) = Seq mkE(r) mkE(s) mkE(r∗) = Empty

Thiemann Derivation Beyond Regular Languages 14 Oct 2018 36 / 77

Auxiliary: Get First Symbol from Parse Tree

fi(·) : r → Maybe Σ fi(p) = Nothing: p is parse tree for empty word fi(p) = Just a: a ∈ Σ is first symbol of parsed word fi(Eps) = Nothing fi(Sym a) = Just a fi(p) = m fi(Inl p) = m fi(p) = m fi(Inr p) = m fi(p) = Just a fi(Seq p q) = Just a fi(p) = Nothing fi(q) = m fi(Seq p q) = m fi(Empty) = Nothing fi(p) = Just a fi(Cons p q) = Just a fi(p) = Nothing fi(q) = m fi(Cons p q) = m

Thiemann Derivation Beyond Regular Languages 14 Oct 2018 37 / 77

Embedding and Projection from Derivation

d(a, 0) = 0

⊢a λy.abort y : 0 → 0 ⊢−

a λy.abort y : 0 → 0

d(a, 1) = 0

⊢a λy.abort y : 0 → 1 ⊢−

a λ() : 1 → 0

d(a, b) = if a = b then 1 else 0

⊢a λy.Sym a : 1 → a ⊢a λy.abort y : 0 → b ⊢−

a λ(Sym a).Eps : a → 1

⊢−

a λ() : b → 0

Thiemann Derivation Beyond Regular Languages 14 Oct 2018 38 / 77

Embedding and Projection from Derivation (2)

d(a, r + s) = d(a, r) + d(a, s)

⊢a tr : r′ → r ⊢a ts : s′ → s ⊢a λy.case y of Inl p → Inl(tr p) Inr q → Inr(ts q) : r′ + s′ → r + s ⊢−

a tr : r → r′

⊢−

a ts : s → s′

⊢−

a λy.case y of Inl p → Inl(tr p)

Inr q → Inr(ts q) : r + s → r′ + s′

Thiemann Derivation Beyond Regular Languages 14 Oct 2018 39 / 77

Embedding and Projection from Derivation (3)

d(a, r.s) = d(a, r).s + N(r).d(a, s)

⊢a tr : r′ → r ⊢a ts : s′ → s ⊢a λy.case y of Inl (Seq p q) → Seq (tr p) q Inr (Seq Eps q) → Seq mkE(r) (ts q) : r′.s + N(r).s′ → r.s ⊢−

a tr : r → r′

⊢−

a ts : s → s′

⊢−

a λ(Seq p q).case fi(p) of Just a → Inl (Seq (tr p) q)

Nothing → Inr (Seq Eps ts q) : r.s → r′.s + N(r).s′

Thiemann Derivation Beyond Regular Languages 14 Oct 2018 40 / 77

Embedding and Projection from Derivation (4)

d(a, r ∗) = d(a, r).r ∗

⊢a tr : r′ → r ⊢a λ(Seq p q).Cons (tr p) q : r′.r∗ → r∗ ⊢−

a tr : r → r′

⊢−

a λy.case y of Cons p q → Seq (tr p) q

: r∗ → r′.r∗ Assumes that parse trees for r∗ are reduced: no unproductive iteration steps, even if N(r) Cons p q implies that fi(p) = Just a

Thiemann Derivation Beyond Regular Languages 14 Oct 2018 41 / 77

Example (cont’d)

Embedding from Derivations

d(a, a∗.b∗) = (1.a∗).b∗ + 1.0 = (1.a∗).b∗

ua = λ(Seq (Seq Eps p) q).Seq (Seq Eps (Cons (Sym a) p)) q t−

a =

λ(Seq (Seq Eps (Cons (Sym a) p)) q).(Seq (Seq Eps p) q)

d(b, a∗.b∗) = 0.b∗ + 1.b∗ = 1.b∗

ub = λ(Seq Eps q).Seq (Cons (Sym b) Empty) q t−

b = λ(Seq (Cons (Sym b) Empty) q).Seq Eps q

Thiemann Derivation Beyond Regular Languages 14 Oct 2018 42 / 77

Step 2: Construtive Embeddings

Coinductive Definition of Embedding

N(s) λEps.mkE(s) : 1 s N(s′) ⊢a t : s′ → s λ(Sym a).t(mkE(s′)) : a s t1 : r1 s t2 : r2 s λy.case y of Inl p → Inl (t1 p) | Inr q → Inr (t2 q) : r1 + r2 s

Thiemann Derivation Beyond Regular Languages 14 Oct 2018 43 / 77

Constructing the Embedding (2)

Coinductive Definition of Embedding (2)

⊢−

a t− a : r1 → r′ 1

⊢a ua : s′ → s t1 : r′

1.r2 s′

t2 : r2 s λ(Seq p1 p2). if fi(p1) = Nothing then (t2 p2) else fi(p1) = Just a; ua(t1(Seq (t−

a p1) p2))

: r1.r2 s

Thiemann Derivation Beyond Regular Languages 14 Oct 2018 44 / 77

Constructing the Embedding (3)

Coinductive Definition of Embedding (3)

⊢−

a t− a : r1 → r′ 1

⊢a ua : s′ → s t1 : r′.r∗ s′ t2 : r∗ s λy.case y of Empty → mkE(s) Cons p q → if fi(p) = Nothing then (t2 q) else fi(p) = Just a; ua(t1(Seq (t−

a p) q))

: r∗ s

Thiemann Derivation Beyond Regular Languages 14 Oct 2018 45 / 77

Example

Embedding for 1 + a∗b a∗b∗

t = λy.case y of Inl p → Inl (t1 p) | Inr q → Inr (t2 q) t1 = λEps.mkE(a∗b∗) : 1 a∗b∗ t2 = λ(Seq p1 p2).case fi(p1) of Nothing → (t22 p2) Just a → ua(t21a(Seq (t−

a p1) p2))

Just b → ub(t21b(Seq (t−

b p1) p2)) : a∗.b a∗.b∗

t21a = λ(Seq p (Sym b)).Seq p Cons (Sym b) Empty : (1.a∗).b (1.a∗).b∗ t21b = λEps.mkE(1.b∗) : 1 1.b∗ t22 = λ(Sym b).t22b(mkE(1.b∗)) : b a∗.b∗ ⊢b t22b = λ(Seq Eps p).Seq Empty (Cons (Sym b) p) : 1.b∗ → a∗.b∗

Thiemann Derivation Beyond Regular Languages 14 Oct 2018 46 / 77

Embedding (continued)

Embeddings From Derivation

⊢a ua = λ(Seq (Seq Eps p) q).Seq (Cons (Sym a) p) q : (1.a∗).b∗ → a∗.b∗ ⊢−

a t− a =

λ(Seq (Cons (Sym a) p) q).Seq (Seq Eps p) q : a∗.b∗ → (1.a∗).b∗ ⊢b ub = λ(Seq Eps q).Seq Empty (Cons (Sym b) q) : 1.b∗ → a∗.b∗ ⊢−

b t− b = λ(Seq Empty (Cons (Sym b) q)).Seq Eps q

: a∗.b∗ → 1.b∗

Thiemann Derivation Beyond Regular Languages 14 Oct 2018 47 / 77

Plan

1 Derivatives for Regular Languages

Brzozowski Derivatives Antimirov’s Partial Derivatives Applications: Equivalence and Containment Computational Interpretation

2 Derivatives for Context-Free Languages

Pragmatic Parsing Deriving Automata Computational Interpretation

3 Derivation for ω-Regular Languages

Attempt #1: Brzozowski-style Derivatives Attempt #2: Extending Partial Derivatives

Thiemann Derivation Beyond Regular Languages 14 Oct 2018 48 / 77

Context-Free Grammars

Context-Free Grammar (CFG)

G = (N, Σ, P, S) N a finite set of nonterminal symbols Σ an alphabet (terminal symbols) P : N → Pfin(N ∪ Σ)∗ a (finite) set of productions S ∈ N distinguished start symbol

Thiemann Derivation Beyond Regular Languages 14 Oct 2018 49 / 77

Plan

1 Derivatives for Regular Languages

Brzozowski Derivatives Antimirov’s Partial Derivatives Applications: Equivalence and Containment Computational Interpretation

2 Derivatives for Context-Free Languages

Pragmatic Parsing Deriving Automata Computational Interpretation

3 Derivation for ω-Regular Languages

Attempt #1: Brzozowski-style Derivatives Attempt #2: Extending Partial Derivatives

Thiemann Derivation Beyond Regular Languages 14 Oct 2018 50 / 77

Derivative of Nonterminal A by a

Algorithm

Let A ∈ N and a ∈ Σ. Let W = {A}. Choose and remove A from W . Exhaustively:

If A → αaβ ∈ P and N(α), then add Aa → β ∈ P. If A → αBβ ∈ P and N(α), then add Aa → Baβ ∈ P; if Ba / ∈ N, then W = W ∪ {B}.

Repeat until W = ∅

Thiemann Derivation Beyond Regular Languages 14 Oct 2018 51 / 77

Derivative of Nonterminal A by a

Algorithm

Let A ∈ N and a ∈ Σ. Let W = {A}. Choose and remove A from W . Exhaustively:

If A → αaβ ∈ P and N(α), then add Aa → β ∈ P. If A → αBβ ∈ P and N(α), then add Aa → Baβ ∈ P; if Ba / ∈ N, then W = W ∪ {B}.

Repeat until W = ∅

Remark

Terminates with L(Aa) = a−1L(A) (prove this!) Iterated derivative does not always terminate Termination can be improved by mapping Aa to an existing nonterminal with equivalent productions

Thiemann Derivation Beyond Regular Languages 14 Oct 2018 51 / 77

Pragmatic Parsing

Algorithm

To parse w compute Sw and test N(Sw).

Remark

Effective procedure Basis for recent work on parsing with derivatives by Adams, Darais, Hollenbeck, Might, Spievak Space leak for many grammars

Thiemann Derivation Beyond Regular Languages 14 Oct 2018 52 / 77

Pragmatic Parsing Example

CFG for a∗b (right recursive)

S → b, S → aS Derivatives: Sa → S, Sb → ε Iterated derivation terminates: Saa ≈ Sa, Sab ≈ Sb

Thiemann Derivation Beyond Regular Languages 14 Oct 2018 53 / 77

Pragmatic Parsing Example

CFG for a∗b (right recursive)

S → b, S → aS Derivatives: Sa → S, Sb → ε Iterated derivation terminates: Saa ≈ Sa, Sab ≈ Sb

CFG for a∗b (left recursive)

S → Ab, A → ε, A → Aa Derivatives: Sa → Aab, Aa → Aaa, Aa → ε, Sb → ε Iterated derivation terminates: Aa ≈ A ⇒ Sa ≈ S

Thiemann Derivation Beyond Regular Languages 14 Oct 2018 53 / 77

Pragmatic Parsing: Context-Free Example

CFG for anbn (a linear language)

S → ε, S → aSb Derivatives: Sa → Sb, nothing for Sb Iterated derivatives: Saa → Sab, Sab → ε Iterated derivatives: Saaa → Saab, Saab → Sabb etc

Thiemann Derivation Beyond Regular Languages 14 Oct 2018 54 / 77

A Context-Free Derivative Construction

[Winter, Bonsangue, Rutten 2011]

Every CFL has a grammar in Greibach Normal Form (GNF) I.e. every production has the form A0 → aA1 . . . An ⇒ Nonterminal A0 has derivative A1 . . . An Drawback: Requires grammar in GNF Every CFG of size n can be converted to a CFG in GNF of size O(n4)

Thiemann Derivation Beyond Regular Languages 14 Oct 2018 55 / 77

Plan

1 Derivatives for Regular Languages

Brzozowski Derivatives Antimirov’s Partial Derivatives Applications: Equivalence and Containment Computational Interpretation

2 Derivatives for Context-Free Languages

Pragmatic Parsing Deriving Automata Computational Interpretation

3 Derivation for ω-Regular Languages

Attempt #1: Brzozowski-style Derivatives Attempt #2: Extending Partial Derivatives

Thiemann Derivation Beyond Regular Languages 14 Oct 2018 56 / 77

Plan

1 Derivatives for Regular Languages

Brzozowski Derivatives Antimirov’s Partial Derivatives Applications: Equivalence and Containment Computational Interpretation

2 Derivatives for Context-Free Languages

Pragmatic Parsing Deriving Automata Computational Interpretation

3 Derivation for ω-Regular Languages

Attempt #1: Brzozowski-style Derivatives Attempt #2: Extending Partial Derivatives

Thiemann Derivation Beyond Regular Languages 14 Oct 2018 57 / 77

Plan

1 Derivatives for Regular Languages

Brzozowski Derivatives Antimirov’s Partial Derivatives Applications: Equivalence and Containment Computational Interpretation

2 Derivatives for Context-Free Languages

Pragmatic Parsing Deriving Automata Computational Interpretation

3 Derivation for ω-Regular Languages

Attempt #1: Brzozowski-style Derivatives Attempt #2: Extending Partial Derivatives

Thiemann Derivation Beyond Regular Languages 14 Oct 2018 58 / 77

ω-Languages

Let ω denote the set of natural numbers Σω

the set of maps from ω → Σ the set of infinite sequences over Σ the set of infinite Σ-words

P(Σω) the set of ω-languages over Σ concatenation only defined for u ∈ Σ∗ and v ∈ Σω

Thiemann Derivation Beyond Regular Languages 14 Oct 2018 59 / 77

Operations on ω Languages

Product

Let U ⊆ Σ∗ and V ⊆ Σω U · V = {u · v | u ∈ U, v ∈ V }

Iteration

Let U ⊆ Σ∗ with ε / ∈ U Uω = U · Uω

Thiemann Derivation Beyond Regular Languages 14 Oct 2018 60 / 77

ω-Regular Languages

ω-regular expressions

Rω

Σ ∋ α, β ::= 0 | α + β | r.α | sω

where r, s ∈ RΣ and N(s) = 0.

Thiemann Derivation Beyond Regular Languages 14 Oct 2018 61 / 77

ω-Regular Languages

ω-regular expressions

Rω

Σ ∋ α, β ::= 0 | α + β | r.α | sω

where r, s ∈ RΣ and N(s) = 0.

Semantics

0 = ∅ α + β = α ∪ β r.α = r · α sω = sω

Thiemann Derivation Beyond Regular Languages 14 Oct 2018 61 / 77

ω-Regular Languages

ω-regular expressions

Rω

Σ ∋ α, β ::= 0 | α + β | r.α | sω

where r, s ∈ RΣ and N(s) = 0.

Semantics

0 = ∅ α + β = α ∪ β r.α = r · α sω = sω

Definition

An ω-language V is regular if there is an ω-regular expression α such that V = α.

Thiemann Derivation Beyond Regular Languages 14 Oct 2018 61 / 77

ω Automata

A nondeterministic B¨ uchi automaton (NBA)

M = (Q, Σ, q0, δ, F) consists of Q finite set of states Σ alphabet I ⊆ Q initial states δ : Q × Σ → PQ transition function F ⊆ Q accepting states M is deterministic (DFA) if |I| = 1 and, for all q, a, |δ(q, a)| = 1.

Thiemann Derivation Beyond Regular Languages 14 Oct 2018 62 / 77

Runs and Languages

Run

A run of M on a1, a2 · · · ∈ Σω is a sequence q0, q1, · · · ∈ Qω s.t. q0 ∈ I, for all i ∈ ω, qi+1 ∈ δ(q, ai). The run is (B¨ uchi-) accepting if qi ∈ F for infinitely many i ∈ ω.

Language of Automaton M

L(M) = {w ∈ Σω | M has an accepting run on w}

Theorem

An ω-language is regular iff it is accepted by an NBA.

Thiemann Derivation Beyond Regular Languages 14 Oct 2018 63 / 77

Plan

1 Derivatives for Regular Languages

Brzozowski Derivatives Antimirov’s Partial Derivatives Applications: Equivalence and Containment Computational Interpretation

2 Derivatives for Context-Free Languages

Pragmatic Parsing Deriving Automata Computational Interpretation

3 Derivation for ω-Regular Languages

Attempt #1: Brzozowski-style Derivatives Attempt #2: Extending Partial Derivatives

Thiemann Derivation Beyond Regular Languages 14 Oct 2018 64 / 77

Derivatives for ω-regular expressions

A Naive Attempt

Derivatives (Attempt #1)

da(0) = da(α + β) = da(α) ⊕ da(β) da(r.α) = (da(r) ⊙ α) ⊕ (da(α) ⊙ N(r)) da(sω) = da(s) ⊙ sω where ⊕, ⊙ are “clever” constructors that normalize modulo ≈.

Thiemann Derivation Beyond Regular Languages 14 Oct 2018 65 / 77

Properties of Brzozowski-style ω-Derivatives

Quotient

Lω(da(α)) = a−1Lω(α)

Representation

Lω(α) =

• a∈Σ

{a} · Lω(da(α))

Finiteness

For each r ∈ Rω

Σ, the set dΣ∗(r)/≈ is finite.

(≈ is a similarity relation on Rω

Σ)

Thiemann Derivation Beyond Regular Languages 14 Oct 2018 66 / 77

Properties of Brzozowski-style ω-Derivatives

Quotient

Lω(da(α)) = a−1Lω(α)

Representation

Lω(α) =

• a∈Σ

{a} · Lω(da(α))

Finiteness

For each r ∈ Rω

Σ, the set dΣ∗(r)/≈ is finite.

(≈ is a similarity relation on Rω

Σ)

ω-RE → ???

The analogous automaton construction does not work out because it yields deterministic automata.

Thiemann Derivation Beyond Regular Languages 14 Oct 2018 66 / 77

Counterexample

Example for DBA NBA: finitely many ’a’s

α = (a + b)∗.bω

Wrong automaton constructed by na¨ ıve extension

α start β a b b a α = (a + b)∗.bω β = α + bω deterministic! accepting states?

Thiemann Derivation Beyond Regular Languages 14 Oct 2018 67 / 77

Plan

1 Derivatives for Regular Languages

Brzozowski Derivatives Antimirov’s Partial Derivatives Applications: Equivalence and Containment Computational Interpretation

2 Derivatives for Context-Free Languages

Pragmatic Parsing Deriving Automata Computational Interpretation

3 Derivation for ω-Regular Languages

Attempt #1: Brzozowski-style Derivatives Attempt #2: Extending Partial Derivatives

Thiemann Derivation Beyond Regular Languages 14 Oct 2018 68 / 77

Insight

Construction for nondeterministic automaton needed!

Thiemann Derivation Beyond Regular Languages 14 Oct 2018 69 / 77

Insight

Construction for nondeterministic automaton needed! Alternative

Construction based on Antimirov’s partial derivatives

Thiemann Derivation Beyond Regular Languages 14 Oct 2018 69 / 77

Refresher: Antimirov’s Algorithm for Partial Derivatives

Step 1: Compute the set of Linear Factors lf(r)

lf(0) = {} lf(1) = {} lf(a) = {a, 1} lf(r.s) = lf(r) ⊙ s ∪ lf(s) ⊙ N(r) lf(r + s) = lf(r) ∪ lf(s) lf(r∗) = lf(r) ⊙ r∗

Thiemann Derivation Beyond Regular Languages 14 Oct 2018 70 / 77

Refresher: Antimirov’s Algorithm for Partial Derivatives

Step 1: Compute the set of Linear Factors lf(r)

lf(0) = {} lf(1) = {} lf(a) = {a, 1} lf(r.s) = lf(r) ⊙ s ∪ lf(s) ⊙ N(r) lf(r + s) = lf(r) ∪ lf(s) lf(r∗) = lf(r) ⊙ r∗

Step 2: Collect linear factors for each input symbol

∂a(r) = {r′ | a, r′ ∈ lf(r)}

Thiemann Derivation Beyond Regular Languages 14 Oct 2018 70 / 77

Extending to ω-Regular Expressions

Step 1: ω-Linear Factors

lfω : Rω

Σ → ℘(Σ × Rω Σ × {0, 1})

lfω(0) = ∅ lfω(α + β) = lfω(α) ∪ lfω(β) lfω(r.α) = lf(r) ⊙ α × {0} ∪ lfω(α) ⊙ N(r) lfω(sω) = lf(s) ⊙ sω × {1}

Thiemann Derivation Beyond Regular Languages 14 Oct 2018 71 / 77

Extending to ω-Regular Expressions

Step 1: ω-Linear Factors

lfω : Rω

Σ → ℘(Σ × Rω Σ × {0, 1})

lfω(0) = ∅ lfω(α + β) = lfω(α) ∪ lfω(β) lfω(r.α) = lf(r) ⊙ α × {0} ∪ lfω(α) ⊙ N(r) lfω(sω) = lf(s) ⊙ sω × {1}

Step 2: Collect ω-linear factors (as before)

∂ω

a (r) = {r′ | a, r′, u ∈ lfω(r)}

Thiemann Derivation Beyond Regular Languages 14 Oct 2018 71 / 77

Success?

NFA constructed by na¨ ıve extension of Antimirov’s algorithm

α start β a, b b b α = (a + b)∗.bω β = bω nondeterministic! Choose F = {β}

Thiemann Derivation Beyond Regular Languages 14 Oct 2018 72 / 77

Success?

NFA constructed by na¨ ıve extension of Antimirov’s algorithm

α start β a, b b b α = (a + b)∗.bω β = bω nondeterministic! Choose F = {β}

Assessment

Conjecture: states of the form sω are accepting Works for this particular expression, but not in general!

Thiemann Derivation Beyond Regular Languages 14 Oct 2018 72 / 77

Counterexample

Another expression for the same language . . .

α = (a + b)∗.(b.b∗)ω β = b∗.(b.b∗)ω

NFA constructed by na¨ ıve extension of Antimirov’s algorithm

α start β a, b b b nondeterministic! Choose F = {β} but why?

Thiemann Derivation Beyond Regular Languages 14 Oct 2018 73 / 77

A Working Approach

α = (a + b)∗.bω

Automaton Constructed from ω-Linear Factors

q0 start q1 start q2 start a a a b b b b lfω(α) = a, (a + b)∗.bω, 0 q0 b, (a + b)∗.bω, 0 q1 b, bω, 1} q2 Q = I = {q0, q1, q2} F = {b, bω, 1} Correct, though not minimal

Thiemann Derivation Beyond Regular Languages 14 Oct 2018 74 / 77

A Working Approach

α = (a + b)∗.bω

Automaton Constructed from ω-Linear Factors

q0 start q1 start q2 start a a a b b b b lfω(α) = a, (a + b)∗.bω, 0 q0 b, (a + b)∗.bω, 0 q1 b, bω, 1} q2 Q = I = {q0, q1, q2} F = {b, bω, 1} Correct, though not minimal Same automaton for α = (a + b)∗.(b.b∗)ω

Thiemann Derivation Beyond Regular Languages 14 Oct 2018 74 / 77

Main Result

Definition

For α ∈ Rω

Σ define the NBA B(α) = (Q, Σ, δ, I, F) by

Q = Q(α) (closure of lfω over symbols); I = lfω(α); F = {a, β, g ∈ Q | g = 1}; and δ(f , a) = ∂ω

a (f ).

Theorem

For α ∈ Rω

Σ: Lω(α) = Lω(B(α)).

Thiemann Derivation Beyond Regular Languages 14 Oct 2018 75 / 77

Conclusions

Derivatives for ω-regular expressions lose too much structure Linear factors retain enough information to construct a correct automaton Future work

identify equivalent states during construction inclusion test?

Thiemann Derivation Beyond Regular Languages 14 Oct 2018 76 / 77

Michael D. Adams, Celeste Hollenbeck, and Matthew Might. On the complexity and performance of parsing with derivatives. In PLDI, pages 224–236. ACM, 2016. Valentin M. Antimirov. Rewriting regular inequalities. In Proc. of FCT’95, volume 965 of LNCS, pages 116–125. Springer, 1995. Valentin M. Antimirov. Partial derivatives of regular expressions and finite automaton constructions. Theoretical Computer Science, 155(2):291–319, 1996. Janusz A. Brzozowski. Derivatives of regular expressions.

• J. ACM, 11(4):481–494, 1964.

Matthew Might, David Darais, and Daniel Spiewak. Parsing with derivatives: a functional pearl. In Proc. of ICFP’11, pages 189–195. ACM, 2011. Martin Sulzmann and Peter Thiemann. A computational interpretation of context-free expressions. In APLAS, volume 10695 of Lecture Notes in Computer Science, pages 387–405. Springer, 2017. Peter Thiemann. Partial derivatives for context-free languages - from µ-regular expressions to pushdown automata. In FoSSaCS, volume 10203 of Lecture Notes in Computer Science, pages 248–264, 2017. Peter Thiemann and Martin Sulzmann. From ω -regular expressions to b¨ uchi automata via partial derivatives. In Proc. of LATA’15, volume 8977 of LNCS, pages 287–298. Springer, 2015. Joost Winter, Marcello M. Bonsangue, and Jan J. M. M. Rutten. Context-free languages, coalgebraically. In CALCO, volume 6859 of LNCS, pages 359–376. Springer, 2011. Thiemann Derivation Beyond Regular Languages 14 Oct 2018 77 / 77