Complexity of Parikhs Theorem Anthony Widjaja Lin Yale-NUS College, - PowerPoint PPT Presentation

Lower bound for semilinear sets for CFGs Proposition : There is an infinite sequence { G n } n ∈ N of CFGs over Σ = { a } s.t. P ( L ( G n )) must have at least 2 Ω( | G n | ) linear sets. The CFG G n generates { a j : j ∈ [0 , 2 n − 1] } : ( 2 n linear sets!!) → S A 0 . . . A n − 1 → for each 0 ≤ i < n A i ε → for each 0 ≤ i < n A i B i → for each 0 < i < n B i B i − 1 B i − 1 → B 0 a ACTS 2015 – 13 / 43

Lower bound for semilinear sets for CFGs Proposition : There is an infinite sequence { G n } n ∈ N of CFGs over Σ = { a } s.t. P ( L ( G n )) must have at least 2 Ω( | G n | ) linear sets. The CFG G n generates { a j : j ∈ [0 , 2 n − 1] } : ( 2 n linear sets!!) → S A 0 . . . A n − 1 → for each 0 ≤ i < n A i ε → for each 0 ≤ i < n A i B i → for each 0 < i < n B i B i − 1 B i − 1 → B 0 a n.b. this kind of encoding for CFG is from (Stockmeyer-Meyer’73) ACTS 2015 – 13 / 43

Intro. Intro. Intro. Complexity Parikh’s Theorem (more precisely) CFG Case NFA Case NFA Case NFA case What about a fixed alphabet size? Normal Form Theorem for Semilinear Sets Normal Form Theorem for Parikh Images of NFA Parikh images of extensions of NFA Conclusion ACTS 2015 – 14 / 43

Exponential lower bound for DFAs Proposition : There is an infinite sequence {A n } n ∈ N of DFAs s.t. P ( L ( A n )) must have at least 2 Ω( |A n | ) linear sets. ACTS 2015 – 15 / 43

Exponential lower bound for DFAs Proposition : There is an infinite sequence {A n } n ∈ N of DFAs s.t. P ( L ( A n )) must have at least 2 Ω( |A n | ) linear sets. A n is over Σ n := { a 1 , . . . , a n +1 } : Σ n · Σ n · · · · · Σ n � �� � n copies ACTS 2015 – 15 / 43

Exponential lower bound for DFAs Proposition : There is an infinite sequence {A n } n ∈ N of DFAs s.t. P ( L ( A n )) must have at least 2 Ω( |A n | ) linear sets. A n is over Σ n := { a 1 , . . . , a n +1 } : Σ n · Σ n · · · · · Σ n � �� � n copies P ( L ( A n )) contains each ( r 1 , . . . , r n +1 ) s.t. � n +1 i =1 r i = n . ACTS 2015 – 15 / 43

Exponential lower bound for DFAs Proposition : There is an infinite sequence {A n } n ∈ N of DFAs s.t. P ( L ( A n )) must have at least 2 Ω( |A n | ) linear sets. A n is over Σ n := { a 1 , . . . , a n +1 } : Σ n · Σ n · · · · · Σ n � �� � n copies P ( L ( A n )) contains each ( r 1 , . . . , r n +1 ) s.t. � n +1 i =1 r i = n . There are � 2 n � ≥ 2 2 n − 1 √ n of these. n ACTS 2015 – 15 / 43

Exponential lower bound for DFAs Proposition : There is an infinite sequence {A n } n ∈ N of DFAs s.t. P ( L ( A n )) must have at least 2 Ω( |A n | ) linear sets. A n is over Σ n := { a 1 , . . . , a n +1 } : Σ n · Σ n · · · · · Σ n � �� � n copies P ( L ( A n )) contains each ( r 1 , . . . , r n +1 ) s.t. � n +1 i =1 r i = n . There are � 2 n � ≥ 2 2 n − 1 √ n of these. n Note: Σ n grows with n ACTS 2015 – 15 / 43

Intro. Intro. Intro. Complexity Parikh’s Theorem (more precisely) What about a fixed alphabet CFG Case NFA Case size? What about a fixed alphabet size? Chrobak-Martinez Kopczynski-Lin Proof outline Normal Form Theorem for Semilinear Sets Normal Form Theorem for Parikh Images of NFA Parikh images of extensions of NFA Conclusion ACTS 2015 – 16 / 43

Unary alphabet case: Chrobak-Martinez Theorem Let us assume that the alphabet is unary, i.e., Σ = { a } . ACTS 2015 – 17 / 43

Unary alphabet case: Chrobak-Martinez Theorem Let us assume that the alphabet is unary, i.e., Σ = { a } . Theorem (Chrobak-Martinez): Descriptional and computational complexity of Parikh Images of unary regular languages are polynomial. ACTS 2015 – 17 / 43

Unary alphabet case: Chrobak-Martinez Theorem Let us assume that the alphabet is unary, i.e., Σ = { a } . Theorem (Chrobak-Martinez): Descriptional and computational complexity of Parikh Images of unary regular languages are polynomial. Note : quadratically many union of arithmetic progressions with periods of linear size � suffice. ACTS 2015 – 17 / 43

Chrobak-Martinez Theorem in Action ACTS 2015 – 18 / 43

Chrobak-Martinez Theorem in Action Parikh image of L ( A ) is 8 + 4 N + 3 N ACTS 2015 – 18 / 43

Chrobak-Martinez Theorem in Action Parikh image of L ( A ) is 8 + 4 N + 3 N which is equal to (8 + 4 N ) ∪ (11 + 4 N ) ∪ (14 + 4 N ) ∪ (17 + 4 N ) ACTS 2015 – 18 / 43

Generalised Chrobak-Martinez Theorem Let Σ := { a 1 , . . . , a k } for fixed k ∈ Z > 0 . ACTS 2015 – 19 / 43

Generalised Chrobak-Martinez Theorem Let Σ := { a 1 , . . . , a k } for fixed k ∈ Z > 0 . Theorem (Kopczynski & Lin’10): Descriptional and computational complexity of Parikh Images of NFAs are polynomial. union of polynomially many linear sets with at most k polynomially-bounded � periods Complexities are exponential in k � Generalizes Chrobak-Martinez Theorem (case k = 1 ). � ACTS 2015 – 19 / 43

Outline of proof Normal-Form Theorem for Semilinear sets. � Normal-Form Theorem for Parikh images of NFA � ACTS 2015 – 20 / 43

Intro. Intro. Intro. Complexity Parikh’s Theorem (more precisely) Normal Form Theorem for CFG Case NFA Case Semilinear Sets What about a fixed alphabet size? Normal Form Theorem for Semilinear Sets Real cones Real cone example Caratheodory’s thm Our theorem Intuition Intuition Higher dim. Normal Form Theorem for Parikh Images of NFA Parikh images of ACTS 2015 – 21 / 43 extensions of NFA

Digression to Convex Geometry: Real Cones Given V = { v 1 , . . . , v n } ⊆ R k , define the real cone over V : cone ( V ) := { Σ n i =1 a i v i : a i ∈ R ≥ 0 } . Note : akin to definition of vector subspace of R k “spanned” by some set V (but ...) ACTS 2015 – 22 / 43

Examples 1 0.8 0.6 0.4 0.2 0 0 0.2 0.4 1 0.6 0.8 0.6 0.8 0.4 0.2 1 0 Real cone of (1 , 1 , 0 . 5) , (1 , 0 . 5 , 1) , (0 . 5 , 1 , 1) ACTS 2015 – 23 / 43

Examples 1 0.8 0.6 0.4 0.2 1 0 0 1 0.5 0 -0.5 -1 -1 Real cone over { 1 , − 1 } × { 1 , − 1 } × { 1 } ACTS 2015 – 23 / 43

Caratheodory’s theorem Theorem : Given a finite V ⊆ R k of rank d ≤ k , we have � cone ( V ′ ) . cone ( V ) = V ′ ⊆ V, | V ′ | = � V ′ � = d Cones over R k can be decomposed into smaller subcones with ≤ k vertices � ACTS 2015 – 24 / 43

Caratheodory’s theorem Theorem : Given a finite V ⊆ R k of rank d ≤ k , we have � cone ( V ′ ) . cone ( V ) = V ′ ⊆ V, | V ′ | = � V ′ � = d Cones over R k can be decomposed into smaller subcones with ≤ k vertices � Note : if k is fixed, there are only polynomially many such subcones. � ACTS 2015 – 24 / 43

Caratheodory’s theorem (example) 1 0.8 0.6 0.4 0.2 1 0 0 1 0.5 0 -0.5 -1 -1 Real cone over { 1 , − 1 } × { 1 , − 1 } × { 1 } ACTS 2015 – 25 / 43

Caratheodory’s theorem (example) 1 0.8 0.6 0.4 0.2 -1 0 0 1 0.5 0 -0.5 1 -1 Real cone over (1 , 1 , 1) , (1 , − 1 , 1) , ( − 1 , 1 , 1) ACTS 2015 – 25 / 43

Caratheodory’s theorem (example) 1 0.8 0.6 0.4 0.2 -1 0 0 1 0.5 0 -0.5 1 -1 Real cone over ( − 1 , − 1 , 1) , (1 , − 1 , 1) , ( − 1 , 1 , 1) ACTS 2015 – 25 / 43

Caratheodory-like theorem for linear sets Question : Does the integer version of Caratheodory’s theorem hold? � ACTS 2015 – 26 / 43

Caratheodory-like theorem for linear sets Question : Does the integer version of Caratheodory’s theorem hold? � Unfortunately, no! � ACTS 2015 – 26 / 43

Caratheodory-like theorem for linear sets Question : Does the integer version of Caratheodory’s theorem hold? � Unfortunately, no! � Fortunately, it does if you allow nonzero offsets :) � ACTS 2015 – 26 / 43

Caratheodory-like theorem for linear sets Theorem : Fix k ∈ N . Given a finite V ⊆ Z k of rank d ≤ k , we can compute in pseudopolynomial time linear sets L ( w 1 ; S 1 ) , . . . , L ( w s ; S s ) s.t. each S i ⊆ V and | S i | = d s � L ( 0 ; V ) = L ( w i ; S i ) . i =1 Each number in w i is polynomially large (in unary) ACTS 2015 – 26 / 43

Caratheodory-like theorem for linear sets Theorem : Fix k ∈ N . Given a finite V ⊆ Z k of rank d ≤ k , we can compute in pseudopolynomial time linear sets L ( w 1 ; S 1 ) , . . . , L ( w s ; S s ) s.t. each S i ⊆ V and | S i | = d s � L ( 0 ; V ) = L ( w i ; S i ) . i =1 Each number in w i is polynomially large (in unary) When V ⊆ N k , each w i ∈ N k ACTS 2015 – 26 / 43

Caratheodory-like theorem for linear sets Theorem : Fix k ∈ N . Given a finite V ⊆ Z k of rank d ≤ k , we can compute in pseudopolynomial time linear sets L ( w 1 ; S 1 ) , . . . , L ( w s ; S s ) s.t. each S i ⊆ V and | S i | = d s � L ( 0 ; V ) = L ( w i ; S i ) . i =1 Each number in w i is polynomially large (in unary) When V ⊆ N k , each w i ∈ N k The same holds when the initial offset is v � = 0 ACTS 2015 – 26 / 43

Caratheodory-like theorem for linear sets Theorem : Fix k ∈ N . Given a finite V ⊆ Z k of rank d ≤ k , we can compute in pseudopolynomial time linear sets L ( w 1 ; S 1 ) , . . . , L ( w s ; S s ) s.t. each S i ⊆ V and | S i | = d s � L ( 0 ; V ) = L ( w i ; S i ) . i =1 Each number in w i is polynomially large (in unary) When V ⊆ N k , each w i ∈ N k The same holds when the initial offset is v � = 0 ACTS 2015 – 26 / 43

Caratheodory-like theorem for linear sets Theorem : Fix k ∈ N . Given a finite V ⊆ Z k of rank d ≤ k , we can compute in pseudopolynomial time linear sets L ( w 1 ; S 1 ) , . . . , L ( w s ; S s ) s.t. each S i ⊆ V and | S i | = d s � L ( 0 ; V ) = L ( w i ; S i ) . i =1 Use Caratheodory’s theorem ACTS 2015 – 26 / 43

Caratheodory-like theorem for linear sets Theorem : Fix k ∈ N . Given a finite V ⊆ Z k of rank d ≤ k , we can compute in pseudopolynomial time linear sets L ( w 1 ; S 1 ) , . . . , L ( w s ; S s ) s.t. each S i ⊆ V and | S i | = d s � L ( 0 ; V ) = L ( w i ; S i ) . i =1 Use Caratheodory’s theorem Use bounds from integer programming (Papadimitriou’83) for estimating the biggest entries in w i ’s ACTS 2015 – 26 / 43

Caratheodory-like theorem for linear sets Theorem : Fix k ∈ N . Given a finite V ⊆ Z k of rank d ≤ k , we can compute in pseudopolynomial time linear sets L ( w 1 ; S 1 ) , . . . , L ( w s ; S s ) s.t. each S i ⊆ V and | S i | = d s � L ( 0 ; V ) = L ( w i ; S i ) . i =1 Use Caratheodory’s theorem Use bounds from integer programming (Papadimitriou’83) for estimating the biggest entries in w i ’s Use dynamic programming to compute all w i ’s ACTS 2015 – 26 / 43

Intuition for dimension 2 15 14 13 12 11 10 9 8 7 6 5 4 3 2 1 0 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 ACTS 2015 – 27 / 43

Intuition for dimension 2 15 14 13 12 11 10 9 8 7 6 5 4 3 2 1 0 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 ACTS 2015 – 27 / 43

b b b b b b b b b b b b b b b b b b Intuition for dimension 2 15 14 13 12 11 10 9 8 7 6 5 4 3 2 1 0 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 ACTS 2015 – 27 / 43

b b b b b b b b b b b b b b b b b b Intuition for dimension 2 15 14 13 12 11 10 9 8 7 6 5 4 3 2 1 0 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 ACTS 2015 – 27 / 43

b b b b b b b b b b b b b b b b b b b b b b b b b b b b b b b Intuition for dimension 2 15 14 13 12 11 10 9 8 7 6 5 4 3 2 1 0 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 ACTS 2015 – 27 / 43

Intuition for dimension 2 Key observations : Each parallelogram has a quadratically many integer points � ACTS 2015 – 28 / 43

Intuition for dimension 2 Key observations : Each parallelogram has a quadratically many integer points � There is a natural ordering on these parallelograms: � The parallelogram above or to the right of a parallelogram is larger. � ACTS 2015 – 28 / 43

Intuition for dimension 2 Key observations : Each parallelogram has a quadratically many integer points � There is a natural ordering on these parallelograms: � The parallelogram above or to the right of a parallelogram is larger. � Once a point “appears”, it stays in the larger parallelograms. � ACTS 2015 – 28 / 43

Intuition for dimension 2 Key observations : Each parallelogram has a quadratically many integer points � There is a natural ordering on these parallelograms: � The parallelogram above or to the right of a parallelogram is larger. � Once a point “appears”, it stays in the larger parallelograms. � So, only need to keep track of minimal representatives M , i.e., � L ( 0 ; { (3 , 1) , (2 , 3) , (2 , 2) } ) = M + (3 , 1) N + (2 , 3) N where: M = { (2 , 2) , (4 , 4) , (6 , 6) , (8 , 8) , (10 , 10) , (12 , 12) } ACTS 2015 – 28 / 43

Intuition for dimension 2 Key observations : Each parallelogram has a quadratically many integer points � There is a natural ordering on these parallelograms: � The parallelogram above or to the right of a parallelogram is larger. � Once a point “appears”, it stays in the larger parallelograms. � So, only need to keep track of minimal representatives M , i.e., � L ( 0 ; { (3 , 1) , (2 , 3) , (2 , 2) } ) = M + (3 , 1) N + (2 , 3) N where: M = { (2 , 2) , (4 , 4) , (6 , 6) , (8 , 8) , (10 , 10) , (12 , 12) } |M| and all numbers in M are “not big”. � ACTS 2015 – 28 / 43

Case of dimension > 2 Need to use linear sets with different periods (unlike d = 2 ) � Replace finding two outermost vectors with Caratheodory’s � The rest are similar: � Dynamic programming, � Bounds from integer programming � ACTS 2015 – 29 / 43

Intro. Intro. Intro. Complexity Parikh’s Theorem (more precisely) Normal Form Theorem for CFG Case NFA Case Parikh Images of NFA What about a fixed alphabet size? Normal Form Theorem for Semilinear Sets Normal Form Theorem for Parikh Images of NFA Proof idea Comp. complex. Parikh images of extensions of NFA Conclusion ACTS 2015 – 30 / 43

Statement of the Theorem Let Σ := { a 1 , . . . , a k } for fixed k ∈ Z > 0 . Theorem (Kopczynski & Lin, LICS 2010): Descriptional and computational complexity of Parikh Images of NFAs are polynomial. poly many union of linear sets with at most k periods with poly-bounded � numbers Complexities are exponential in k . � Generalizes Chrobak-Martinez Theorem (case k = 1 ). � ACTS 2015 – 31 / 43

Proof idea: Path types Notation : Q = { q 0 , . . . , q n − 1 } with initial state q 0 and final state q n − 1 . ACTS 2015 – 32 / 43

Proof idea: Path types Notation : Q = { q 0 , . . . , q n − 1 } with initial state q 0 and final state q n − 1 . Given a path π and simple cycles C 1 , . . . , C m meeting with π : C 1 C 2 C m p π q ACTS 2015 – 32 / 43

Proof idea: Path types Notation : Q = { q 0 , . . . , q n − 1 } with initial state q 0 and final state q n − 1 . Given a path π and simple cycles C 1 , . . . , C m meeting with π : C 1 C 2 C m p π q The path type T π of π is the linear set L ( P ( π ); {P ( C i ) } m i =1 ) . ACTS 2015 – 32 / 43

Proof idea: Path types Notation : Q = { q 0 , . . . , q n − 1 } with initial state q 0 and final state q n − 1 . Given a path π and simple cycles C 1 , . . . , C m meeting with π : C 1 C 2 C m p π q The path type T π of π is the linear set L ( P ( π ); {P ( C i ) } m i =1 ) . There are at most n k O (1) many periods. ACTS 2015 – 32 / 43

Proof idea Characterization of Parikh images of L ( A ) Lemma: P ( L ( A )) = � π T π , where π ranges over paths from initial to final state of length at most O ( n 2 ) . ACTS 2015 – 33 / 43

Proof idea Characterization of Parikh images of L ( A ) Lemma: P ( L ( A )) = � π T π , where π ranges over paths from initial to final state of length at most O ( n 2 ) . Problem: There are 2 ( n + k ) O (1) such π . ACTS 2015 – 33 / 43

Proof idea Characterization of Parikh images of L ( A ) Lemma: P ( L ( A )) = � π T π , where π ranges over paths from initial to final state of length at most O ( n 2 ) . Problem: There are 2 ( n + k ) O (1) such π . BUT: can apply Caratheodory-like theorem on each T π : ACTS 2015 – 33 / 43

Proof idea Characterization of Parikh images of L ( A ) Lemma: P ( L ( A )) = � π T π , where π ranges over paths from initial to final state of length at most O ( n 2 ) . Problem: There are 2 ( n + k ) O (1) such π . BUT: can apply Caratheodory-like theorem on each T π : Corollary: P ( L ( A )) coincides with a union of polynomially many linear sets with polynomially large offsets and at most k polynomially large periods. ACTS 2015 – 33 / 43

Computational complexity Theorem: P ( L ( A )) coincides with a union of polynomially many linear sets with polynomially large offsets and at most k polynomially large periods. A naive implementation takes exponential time. Can improve to PTIME using dynamic programming! ACTS 2015 – 34 / 43

Intro. Intro. Intro. Complexity Parikh’s Theorem (more precisely) Parikh images of extensions of CFG Case NFA Case NFA What about a fixed alphabet size? Normal Form Theorem for Semilinear Sets Normal Form Theorem for Parikh Images of NFA Parikh images of extensions of NFA LG LG RBCA Conclusion ACTS 2015 – 35 / 43

Linear grammars → S A (5) → A aBb (6) → B aAb (7) → B ε (8) ACTS 2015 – 36 / 43

Linear grammars → S A (5) → A aBb (6) → B aAb (7) → B ε (8) The r.h.s. of a rule has at most 1 variable. ACTS 2015 – 36 / 43

Linear grammars → S A (5) → A aBb (6) → B aAb (7) → B ε (8) The r.h.s. of a rule has at most 1 variable. Proposition: Generalised Chrobak-Martinez Theorem extends to Linear Grammars. ACTS 2015 – 36 / 43

Linear grammars → S A (5) → A aBb (6) → B aAb (7) → B ε (8) The r.h.s. of a rule has at most 1 variable. Proposition: Generalised Chrobak-Martinez Theorem extends to Linear Grammars. Proof : Linear grammars are essentially NFA ... ACTS 2015 – 36 / 43

CFG of fixed “dimensions” A 1 A 1 A 2 a c A 1 a Theorem (Esparza-Ganty-Kiefer-Luttenberger’11): Generalised Chrobak-Martinez Theorem extends to CFG of fixed dimensions. Dimensions measure how many times “doubling tricks” possibly get used in a CFG. ACTS 2015 – 37 / 43

CFG of fixed “dimensions” A 1 A 1 A 2 a c A 1 a Theorem (Esparza-Ganty-Kiefer-Luttenberger’11): Generalised Chrobak-Martinez Theorem extends to CFG of fixed dimensions. Dimensions measure how many times “doubling tricks” possibly get used in a CFG. Linear grammars have dimension 0. ACTS 2015 – 37 / 43

CFG of fixed “dimensions” A 1 A 1 A 2 a c A 1 a Theorem (Esparza-Ganty-Kiefer-Luttenberger’11): Generalised Chrobak-Martinez Theorem extends to CFG of fixed dimensions. Dimensions measure how many times “doubling tricks” possibly get used in a CFG. Linear grammars have dimension 0. Proof : Compute an equivalent NFA of polynomial size and use Generalised Chrobak-Martinez Theorem. ACTS 2015 – 37 / 43