Review Languages and Grammars Alphabets, strings, languages - - PDF document

1

CS 301 - Lecture 27 Complexity

Fall 2008

Review

• Languages and Grammars

– Alphabets, strings, languages

• Regular Languages

– Deterministic Finite and Nondeterministic Automata – Equivalence of NFA and DFA – Regular Expressions and Regular Grammars – Properties of Regular Languages – Languages that are not regular and the pumping lemma

• Context Free Languages

– Context Free Grammars – Derivations: leftmost, rightmost and derivation trees – Parsing, Ambiguity, Simplifications and Normal Forms – Nondeterministic Pushdown Automata – Pushdown Automata and Context Free Grammars – Deterministic Pushdown Automata – Pumping Lemma for context free grammars – Properties of Context Free Grammars

• Turing Machines

– Definition, Accepting Languages, and Computing Functions – Combining Turing Machines and Turing’s Thesis – Turing Machine Variations, Universal Turing Machine, and Linear Bounded Automata – Recursive and Recursively Enumerable Languages, Unrestricted Grammars – Context Sensitive Grammars and the Chomsky Hierarchy

• Computational Limits and Complexity

– Computability and Decidability – Complexity

Computational Complexity

Example: Given a set of n numbers x1, x2, …, xn is number y in the set? Algorithm to answer this:

• If x = x1, stop and answer yes
• If x = x2, stop and answer yes
• If x = x3, stop and answer yes
• …..
• If x = xn, stop and answer yes
• Answer no.

Time Required depends on n worst case requires n compares Let’s formalize this idea…

2

Time Complexity: Space Complexity: The number of steps during a computation Space used during a computation

Time Complexity

• Model Computation Using Turing machine
• We count the number of steps until

a string is accepted

• We use the O(n) notation

Example:

} : { ≥ = n b a L

n n

Algorithm to accept a string :

w

• Use a two-tape Turing machine
• Copy the on the second tape
• Compare the and

a a b |) (| w O

• Copy the on the second tape
• Compare the and

a a b

Time needed:

|) (| w O

Total time:

|) (| w O } : { ≥ = n b a L

n n

3

For string of length time needed for acceptance:

} : { ≥ = n b a L

n n

n ) (n O

Language class:

) (n DTIME

A Deterministic Turing Machine accepts each string of length in time

) (n DTIME

1

L

2

L

3

L ) (n O n ) (n DTIME } : { ≥ n b a

n n

In a similar way we define the class

)) ( ( n T DTIME ) (n T

for any time function: Examples:

),... ( ), (

3 2

n DTIME n DTIME

4

Example: The membership problem for context free languages

} grammar by generated is : { G w w L = ) ( 3 n DTIME L∈

(CYK - algorithm)

Polynomial time Theorem:

) ( k n DTIME ) (

1 + k

n DTIME ) ( ) (

1 k k

n DTIME n DTIME ⊂

+

Polynomial time algorithms:

) ( k n DTIME

Represent tractable algorithms: For small we can compute the result fast

k

Example: The Satisfiability Problem Boolean expressions in Conjunctive Normal Form:

k

t t t t ∧ ∧ ∧ ∧ 

3 2 1 p i

x x x x t ∨ ∨ ∨ ∨ = 

3 2 1

Variables Question: is expression satisfiable?

5

) ( ) (

3 1 2 1

x x x x ∨ ∧ ∨

Satisfiable? Can you find values for x1, x2, and x3 so that

1 ) ( ) (

3 1 2 1

= ∨ ∧ ∨ x x x x

Example:

) ( ) (

3 1 2 1

x x x x ∨ ∧ ∨

Satisfiable:

1 , 1 ,

3 2 1

= = = x x x 1 ) ( ) (

3 1 2 1

= ∨ ∧ ∨ x x x x

Example:

2 1 2 1

) ( x x x x ∧ ∧ ∨

Example: Satisfiable? Can you find values for x1, x2, and x3 so that

2 1 2 1

) ( x x x x ∧ ∧ ∨

Not satisfiable Example:

6

e} satisfiabl is expression : { w w L =

For variables, combinations to try

n

Algorithm: search exhaustively all the possible binary values of the variables Exponential, can we do better? Exponential time algorithms:

) 2 ( n DTIME

Represent intractable algorithms: Some problem instances may take centuries to solve Example: The satisfiability problem Non-Deterministic algorithm:

• Guess an assignment of the variables

e} satisfiabl is expression : { w w L =

• Check if this is a satisfying assignment

Time for variables:

n ) (n O e} satisfiabl is expression : { w w L =

Total time:

• Guess an assignment of the variables
• Check if this is a satisfying assignment

) (n O ) (n O

7

Non-Determinism

Language class:

) (n NTIME

A Non-Deterministic Turing Machine accepts each string of length in time

) (n NTIME

1

L

2

L

3

L ) (n O n

Example:

} {ww L =

Non-Deterministic Algorithm to accept a string :

ww

• Use a two-tape Turing machine
• Guess the middle of the string

and copy on the second tape

• Compare the two tapes

w |) (| w O

Time needed:

|) (| w O

Total time:

|) (| w O } {ww L =

• Use a two-tape Turing machine
• Guess the middle of the string

and copy on the second tape

• Compare the two tapes

w ) (n NTIME } {ww L =

8

In a similar way we define the class

)) ( ( n T NTIME ) (n T

for any time function: Examples:

),... ( ), (

3 2

n NTIME n NTIME

Non-Deterministic Polynomial time algorithms:

) ( k n NTIME L∈

Selecting the “Right” Machine

• Model Computation Using Turing machine
• But the choice of machine seems to

matter a great deal!

• Two tapes machines might take fewer

steps than one tape machines

• Non-Deterministic machines might take

fewer steps than Deterministic ones

• Is there a “right” choice to make?

Selecting the “Right” Machine

• If a two-tape machine takes steps, a one

tape machine can simulate this in

• If a non-deterministic machine takes

steps, a deterministic one can simulate this in

So do we need classes for each machine type??

9

What’s Next

• Read

– Linz Chapter 1,2.1, 2.2, 2.3, (skip 2.4), 3, 4, 5, 6.1, 6.2, (skip 6.3), 7.1, 7.2, 7.3, (skip 7.4), 8, 9, 10, 11, 12.1, 12.2, (skip 12.3, 12.4, 12.5, 13), 14.1, 14.2, and 14.3 – JFLAP Chapter 1, 2.1, (skip 2.2), 3, 4, 5, 6, 7, (skip 8), 9, (skip 10), 11

• Next Lecture Topics from 14.4 - 14.7

– P, NP, Reductions, and NP-complete problems

• Quiz 4 Wednesday 12/3

– Closed book, but you may bring one sheet of 8.5 x 11 inch paper with any notes you like.

• Homework

– Homework 13 Due Today – Homework 14 Due Thursday – Homework 15 Due Tuesday 12/9 – Homework 16 = study for the final!!