BU CS 332 Theory of Computation Lecture 18: Reading: Time - - PowerPoint PPT Presentation

▶
bu cs 332 theory of computation
SMART_READER_LITE
LIVE PREVIEW

BU CS 332 Theory of Computation Lecture 18: Reading: Time - - PowerPoint PPT Presentation

Mar 23, 2023 381 likes •680 views

BU CS 332 Theory of Computation Lecture 18: Reading: Time Complexity Sipser Ch 7.1 7.2 Complexity Class P Mark Bun April 6, 2020 Where we are in CS 332 Automata & Formal Languages Computability Complexity Previous unit:

slide-1
SLIDE 1

BU CS 332 – Theory of Computation

Lecture 18:

  • Time Complexity
  • Complexity Class P

Reading: Sipser Ch 7.1‐7.2

Mark Bun April 6, 2020

slide-2
SLIDE 2

Where we are in CS 332

4/6/2020 CS332 ‐ Theory of Computation 2

Automata & Formal Languages Computability Complexity

Previous unit: Computability theory What kinds of problems can / can’t computers solve? Final unit: Complexity theory What kinds of problems can / can’t computers solve under constraints on their computational resources?

slide-3
SLIDE 3

First topic: Time complexity

Today: Answering the basic questions

  • 1. How do we measure complexity? (as in CS 330)
  • 2. Asymptotic notation (as in CS 330)
  • 3. How robust is the TM model when we care about

measuring complexity?

  • 4. How do we mathematically capture our intuitive

notion of “efficient algorithms”?

4/6/2020 CS332 ‐ Theory of Computation 3

slide-4
SLIDE 4

Running time analysis

Time complexity of a TM (algorithm) = maximum number of steps it takes on a worst‐case input Formally: Let . A TM runs in time if on every input

∗,

halts on within at most steps ‐ Focus on worst‐case running time: Upper bound of must hold for all inputs of length ‐ Exact running time does not translate well between computational models / real computers. Instead focus on asymptotic complexity.

4/6/2020 CS332 ‐ Theory of Computation 4

slide-5
SLIDE 5

Example

How much time does it take for a basic single‐tape TM to decide

  • ?

Let’s analyze one particular TM : = “On input :

  • 1. Scan input and reject if not of the form ∗

∗

  • 2. While input contains both ’s and ’s:

Cross off one and one

  • 3. Accept if no 0’s and no 1’s left. Otherwise, reject.”

4/6/2020 CS332 ‐ Theory of Computation 5

slide-6
SLIDE 6

Review of asymptotic notation

‐notation (upper bounds) means: There exist constants

  • such that

for every

  • Example:
  • (

, )

4/6/2020 CS332 ‐ Theory of Computation 6

slide-7
SLIDE 7

Caution: does not mean “equals”

Not reflexive: does not mean Example:

  • Alternative (better) notation:

4/6/2020 CS332 ‐ Theory of Computation 7

slide-8
SLIDE 8

Examples

4/6/2020 CS332 ‐ Theory of Computation 8

  • 6

3 2

slide-9
SLIDE 9

Review of asymptotic notation

‐notation (lower bounds) means: There exist constants

  • such that

for every

  • Example:

( , )

4/6/2020 CS332 ‐ Theory of Computation 9

slide-10
SLIDE 10

When should we use vs. ?

Upper bounds: Use “The merge‐sort algorithm uses at most comparisons in the worst case” Lower bounds: Use “Every comparison‐based sorting algorithm requires at least comparisons in the worst case”

4/6/2020 CS332 ‐ Theory of Computation 10

slide-11
SLIDE 11

Review of asymptotic notation

‐notation (tight bounds) means: AND Example:

  • Generally, polynomials are easy:

𝑒 𝑒

1

𝑒

4/6/2020 CS332 ‐ Theory of Computation 11

slide-12
SLIDE 12

Little‐oh and little‐omega

4/6/2020 CS332 ‐ Theory of Computation 12

‐notation and ‐notation are like  and ; ‐notation and ‐notation are like and Example:

  • (

) means: For every constant there exists such that for every

slide-13
SLIDE 13

A handy‐dandy chart

4/6/2020 CS332 ‐ Theory of Computation 13

Notation … means … Think… Example lim

←

  • f(n)=O(g(n))

∃ c>0, n0>0, ∀ n > n0 : f(n) < cg(n) Upper bound 100n2 = O(n3) If it exists, it is < ∞ f(n)=(g(n)) ∃c>0, n0>0, ∀ n > n0 : cg(n) < f(n) Lower bound 2n = (n100) If it exists, it is > 0 f(n)=(g(n)) both of the above: f=(g) and f = O(g) Tight bound log(n!) = (n log n) If it exists, it is > 0 and < ∞ f(n)=o(g(n)) ∀ c>0, ∃ n0>0, ∀ n > n0 : f(n) < cg(n) Strict upper bound n2 = o(2n) Limit exists, =0 f(n)=(g(n)) ∀ c>0, ∃n0>0, ∀ n > n0 : cg(n) < f(n) Strict lower bound n2 = (log n) Limit exists, =∞

slide-14
SLIDE 14

Asymptotic notation within expressions

Asymptotic notation within an expression is shorthand for an unspecified function satisfying the statement Examples:

  • 4/6/2020

CS332 ‐ Theory of Computation 14

slide-15
SLIDE 15

FAABs: Frequently asked asymptotic bounds

4/6/2020 CS332 ‐ Theory of Computation 15

  • Polynomials.

1 𝑒 𝑒 is  𝑒

if

𝑒

  • Logarithms.

𝑏

𝑐

for all constants For every ,

  • Exponentials. For all

and all ,

𝑒 𝑜

  • Factorial.

By Stirling’s formula,

slide-16
SLIDE 16

Time Complexity

4/6/2020 CS332 ‐ Theory of Computation 16

slide-17
SLIDE 17

Time complexity classes

Let is a class (i.e., set) of languages: A language if there exists a basic single‐ tape (deterministic) TM that 1) Decides , and 2) Runs in time

4/6/2020 CS332 ‐ Theory of Computation 17

slide-18
SLIDE 18

Example

𝐵 01 𝑛 0 𝑁 = “On input 𝑥:

  • 1. Scan input and reject if not of the form 0∗1∗
  • 2. While input contains both 0’s and 1’s:

Cross off one 0 and one 1

  • 3. Accept if no 0’s and no 1’s left. Otherwise, reject.”
  • runs in time
  • Is there a faster algorithm?

4/6/2020 CS332 ‐ Theory of Computation 18

slide-19
SLIDE 19

Example

𝐵 01 𝑛 0 𝑁′ = “On input 𝑥:

  • 1. Scan input and reject if not of the form 0∗1∗
  • 2. While input contains both 0’s and 1’s:
  • Reject if the total number of 0’s and 1’s remaining is odd
  • Cross off every other 0 and every other 1
  • 3. Accept if no 0’s and no 1’s left. Otherwise, reject.”
  • Running time of

:

  • Is there a faster algorithm?

4/6/2020 CS332 ‐ Theory of Computation 19

slide-20
SLIDE 20

Example

Running time of : Theorem (Sipser, Problem 7.49): If can be decided in time on a 1‐tape TM, then is regular

4/6/2020 CS332 ‐ Theory of Computation 20

slide-21
SLIDE 21

Does it matter that we’re using the 1‐tape model for this result?

It matters: 2‐tape TMs can decide faster

𝑁′′ = “On input 𝑥:

  • 1. Scan input and reject if not of the form 0∗1∗
  • 2. Copy 0’s to tape 2
  • 3. Scan tape 1. For each 1 read, cross of a 0 on tape 2
  • 4. If 0’s on tape 2 finish at same time as 1’s on tape 1, accept.

Otherwise, reject.”

Analysis: is decided in time

  • n a 2‐tape TM

Moral of the story (part 1): Unlike decidability, time complexity depends on the TM model

4/6/2020 CS332 ‐ Theory of Computation 21

slide-22
SLIDE 22

How much does the model matter?

Theorem: Let be a function. Every multi‐tape TM running in time has an equivalent single‐tape TM running in time

  • Proof idea:

We already saw how to simulate a multi‐tape TM with a single‐tape TM Need a runtime analysis of this construction Moral of the story (part 2): Time complexity doesn’t depend too much on the TM model (as long as it’s deterministic, sequential)

4/6/2020 CS332 ‐ Theory of Computation 22

slide-23
SLIDE 23

Simulating Multiple Tapes

Implementation‐Level Description On input

  • 1. Format tape into
  • 2. For each move of

: Scan left‐to‐right, finding current symbols Scan left‐to‐right, writing new symbols, Scan left‐to‐right, moving each tape head If a tape head goes off the right end, insert blank If a tape head goes off left end, move back right

4/6/2020 CS332 ‐ Theory of Computation 23

slide-24
SLIDE 24

How much does the model matter?

Theorem: Let be a function. Every multi‐tape TM running in time has an equivalent single‐tape TM running in time

  • Proof: Time analysis of simulation
  • Time to initialize (i.e., format tape):
  • Time to simulate one step of multi‐tape TM:
  • Number of steps to simulate:

=> Total time:

4/6/2020 CS332 ‐ Theory of Computation 24

slide-25
SLIDE 25

Extended Church‐Turing Thesis

Every “reasonable” model of computation can be simulated by a basic, single‐tape TM with only a polynomial slowdown. E.g., doubly infinite TMs, multi‐tape TMs, RAM TMs Does not include nondeterministic TMs (not reasonable) Possible counterexamples? Randomized computation, parallel computation, DNA computing, quantum computation

4/6/2020 CS332 ‐ Theory of Computation 25

slide-26
SLIDE 26

Complexity Class

4/6/2020 CS332 ‐ Theory of Computation 26

slide-27
SLIDE 27

Complexity class

Definition: is the class of languages decidable in polynomial time on a basic single‐tape (deterministic) TM

  • Class doesn’t change if we substitute in another

reasonable deterministic model (Extended Church‐Turing)

  • Cobham‐Edmonds Thesis: Captures class of problems that

are feasible to solve on computers

4/6/2020 CS332 ‐ Theory of Computation 27

slide-28
SLIDE 28

Examples of languages in

  • 𝑄𝐵𝑈𝐼

𝐻, 𝑡, 𝑢 𝐻 is a directed graph with a directed path from 𝑡 to 𝑢

  • 𝐵

𝐸, 𝑥 𝐸 is a DFA that accepts input 𝑥

  • 𝑆𝐹𝑀𝑄𝑆𝐽𝑁𝐹

𝑦, 𝑧 𝑦 and 𝑧 are relatively prime

  • 𝑄𝑆𝐽𝑁𝐹𝑇

𝑦 𝑦 is prime

  • Every context‐free language (section tomorrow)

4/6/2020 CS332 ‐ Theory of Computation 28

2006 Gödel Prize citation The 2006 Gödel Prize for outstanding articles in theoretical computer science is awarded to Manindra Agrawal, Neeraj Kayal, and Nitin Saxena for their paper "PRIMES is in P." In August 2002 one of the most ancient computational problems was finally solved….