Announcements Final Exam Dates have been Computational Complexity - PDF document

Announcements  Final Exam Dates have been Computational Complexity announced  Tuesday, November 11  12:30 – 2:30pm  Room 70-2690  Conflicts? Let me know. Announcements Plan for this week  Today  Reminder: Exam 2  1st half  TMs and Languages  Will be returned after break  2nd Half  Computatbility  Course evaluations  Wednesday  Online  Complexity  Please complete!  Problem session. Plan for remainder of quarter Before We Start  Note change  Any questions?  Next week  Tuesday:  Cook’s Theorem  NP-Complete Problems  Thursday  Final Exam Review  Final problem session 1

Our complete picture: The Turing Machine Recursively Enumerable  Motivating idea Recursive  Build a theoretical a “human computer” Context Sensitive  Likened to a human with a paper and pencil that can solve problems in an algorithmic way Context Free  The theoretical machine provides a means to Deterministic Context Free determine:  If an algorithm or procedure exists for a given problem Finite  What that algorithm or procedure looks like  How long would it take to run this algorithm or Regular procedure. The Church-Turing Thesis (1936) Decision Problem  Any algorithmic procedure that can be  Let’s formalize this a bit carried out by a human or group of  A decision problem is a problem that has a humans can be carried out by some yes/no answer Turing Machine”  Equating algorithm with running on a TM  Example:  Turing Machine is still a valid  Is a given string x a palindrome (Is x ∈ pal?) computational model for most modern  Is a given context free language empty? computers. What makes a good Decision Problem algorithm?  Running a decision problem on a TM.  Suppose we know that a decision problem is decidable, how do we know  The problem must first be encoded that there is a good algorithm that  Example: solves the problem?  Is a given string x a palindrome (Is x ∈ pal?)  x is an instance of the probkem  Consider running time on a TM  Is a given context free language empty?  Actually, consider the complexity of a TM  Instance of a problem is a CFG…must be encoded. that can solve the problem. 2

Complexity Units  Complexity refers to the rate at which the  What does T(n) return? storage or time grows as a function of the  A “basic operation” can be defined as a input to the algorithm move a TM makes in accepting / rejecting  T(n) = time complexity (amount of time an a string algorithm will take based on input)  S(n) = space complexity (amount of space an  T(n) = number of TM moves algorithm will take based on input)  For the rest of this lecture, we’ll only consider T(n) though the discussion can also be applied to S(n) Asymptotic Analysis Asymptotic Analysis  Based on the idea that as the input to  Big-O (order of) an algorithm gets large  T(n) = O(f(n)) if and only if there are constants c 0 and n 0 such that  The complexity will become proportional to a known function.  T(n) ≤ c 0 f(n) for all n ≥ n 0  What this really means  Notations:  Eventually, when the size of the input gets  O (Big-O) – upper bounds on the complexity large enough, the upper bounds on the runtime  Θ (Big-Theta) – tight bounds on the complexity will be proportional to the function f(n). Algorithm Efficiencies Are you a good witch?  So what should be the cutoff between a “good” algorithm and a “bad” algorithm?  In the 1960s, Jack Edmonds proposed:  A “good” algorithm is one whose running time is a polynomial function of the size of the input  Other algorithms are “bad”  This definition was adopted:  A problem is called tractable if there exists a “good” (polynomial time) algorithm that solves it.  A problem is called intractable otherwise. 3

Is this a valid cutoff? The class P  The class P contains all decision problems that are decidable by an “algorithm” that runs in polynomial time.  Does this define a class of languages?  Yes…  The set of all problems whose encodings of “yes instances” (a language) is recognized by a TM M  M recognizes the above language in Polynomial Time.  Recall: These languages are recursive The class P The class P  Are there actually problems in P?  If we take the Church-Turing Thesis to be true:  Most everyday algorithms (e.g. sorting, searching) are in P  P is robust:  Are there problems not in P:  The problems contained in P remain in P even  Yes, we know that Self-Accepting is not in if we change our basic model of computation. P since it isn’t even solvable  A basic TM  A Sun, a Mac, even a PC running Windows Vista!  Are there decidable problems not in P?  Let’s take a look… Satisfiability (SAT) Satisfiability (SAT)  INSTANCE  Example of an instance  A Logical expression containing  (x 1 | x 2 | x 3 ) & (x 4 | !x 5 ) & !x 6 & (x 7 | x 8 )  variables x i  logical connectors &, |, and !  One Solution:  In conjuction normal form (C 1 & C 2 & C 3 … & C n )  PROBLEM  x 1 = true x 5 = false  Is there an assignment of truth values to each of  x 2 = false x 6 = false the variables such that the expression will  x 3 = false x 7 = true evaluate to true.  x 4 = false x 8 = false 4

Satisfiability (SAT) Satisfiability (SAT)  Naïve algorithm to solve  Can we do better?  Systematically consider all combinations of  No known polynomial algorithm assignments of True and False values for  Either one doesn’t exist all variables and test the expression  One exists and we haven’t found it yet.  In the worst case, for n variables  T(n) = O (2 n ) Non-deterministic TM Non-deterministic TM  Let’s reconsider the NDTM  A non-determininstic TM will accept w if there is at least one sequence of  Same as the ordinary TM except:  The transition function will return a set of triplets moves:  (q, x, D)  q 0 w  * α p β and p ∈ F  For each state / symbol combination, 0 or more transitions can be defined.  The machine can “choose” which transition to take. Non- deterministic TM  δ : Q x Γ → 2 Q x Γ x {R, L} Non-deterministic TM Non-deterministic TM  Simulating a NDTM on a TM  Simple backtracking to consider all paths through the TM.  One way to think about it…  A TM “replicates” itself whenever there is a choice.  Each “replication” continues computation along a given path 5

Non-deterministic TM The class NP  The class NP contains all decision problems that are  If we map all paths that can possibly be decidable by a non-deterministic “algorithm” that taken: runs in polynomial time.  For each w accepted, there is at least one accepting  Number of paths will be exponential sequence that will run in polynomial time.  Example:  Does this define a class of languages? -- Yes…  Suppose at each configuration there are 2  The set of all problems whose encodings of “yes instances” (a language) is recognized by a NDTM M possible moves.  M recognizes the above language in Polynomial Time.  Number of paths 2 n  Are there problems in NP  Well, for one SAT is Satisfiability (SAT) The class NP  Running SAT on a non-deterministic TM.  Clearly P is a subset of NP  Step 1: “Guess” a set of boolean assignments to each variable (this is the non-deterministic part – 2 n different choices)  Step 2: Evaluate the truth of the entire expression using the guessed assignment (this part is deterministic) NP  Can certainly do each step in polynomial time P Is there something in After 40 years of research…nobody here? knows Reducing one language to Reducing one language to another another  Formally (for complexity)  Worked well for decidability…Let’s use it here.  Basic idea  Let L 1 and L 2 be languages over Σ 1 and Σ 2  We say L 1 is polynomial time reducible to  Take an encoding of one problem L 2 (L 1 ≤ p L 2 ) if  Convert it to another problem we know to be in P  There exists a Turing computable function or NP  f: Σ 1 * → Σ 2 * such that  Conversion must be done in polynomial time!  x ∈ L 1 iff f(x) ∈ L 2  f can be computed in polynomial time. 6