Homework Homework 6 & 7 Computational Complexity Returned - PDF document

Homework • Homework 6 & 7 Computational Complexity – Returned today • Homework 8 – Due today if you would like it back by Thursday. – Otherwise due Thursday. Before we start Before we start • Pumping lemma for CFLs • Post Correspondence Problem – Choose one appropriate u such that |u| ≥ n – A solution to the PCP is a finite set of indicies – Must consider all possible assignments of – Repeating of indicies is allowed vwxyz for that particular u. – Set of solutions is infinite. – Must show that vw k xy k z fails for any k ≥ 0. Before we start The Turing Machine • Any questions? • Motivating idea – Build a theoretical a “human computer” – Likened to a human with a paper and pencil that can solve problems in an algorithmic way – The theoretical machine provides a means to determine: • If an algorithm or procedure exists for a given problem • What that algorithm or procedure looks like • How long would it take to run this algorithm or procedure. 1

The Church-Turing Thesis (1936) Decision Problem • Any algorithmic procedure that can be • Let’s formalize this a bit carried out by a human or group of humans – A decision problem is a problem that has a can be carried out by some Turing yes/no answer Machine” – Equating algorithm with running on a TM – Example: • Is a given string x a palindrome (Is x ∈ pal?) – Turing Machine is still a valid computational • Is a given context free language empty? model for most modern computers. Decision Problem What makes a good algorithm? • Running a decision problem on a TM. • Suppose we know that a decision problem is decidable, how do we know that there is a – The problem must first be encoded good algorithm that solves the problem? – Example: • Is a given string x a palindrome (Is x ∈ pal?) – Consider running time on a TM – x is an instance of the probkem – Actually, consider the complexity of a TM that • Is a given context free language empty? can solve the problem. – Instance of a problem is a CFG…must be encoded. Complexity Units • Complexity refers to the rate at which the storage • What does T(n) return? or time grows as a function of the input to the – A “basic operation” can be defined as a move a algorithm TM makes in accepting / rejecting a string – T(n) = time complexity (amount of time an algorithm will take based on input) – T(n) = number of TM moves – S(n) = space complexity (amount of space an algorithm will take based on input) • For the rest of this lecture, we’ll only consider T(n) though the discussion can also be applied to S(n) 2

Asymptotic Analysis Asymptotic Analysis • Based on the idea that as the input to an • Big-O (order of) algorithm gets large – T(n) = O(f(n)) if and only if there are constants c 0 and n 0 such that – The complexity will become proportional to a • T(n) ≤ c 0 f(n) for all n ≥ n 0 known function. – What this really means – Notations: • Eventually, when the size of the input gets large • O (Big-O) – upper bounds on the complexity enough, the upper bounds on the runtime will be • Θ (Big-Theta) – tight bounds on the complexity proportional to the function f(n). Algorithm Efficiencies Are you a good witch? • So what should be the cutoff between a “good” algorithm and a “bad” algorithm? – In the 1960s, Jack Edmonds proposed: • A “good” algorithm is one whose running time is a polynomial function of the size of the input • Other algorithms are “bad” – This definition was adopted: • A problem is called tractable if there exists a “good” (polynomial time) algorithm that solves it. • A problem is called intractable otherwise. Is this a valid cutoff? The class P • The class P contains all decision problems that are decidable by an “algorithm” that runs in polynomial time. – Does this define a class of languages? – Yes… • The set of all problems whose encodings of “yes instances” (a language) is recognized by a TM M • M recognizes the above language in Polynomial Time. 3

The class P The class P • Are there actually problems in P? • If we take the Church-Turing Thesis to be true: – Most everyday algorithms (e.g. sorting, searching) are in P – P is robust: • Are there problems not in P: • The problems contained in P remain in P even if we – Yes, we know that Self-Accepting is not in P change our basic model of computation. since it isn’t even solvable – A basic TM – A Sun, a Mac, even a PC running Windows XP! • Are there decidable problems not in P? – Let’s take a look… Satisfiability (SAT) Satisfiability (SAT) • INSTANCE • Example of an instance – A Logical expression containing – (x 1 | x 2 | x 3 ) & (x 4 | !x 5 ) & !x 6 & (x 7 | x 8 ) • variables x i • logical connectors &, |, and ! – One Solution: • In conjuction normal form (C 1 & C 2 & C 3 … & C n ) • x 1 = true x 5 = false • PROBLEM • x 2 = false x 6 = false – Is there an assignment of truth values to each of • x 3 = false x 7 = true the variables such that the expression will • x 4 = false x 8 = false evaluate to true. Satisfiability (SAT) Satisfiability (SAT) • Naïve algorithm to solve • Can we do better? – Systematically consider all combinations of – No known polynomial algorithm assignments of True and False values for all • Either one doesn’t exist variables and test the expression • One exists and we haven’t found it yet. – In the worst case, for n variables • T(n) = O (2 n ) 4

Non-deterministic TM Non-deterministic TM • Let’s reconsider the NDTM • If we map all paths that can possibly be taken: – Same as the ordinary TM except: • The transition function will return a set of triplets – Number of paths will be exponential – (q, x, D) – Example: • For each state / symbol combination, 0 or more • Suppose at each of n states, for each character, there transitions can be defined. are 2 possible moves. • The machine can “choose” which transition to take. • Number of paths 2 n Satisfiability (SAT) The class NP • Running SAT on a non-deterministic TM. • The class NP contains all decision problems that are decidable by a non-deterministic “algorithm” – Step 1: “Guess” a set of boolean assignments that runs in polynomial time. to each variable (this is the non-deterministic part – 2 n different choices) • Represents algorithms that run in exponential time – Does this define a class of languages? -- Yes… – Step 2: Evaluate the truth of the entire • The set of all problems whose encodings of “yes instances” (a expression using the guessed assignment (this language) is recognized by a NDTM M part is deterministic) • M recognizes the above language in Polynomial Time. • Are there problems in NP – Can certainly do each step in polynomial time – Well, for one SAT is The class NP Reducing one language to another • Clearly P is a subset of NP • Worked well for deciability…Let’s use it here. • Basic idea – Take an encoding of one problem NP – Convert it to another problem we know to be in P or N P – Conversion must be done in polynomial time! Is there something in After 40 years of research…nobody here? knows 5

Reducing one language to another Reducing one language to another • Formally (for complexity) • Informally (for complexity) – Let L 1 and L 2 be languages over Σ 1 and Σ 2 – We can take any encoded instance of one problem • Use a TM to compute a corresponding encoded instance of – We say L 1 is polynomial time reducible to L 2 another problem. (L 1 ≤ p L 2 ) if – In polynomial time • There exists a Turing computable function • If this other problem has a TM that recognizes the set of “yes * → Σ 2 * such that • f: Σ 1 encodings” in polynomial time (P), we can run that TM to solve the first problem. • If this other problem has a NDTM that recognizes the set of • x ∈ L 1 iff f(x) ∈ L 2 “yes encodings” in polynomial time (NP), we can run that TM • f can be computed in polynomial time. to solve the first problem. The class NP-complete The class NP-complete • The hardest of the problems in class NP are • Implications in the class of NP-complete problems: – Hardest problem • A language L is in NP if – It’s probably not worth looking for a solution – L ∈ NP for an NP-complete problem – For every other language L 1 ∈ NP – How do we show a problem to be NP-complete • L 1 ≤ p L • Reduction • All NP-complete problems are “equally” • We need a NP-complete problem to kick things off. difficult. The class NP-complete NP-Complete and P • Guess what? • If any a polynomial algorithm is found for any NP-complete problem – SAT can be shown to be NP-complete – A polynomial algorithm can be found for all NP- – Cook’s Theorem complete problems (via polynomial reduction). • In other words – For any L ∈ NP-Complete, • If L ∈ P then • P = NP • Finding such a language is still an open problem. 6