Comparing Problems Remember the concepts of Problem, Algorithm, and - PDF document

Comparing Problems Remember the concepts of Problem, Algorithm, and Program. We’ve gotten pretty good at comparing algorithms. How do we compare problems? Sorted Array Search Sorting Integer Factorization Integer Multiplication Selection Maximum Matching Minimum Vertex Cover CS 355 (USNA) Unit 7 Spring 2012 1 / 42 Computational Complexity The difficulty of a problem is the worst-case cost of the best possible algorithm that solves that problem. Computational complexity is the study and classification of problems according to their inherent difficulty. Why study this? Want to know when an algorithm is as good as possible. Sometimes we want problems to be difficult! CS 355 (USNA) Unit 7 Spring 2012 2 / 42 How to compare problems Big- O , big-Θ, and big-Ω are used to compare two functions. How can we compare two problems? Example: Sorting vs. Selection Forget about any specific algorithms for these problems. Instead, develop algorithms to solve one problem by using any algorithm for the other problem . Solving selection using a sorting algorithm: Solving sorting using a selection algorithm: Conclusion? CS 355 (USNA) Unit 7 Spring 2012 3 / 42

Defining tractable and intractable Cobham-Edmonds thesis: A problem is tractable only if it can be solved in polynomial time. What can we say about intractable problems? Maybe they’re undecidable (e.g., the halting problem) Maybe they just seem impossible (e.g., regexp equivalence) But not always! (e.g., integer factorization) Million-dollar question : Can any problems be verified quickly but not solved quickly? CS 355 (USNA) Unit 7 Spring 2012 4 / 42 Fair comparisons: Machine models Proving lower bounds on problems requires a careful model of computation. Candidates: Turing machine Clock cycles on your phone MIPS instructions “Primitive operations” Theorem These models are all polynomial-time equivalent. CS 355 (USNA) Unit 7 Spring 2012 5 / 42 Fair comparisons: Bit-length Input size is our measure of difficulty ( n ). It must be measured the same between different problems! Past examples: Factorization Θ( √ n ) vs. HeapSort Θ( n log n ) Karatsuba’s Θ( n 1 . 59 ) vs. Strassen’s Θ( n 2 . 81 ) Dijkstra’s Θ( n 2 ) vs Dijkstra’s Θ(( n + m ) log n ) Only measure for this unit: length in bits of the input CS 355 (USNA) Unit 7 Spring 2012 6 / 42

Fair comparisons: Decision problems What about the size of the output? We’ll consider only : Definition: Decision Problems Problems whose output is YES or NO Is this a big restriction? Selection EI Scheduling Integer factorization Minimum vertex cover CS 355 (USNA) Unit 7 Spring 2012 7 / 42 Decision problem comparison Compare regular factorization with decision problem version: 1 Given instance ( N , k ) of decision problem, use computational version to solve it: 2 Given instance N of computational problem, use decision problem to solve it: CS 355 (USNA) Unit 7 Spring 2012 8 / 42 Formal Problem Definitions Page 1 SHORTPATH(G,k) Input : Graph G = ( V , E ), integer k Output : Does G have a path of length at most k ? Input size and encoding: LONGPATH(G,k) Input : Graph G = ( V , E ), integer k Output : Does G have a path of length at least k ? Input size and encoding: CS 355 (USNA) Unit 7 Spring 2012 9 / 42

Formal Problem Definitions Page 2 FACT(N,k) Input : Integers N and k Output : Does N have a prime factor less than k ? Input size and encoding: VC(G,k) Input : Graph G = ( V , E ), integer k Output : Does G have a vertex cover with at most k nodes? Input size and encoding: CS 355 (USNA) Unit 7 Spring 2012 10 / 42 Our first complexity class Complexity theory is all about classifying problems based on difficulty. Definition The complexity class P consists of all decision problems that can be solved by an algorithm whose worst-case cost is O ( n k ), for some constant k , and where n is the bit-length of the input instance. This is the “polynomial-time” class. Can you name some members? CS 355 (USNA) Unit 7 Spring 2012 11 / 42 Certificates A certificate for a decision problem is some kind of digital “proof” that the answer is YES . The certificate is usually what the output would be from the “computational version”. Examples (informally): Integer factorization Minimum vertex cover Shortest path Longest path CS 355 (USNA) Unit 7 Spring 2012 12 / 42

Nice properties of P When we just worry about polynomial-time, we can be really lazy in analysis! Polynomial-time is closed under: Addition : n k + n ℓ ∈ O ( n max( k ,ℓ ) ) In terms of algorithms: one after the other. Multiplication : n k · n ℓ ∈ O ( n k + ℓ ) In terms of algorithms: calls within loops. Composition : n k ◦ n ℓ ∈ O ( n k ℓ ) In terms of algorithms: replace every primitive op. with a function call CS 355 (USNA) Unit 7 Spring 2012 13 / 42 Verifiers A verifier is an algorithm that takes: 1 Problem instance (input) for some decision problem 2 An alleged certificate that the answer is YES and returns YES iff the certificate is legit. Principle comes from “guess-and-check” algorithms: Finding the answer is tough, but checking the answer is easy. We can write fast verifiers for hard problems! CS 355 (USNA) Unit 7 Spring 2012 14 / 42 Our second complexity class Definition The complexity class NP consists of all decision problems that have can be verified in polynomial-time in the bit-size of the original problem input. Steps for an NP -proof: 1 Define a notion of certificate 2 Prove that certificates have length O ( n k ) for some constant k 3 Come up with a verifier algorithm 4 Prove that the algorithm runs in time O ( n k ) for some (other) constant k CS 355 (USNA) Unit 7 Spring 2012 15 / 42

VC is in NP VC(G,k) : “Does G have a vertex cover with at most k vertices?” 1 Certificate: 2 Certificate size: 3 Verifier algorithm: 4 Algorithm cost: CS 355 (USNA) Unit 7 Spring 2012 16 / 42 FACT is in NP FACT(N,k) : “Does N have a prime factor less than k ?” 1 Certificate: 2 Certificate size: 3 Verifier algorithm: 4 Algorithm cost: CS 355 (USNA) Unit 7 Spring 2012 17 / 42 How to get rich The BIG question is: Does P equal NP ? The Clay Institute offers $1,000,000 for a proof either way. What you would need to prove P = NP : What you would need to prove P � = NP : In a nutshell: Is guess-and-check ever the best algorithm? CS 355 (USNA) Unit 7 Spring 2012 18 / 42

Alternate meaning of NP Meaning of the name NP : “Non-deterministic polynomial time” Non-deterministic Turing machine Turing machine with (possibly) multiple transitions for the same current state and current tape symbol Like a computer program with “guesses” Connection to randomness? Why is this equivalent to our definition with certificates and verifiers? CS 355 (USNA) Unit 7 Spring 2012 19 / 42 Reductions Recall that a reduction from problem A to problem B is a way of solving problem A using any algorithm for problem A. Then we know that A is not more difficult than B. Formally, a reduction from A to B : 1 Takes an instance of problem A as input 2 Uses this to create m instances of problem B 3 Uses the solutions to those m problem B’s to recover the solution for the original problem A CS 355 (USNA) Unit 7 Spring 2012 20 / 42 Example Linear-Time Reduction Two problems: MMUL(A,B) : Compute the product of matrices A and B MSQR(A,B) : Compute the matrix square A 2 Show that the inherent difficulty of MMUL and MSQR is the same. CS 355 (USNA) Unit 7 Spring 2012 21 / 42

Polynomial-Time Reduction Ingredients for analyzing a reduction : (All will be functions of n , the input size for problem A) Number ( m ) of problem B instances created Maximum bit-size of a problem B instance Amount of extra work to do the actual reduction. Polynomial-time reduction : all three ingredients are O ( n k ) (Often m = 1, sometimes called a “strong reduction”.) We write A ≤ P B , meaning “A is polynomial-time reducible to B”. CS 355 (USNA) Unit 7 Spring 2012 22 / 42 Formal Problem Definitions Page 3 Minimum Hitting Set: HITSET(L,k) Input : List L of sets S 1 , S 2 , . . . , S m , integer k . Output : Is there a set H with size at most k such that every S i ∩ H is not empty? Input size and encoding: HAMCYCLE(G) Input : Graph G = ( V , E ) Output : Does G have a cycle that touches every vertex? Input size and encoding: CS 355 (USNA) Unit 7 Spring 2012 23 / 42 VC reduces to HITSET CS 355 (USNA) Unit 7 Spring 2012 24 / 42

HAMCYCLE reduces to LONGPATH CS 355 (USNA) Unit 7 Spring 2012 25 / 42 Completeness Definition A problem B is NP -hard if A ≤ P B for every problem A ∈ NP . Informally: NP -hard means “at least as difficult as every problem in NP ” Definition A problem B is NP -complete if B is NP -hard and B ∈ NP . What is the hardest problem in NP ? CS 355 (USNA) Unit 7 Spring 2012 26 / 42 An easy NP -hard proof Theorem : The halting problem is NP -hard. Proof : CS 355 (USNA) Unit 7 Spring 2012 27 / 42