TDDE25 Computational Complexity Theory F 9 Chap 5: Algorithms - PowerPoint PPT Presentation

https://liuonline.sharepoint.com/sites/Lisam_TDDE25_2020HT_FZ Algorithmics and Computability Part II TDDE25 Computational Complexity Theory Fö 9 Chap 5: Algorithms Chap 12: Theory of Computation Patrick Doherty Dept of Computer and Information Science Artificial Intelligence and Integrated Computer Systems Division 1 2 Traveling Salesman Problem Computational Problems Computational Problems that CANNOT be solved by any algorithm Computability Theory Computational Problems that CAN be solved by an algorithm Computational Problems that CAN be solved by an algorithm • The Traveling Salesman Problem is one of the most intensively studied problems in computational mathematics. Intractable Class NP CANNOT be solved in any practical • A traveling salesman has n number of cities to visit. He wants to know the shortest sense due to excessive time/space route which will allow him to visit all cities one time and return to his starting point. Computational requirements • Solving this problem becomes MUCH harder as the number of cities increases; Complexity Theory the figure in the middle shows the solution for the 13,509 cities and towns in the CAN be solved in a practical US that have more than 500 residents. sense with reasonable time/space Class P Tractable requirements • The TSP problem is in the complexity class NP-Complete . 3 4

Traveling Salesman Problem Sequential Search: Telephone Book • Suppose a telephone book has N=1000000 entries. • Given the name Y , search the telephone book sequentially for Y ’s telephone number • Suppose there are n cities to visit. • Entries <X 1 , T 1 >, <X 2 , T 2 >, ... X 1000000 , T 1000000 > • The number of possible itineraries is (n-1)! • At each iteration Y is compared with X i • For n=10 cities, there are 9!=362,880 itineraries. • Assume time increases relative to the number of comparisons, so we are • What if n=40 ? For brute force methods: counting comparison instructions. (there may be other instructions...) • There are now 39! itineraries to check which is greater then 10 45 • In the worst case, 1 000 000 comparisons may have to be made. • Examining 10 15 tours per second, the required time would be several billion lifetimes of the • Call the algorithm A . We say it has a worst case running time which is on the universe (universe is estimated to be 13.8 billion years old) order of N . • In fact, no supercomputer, existing or projected can run this fast. • A runs in time O(N) in the worst case, where N is the number of entries in the • There are other methods though (branch-bound, progressive improvement, heuristics) that can compute up to 200 cities in “reasonable” time. Current record (?): 85,900 cities for a problem instance. telephone book. • (n-1)! grows faster than 2 n . So the time it takes to solve the problem grows exponentially with the size of • In other words, the time complexity of A is dependent on the size of the input. the input. • A has worst case behavior which is linear in the size of the input to A . 5 6 Linear Family of Functions O Big- Notation N 3 N 100 N • We do not care whether the algorithm takes time , , , or N N /6 even a fraction of : • The only thing that matters is that the running time of the algorithm N grows linearly with . • In other words, there is some constant such that the algorithm k k * N runs in time that is no more than in the worst case T ( n ) n • Let be a function on (the size of the input to an algorithm) T ( n ) f ( n ) then is on the order of if: T ( n ) characterizes the running time of the algorithm, i.e, lines of code, # of additions, etc. as a function of input n. T ( n ) usually characterizes worst case running time. Asymptotic Analysis T ( n ) O ( f ( n )) T ( n ) < k * f ( n ) k is if for some , n > n 0 for all Multiplicative constants only change the slope of the curve and not its shape. 7 8

Binary Search: Telephone Book Binary Search Example: Telephone Book Find Casy 1 Allen 2 Baley Casy > Boyer 3 Boyer Round up on non whole numbers 4 Casy Eureka! Casy < Davis 5 Davis A: Sorted List as content 6 Davison Casy < Davison t: Number to be found 7 Glen n: size of list 8 Greer n=7 0 1 2 3 4 5 6 9 Haley 10 Hanson Start 11Harrison Casy < Harrison 12 Listor 13 Mendel 14 Morgenstern 15 Patton 16 Perkins Size of the problem is Length = 20 17 Quinn reduced by half each 18 Reed # Comparisons = 5 19 Schmidt comparison! 20 Woolf 9 10 Comparing Linear vs. Log Binary Search: 1 000 000 Entries 20 1000000 Length = Length = Sequential search Binary search 5 # Comparisons = # Comparisons = ?? N 1+ log 2 N • Each comparison reduces the length of the input list by half. 10 4 • The process terminates when or before the list becomes empty 100 7 • So, the worst case number of comparisons is obtained by figuring N out how many times a number can be repeatedly divided by two 1000 10 before it is reduced to 0 (ignoring fractions) 1000000 20 a million 2 20 1000000 is about 1000000000 30 a billion log 2 (1000000) is about 20 1000000000000000000 60 a billion billion log 2 ( N ) N 1 0 counts the number required to reduce to ( not ) 1 + log 2 ( N ) In the worst case, # of comparisons required is , O ( log 2 ( N )) or 11 12

Various Runtimes of different algorithms O ( n 2 ) O ( n 3 ) Family of Quadratic / Cubic Functions Linear O ( n ) O ( n 2 ) Quadratic Cubic O ( n 3 ) Cubic Logarithmic O ( log 2 ( n )) Quadratic O (2 n ) Exponential Square O ( sqrt ( n )) Root 13 14 Polynomials Some Values of some Representative Functions The basic shape of a polynomial function is determined by the highest N= 10 50 100 300 1000 valued exponent in the polynomial (called the order of the polynomial). 5N 50 250 500 1500 5000 N x log 2 N 33 282 665 2469 9966 Tractable 1 million N 2 100 2500 10000 90000 (7 digits) Polynomial 1 million 27 million 1 billion N 3 1000 125000 (7 digits) (8 digits) (10 digits) a 16 digit a 31 digit a 91 digit a 302 digit 2 N 1024 number number number number Intractable 3.6 million a 65 digit a 161 digit a 623 digit unimaginably N! Super Polynomial large (7 digits) number number number 10 billion an 85 digit a 201 digit a 744 digit unimaginably N N large (11 digits) number number number The number of protons in the known universe has 126 digits The number of microseconds since the Big Bang has 24 digits 15 16

Dominant Terms of a Polynomial ( O ( p ( n )) Dominant Terms of a Polynomial ( O ( p ( n )) Only the dominant terms of a polynomial matter in the long run. Lower-order terms fade to insignificance as the problem size increases. As n grows 17 18 O ( n 3 ) Sorting Algorithms O ( log 2 ( n )) vs. O ( n * log 2 ( n )) The best sorting algorithms (such as mergesort) run in time. Polynomial curves will always overtake logarithmic curves eventually, when the Slower ones (such as bubble sort, selection sort, and insertion sort), take problem size gets big enough, regardless of the multiplicative constants involved. O ( n 2 ) time. 19 20

P The Class of Computational Problems NP The Class of Computational Problems • A function f ( N ) is said to be bounded from above by another function g ( N ) , if for all N • Most exponential time algorithms are merely variations on exhaustive from a certain point on, f ( N ) is no greater than g ( N ) . search. log 2 N is bounded by N • • Generate and test N 2 N * log 2 N is bounded by • • Incrementally generate partial solution and Backtrack 2 N N ! is bounded by • • Some examples of problems: N N N ! is bounded by • N k N • Traveling salesman problem • For our purposes a polynomial function of is one that is bounded from above by for some fixed . k • find the shortest path passing through all nodes in a graph only once. • All other functions are super polynomial (exponential) • shortest path problem • Recall definition of a polynomial function: • is an expression of finite length constructed from variables and constants, using only • 3-coloring map problem the operations of addition, subtraction, multiplication, and non-negative integer exponents. • Is there a Hamiltonian path in a graph? • An algorithm whose order-of-magnitude time performance is bounded from above by a • is there a path passing through all nodes in a graph only once. polynomial function of N , where N is the size of its inputs, is called a polynomial time O ( p ( N )) p algorithm ( ), where is a polynomial function) • Satisfiability problem 21 22 The Satisfiability Problem Formula to satisfy: A ∧ (¬ B ∨ C ) ∧ (¬ C ∨ B ) • Satisfiability Problem • Find a truth assignment that satisfies a sentence in A:True A:False the propositional calculus • Existing algorithms are exponential in the size of B:True B:False B:True B:False the input formula. • But! .... C:True C:False C:True C:False C:True C:False C:True C:False • If one has a truth assignment for a formula, certifying that it is in fact a valid truth assignment A:True A:True A:True A:True A:False A:False A:False A:False B:True B:True B:False B:False B:True B:True B:False B:False can be checked in polynomial time. C:True C:False C:True C:False C:True C:False C:True C:False • Let’s call such a solution a “short” certificate (it’s Short 2 3 Models Certificate size is bounded by a polynomial) 23 24