CS 401: Computer Algorithm I Complexity / Graphs Xiaorui Sun 1 - PowerPoint PPT Presentation

CS 401: Computer Algorithm I Complexity / Graphs Xiaorui Sun 1

Complexity Given two positive functions f and g • f(N) is O(g(N)) iff there is a constant c > 0 s.t., f(N) is eventually always £ c g(N) • f(N) is W (g(N)) iff there is a constant e > 0 s.t., f(N) is eventually always ³ e g(N) • f(N) is Q (g(N)) iff there are constants c 1 , c 2 >0 so that eventually always c 1 g(N) £ f(N) £ c 2 g(N) 2

A Survey of Common Running Times

Linear Time: O(n) Linear time. Running time is at most a constant factor times the size of the input. Computing the maximum. Compute maximum of n numbers a 1 , …, a n . max ¬ a 1 for i = 2 to n { if (a i > max) max ¬ a i } 4

Linear Time: O(n) Merge. Combine two sorted lists A = a 1 ,a 2 ,…,a n with B = b 1 ,b 2 ,…,b n into sorted whole. i = 1, j = 1 while (both lists are nonempty) { if (a i £ b j ) append a i to output list and increment i else(a i > b j )append b j to output list and increment j } append remainder of nonempty list to output list Claim. Merging two lists of size n takes O(n) time. Pf. After each comparison, the length of output list increases by 1. 5

O(n log n) Time O(n log n) time. Arises in divide-and-conquer algorithms. also referred to as linearithmic time Sorting. Mergesort and heapsort are sorting algorithms that perform O(n log n) comparisons. Largest empty interval. Given n time-stamps x 1 , …, x n on which copies of a file arrive at a server, what is largest interval of time when no copies of the file arrive? O(n log n) solution. Sort the time-stamps. Scan the sorted list in order, identifying the maximum gap between successive time-stamps. 6

Quadratic Time: O(n 2 ) Quadratic time. Enumerate all pairs of elements. Closest pair of points. Given a list of n points in the plane (x 1 , y 1 ), …, (x n , y n ), find the pair that is closest. O(n 2 ) solution. Try all pairs of points. min ¬ (x 1 - x 2 ) 2 + (y 1 - y 2 ) 2 for i = 1 to n { for j = i+1 to n { d ¬ (x i - x j ) 2 + (y i - y j ) 2 if (d < min) min ¬ d } } see chapter 5 Remark. W (n 2 ) seems inevitable, but this is just an illusion.

Polynomial Time: O(n k ) Time Independent set of size k. Given a graph, are there k nodes such that no two are joined by an edge? k is a constant O(n k ) solution. Enumerate all subsets of k nodes. foreach subset S of k nodes { check whether S in an independent set if (S is an independent set) report S is an independent set } } Check whether S is an independent set = O(k 2 ). æ ö ÷ = n ( n - 1) ( n - 2) " ( n - k + 1) £ n k Number of k element subsets = n ç k ( k - 1) ( k - 2) " (2) (1) è ø O(k 2 n k / k!) = O(n k ). k k ! !! poly-time for k=17, 8 but not practical

Exponential Time Independent set. Given a graph, what is maximum size of an independent set? O(n 2 2 n ) solution. Enumerate all subsets. S* ¬ f foreach subset S of nodes { check whether S in an independent set if (S is largest independent set seen so far) update S* ¬ S } } 9

Efficiency An algorithm runs in polynomial time if ! " = " $(&) . Equivalently, ! " = ((" ) ) for some constant d. Suppose we can do 1 million operations per second. not only get very big, but do so abruptly , which likely yields erratic performance on small instances Outdated: Nvidia announced a “computer” this Tue that do 2 quadrillion ( 2×10 &. ) operations/sec. It brings down the 31,710 years to 500 sec. 10 However, 2 &// operations still takes millions of years.

Why “Polynomial”? Point is not that n 2000 is a practical bound, or that the differences among n and 2n and n 2 are negligible. Rather, simple theoretical tools may not easily capture such differences, whereas exponentials are qualitatively different from polynomials, so more amenable to theoretical analysis. • “My problem is in P” is a starting point for a more detailed analysis • “My problem is not in P” may suggest that you need to shift to a more tractable variant 11

Graph algorithms

Undirected Graphs G=(V,E) Notation. G = (V, E) • V = nodes (or vertices) • E = edges between pairs of nodes • Captures pairwise relationship between objects • Graph size parameters: n = |V|, m = |E| V = {1, 2, 3, 4, 5 ,6, 7, 8} E = {(1,2), (1,3), (2,3), (2,4), (2,5), (3,5), (3,7), aaaa(3,8), (4,5), (5,6), (7,8)} m=11, n=8

Undirected Graphs G=(V,E) A 2 10 3 Disconnected graph 11 12 4 8 Multi edges 13 B 9 Isolated vertices 6 Self loop 7 14

Graphs 15

Graph applications 16

Directed Graphs 1 2 10 3 self loop 11 12 4 8 13 5 Multi edge 9 6 7 17

Terminology 3 • Path: A sequence of vertices 5 s.t. each vertex is connected 6 to the next vertex with an edge 4 1 • Cycle: Path of length > 2 that has 2 the same start and end 10 1 • Tree: A connected graph with no cycles 2 5 3 4 6 18

Terminology (cont’d) • Degree of a vertex: # edges that touch that vertex 3 5 1 4 6 deg(6)=3 2 7 10 • Connected: Graph is connected if there is a path between every two vertices • Connected component: Maximal set of connected vertices 19

Degree Sum Claim: In any undirected graph, the number of edges is 1 2 ∑ %&'(&) * deg(/) ⁄ equal to Pf: ∑ %&'(&) * deg(/) counts every edge of the graph exactly twice; once from each end of the edge. 3 5 1 4 6 2 |E|=8 7 10 1 deg / = 2 + 2 + 1 + 1 + 3 + 2 + 3 + 2 = 16 %&'(&) * 20

Odd Degree Vertices Claim: In any undirected graph, the number of odd degree vertices is even Pf: In previous claim we showed sum of all vertex degrees is even. So there must be even number of odd degree vertices, because sum of odd number of odd numbers is odd. 3 5 1 4 6 2 4 odd degree vertices 7 10 3, 4, 5, 6 21

#edges Let ! = ($, &) be a graph with ( = |$| vertices and * = & edges. . = - -/0 - = 1(( . ) Claim: 0 ≤ * ≤ . Pf: Since every edge connects two distinct vertices (i.e., G has no loops) and no two edges connect the same pair of vertices (i.e., G has no multi-edges) It has at most - . edges. 22

Degree 1 vertices Claim: If G has no cycle, then it has a vertex of degree ≤ 1 (Every tree has a leaf) Proof: (By contradiction) Suppose every vertex has degree ≥ 2. Start from a vertex & ' and follow a path, & ' , … , & * when we are at & * we choose the next vertex to be different from & *+' . We can do so because deg & * ≥ 2. The first time that we see a repeated vertex ( & / = & * ) we get a cycle. We always get a repeated vertex because 2 has finitely many vertices & 4 & ' & 5 & 3 & 6 23

Trees and Induction Claim: Show that every tree with ! vertices has ! − 1 edges. Proof: (Induction on ! .) Base Case: ! = 1 , the tree has no edge Inductive Step: Let % be a tree with ! vertices. So, % has a vertex & of degree 1 . Remove & and the neighboring edge, and let %’ be the new graph. We claim %’ is a tree: It has no cycle, and it must be connected. So, %’ has ! − 2 edges and % has ! − 1 edges. 24