Computational Complexity (Continued) 15-150 1 Story so far We - PowerPoint PPT Presentation

Computational Complexity (Continued) 15-150 1

Story so far • We need to model the efficiency of our algorithms. • Complexity models (usually) follow the structure of the algorithm. • Accounting for the most important operations are usually sufficient. • For recursively expressed algorithms, models lead to recurrences. • (Simple) Recurrences can be solved by simple algebraic expansions. • Models can be expressed with the big-O notation. 2

Plan for today • Setting up recurrences for more complicated recursive algorithms. 3

Analyzing Mergesort • How does Mergesort work? • Split given list into two (roughly) equal parts. • Recursively sort each part using (smaller) mergesorts • Merge the two sorted lists into one sorted list • All real work is done during merging • Mergesort is a Divide and Conquer algorithm 4

Mergesort 5

Mergesort -- Big Picture 6

Merging two sorted lists In general the two sorted input lists can be of different sizes. merge ([1,3,5],[2,4,6,8]) ↪ [1,2,3,4,5,6,8] 7

Splitting a list Note that this not splitting a list in the middle! 8

Complexity of Split (Work/Span) split ([1,2,3,4,5] ↪ ([1,3,5],[2,4]) Why? Why? 𝑇 !"#$% (𝑜) = 𝑃(𝑜) 𝑋 !"#$% (𝑜) = 𝑃(𝑜) 9

Complexity of Merge 𝑇 &'()' (𝑜) = 𝑃(𝑜) 𝑋 &'()' (𝑜) = 𝑃(𝑜) 10

Complexity of Mergesort Why? 11

Complexity of Mergesort 𝑦 = log + 𝑜 2 * = 𝑜 ( n 2 * ) ( n 2 * ) 12

Complexity of Mergesort Why? &./ 𝑠 $ = 𝑠 & − 1 2 012 3 − 1 / = 𝑜 − 1 𝑠 − 1 2 − 1 $,- Geometric Series 𝑙 + 𝑜 log 𝑜 Why? Lower order terms 𝑋 &!4(% 𝑜 = 𝑙 + 𝑜 log 𝑜 + 𝑙 - + 𝑙 / 𝑜 − 𝑙 / = O(n log n) *All logs are log + 13

How about Span? Big Picture 𝑡𝑞𝑏𝑜 !"#$% (𝑜) split l 3 3 msort l1 msort l2 max(𝑡𝑞𝑏𝑜 &!4(% + , 𝑡𝑞𝑏𝑜 &!4(% ( + ) ) 𝑡𝑞𝑏𝑜 &'()' (𝑜) merge 14

How about Span? Big Picture 𝑇 !"#$% (𝑜) split l 3 3 msort l2 msort l1 max(𝑇 &!4(% + , 𝑇 &!4(% ( + ) ) 𝑇 &'()' (𝑜) merge 15

How about Span?

Spans for Split and Merge This split is purely sequential. ⟹ 𝑇 !"#$% 𝑜 = 𝑃(𝑜) This merge is purely sequential. ⟹ 𝑇 &'()' 𝑜 = 𝑃(𝑜) 17

Span for Mergesort 18

Span for Mergesort ) ) 19

Span for Mergesort 012 ! 3 ./ 1 5 1 / 2 $ < / 2 $ = 2 ⟹ 𝑇 &!4(% 𝑜 < 𝑙 - + 𝑙 / log + 𝑜 + 2𝑙 + 𝑜 = 𝑃(𝑜) $,- $,- Observation: Although we can run the two recursive calls to msort in parallel, the sequential algorithms for split and merge limit the overall parallelism! 20

Summary • We have seen that 𝑋 !"#$% 𝑜 = 𝑃(𝑜 log 𝑜) • and 𝑇 !"#$% 𝑜 = 𝑃(𝑜) 21

Can we do any better? • We can not asymptotically improve the work. • It turns out that any comparison-based algorithm for sorting has to make at least 𝑑 𝑜 log 𝑜 comparisons. • We already can sort using at most 𝑑 & 𝑜 log 𝑜 work. • That is what we showed. • So, we can not improve work, asymptotically. • Upper and lower bounds are asymptotically same! 22

Can we do any better? It turns out we can improve the spans for split and merge to 𝑇 ?@ABC 𝑜 = 𝑃(log 𝑜 ) 𝑇 DEFGE 𝑜 = 𝑃(log 𝑜 ) These will give us a 𝑇 !"#$% 𝑜 = 𝑃(log & 𝑜 ) Think about why we have the square of the log. 23

Binary search trees • A Binary Search Tree (BST) is • A binary tree with a value in each node • All values in the left subtree < value in the root • All values in the right subtree > value in the root • The last two conditions hold at each node. 24

What is the issue? 6 3 7 1 8 6 9 I can not just attach these two subtrees to 6? I have to make sure everything on the left < 6 and everything on the right > 6 I need to keep track of the max value of the left and the min value of the right!! 25

Checking if a Binary Tree is a BST 𝑃(𝑜) Why? What is the minimum work needed? 26

Checking if a Binary Tree is a BST Assumptions 𝑙 / 𝑗𝑔 𝑜 = 0 𝑜 = 2 & − 1 for some m 𝑜 • 𝑋 𝑜 = H 𝑙 + + 2 𝑋 𝑜 > 0 Both subtrees are the same • 2 size at each level 27

Work Assumptions 𝑙 / 𝑗𝑔 𝑜 = 0,1 𝑜 = 2 & − 1 for some m • 𝑜 𝑋 𝑜 = H Both subtrees are the same 𝑙 + + 2 𝑋 𝑜 > 1 • 2 size at each level Assumptions 𝑜 = 2 & for some m 𝑜 ≤ 𝑋 𝑜 2 ⟹ 𝑋 𝑜 ≤ 𝑙 + +2 𝑋 𝑜 • 𝑋 2 for n > 1 Both subtrees are (about) • 2 the same size at each level 𝑋 𝑜 ≤ 𝑙 + + 2 𝑋 𝑜 2 3 = 𝑙 + + 2𝑙 + + 4𝑋 6 𝑋 𝑜 ≤ 𝑜 − 1 𝑙 + + 𝑙 / 𝑜 = 𝑃(𝑜) … = 1 + 2 + 4 + ⋯ 2 "./ 𝑙 + + 2 " 𝑋 3 + " 𝑥ℎ𝑓𝑜 𝑞 = 𝑛 1 + 2 + 4 + ⋯ 2 &./ 𝑙 + + 𝑜𝑋 1 = 28

Span Parallel Intuitively – the span should be proportional to the depth of the tree! 29

Span 𝑇 1 = 𝑙 / 𝑇 𝑜 ≤ 𝑙 + + 𝑇 𝑜 2 𝑔𝑝𝑠 𝑜 > 1 ⟹ 𝑇 𝑜 = 𝑃(log 𝑜) Left as an exercise. 30

Retrospective View • How much effort/time do computers need to solve a given problem? • Can we approximate this without really solving the problem? • How bad does complexity get? 31

Here is a simple set of US cities • Major Cities in the US = {New York, Boston, Miami, Houston, Pittsburgh, Chicago, Baltimore, Dallas, Houston, San Francisco, Seattle, Denver, Austin, Atlanta, St. Louis, Las Vegas, San Diego, Albuquerque, Philadelphia, Washington DC,……} • Let’s assume there are 51 cities in this set! 32

Searching for a City • Is Pittsburgh a city in the US? • More abstractly, is x e S ? • S is a set of N objects. • This looks like a very “ simple ” problem. • anyone should be able to answer it immediately. 33

Sorting Cities by Name • List the cities in the USA in reverse alphabetical order. • {New York, Boston, Miami, Houston, Pittsburgh, Chicago, Baltimore, Dallas, Houston, San Francisco, Seattle, Denver, Austin, Atlanta, St. Louis, Las Vegas, San Diego, Albuquerque, Philadelphia, Washington DC,……} 34

Sorting Cities by Name • List the cities in the USA in reverse alphabetical order. • Somehow you can NOT do this immediately, it is “ harder ” than the previous problem. • But after a “ short ” time you probably can do it. 35

Optimizing Concert Tours • Suppose a concert band wants to tour the 51 cities. • But they do not want to travel too much. 36

Travelling Rock Band Problem • Find the shortest tour of cities in the US, such that • starting with New York, • you visit each city exactly once and • return back to New York. • e.g., New York, Philadelphia, Baltimore, Washington DC, …., Boston, New York. 37

Travelling Rock Band Problem • Find the shortest tour of cities in US, such that starting with New York, you visit each city exactly once and return back to New York. • This seems to be a very “ complex ” problem. • There are (N-1)! possible tours to consider! (you can fly if you want) • Why? 38

Travelling Rock Band Problem • Let ʼ s put n! into perspective. • It turns out that log n! » n log n • So 51 cities, we have 50! different tours • log 10 50! » 50 log 10 50 » 85 • So 50! has » 85 digits » 10 84 • This is a large, really large number. 39

How large is it? * 50! has » 85 digits » 10 84 * Let us assume we can check 10 9 (1 billion) tours every second using a fast computer * We need » 10 84 / 10 9 = 10 75 seconds * » 10 70 days * » 2.5 * 10 67 years * » 2.5 * 10 65 centuries * For reference: Big Bang is estimated to be about 1.5 *10 7 centuries ago. 40

How large is really large? 41

Some Complexity Designations Faster • Constant: 𝑃(1) • Logarithmic: 𝑃(log 𝑜) • Linear: 𝑃(𝑜) • Quadratic: 𝑃(𝑜 2 ) • Polynomial: 𝑃 𝑜 X 𝑔𝑝𝑠 𝑙 > 1 Slower • Exponential: 𝑃 𝑑 𝑜 𝑔𝑝𝑠 𝑑 > 1 • Superexponential: Grows faster than 𝑃 𝑜 Y , 𝑃(𝑜!) any exponential e.g. Glacial • Diabolical ( J ): Stack of n 2’s 𝑃(2 ! ⋰"" ) Tectonic 42