Computational Complexity (Continued) 15-150 1 Story so far We - - PowerPoint PPT Presentation

β–Ά
computational complexity
SMART_READER_LITE
LIVE PREVIEW

Computational Complexity (Continued) 15-150 1 Story so far We - - PowerPoint PPT Presentation

Jan 23, 2024 99 likes β€’538 views

Computational Complexity (Continued) 15-150 1 Story so far We need to model the efficiency of our algorithms. Complexity models (usually) follow the structure of the algorithm. Accounting for the most important operations are

slide-1
SLIDE 1

Computational Complexity

(Continued)

15-150

1

slide-2
SLIDE 2

Story so far

  • We need to model the efficiency of our algorithms.
  • Complexity models (usually) follow the structure of

the algorithm.

  • Accounting for the most important operations are

usually sufficient.

  • For recursively expressed algorithms, models lead

to recurrences.

  • (Simple) Recurrences can be solved by simple algebraic

expansions.

  • Models can be expressed with the big-O notation.

2

slide-3
SLIDE 3

Plan for today

  • Setting up recurrences for more complicated

recursive algorithms.

3

slide-4
SLIDE 4

Analyzing Mergesort

  • How does Mergesort work?
  • Split given list into two (roughly) equal parts.
  • Recursively sort each part using (smaller) mergesorts
  • Merge the two sorted lists into one sorted list
  • All real work is done during merging
  • Mergesort is a Divide and Conquer algorithm

4

slide-5
SLIDE 5

Mergesort

5

slide-6
SLIDE 6

Mergesort -- Big Picture

6

slide-7
SLIDE 7

Merging two sorted lists

7

In general the two sorted input lists can be of different sizes. merge ([1,3,5],[2,4,6,8]) β†ͺ [1,2,3,4,5,6,8]

slide-8
SLIDE 8

Splitting a list

8

Note that this not splitting a list in the middle!

slide-9
SLIDE 9

Complexity of Split (Work/Span)

9

Why? Why? 𝑋

!"#$%(π‘œ) = 𝑃(π‘œ)

𝑇!"#$%(π‘œ) = 𝑃(π‘œ) split ([1,2,3,4,5] β†ͺ ([1,3,5],[2,4])

slide-10
SLIDE 10

Complexity of Merge

10

𝑋

&'()'(π‘œ) = 𝑃(π‘œ)

𝑇&'()'(π‘œ) = 𝑃(π‘œ)

slide-11
SLIDE 11

Complexity of Mergesort

11

Why?

slide-12
SLIDE 12

Complexity of Mergesort

12

( n 2* ) ( n 2* ) 𝑦 = log+ π‘œ 2* = π‘œ

slide-13
SLIDE 13

Complexity of Mergesort

13

Why? *All logs are log+ Why? /

$,- &./

𝑠$ = 𝑠& βˆ’ 1 𝑠 βˆ’ 1

Geometric Series

2012 3 βˆ’ 1 2 βˆ’ 1 = π‘œ βˆ’ 1 𝑋

&!4(% π‘œ = 𝑙+π‘œ log π‘œ + 𝑙- + 𝑙/ π‘œ βˆ’ 𝑙/ = O(n log n)

𝑙+π‘œ log π‘œ

Lower order terms

slide-14
SLIDE 14

How about Span? Big Picture

14

split l

msort l1

msort l2 merge π‘‘π‘žπ‘π‘œ!"#$%(π‘œ) max(π‘‘π‘žπ‘π‘œ&!4(%

3 + , π‘‘π‘žπ‘π‘œ&!4(%( 3 +))

π‘‘π‘žπ‘π‘œ&'()'(π‘œ)

slide-15
SLIDE 15

How about Span? Big Picture

15

msort l2 split l merge msort l1 𝑇!"#$%(π‘œ) max(𝑇&!4(%

3 + , 𝑇&!4(%( 3 +))

𝑇&'()'(π‘œ)

slide-16
SLIDE 16

How about Span?

slide-17
SLIDE 17

Spans for Split and Merge

17

This split is purely sequential. ⟹ 𝑇!"#$% π‘œ = 𝑃(π‘œ) This merge is purely sequential. ⟹ 𝑇&'()' π‘œ = 𝑃(π‘œ)

slide-18
SLIDE 18

Span for Mergesort

18

slide-19
SLIDE 19

Span for Mergesort

19

) )

slide-20
SLIDE 20

Span for Mergesort

20

/

$,- 012! 3 ./ 1

2$ < /

$,- 5 1

2$ = 2 ⟹ 𝑇&!4(% π‘œ < 𝑙- + 𝑙/ log+ π‘œ + 2𝑙+π‘œ = 𝑃(π‘œ) Observation: Although we can run the two recursive calls to msort in parallel, the sequential algorithms for split and merge limit the overall parallelism!

slide-21
SLIDE 21

Summary

  • We have seen that
  • and

21

𝑋

!"#$% π‘œ = 𝑃(π‘œ log π‘œ)

𝑇!"#$% π‘œ = 𝑃(π‘œ)

slide-22
SLIDE 22

Can we do any better?

  • We can not asymptotically improve the work.
  • It turns out that any comparison-based algorithm

for sorting has to make at least 𝑑 π‘œ log π‘œ comparisons.

  • We already can sort using at most 𝑑&π‘œ log π‘œ work.
  • That is what we showed.
  • So, we can not improve work, asymptotically.
  • Upper and lower bounds are asymptotically same!

22

slide-23
SLIDE 23

Can we do any better?

23

It turns out we can improve the spans for split and merge to 𝑇?@ABC π‘œ = 𝑃(log π‘œ ) 𝑇DEFGE π‘œ = 𝑃(log π‘œ )

These will give us a 𝑇!"#$% π‘œ = 𝑃(log& π‘œ )

Think about why we have the square of the log.

slide-24
SLIDE 24

Binary search trees

  • A Binary Search Tree

(BST) is

  • A binary tree with a

value in each node

  • All values in the left

subtree < value in the root

  • All values in the right

subtree > value in the root

  • The last two conditions

hold at each node.

24

slide-25
SLIDE 25

What is the issue?

25

7 6 9 3 1 8 6 I can not just attach these two subtrees to 6? I have to make sure everything on the left < 6 and everything on the right > 6 I need to keep track of the max value of the left and the min value of the right!!

slide-26
SLIDE 26

Checking if a Binary Tree is a BST

26

What is the minimum work needed? 𝑃(π‘œ) Why?

slide-27
SLIDE 27

Checking if a Binary Tree is a BST

27

𝑋 π‘œ = H 𝑙/ 𝑗𝑔 π‘œ = 0 𝑙+ + 2 𝑋 π‘œ 2 π‘œ > 0 Assumptions

  • π‘œ = 2& βˆ’ 1 for some m
  • Both subtrees are the same

size at each level

slide-28
SLIDE 28

Work

28

𝑋 π‘œ = H 𝑙/ 𝑗𝑔 π‘œ = 0,1 𝑙+ + 2 𝑋 π‘œ 2 π‘œ > 1 Assumptions

  • π‘œ = 2& βˆ’ 1 for some m
  • Both subtrees are the same

size at each level 𝑋 π‘œ 2 ≀ 𝑋 π‘œ 2 ⟹ 𝑋 π‘œ ≀ 𝑙+ +2 𝑋 π‘œ 2 for n > 1 Assumptions

  • π‘œ = 2& for some m
  • Both subtrees are (about)

the same size at each level 𝑋 π‘œ ≀ 𝑙+ + 2 𝑋 π‘œ 2 = 𝑙+ + 2𝑙+ + 4𝑋

3 6

… = 1 + 2 + 4 + β‹― 2"./ 𝑙+ + 2"𝑋

3 +"

π‘₯β„Žπ‘“π‘œ π‘ž = 𝑛 = 1 + 2 + 4 + β‹― 2&./ 𝑙+ + π‘œπ‘‹ 1 𝑋 π‘œ ≀ π‘œ βˆ’ 1 𝑙+ + 𝑙/π‘œ = 𝑃(π‘œ)

slide-29
SLIDE 29

Span

29

Intuitively – the span should be proportional to the depth of the tree! Parallel

slide-30
SLIDE 30

Span

30

𝑇 π‘œ ≀ 𝑙+ + 𝑇 π‘œ 2 𝑔𝑝𝑠 π‘œ > 1 ⟹ 𝑇 π‘œ = 𝑃(log π‘œ) 𝑇 1 = 𝑙/ Left as an exercise.

slide-31
SLIDE 31
  • How much effort/time do computers need to

solve a given problem?

  • Can we approximate this without really solving

the problem?

  • How bad does complexity get?

31

Retrospective View

slide-32
SLIDE 32
  • Major Cities in the US = {New York, Boston,

Miami, Houston, Pittsburgh, Chicago, Baltimore, Dallas, Houston, San Francisco, Seattle, Denver, Austin, Atlanta, St. Louis, Las Vegas, San Diego, Albuquerque, Philadelphia, Washington DC,……}

  • Let’s assume there are 51 cities in this set!

32

Here is a simple set of US cities

slide-33
SLIDE 33
  • Is Pittsburgh a city in the US?
  • More abstractly, is x e S ?
  • S is a set of N objects.
  • This looks like a very β€œsimple” problem.
  • anyone should be able to answer it

immediately.

33

Searching for a City

slide-34
SLIDE 34
  • List the cities in the USA in reverse

alphabetical order.

  • {New York, Boston, Miami, Houston, Pittsburgh,

Chicago, Baltimore, Dallas, Houston, San Francisco, Seattle, Denver, Austin, Atlanta, St. Louis, Las Vegas, San Diego, Albuquerque, Philadelphia, Washington DC,……}

34

Sorting Cities by Name

slide-35
SLIDE 35
  • List the cities in the USA in reverse

alphabetical order.

  • Somehow you can NOT do this

immediately, it is β€œharder” than the previous problem.

  • But after a β€œshort” time you probably can

do it.

35

Sorting Cities by Name

slide-36
SLIDE 36
  • Suppose a concert band wants to tour the 51

cities.

  • But they do not want to travel too much.

36

Optimizing Concert Tours

slide-37
SLIDE 37
  • Find the shortest tour of cities in the US,

such that

  • starting with New York,
  • you visit each city exactly once and
  • return back to New York.
  • e.g., New York, Philadelphia, Baltimore,

Washington DC, …., Boston, New York.

37

Travelling Rock Band Problem

slide-38
SLIDE 38
  • Find the shortest tour of cities in US,

such that starting with New York, you visit each city exactly once and return back to New York.

  • This seems to be a very β€œcomplex”

problem.

  • There are (N-1)! possible tours to consider!

(you can fly if you want)

  • Why?

38

Travelling Rock Band Problem

slide-39
SLIDE 39
  • LetΚΌs put n! into perspective.
  • It turns out that log n! Β» n log n
  • So 51 cities, we have 50! different tours
  • log10 50! Β» 50 log10 50 Β» 85
  • So 50! has Β» 85 digits Β» 1084
  • This is a large, really large number.

39

Travelling Rock Band Problem

slide-40
SLIDE 40

* 50! has Β» 85 digits Β» 1084 * Let us assume we can check 109 (1 billion) tours every second using a fast computer * We need Β» 1084/109 = 1075 seconds

* Β» 1070 days * Β» 2.5 * 1067 years * Β» 2.5 * 1065 centuries

* For reference: Big Bang is estimated to be about 1.5 *107 centuries ago.

40

How large is it?

slide-41
SLIDE 41

How large is really large?

41

slide-42
SLIDE 42
  • Constant: 𝑃(1)
  • Logarithmic: 𝑃(log π‘œ)
  • Linear: 𝑃(π‘œ)
  • Quadratic: 𝑃(π‘œ2)
  • Polynomial: 𝑃 π‘œX 𝑔𝑝𝑠 𝑙 > 1
  • Exponential: 𝑃 π‘‘π‘œ 𝑔𝑝𝑠 𝑑 > 1
  • Superexponential: Grows faster than

any exponential e.g. 𝑃 π‘œY , 𝑃(π‘œ!)

  • Diabolical (J):

42

Some Complexity Designations

Faster Slower Glacial Tectonic 𝑃(2!β‹°"" )

Stack of n 2’s