Computational Complexity 15-150 1 Desirable Properties of - - PowerPoint PPT Presentation

▶
computational complexity
SMART_READER_LITE
LIVE PREVIEW

Computational Complexity 15-150 1 Desirable Properties of - - PowerPoint PPT Presentation

Nov 02, 2023 42 likes •365 views

Computational Complexity 15-150 1 Desirable Properties of Programs What is the most desirable property of a program? Correctness Nobody cares about incorrect programs. Assuming you have a correct program, what is the next

slide-1
SLIDE 1

Computational Complexity

15-150

1

slide-2
SLIDE 2

Desirable Properties of Programs

  • What is the most desirable property of a program?
  • Correctness
  • Nobody cares about incorrect programs.
  • Assuming you have a correct program, what is the

next desirable property?

  • Efficiency – Programs should not be using unnecessary

resources

  • Total computational effort (time/steps etc.)
  • Memory
  • Communication

2

slide-3
SLIDE 3

Why do we care?

  • Suppose you have two Programs P1 and P2 for sorting n

integers

  • Assume when n=1000,
  • P1 takes 10,000 units of time
  • P2 takes 1,000,000 units of time
  • Which is preferable?
  • Obviously P1.
  • How do you find out? We obviously do NOT really want

to run these two programs to find out P2 takes too much time.

  • We need a simple but reasonably accurate way to find this
  • ut!

3

slide-4
SLIDE 4

Why do we care?

  • Less running time -> Less waiting for results
  • Less running time -> Less power consumption
  • > Lower cost
  • Less power consumption -> Less heat produced
  • > Less cooling
  • > Lower cost

4

slide-5
SLIDE 5

How? Big Picture

  • Model the effort that needs to be expended when

running a program

  • Work (models overall cost)
  • Model the opportunities for (ideal) parallel

execution

  • Span (models time waiting for results)

5

slide-6
SLIDE 6

How? Big Picture

6

slide-7
SLIDE 7

What do we mean by “Model”?

  • What do we model?
  • Actual time?
  • Depends on
  • Actual hardware, (CPU Speed, Cache Size, etc.)
  • Compiler, optimizer etc.
  • Compiled vs Interpreted execution mode.
  • Way too complicated to model and measure
  • Instead
  • Model the algorithm instead of the program.
  • Model some other proxy for actual effort.
  • For example, the count of most frequently executed operation
  • Model accurately enough so that algorithms can be compared.
  • We are NOT really concerned about all details.

7

slide-8
SLIDE 8

Some elaborations

  • Model the algorithm instead of the program.
  • This is usually sufficient.
  • A very efficient implementation of an otherwise inefficient

algorithm is not a good idea

  • Unless your inputs are always expected to be very small.
  • Model some other proxy for actual effort.
  • Number of steps,
  • Number of important operations
  • Number of instructions executed?
  • Model accurately enough so that algorithms can be

compared.

  • How does work or span vary as input size (not value) increases?
  • For example, if my input doubles in size, how does my work

increase?

  • Doubles the first effort?
  • Square of the first effort?

8

slide-9
SLIDE 9

Example

How many steps are needed for evalution as a function of the value of n? (This is really wrong but convenient – bear with me.) 𝑋 𝑜 = Number of steps executed when argument has value 𝑜

9

slide-10
SLIDE 10

Example

Do you see a pattern here?

10

slide-11
SLIDE 11

Example

Do you see a pattern here?

  • We always compute for

argument 1 less

  • We have 3 additional steps.
  • Except for the base case! (1

step there)

11

slide-12
SLIDE 12

Example

This is called a recurrence.

  • Reflects the structure of the code
  • NOT a closed form -- need to repeatedly expand
  • With some kindergarten algebra, we can easily find a

closed form

12

slide-13
SLIDE 13

Example

n steps Repeated Expansion

13

slide-14
SLIDE 14

Example

Case Definition Solution 𝑜 = 0 1 1 𝑜 > 0 𝑋

!"# 𝑜 = 3 + 𝑋 !"#(𝑜 − 1)

= 3 + 3 𝑜 − 1 + 1 = 3𝑜 + 3 − 3 + 1 = 3𝑜 + 1 3𝑜 + 1 𝑋

!"# 𝑜 = 3𝑜 + 1

14

slide-15
SLIDE 15

Another Example

What is exp’ computing?

𝑓𝑦𝑞! 𝑜 = 2"

What is the cost of square?

2 (body and *)

15

slide-16
SLIDE 16

Another Example

What is the recurrence for 𝑋

!"#$(𝑜) ? (𝑜 is again the value of the argument)

How many cases should we consider? 3 Why 5? Why 6? Function body, mod, compare =, if, div Function body, mod, compare =, if, -, *

16

slide-17
SLIDE 17

Another Example

Why? Why?

17

slide-18
SLIDE 18

Another Example

In general we do really care about the constants (1, 7 or 13)

  • They do not impact the growth rates
  • They just move the function (cost vs n) up/down the y axis.
  • We abstract away

18

slide-19
SLIDE 19

Another Example

Do you see another simplification? For odd 𝑜, 𝑜 − 1 𝑒𝑗𝑤 2 = 𝑜 𝑒𝑗𝑤 2 where you can pick 𝑙 = max(𝑙$, 𝑙%)

19

slide-20
SLIDE 20

Another Example

Again with some kindergarten algebra, we can easily find a closed form 1 + log% 𝑜 steps 𝑋!"#! 𝑜 = 𝑙 log% 𝑜 + 1 + 𝑙& = 𝑙 log% 𝑜 + 𝑙 + 𝑙& = 𝑙 log% 𝑜 + 𝑙′ For modeling purposes, it is convenient to assume that 𝑜 = 2' for some 𝑙 ≥ 0

20

slide-21
SLIDE 21

Let’s Compare the Two Algorithms

50 100 150 200 250 300 350 10 20 30 40 50 60 70 80 90 100 3n+1 13log_2(n)+15

For small n, the first algorithm seems to be better As n increases, the second algorithm, is much better. We are making one very important but incorrect assumption?

  • Can you spot it? Let me know(J)

21

slide-22
SLIDE 22

Comparing Algorithms

  • We really compare the functions’ growth rate.
  • Constant factors do not matter
  • Lower order terms do not matter
  • We use the big-O notation.
  • We say things like
  • 𝑋 𝑜 = 3𝑜 + 1 = 𝑃(𝑜) – Function grows linearly
  • 𝑋 𝑜 = 𝑙 log% 𝑜 + 𝑙$ = 𝑃(log 𝑜)
  • Function grows logarithmically
  • The base of the logarithm does not matter (Why?)
  • log 𝑜 grows much slower than 𝑜 regardless of the

constant multipliers.

22

slide-23
SLIDE 23

The big-O notation

  • We say a function 𝑔 𝑜 = 𝑃(𝑕 𝑜 ) iff there are

constants 𝑑 > 0 and 𝑜' ≥ 0, such that for all 𝑜 ≥ 𝑜', 𝑑𝑕 𝑜 ≥ 𝑔 𝑜 .

  • You can also see uses as 𝑔 𝑜 ∈ 𝑃(𝑕 𝑜 )
  • The set of all functions that grow at most as 𝑕 𝑜 .
  • We say 𝑑𝑕 𝑜 is an upper bound for 𝑔(𝑜).

23

slide-24
SLIDE 24

The big-O notation

24

slide-25
SLIDE 25

What is the argument of the work model?

  • The first examples took n to be the value.
  • This is technically wrong!

25

slide-26
SLIDE 26

The big-O notation

  • 1000 𝑜! = O n!
  • 𝑜! + 1000000𝑜 = 𝑃 𝑜!
  • 5𝑜 log! 𝑜 = 𝑃(𝑜 log 𝑜)
  • 5𝑜 log! 𝑜"#$ = 𝑃(𝑜 log 𝑜) (Why?)
  • 5𝑜 = 𝑃(𝑜%) (But this is not a very useful statement)
  • Why?
  • We never say things like:
  • 3𝑜 = 𝑃 5𝑜 Why?
  • 5𝑜% + 7 = 𝑃(𝑜 + 7) Why?
  • 𝑃 𝑜 = 5𝑜 + 7
  • 𝑃

is always on the right side of =/∈

26

slide-27
SLIDE 27

Another Example – List Reversal

We also need to know the complexity of the @ function

27

slide-28
SLIDE 28

Complexity of the Append

What does 𝑜 represent?

The length/size of the first list.

The size of the second list is not important. (Why?) This is indeed the correct way to model work (and span).

  • Work and span should be expressed as a function of the size of the input
  • Not as a function of the input!! (Remember my earlier comment)

𝑋

@ 𝑜 = 𝑏;𝑜 + 𝑏< = 𝑃(𝑜)

28

slide-29
SLIDE 29

Complexity of rev

Substituting 𝑋

@ 𝑜 = 𝑏"𝑜 + 𝑏# and combining constants we get

29

slide-30
SLIDE 30

Complexity of rev

𝑋

$%& 𝑜 = 𝑏"

2 𝑜' + (𝑐"

(− 𝑏"

2 )n + 𝑐# = 𝑃(𝑜') Lower order terms

30

slide-31
SLIDE 31

Complexity of trev

You only need to know the complexity of trevh 𝑋

ABCDE 𝑜 = 𝑑;𝑜 + 𝑑< = 𝑃(𝑜)

Tail recursive reverse is substantially more efficient.

31

slide-32
SLIDE 32

Summary

  • We need to model the efficiency of our algorithms.
  • Complexity models are follow to the structure of

the algorithm.

  • Accounting for the most important operations are

usually sufficient.

  • For recursively expressed algorithms models lead to

recurrences.

  • (Simple) Recurrences can be solved by simple algebraic

expansions.

  • Models can be expressed with the big-O notation.

32