Analysis of Algorithms Growth of Functions Asymptotic Notation : - - PowerPoint PPT Presentation

▶
analysis of algorithms
SMART_READER_LITE
LIVE PREVIEW

Analysis of Algorithms Growth of Functions Asymptotic Notation : - - PowerPoint PPT Presentation

Dec 03, 2023 690 likes •799 views

Analysis of Algorithms Growth of Functions Asymptotic Notation : , , , , Asymptotic Notation Properties Growth of Functions http://www.cp.eng.chula.ac.th/faculty/spj Growth of Functions Growth Rates

slide-1
SLIDE 1

Analysis of Algorithms

Growth of Functions

http://www.cp.eng.chula.ac.th/faculty/spj

  • Growth of Functions

Asymptotic Notation : Ο, Ω, Θ, ο, ω Asymptotic Notation Properties

http://www.cp.eng.chula.ac.th/faculty/spj

Growth of Functions

n

0.5n + 3 2n-7 n

Linear

http://www.cp.eng.chula.ac.th/faculty/spj

Growth Rates

n

log n n n2 2n log n n n2 2n

slide-2
SLIDE 2

http://www.cp.eng.chula.ac.th/faculty/spj

Growth Rates

lim f (n) / g(n) =

n →

∞

0 f (n) grows slower than g(n)

∞ f (n) grows faster than g(n)

  • therwise f (n) and g(n) have the

same growth rate

http://www.cp.eng.chula.ac.th/faculty/spj

f (n) g(n)

lim f (n) / g(n) = 0

n →

∞

f (n) g(n) : f (n) grows slower than g(n) 0.5n 1 log n log6 n n0.5 n3 2n n!

http://www.cp.eng.chula.ac.th/faculty/spj

l'Hôpital's Rule

If f (n) and g(n) are differentiable, lim f (n) = ∞ ,

n →

∞

lim g(n) = ∞ , and lim f ' (n) / g ' (n) exists, then

n →

∞ n → ∞

lim f (n) / g(n) = lim f ' (n) / g ' (n)

n →

∞ n → ∞

http://www.cp.eng.chula.ac.th/faculty/spj

lg n vs. n

n n g n n f = = ) ( lg ) ( = n n

n

lg lim

∞ →

n n

n

ln lim 2 ln 1

∞ →

=

( )

n n

n

2 1 1 lim 2 ln 1

∞ →

= n

n

2 lim 2 ln 1

∞ →

=

Every sublinear function grows faster than any polylogarithmic function e.g., n0.01 vs. ( log n )100

n n

n

2 ln ln lim

∞ →

=

d ln n d n = l / n d n d n = 1/(2 n )

slide-3
SLIDE 3

http://www.cp.eng.chula.ac.th/faculty/spj

Asymptotic

y = 1 + x2 asymptotes Asymptotic : any approximation value that gets closer and closer to the truth, when some parameter approaches a limiting value.

http://www.cp.eng.chula.ac.th/faculty/spj

Asymptotic Notations

Deal with the behaviour of functions in the limit (for sufficiently large value of its parameters) Permit substantial simplification (napkin mathematic, rough order of magnitude) Classify functions by their growth rates

http://www.cp.eng.chula.ac.th/faculty/spj

Same Growth Rates

1 100 3100 2 log n 50 log n + 9 12n0.5 + log n 2n0.5 + 107 2n5 + n3 6nlog n + n 9n5 + 2 7 log n!

Θ (1) Θ (log n) Θ (n0.5) Θ (n log n) Θ (n5)

... ... ... ... ... ... ...

http://www.cp.eng.chula.ac.th/faculty/spj

No Faster Than Growth Rates

1 100 3100 2 log n 50 log n + 9 12n0.5 + log n 2n0.5 + 107 2n5 + n3 6nlog n + n 9n5 + 2 7 log n! ... ... ... ... ... ... ...

Ο (1) Ο (log n) Ο (n0.5) Ο (n log n) Ο (n5)

slide-4
SLIDE 4

http://www.cp.eng.chula.ac.th/faculty/spj

Slower Than Growth Rates

1 100 3100 2 log n 50 log n + 9 12n0.5 + log n 2n0.5 + 107 2n5 + n3 6nlog n + n 9n5 + 2 7 log n! ... ... ... ... ... ... ...

ο (log n) ο (n0.5) ο (n) ο (n3) ο (2n)

http://www.cp.eng.chula.ac.th/faculty/spj

No Slower Than Growth Rates

1 100 3100 2 log n 50 log n + 9 12n0.5 + log n 2n0.5 + 107 2n5 + n3 6nlog n + n 9n5 + 2 7 log n! ... ... ... ... ... ... ...

Ω (n5) Ω (n log n) Ω (n0.5) Ω (log n) Ω (1)

http://www.cp.eng.chula.ac.th/faculty/spj

Faster Than Growth Rates

1 100 3100 2 log n 50 log n + 9 12n0.5 + log n 2n0.5 + 107 2n5 + n3 6nlog n + n 9n5 + 2 7 log n! ... ... ... ... ... ... ...

ω (n4) ω (n) ω (n0.1) ω (1)

http://www.cp.eng.chula.ac.th/faculty/spj

Special Orders of Growth

constant : Θ( 1 ) logarithmic : Θ( log n ) polylogarithmic : Θ( log c n ) , c ≥ 1 sublinear : Θ( na ) , 0 < a < 1 linear : Θ( n ) quadratic : Θ( n2 ) polynomial : Θ( nc ) , c ≥ 1 exponential : Θ( cn ) , c > 1

slide-5
SLIDE 5

http://www.cp.eng.chula.ac.th/faculty/spj

Analogy

f (n) and g(n) f and g (real numbers) f (n) = Θ(g(n))

≈

f = g f (n) = Ο(g(n))

≈

f ≤ g f (n) = ο(g(n))

≈

f < g f (n) = Ω(g(n))

≈

f ≥ g f (n) = ω(g(n))

≈

f > g

Not all functions are asymptotically comparable

n vs. n1+ sin n

http://www.cp.eng.chula.ac.th/faculty/spj

O-notation

O( g(n)) = { f (n) :there exist positive constants c and n0 such that 0 ≤ f (n) ≤ c g(n) for all n ≥ n0 } 27n2 + 0.5n ∈

O( n3 ) ?

27n2 + 0.5n ≤ n3 for all n ≥ 100 0 999 0 10 3 ??? ?

http://www.cp.eng.chula.ac.th/faculty/spj

Asymptotic Upper Bound

n f (n) c g(n) n0

f (n) = O( g(n) )

http://www.cp.eng.chula.ac.th/faculty/spj

Ω-notation

Ω( g(n)) = { f (n) :there exist positive constants

c and n0 such that 0 ≤ c g(n) ≤ f (n) for all n ≥ n0 } 27n2 + 0.5n ∈

Ω( n ) ? n ≤ 27n2 + 0.5n for all n ≥

1 0 26 0 100 4 ??? ?

slide-6
SLIDE 6

http://www.cp.eng.chula.ac.th/faculty/spj

Asymptotic Lower Bound

n c g(n) n0

f (n) = Ω( g(n) )

f (n)

http://www.cp.eng.chula.ac.th/faculty/spj

Θ-notation

Θ( g(n)) = { f (n) :there exist positive constants

c1, c2 , and n0 such that 0 ≤ c1g(n) ≤ f (n) ≤ c2 g(n) for all n≥ n0 } 27n2 + 0.5n ∈

Θ( n2 ) ? n2 ≤ 27n2 + 0.5n ≤ n2 for all n ≥

1 999 0 27 28 0 ??? ??? ?

http://www.cp.eng.chula.ac.th/faculty/spj

Asymptotic Tight Bound

n

f (n) = Θ( g(n) )

f (n) c1 g(n) n0 c2 g(n)

http://www.cp.eng.chula.ac.th/faculty/spj

ο-notation and ω-notation

lim

n → ∞

lim = 0

n → ∞

f (n) g(n) f (n) = ο( g(n) ) lim

n → ∞

lim = ∞

n → ∞

f (n) g(n) f (n) = ω( g(n) )

slide-7
SLIDE 7

http://www.cp.eng.chula.ac.th/faculty/spj

Asymptotic Notation Properties

Transitivity Reflexivity Symmetry Transpose symmetry

http://www.cp.eng.chula.ac.th/faculty/spj

Transitivity

f (n) = Θ(g(n)) and g(n) = Θ(h(n)) imply f (n) = Θ(h(n)) f (n) = Ο(g(n)) and g(n) = Ο(h(n)) imply f (n) = Ο(h(n)) f (n) = Ω(g(n)) and g(n) = Ω(h(n)) imply f (n) = Ω(h(n)) f (n) = ο(g(n)) and g(n) = ο(h(n)) imply f (n) = ο(h(n)) f (n) = ω(g(n)) and g(n) = ω(h(n)) imply f (n) = ω(h(n))

http://www.cp.eng.chula.ac.th/faculty/spj

Reflexivity

f (n) = Θ( f (n) ) f (n) = Ο( f (n) ) f (n) = Ω( f (n) )

http://www.cp.eng.chula.ac.th/faculty/spj

Symmetry and Transpose Symmetry

f (n) = Θ( g (n) ) if and only if g(n) = Θ( f (n) ) f (n) = Ο( g (n) ) if and only if g(n) = Ω( f (n) ) f (n) = ο( g (n) ) if and only if g(n) = ω( f (n) )

slide-8
SLIDE 8

http://www.cp.eng.chula.ac.th/faculty/spj

Θ, Ω, Ο

f (n) = Θ( g (n) ) if and only if

f (n) = Ω( g (n) ) and f (n) = Ο( g (n) )

http://www.cp.eng.chula.ac.th/faculty/spj

Example

( )

1 1 + =

Θ =

∑

k n i k

n i

∑ ∑

= =

≤

n i k n i k

n i

1 1 1 +

=

k

n

( )

1 +

Ο =

k

n

 

∑

=

      ≥

n n i k

n

2 /

2

 

∑ ∑

= =

≥

n n i k n i k

i i

2 / 1 k

n n       ≥ 2 2

1 1

2 +

+

=

k k

n

( )

1 +

Ω =

k

n

Ο Ω

http://www.cp.eng.chula.ac.th/faculty/spj

Example

log n ! = Θ( n log n ) n ! = n × (n-1) × ...× 2 × 1 ≤ n × n × ...× n × n = n n log n ! ≤ log n n = O( n log n ) n ! = n × (n-1) × ...× (n/2) × (n/2-1) ×...× 2 × 1 ≥ n/2 × n/2 × ...× n/2 × 1 ×...× 1 × 1

≥ (n/2) n/2

log n! ≥ (n/2) log (n/2) = Ω( n log n )

Ο Ω

http://www.cp.eng.chula.ac.th/faculty/spj

Logarithms

Asymptotically, in logarithm,

the base of the log does not matter log b n = ( log c n ) / ( log c b ) log 10 n = ( log 2 n ) / ( log 2 10 ) = Θ( log n ) any polynomial function of n does not matter log n30 = 30 log n = Θ( log n )

lg, ln, log

(CS) (math) (asymptotic)

slide-9
SLIDE 9

http://www.cp.eng.chula.ac.th/faculty/spj

Polylog, Polynomial, Exponential

Any positive exponential function grows faster than any polynomial function

  • f (n) : monotically growing function,
  • ( f (n) ) c = o( a f (n) ) , c > 0, a > 1

Any positive polynomial function grows faster than any polylogarithmic function

  • log b n = o( nc )

http://www.cp.eng.chula.ac.th/faculty/spj

One-Way Equality

n = O(n2) ⇒ n ∈ O(n2) O( n2 ) = n Why dont we use ∈

? (GKP)

tradition : the practice stuck tradition : CS is used to abuse equal sign tradition : read = as is, is is one-way natural : when we do asymptotic calculation

http://www.cp.eng.chula.ac.th/faculty/spj

Asymptotic Notaton in Equations

Hn = 1/1 + 1/2 +...+ 1/n = ln n + γ + O(1/n) 2n2 + 3n + 1 = 2n2 + Θ(n) 2n2 + 3n + 1 = 2n2 + f (n), f (n) = Θ(n) Eliminate inessential detail e.g., MergeSort : T(n) = 2T(n/2) + n ?

  • T(n) = 2T(n/2) + (n-1) ?
  • T(n) = 2T(n/2) + Θ(n) = Θ(nlog n)

http://www.cp.eng.chula.ac.th/faculty/spj

Manipulating Asymptotic Notations

c O( f (n) ) = O( f (n) ) O(O ( f (n) )) = O( f (n) ) O( f (n) )O( g (n) ) = O( f (n) g(n) ) O( f (n) g(n) ) = f (n) O( g(n) ) O( f (n) + g(n) ) = O( max( f (n), g(n) ) )

n

Σ

k = 1

O( f (k) ) = O( f (k) )

n

Σ

k = 1

slide-10
SLIDE 10

http://www.cp.eng.chula.ac.th/faculty/spj

Example : BuildHeap

∑

=

=      

k h h

h O n ) ( 2

 

n k lg =       ∑

= k h h

h n O

0 2

      =

∑

∞ =0 2 h h

h n O

= O(2n) = O(n)

http://www.cp.eng.chula.ac.th/faculty/spj

Conclusion

Growth of functions

give a simple characterization of functions behavior allow us to compare the relative growth rates of functions

Use asymptotic notation to classify functions by their growth rates Asymptotics is the art of knowing where to be sloppy and where to be precise

http://www.cp.eng.chula.ac.th/faculty/spj

Don Knuth

Father of analysis of algorithm Author of The Art of Computer Programming Programmer of the TEX and METAFONT ...