[PPT] - C o mp l e x i t y ( W e i s s c h a p t e r 2 PowerPoint Presentation

SLIDE 1

C

mp

l e x i t y ( W e i s s c h a p t e r 2 )

SLIDE 2

C

m

p l e x i t y

Tii s l e c t u r e i s a l l a b

u

t h

w

t

d

e s c r i b e t h e p e r f

r

m a n c e

f

a n a l g

r

i t h m G i v e n a n a l g

r

i t h m , a n d ( e . g . ) t h e s i z e

f

t h e i n p u t , c a n w e c

m

e u p w i t h a f

r

m u l a f

r

t h e r u n t i m e

f

t h e a l g

r

i t h m ?

P

r

b

l e m : r u n t i m e m a y v a r y b a s e d

n

e x a c t i n p u t – s

l

u t i

n

: l

k

a t w

r

s t

c

a s e r u n t i m e f

r

a g i v e n s i z e

P

r

b

l e m : c a l c u l a t i n g a n e x a c t r u n t i m e r e q u i r e s d e e p k n

w

l e d g e

f

t h e m a c h i n e t h e p r

g

r a m w i l l b e r u n

n

– s

l

u t i

n

: c

u

n t n u m b e r

f

s t e p s i n s t e a d

P

r

b

l e m : t h e f

r

m u l a i s u s u a l l y v e r y l a r g e a n d a n n

y

i n g t

c

a l c u l a t e – s

l

u t i

n

: t h e r e s t

f

t h i s l e c t u r e !

I d e a : a s y m p t

t

i c c

m

p l e x i t y – w h a t i s t h e p e r f

r

m a n c e l i k e w h e n n i s l a r g e ?

SLIDE 3

≈ n

2

/ 2 2 n ≈ n

2

/ 2 O b s e r v a t i

n

1 : c

n

s t a n t f a c t

r

s d

n

’ t u s u a l l y m a t t e r !

SLIDE 4

n ( n

1

) / 2 = n

2

/ 2

n

/ 2 n

2

/ 2 O b s e r v a t i

n

2 :

n

l y l e a d i n g t e r m s a r e s i g n i fj c a n t

SLIDE 5

B i g

O

n

t

a t i

n

Wh e n n i s l a r g e :

o

n l y l e a d i n g t e r m s a r e s i g n i fj c a n t

c
n

s t a n t f a c t

r

s d

n

’ t ( u s u a l l y ) m a t t e r

M a i n c

n

c e p t i n t h i s l e c t u r e : b i g

O

n

t

a t i

n

, w h i c h a l l

w

s u s t

i

g n

r

e a l l t h

s

e d e t a i l s i n

u

r f

r

m u l a s Tie r u n t i m e

f

t h e t h r e e fj l e c

p

y i n g p r

g

r a m s i s :

Tie

fj r s t

n

e : n ( n

1

) / 2 i s O ( n

2

) ( “ b i g

O

n

s

q u a r e d ” )

Tie

s e c

n

d

n

e : n ( n

1

) / 2 i s O ( n

2

) t

Tie

t h i r d

n

e : 2 n i s O ( n )

O

( … ) m e a n s r

u

g h l y : “ p r

p
r

t i

n

a l t

…

, w h e n n i s l a r g e e n

u

g h ”

SLIDE 6

O ( n

2

) O ( n ) O ( n

2

) B i g

O

n

t

a t i

n

e x p l a i n s t h e d i fg e r e n c e i n p e r f

r

m a n c e !

SLIDE 7

T i m e c

m

p l e x i t y

W i t h b i g

O

n

t

a t i

n

, i t d

e

s n ' t m a t t e r w h e t h e r w e c

u

n t s t e p s

r

t i m e ! A s l

n

g a s e a c h s t e p t a k e s a c

n

s t a n t a m

u

n t

f

t i m e :

i

f t h e n u m b e r

f

s t e p s i s p r

p
r

t i

n

a l t

n

2

t

h e n t h e a m

u

n t

f

t i m e i s p r

p
r

t i

n

a l t

n

2

W e s a y t h a t t h e a l g

r

i t h m h a s O ( n

2

) t i m e c

m

p l e x i t y

r

s i m p l y c

m

p l e x i t y

SLIDE 8

C

m

m

n

c

m

p l e x i t i e s

SLIDE 9

SLIDE 10

Q u i z

A n a l g

r

i t h m t a k e s O ( n ) t i m e t

r

u n . Wh a t h a p p e n s t

t

h e r u n t i m e i f t h e s i z e

f

t h e i n p u t i s d

u

b l e d ? Wh a t a b

u

t i f t h e a l g

r

i t h m t a k e s O ( n

2

) t i m e t

r

u n ? H

w

d

e

s t h i s e x p l a i n t h e f

l

l

w

i n g f a c t s :

I

n t h e s l

w

fj l e

c
p

y i n g p r

g

r a m , i t s t a r t e d q u i c k l y b u t g r a d u a l l y g

t

s l

w

e r a s i t r e a d t h e fj l e

I

n t h e f a s t fj l e

c
p

y i n g p r

g

r a m , i t c a r r i e d

n

a t a c

n

s t a n t r a t e

SLIDE 11

G r

w

t h r a t e s

I m a g i n e t h a t w e d

u

b l e t h e i n p u t s i z e f r

m

n t

2

n . I f a n a l g

r

i t h m i s . . .

O

( 1 ) , t h e n i t t a k e s t h e s a m e t i m e a s b e f

r

e

O

( l

g

n ) , t h e n i t t a k e s a c

n

s t a n t a m

u

n t m

r

e

O

( n ) , t h e n i t t a k e s t w i c e a s l

n

g

O

( n l

g

n ) , t h e n i t t a k e s t w i c e a s l

n

g p l u s a l i t t l e b i t m

r

e

O

( n

2

) , t h e n i t t a k e s f

u

r t i m e s a s l

n

g

– Tii

s e x p l a i n s w h y t h e s l

w

fj l e r e a d i n g p r

g

r a m s s t a r t e d q u i c k l y , b u t t h e n g r a d u a l l y s l

w

e d d

w

n a s t h e y c

n

t i n u e d r e a d i n g t h e fj l e . H

w

?

I f a n a l g

r

i t h m i s O ( 2

n

) , t h e n a d d i n g

n

e e l e m e n t m a k e s i t t a k e t w i c e a s l

n

g B i g O t e l l s y

u

h

w

t h e p e r f

r

m a n c e

f

a n a l g

r

i t h m s c a l e s w i t h t h e i n p u t s i z e

SLIDE 12

B i g O m a t h e m a t i c a l l y

SLIDE 13

B i g O , f

r

m a l l y

B i g O m e a s u r e s t h e g r

w

t h

f

a m a t h e m a t i c a l f u n c t i

n
T

y p i c a l l y a f u n c t i

n

T ( n ) g i v i n g t h e n u m b e r

f

s t e p s t a k e n b y a n a l g

r

i t h m

n

i n p u t

f

s i z e n

B

u t c a n a l s

b

e u s e d t

m

e a s u r e s p a c e c

m

p l e x i t y ( m e m

r

y u s a g e )

r

a n y t h i n g e l s e

S

f
r

t h e fj l e

c
p

y i n g p r

g

r a m :

T

( n ) = n ( n

1

) / 2

T

( n ) i s O ( n

2

)

I

n g e n e r a l , T ( n ) i s O ( f ( n ) ) , f

r

s

m

e f u n c t i

n

f

W

e

f

t e n a b u s e n

t

a t i

n

a n d w r i t e “ T ( n ) = O ( f ( n ) ) ”

SLIDE 14

B i g O , f

r

m a l l y

Wh a t d

e

s i t m e a n t

s

a y “ T ( n ) i s O ( f ( n ) ) ” ?

e

. g . T ( n ) i s O ( n

2

)

W e c

u

l d s a y i t m e a n s T ( n ) i s p r

p
r

t i

n

a l t

f

( n )

i

. e . T ( n ) = k × f ( n ) f

r

s

m

e k

e

. g . T ( n ) = n

2

/ 2 i s O ( n

2

) ( l e t k = ½ )

B u t t h i s i s t

r

e s t r i c t i v e !

W

e w a n t T ( n ) = n ( n

1

) / 2 t

b

e O ( n

2

)

W

e w a n t T ( n ) = n

2

+ 1 t

b

e O ( n

2

)

SLIDE 15

B i g O , f

r

m a l l y

I n s t e a d , w e s a y t h a t T ( n ) i s O ( f ( n ) ) i f :

T

( n ) ≤ k × f ( n ) f

r

s

m

e k , i . e . T ( n ) i s p r

p
r

t i

n

a l t

f

( n )

r

l

w

e r !

Tii

s

n

l y h a s t

h
l

d f

r

b i g e n

u

g h n : i . e . f

r

a l l n a b

v

e s

m

e t h r e s h

l

d n

I f y

u

d r a w t h e g r a p h s

f

T ( n ) a n d k × f ( n ) , a t s

m

e p

i

n t t h e g r a p h

f

k × f ( n ) m u s t p e r m a n e n t l y

v

e r t a k e t h e g r a p h

f

T ( n )

I

n

t

h e r w

r

d s , T ( n ) g r

w

s m

r

e s l

w

l y t h a n k × f ( n )

N

t

e t h a t b i g

O

n

t

a t i

n

i s a l l

w

e d t

v

e r e s t i m a t e t h e c

m

p l e x i t y !

k

× f ( n ) i s a n u p p e r b

u

n d

n

T ( n )

SLIDE 16

A n e x a m p l e : n

2

+ 2 n + 3 i s O ( n

2

)

n

2

+ 2 n + 3 ≤ 2 n

2

f

r

n ≥ 3

SLIDE 17

Q u i z

I

s 3 n + 5 i n O ( n ) ?

I

s n

2

+ 2 n + 3 i n O ( n

3

) ?

I

s i t i n O ( n

2

) ?

I

s i t i n O ( n ) ?

Wh

y d

w

e n e e d t h e t h r e s h

l

d n ?

SLIDE 18

SLIDE 19

A d d i n g b i g O

S

m

e f u n c t i

n

s g r

w

f a s t e r t h a n

t

h e r s : O ( 1 ) < O ( l

g

n ) < O ( n ) < O ( n l

g

n ) < O ( n

2

) < O ( n

3

) < O ( 2

n

) Wh e n a d d i n g t w

f

u n c t i

n

s , t h e f a s t e r

g

r

w

i n g f u n c t i

n

“ w i n s ” :

O ( 1 ) + O ( l

g

n ) = O ( l

g

n ) O ( l

g

n ) + O ( n

k

) = O ( n

k

) ( i f k ≥ ) O ( n

j

) + O ( n

k

) = O ( n

k

) , i f j ≤ k O ( n

k

) + O ( 2

n

) = O ( 2

n

)

SLIDE 20

A n e x a m p l e : n

2

+ 2 n + 3 i s O ( n

2

)

U s e h i e r a r c h y : n

2

+ 2 n + 3 = O ( n

2

) + O ( n ) + O ( 1 ) = O ( n

2

)

SLIDE 21

Q u i z

Wh a t a r e t h e s e i n B i g O n

t

a t i

n

( s i m p l i fj e d a s f a r a s p

s

s i b l e ) ?

n

2

+ 1 1

2

n

3

+ 3 n + 1

n

4

+ 2

n

SLIDE 22

J u s t u s e h i e r a r c h y !

n

2

+ 1 1 = O ( n

2

) + O ( 1 ) = O ( n

2

) 2 n

3

+ 3 n + 1 = O ( n

3

) + O ( n ) + O ( 1 ) = O ( n

3

) n

4

+ 2

n

= O ( n

4

) + O ( 2

n

) = O ( 2

n

)

SLIDE 23

M u l t i p l y i n g b i g O

O ( f ( n ) ) × O ( g ( n ) ) = O ( f ( n ) × g ( n ) )

e

. g . , O ( n

2

) × O ( l

g

n ) = O ( n

2

l

g

n )

Y

u

c a n d r

p

c

n

s t a n t f a c t

r

s :

k

× O ( f ( n ) ) = O ( f ( n ) ) , i f k i s c

n

s t a n t

e

. g . 2 × O ( n ) = O ( n )

( E x e r c i s e : s h

w

t h a t t h e s e a r e t r u e )

SLIDE 24

Q u i z

Wh a t i s ( n

2

+ 3 ) ( 2

n

× n ) + l

g

1

n i n B i g O n

t

a t i

n

?

SLIDE 25

A n s w e r

( n

2

+ 3 ) ( 2

n

× n ) + l

g

1

n = O ( n

2

) × O ( 2

n

× n ) + O ( l

g

n ) = O ( 2

n

× n

3

) + O ( l

g

n ) ( m u l t i p l i c a t i

n

) = O ( 2

n

× n

3

) ( h i e r a r c h y )

l

g

1

n = l

g

n / l

g

1 i . e . l

g

n t i m e s a c

n

s t a n t f a c t

r

SLIDE 26

R e a s

n

i n g a b

u

t p r

g

r a m s

SLIDE 27

C

m

p l e x i t y

f

a p r

g

r a m

M

s

t “ p r i m i t i v e ”

p

e r a t i

n

s t a k e O ( 1 ) t i m e : int add(int x, int y) { return x + y; } ( E x c e p t i

n

: c r e a t i n g a n a r r a y

f

l e n g t h n t a k e s O ( n ) t i m e )

Tii s i s c a l l e d t h e u n i f

r

m c

s

t m

d

e l , b e c a u s e a l l p r i m i t i v e

p

e r a t i

n

s a r e a s s i g n e d t h e s a m e c

s

t

SLIDE 28

C

m

p l e x i t y

f

a p r

g

r a m

Wh a t a b

u

t l

p

s ? ( A s s u m e t h e a r r a y s i z e i s n )

boolean member(Object[] array, Object x) { for (int i = 0; i < array.length; i++) if (array[i].equals(x)) return true; return false; }

SLIDE 29

C

m

p l e x i t y

f

a p r

g

r a m

Wh a t a b

u

t l

p

s ? ( A s s u m e t h e a r r a y s i z e i s n )

boolean member(Object[] array, Object x) { for (int i = 0; i < array.length; i++) if (array[i].equals(x)) return true; return false; } L

p

r u n s O ( n ) t i m e s L

p

b

d

y t a k e s O ( 1 ) t i m e O ( 1 ) × O ( n ) = O ( n )

SLIDE 30

C

m

p l e x i t y

f

l

p

s

Tie c

m

p l e x i t y

f

a l

p

i s : t h e n u m b e r

f

t i m e s i t r u n s t i m e s t h e c

m

p l e x i t y

f

t h e b

d

y F

r

n e s t e d l

p

s , s t a r t f r

m

t h e i n n e r m

s

t l

p

a n d w

r

k y

u

r w a y

u

t w a r d s !

SLIDE 31

W h a t a b

u

t t h i s

n

e ?

boolean unique(Object[] a) { for(int i = 0; i < a.length; i++) for (int j = 0; j < a.length; j++) if (a[i].equals(a[j]) && i != j) return false; return true; }

SLIDE 32

W h a t a b

u

t t h i s

n

e ?

boolean unique(Object[] a) { for(int i = 0; i < a.length; i++) for (int j = 0; j < a.length; j++) if (a[i].equals(a[j]) && i != j) return false; return true; }

L

p

b

d

y : O ( 1 ) I n n e r l

p

r u n s n t i m e s : O ( n ) × O ( 1 ) = O ( n ) O u t e r l

p

r u n s n t i m e s : O ( n ) × O ( n ) = O ( n

2

)

SLIDE 33

W h a t a b

u

t t h i s

n

e ?

void function(int n) { for(int i = 0; i < n*n; i++) for (int j = 0; j < n/2; j++) “something taking O(1) time” }

SLIDE 34

W h a t a b

u

t t h i s

n

e ?

void function(int n) { for(int i = 0; i < n*n; i++) for (int j = 0; j < n/2; j++) “something taking O(1) time” }

L

p

b

d

y : O ( 1 ) I n n e r l

p

r u n s n / 2 = O ( n ) t i m e s : O ( n ) × O ( 1 ) = O ( n ) O u t e r l

p

r u n s n

2

t i m e s : O ( n

2

) × O ( n ) = O ( n

3

)

SLIDE 35

H e r e ' s a n e w

n

e

boolean unique(Object[] a) { for(int i = 0; i < a.length; i++) for (int j = 0; j < i; j++) if (a[i].equals(a[j])) return false; return true; }

SLIDE 36

H e r e ' s a n e w

n

e

boolean unique(Object[] a) { for(int i = 0; i < a.length; i++) for (int j = 0; j < i; j++) if (a[i].equals(a[j])) return false; return true; }

B

d

y i s O ( 1 ) I n n e r l

p

i s i × O ( 1 ) = O ( i ) ? ? B u t i t s h

u

l d b e i n t e r m s

f

n ?

SLIDE 37

H e r e ' s a n e w

n

e

boolean unique(Object[] a) { for(int i = 0; i < a.length; i++) for (int j = 0; j < i; j++) if (a[i].equals(a[j])) return false; return true; }

B

d

y i s O ( 1 ) i < n , s

i

i s O ( n ) S

l
p

r u n s O ( n ) t i m e s , c

m

p l e x i t y : O ( n ) × O ( 1 ) = O ( n )

SLIDE 38

H e r e ' s a n e w

n

e

boolean unique(Object[] a) { for(int i = 0; i < a.length; i++) for (int j = 0; j < i; j++) if (a[i].equals(a[j])) return false; return true; }

B

d

y i s O ( 1 ) i < n , s

i

i s O ( n ) S

l
p

r u n s O ( n ) t i m e s , c

m

p l e x i t y : O ( n ) × O ( 1 ) = O ( n ) O u t e r l

p

r u n s n t i m e s : O ( n ) × O ( n ) = O ( n

2

)

SLIDE 39

T h r e e n e s t e d l

p

s

void something(Object[] a) { for(int i = 0; i < a.length; i++) for (int j = 0; j < i; j++) for (int k = 0; k < j; k++) “something that takes 1 step” }

i < n , j < n , k < n , s

a

l l t h r e e l

p

s r u n O ( n ) t i m e s T

t

a l r u n t i m e i s O ( n ) × O ( n ) × O ( n ) × O ( 1 ) = O ( n

3

)

SLIDE 40

W h a t ' s t h e c

m

p l e x i t y ?

void something(Object[] a) { for(int i = 0; i < a.length; i++) for (int j = 1; j < a.length; j *= 2) … // something taking O(1) time }

SLIDE 41

W h a t ' s t h e c

m

p l e x i t y ?

void something(Object[] a) { for(int i = 0; i < a.length; i++) for (int j = 1; j < a.length; j *= 2) … // something taking O(1) time }

A l

p

r u n n i n g t h r

u

g h i = 1 , 2 , 4 , … , n r u n s O ( l

g

n ) t i m e s !

I n n e r l

p

i s O ( l

g

n ) O u t e r l

p

i s O ( n l

g

n )

SLIDE 42

W h i l e l

p

s

long squareRoot(long n) { long i = 0; long j = n; while (i < j) { long k = (i + j) / 2; if (k*k <= n) i = k; else j = k-1; } return i; }

E a c h i t e r a t i

n

t a k e s O ( 1 ) t i m e . . . b u t h

w

m a n y t i m e s d

e

s t h e l

p

r u n ?

SLIDE 43

W h i l e l

p

s

long squareRoot(long n) { long i = 0; long j = n; while (i < j) { long k = (i + j) / 2; if (k*k <= n) i = k; else j = k-1; } return i; }

E a c h i t e r a t i

n

t a k e s O ( 1 ) t i m e . . . a n d h a l v e s j-i, s

O

( l

g

n ) i t e r a t i

n

s

SLIDE 44

S u m m a r y : l

p

s

B a s i c r u l e f

r

c

m

p l e x i t y

f

l

p

s :

N

u m b e r

f

i t e r a t i

n

s t i m e s c

m

p l e x i t y

f

b

d

y

f
r

( i n t i = ; i < n ; i + + ) … : n i t e r a t i

n

s

f
r

( i n t i = 1 ; i ≤ n ; i * = 2 ) : O ( l

g

n ) i t e r a t i

n

s

Wh

i l e l

p

s : h a v e t

w
r

k

u

t n u m b e r

f

i t e r a t i

n

s

I f t h e c

m

p l e x i t y

f

t h e b

d

y d e p e n d s

n

t h e v a l u e

f

t h e l

p

c

u

n t e r :

e

. g . O ( i ) , w h e r e ≤ i < n

Y
u

c a n s a f e l y r

u

n d i u p t

O

( n ) !

SLIDE 45

S e q u e n c e s

f

s t a t e m e n t s

Wh a t ' s t h e c

m

p l e x i t y h e r e ? ( A s s u m e t h a t t h e l

p

b

d

i e s a r e O ( 1 ) ) for (int i = 0; i < n; i++) … for (int i = 1; i < n; i *= 2) …

SLIDE 46

S e q u e n c e s

f

s t a t e m e n t s

Wh a t ' s t h e c

m

p l e x i t y h e r e ? ( A s s u m e t h a t t h e l

p

b

d

i e s a r e O ( 1 ) ) for (int i = 0; i < n; i++) … for (int i = 1; i < n; i *= 2) … F i r s t l

p

: O ( n ) S e c

n

d l

p

: O ( l

g

n ) T

t

a l : O ( n ) + O ( l

g

n ) = O ( n ) F

r

s e q u e n c e s , a d d t h e c

m

p l e x i t i e s !

SLIDE 47

M

d

e l l i n g a s l

w

d y n a m i c a r r a y

int[] array = {}; for (int i = 0; i < n; i+=100) { int[] newArray = new int[array.length+100]; for (int j = 0; j < i; j++) newArray[j] = array[j]; newArray = array; }

SLIDE 48

M

d

e l l i n g a s l

w

d y n a m i c a r r a y

int[] array = {}; for (int i = 0; i < n; i+=100) { int[] newArray = new int[array.length+100]; for (int j = 0; j < i; j++) newArray[j] = array[j]; newArray = array; }

I n n e r l

p

O ( n ) R e s t

f

l

p

b

d

y O ( 1 ) , s

l
p

b

d

y O ( 1 ) + O ( n ) = O ( n ) O u t e r l

p

: n i t e r a t i

n

s , O ( n ) b

d

y , s

O

( n

2

)

SLIDE 49

M

d

e l l i n g a f a s t d y n a m i c a r r a y

int[] array = {0}; for (int i = 1; i <= n; i=2) { int[] newArray = new int[array.length2]; for (int j = 0; j < i; j++) newArray[j] = array[j]; newArray = array; }

SLIDE 50

M

d

e l l i n g a f a s t d y n a m i c a r r a y

int[] array = {0}; for (int i = 1; i <= n; i=2) { int[] newArray = new int[array.length2]; for (int j = 0; j < i; j++) newArray[j] = array[j]; newArray = array; }

O u t e r l

p

: l

g

n i t e r a t i

n

s , O ( n ) b

d

y , s

O

( n l

g

n ) ? ?

SLIDE 51

M

d

e l l i n g a f a s t d y n a m i c a r r a y

int[] array = {0}; for (int i = 1; i <= n; i=2) { int[] newArray = new int[array.length2]; for (int j = 0; j < i; j++) newArray[j] = array[j]; newArray = array; }

H e r e w e “ r

u

n d u p ” O ( i ) t

O

( n ) . Tii s c a u s e s a n

v

e r e s t i m a t e !

SLIDE 52

A c

m

p l i c a t i

n

O u r a l g

r

i t h m h a s O ( n ) c

m

p l e x i t y , b u t w e ' v e c a l c u l a t e d O ( n l

g

n )

A

n

v

e r e s t i m a t e , b u t n

t

a s e v e r e

n

e ( I f n = 1 t h e n n l

g

n = 2 n )

Tii

s c a n h a p p e n b u t i s n

r

m a l l y n

t

s e v e r e

T
g

e t t h e r i g h t a n s w e r : d

t

h e m a t h s

G

d

n e w s : f

r

“ n

r

m a l ” l

p

s t h i s d

e

s n ' t h a p p e n

I

f a l l b

u

n d s a r e n ,

r

n

2

,

r

a n

t

h e r l

p

v a r i a b l e ,

r

a l

p

v a r i a b l e s q u a r e d ,

r

…

M a i n e x c e p t i

n

: l

p

v a r i a b l e i d

u

b l e s e v e r y t i m e , b

d

y c

m

p l e x i t y d e p e n d s

n

i

SLIDE 53

D

i

n g t h e s u m s

I n

u

r e x a m p l e :

Tie

i n n e r l

p

' s c

m

p l e x i t y i s O ( i )

I

n t h e

u

t e r l

p

, i r a n g e s

v

e r 1 , 2 , 4 , 8 , … , 2

a

I n s t e a d

f

r

u

n d i n g u p , w e w i l l a d d u p t h e t i m e f

r

a l l t h e i t e r a t i

n

s

f

t h e l

p

:

1 + 2 + 4 + 8 + … + 2

a

= 2 × 2

a

– 1 < 2 × 2

a

S i n c e 2

a

≤ n , t h e t

t

a l t i m e i s a t m

s

t 2 n , w h i c h i s O ( n )

SLIDE 54

A l a s t e x a m p l e

for (int i = 1; i <= n; i = 2) { for (int j = 0; j < nn; j++) for (int k = 0; k <= j; k++) // O(1) for (int j = 0; j < n; j++) // O(1) }

SLIDE 55

A l a s t e x a m p l e

for (int i = 1; i <= n; i = 2) { for (int j = 0; j < nn; j++) for (int k = 0; k <= j; k++) // O(1) for (int j = 0; j < n; j++) // O(1) } T

t

a l : O ( l

g

n ) × ( O ( n

2

) × O ( n

2

) + O ( n ) ) = O ( n

4

l

g

n )

k <= j < n*n s

t

h i s l

p

i s O ( n

2

) Tie

u

t e r l

p

r u n s O ( l

g

n ) t i m e s Tii s l

p

i s O ( n ) Tie j

l
p

r u n s n

2

t i m e s

SLIDE 56

A c

u

p l e

f

l

s

e e n d s

SLIDE 57

B i g Ω

R e c a l l t h a t b i g

O

a l l

w

s u s t

v

e r e s t i m a t e t h e g r

w

t h r a t e

f

a f u n c t i

n

:

2

n

2

+ 3 n + 1 i s O ( n

2

) , b u t a l s

O

( n

3

) B i g

O

h a s a c

u

s i n , b i g

(

“ b i g

m

e g a ” ) , w h i c h a l l

w

s u s Ω t

u

n d e r e s t i m a t e t h e g r

w

t h r a t e :

2

n

2

+ 3 n + 1 i s ( n Ω

2

) , b u t a l s

(

n ) Ω F

r

m a l l y w e j u s t r e p l a c e a ≤ w i t h a ≥ i n t h e d e fj n i t i

n
f

b i g

O

:

T

( n ) i s O ( n

2

) i f T ( n ) ≤ k n

2

f

r

s

m

e k , f

r

b i g e n

u

g h n

T

( n ) i s ( n Ω

2

) i f T ( n ) ≥ k n

2

f

r

s

m

e k , f

r

b i g e n

u

g h n

SLIDE 58

B i g Θ

Tie r e i s a l s

b

i g

(

“ b i g

t

h e t a ” ) , w h i c h i s l i k e b i g

O

Θ b u t r e q u i r e s t h e c

m

p l e x i t y g i v e n t

b

e t i g h t :

F
r

e x a m p l e , 2 n

2

+ 3 n + 1 i s ( n Θ

2

) ( a n d n

t

h i n g e l s e )

T

( n ) i s ( f ( n ) ) i f T ( n ) i s b

t

h O ( f ( n ) ) a n d ( f ( n ) ) Θ Ω Y

u

s h

u

l d r e c

g

n i s e a l l t h r e e n

t

a t i

n

s , b u t w e w i l l m

s

t l y s t i c k t

b

i g

O

i n t h i s c

u

r s e

Tie
t

h e r t w

a

r e g e n e r a l l y h a r d e r t

c

a l c u l a t e a c c u r a t e l y

B

i g

i

s m

s

t l y u s e f u l f

r

d e fj n i n g b i g

Ω

Θ

B

i g

O

g i v e s y

u

a n u p p e r b

u

n d , w h i c h c a n t e l l y

u

t h a t a n a l g

r

i t h m i s f a s t e n

u

g h

SLIDE 59

A m

r

t i s e d t i m e c

m

p l e x i t y

H

w

l

n

g d

e

s i t t a k e t

a

d d

n

e e l e m e n t t

a

d y n a m i c a r r a y ?

S

i m p l e a n s w e r : O ( n )

B

u t a d d i n g n e l e m e n t s t

a

n e m p t y a r r a y t a k e s O ( n ) t i m e , O ( 1 ) “ p e r e l e m e n t ” . S

i

t ’ s s

m

e h

w

O ( 1 ) “

n

a v e r a g e ” ?

I

f w e m e a s u r e t h e r u n t i m e

f

a p r

g

r a m u s i n g d y n a m i c a r r a y s , i t w i l l l

k

a s i f e a c h

p

e r a t i

n

t

k

O ( 1 ) t i m e !

T

c

a p t u r e t h i s , w e s a y t h a t a d d i n g a n e l e m e n t t

a

d y n a m i c a r r a y h a s O ( 1 ) a m

r

t i s e d c

m

p l e x i t y

A

n

p

e r a t i

n

h a s O ( f ( n ) ) a m

r

t i s e d c

m

p l e x i t y i f , f

r

a n y s e q u e n c e

f
p

e r a t i

n

s , t h e t

t

a l r u n t i m e i s a s i f e a c h

p

e r a t i

n

t

k

O ( f ( n ) ) t i m e

e

. g . : O ( l

g

n ) a m

r

t i s e d c

m

p l e x i t y n

p

e r a t i

n

s t a k e O ( n l

g

n ) t i m e →

A

m

r

t i s e d c

m

p l e x i t y c a n

c

c u r w h e n a n e x p e n s i v e

p

e r a t i

n

i s a l w a y s b a l a n c e d

u

t b y m a n y c h e a p

n

e s

B e c a r e f u l t

d

i s t i n g u i s h a m

r

t i s e d f r

m

“ n

r

m a l ” c

m

p l e x i t y

I

f y

u

r p r

g

r a m h a s r e a l

t

i m e c

n

s t r a i n t s , t h e n a d a t a s t r u c t u r e w i t h a m

r

t i s e d c

m

p l e x i t y m a y b e t

t

a l l y u n s u i t a b l e

B

u t f

r

m

s

t a p p l i c a t i

n

s , i t w

r

k s j u s t fj n e

SLIDE 60

T h e u n i f

r

m c

s

t m

d

e l

W e a s s u m e d t h a t a l l p r i m i t i v e

p

e r a t i

n

s t

k

c

n

s t a n t t i m e – t h i s i s c a l l e d t h e u n i f

r

m c

s

t m

d

e l B u t – i f y

u

r p r

g

r a m m i n g l a n g u a g e s u p p

r

t s i n t e g e r s

f

u n b

u

n d e d s i z e – t h e n a r i t h m e t i c

n

b i g g e r n u m b e r s t a k e s l

n

g e r !

M
s

t a r i t h m e t i c

p

e r a t i

n

s g r

w

a s O ( l

g

n ) , w h e r e n i s t h e m a g n i t u d e

f

t h e n u m b e r

Tii

s i s c a l l e d t h e l

g

a r i t h m i c c

s

t m

d

e l

I

t i s c

m

m

n

w h e n i n t e g e r s c a n b e u n b

u

n d e d s i z e , a n d a l s

i

n s

m

e s p e c i a l i s e d a p p l i c a t i

n

s l i k e c r y p t

g

r a p h y

SLIDE 61

L i f e w i t h

u

t b i g O n

t

a t i

n

SLIDE 62

W h a t h a p p e n s w i t h

u

t b i g O ?

H

w

m a n y s t e p s d

e

s t h i s f u n c t i

n

t a k e

n

a n a r r a y

f

l e n g t h n ( i n t h e w

r

s t c a s e ) ?

boolean unique(Object[] a) { for(int i = 0; i < a.length; i++) for (int j = 0; j < a.length; j++) if (a[i].equals(a[j]) && i != j) return false; return true; }

A s s u m e t h a t l

p

b

d

y t a k e s 1 s t e p

SLIDE 63

W h a t h a p p e n s w i t h

u

t b i g O ?

H

w

m a n y s t e p s d

e

s t h i s f u n c t i

n

t a k e

n

a n a r r a y

f

l e n g t h n ( i n t h e w

r

s t c a s e ) ?

boolean unique(Object[] a) { for(int i = 0; i < a.length; i++) for (int j = 0; j < a.length; j++) if (a[i].equals(a[j]) && i != j) return false; return true; }

O u t e r l

p

r u n s n t i m e s E a c h t i m e , i n n e r l

p

r u n s n t i m e s T

t

a l : n × n = n

2

SLIDE 64

W h a t a b

u

t t h i s

n

e ?

boolean unique(Object[] a) { for(int i = 0; i < a.length; i++) for (int j = 0; j < i; j++) if (a[i].equals(a[j])) return false; return true; }

L

p

r u n s t

i

i n s t e a d

f

n

SLIDE 65

S

m

e h a r d s u m s

Wh e n i = , i n n e r l

p

r u n s t i m e s Wh e n i = 1 , i n n e r l

p

r u n s 1 t i m e … Wh e n i = n

1

, i n n e r l

p

r u n s n

1

t i m e s T

t

a l :

=

+ 1 + 2 + … + n

1

w h i c h i s n ( n

1

) / 2

∑

i=0 n−1

i

SLIDE 66

W h a t a b

u

t t h i s

n

e ?

boolean unique(Object[] a) { for(int i = 0; i < a.length; i++) for (int j = 0; j < i; j++) if (a[i].equals(a[j])) return false; return true; }

A n s w e r : n ( n

1

) / 2

SLIDE 67

W h a t a b

u

t t h i s

n

e ?

void something(Object[] a) { for(int i = 0; i < a.length; i++) for (int j = 0; j < i; j++) for (int k = 0; k < j; k++) “something that takes 1 step” }

SLIDE 68

M

r

e h a r d s u m s

O u t e r l

p

: i g

e

s f r

m

t

n
1

M i d d l e l

p

: j g

e

s f r

m

t

i
1

I n n e r l

p

: k g

e

s f r

m

t

j
1

C

u

n t s : h

w

m a n y v a l u e s i , j , k w h e r e ≤ i < n , ≤ j < i , ≤ k ≤ j

∑

i=0 n−1

∑

j=0 i−1

∑

k=0 j−1

1

SLIDE 69

M

r

e h a r d s u m s

C

u

n t s : h

w

m a n y v a l u e s i , j , k w h e r e ≤ i < n , ≤ j < i , ≤ k ≤ j

∑

i=0 n−1

∑

j=0 i−1

∑

k=0 j−1

1

A n s w e r ( I l

k

e d i t u p ) : n ( n

1

) ( n

2

) / 6

SLIDE 70

W h a t a b

u

t t h i s

n

e ?

void something(Object[] a) { for(int i = 0; i < a.length; i++) for (int j = 0; j < i; j++) for (int k = 0; k < j; k++) “something that takes 1 step” }

A n s w e r : n ( n

1

) ( n

2

) / 6

SLIDE 71

S u m s v s i n t e g r a l s

F

r

e x a m p l e : N

t

q u i t e t h e s a m e , b u t c l

s

e ! ( u s u a l l y g i v e s t h e r i g h t c

m

p l e x i t y ) A b e t t e r a p p r

a

c h : “ F i n i t e c a l c u l u s : a t u t

r

i a l f

r

s

l

v i n g n a s t y s u m s ”

a

d a p t s r u l e s

f

c a l c u l u s t

w
r

k w i t h s u m s i n s t e a d

f

i n t e g r a l s

∑

x=a b

f (x)≈∫

a b

f (x)

∑

i=0 n

i=n(n+1)/2

∫

n

x dx=n

2/2

SLIDE 72

B i g O i n r e t r

s

p e c t

W e d

l
s

e s

m

e p r e c i s i

n

b y t h r

w

i n g a w a y c

n

s t a n t f a c t

r

s

.

. . y

u

p r

b

a b l y d

c

a r e a b

u

t a f a c t

r
f

1 p e r f

r

m a n c e i m p r

v

e m e n t

.

. . b u t i n p r a c t i c e t h e c

n

s t a n t f a c t

r

s d

n

’ t g e t m u c h h i g h e r t h a n 2 ,

O n t h e

t

h e r h a n d , l i f e g e t s m u c h s i m p l e r :

A

s m a l l p h r a s e l i k e O ( n

2

) t e l l s y

u

e x a c t l y h

w

t h e p e r f

r

m a n c e s c a l e s w h e n t h e i n p u t g e t s b i g

I

t ' s a l

t

e a s i e r t

c

a l c u l a t e b i g

O

c

m

p l e x i t y t h a n a p r e c i s e f

r

m u l a ( l

t

s

f

g

d

r u l e s t

h

e l p y

u

)

B i g O i s n

r

m a l l y a n e x c e l l e n t c

m

p r

m