Midterm Review Tyler Moore CSE 3353, SMU, Dallas, TX , 2013 - - PDF document

Midterm Review

Tyler Moore

CSE 3353, SMU, Dallas, TX

, 2013

Portions of these slides have been adapted from the slides written by Prof. Steven Skiena at SUNY Stony Brook, author

• f Algorithm Design Manual. For more information see http://www.cs.sunysb.edu/~skiena/

Administrivia

Extra office hours next week

Monday 12:30pm-1:30pm Monday 5-5:30pm You may pick up graded HW2 and answer key then

Midterm next Tuesday March 5

You may use one side of 1/2 sheet of letter paper for handwritten notes No calculators Review today

2 / 34

Defining bounding functions

f (n) = O(g(n)) means c · g(n) is an upper bound on f (n). Thus there exists some constant c such that f (n) is always ≤ c · g(n) for n ≥ no for some constant n0. f (n) = Ω(g(n)) means c · g(n) is a lower bound on f (n). Thus there exists some constant c such that f (n) is always ≥ c · g(n) for n ≥ no. for some constant n0. f (n) = Θ(g(n)) means c1 · g(n) is an upper bound on f (n) and c2 · g(n) is a lower bound on f (n). Thus there exists some constant c1 and c2 such that f (n) ≤ c1 · g(n) and f (n) ≥ c2 · g(n) for n ≥ no for some constant n0.

3 / 34

Dominance and little oh

We say that f (n) dominates g(n) if limn→∞

g(n) f (n) = 0. Otherwise f (n)

does not dominate g(n). We say that f (n) = o(g(n)) ⇐ ⇒ g(n) dominates f (n) So n2 = o(n3) since n3 dominates n2.

4 / 34

Notes Notes Notes Notes

Dominance examples

f (n) = n2, g(n) = n. Does f (n) dominate g(n)? Is n2 o(n)? Is n o(n2)? Is n o(n)?

5 / 34

Dominance relations

You should come to accept the dominance ranking of the basic functions: n! ≫ 2n ≫ n3 ≫ n2 ≫ n log n ≫ n ≫ √n ≫ log n ≫ 1

6 / 34

Discussion on Analysis

If the question asks you to explain why f (n) = Ω(g(n)), etc., then provide a value for c and n0 where the relationship holds, perhaps doing a bit of algebra to make the point clear. Best-case/worst-case/average-case does not correspond to Oh/Omega/Theta. Why do we see each of the following cost functions? (log n, n, n2, 2n, n!)

7 / 34

Useful rules on exponents and logarithms

loga(xy) = loga(x) + loga(y) loga(b) = logc b

logc a

loga(nb) = b · loga(n) xaxb = xa+b

xa xb = xa−b

xa · ya = (xy)a

xa ya = ( x y )a

8 / 34

Notes Notes Notes Notes

Python

If you have to write code on the exam, in most circumstances pseudo-code or Python is fine There are a few Python-specific points worth noting Python’s handling of mutable/immutable objects List comprehensions You do NOT need to worry about user-defined classes in Python (e.g., the code I showed on Binary Search Trees)

9 / 34

Variables in Python

Better thought of as names or identifiers attached to an object. A nice explanation: http://python.net/~goodger/projects/pycon/2007/ idiomatic/handout.html#other-languages-have-variables

10 / 34

Key distinction: mutable vs. immutable objects

Immutable: objects whose value cannot change

1

Tuples (makes sense)

2

Booleans (surprise?)

3

Numbers (surprise?)

4

Strings (surprise?)

Mutable: objects whose value can change

1

Dictionaries

2

Lists

3

User-defined objects (unless defined as immutable)

This distinction matters because it explains seemingly contradictory behavior

11 / 34

Variable assignment in action

>>> #variables are really names ... c = 4 >>> d = c >>> c+=1 >>> c 5 >>> d #d does not change because numbers are immutable 4 >>> #lists are mutable ... a = [1,4,2] >>> b = a #so this assigns the name b to the object attached to name a >>> a.append(3) >>> a [1, 4, 2, 3] >>> b #b still points to the same object, its contents have just changed. [1, 4, 2, 3]

12 / 34

Notes Notes Notes Notes

Im/mutablility and function calls

>>> #let’s try this in a function ... def increment(n): #n is a name assigned to the function argument when called ... #because numbers are immutable, the following ... #reassigns n to the number represented by n+1 ... n+=1 ... return n ... >>> a = 3 >>> increment(a) 4 >>> #a does not change ... a 3

13 / 34

Im/mutablility and function calls

>>> def sortfun(s): ... s.sort() ... return s ... >>> def sortfun2(s): ... l = list(s) ... l.sort() ... return l ... >>> a = [1,4,2] >>> sortfun(a) [1, 2, 4] >>> a [1, 2, 4] >>> b = [3,9,1] >>> sortfun2(b) [1, 3, 9] >>> b [3, 9, 1]

14 / 34

Im/mutablility and function calls

def selection_sort(s): """ Input: list s to be sorted Output: sorted list """ for i in range(len(s)): #don’t name min since reserved word minidx=i for j in range(i+1,len(s)): if s[j]<s[minidx]: minidx=j s[i],s[minidx]=s[minidx],s[i] return s >>> b [3, 9, 1] >>> selection_sort(b) [1, 3, 9] >>> b [1, 3, 9]

15 / 34

List comprehensions

Recall set-builder notation from Discrete Math: S = {3x|x ∈ N, x > 5} We can approximate that in Python

> > > range (1 ,11) [ 1 , 2 , 3 , 4 , 5 , 6 , 7 , 8 , 9 , 10] > > > {3∗ x f o r x i n range (1 ,11) i f x>5} s e t ( [ 2 4 , 18 , 27 , 21 , 3 0 ] ) > > > [3∗ x f o r x i n range (1 ,11) i f x >5] [18 , 21 , 24 , 27 , 30] Comprehensions arise in very common coding scenarios

16 / 34

Notes Notes Notes Notes

List comprehensions

Here’s a common coding task: iterate over some list, perform some action

• n each element of that list, and store the results in a new list. Here’s an

example:

> > > c h e e s e s = [ ’ s w i s s ’ , ’ g r u y e r e ’ , ’ cheddar ’ , ’ s t i l t o n ’ , ’ r o q u e f o r t ’ , ’ b r i e ’ ] > > > c h e e s e l e n =[] > > > f o r c i n c h e e s e s : . . . c h e e s e l e n . append ( l e n ( c )) . . . > > > c h e e s e l e n [ 5 , 7 , 7 , 7 , 9 , 4]

We can do this on a single line:

c h e e s e l e n =[ l e n ( c ) f o r c i n c h e e s e s ] > > > c h e e s e l e n [ 5 , 7 , 7 , 7 , 9 , 4]

17 / 34

List comprehensions

But wait, there’s more! Suppose you only want to add items to the list if they meet a certain condition, say if the item begins with the letter s. Well here’s the long way:

> > > s c h e e s e l e n =[] > > > f o r c i n c h e e s e s : . . . i f c[0]== ’ s ’ : . . . s c h e e s e l e n . append ( l e n ( c ) ) . . . > > > s c h e e s e l e n [ 5 , 7]

You can add a condition at the end of the list comprehension:

c h e e s e l e n =[ l e n ( c ) f o r c i n c h e e s e s i f c[0]==” s ” ] > > > s c h e e s e l e n [ 5 , 7]

18 / 34

More list comprehension examples

1 > > > s p o r t s 2 [ ’ f o o t b a l l ’ , ’ t e n n i s ’ , ’ i c e hockey ’ , ’ l a c r o s s e ’ , ’ f i e l d hockey ’ , ’ b a s k e t b a l l ’ , ’ b a s e b a l l ’ , ’ swimming ’ ] 3 > > > [ s f o r s i n s p o r t s i f l e n ( s ) >8] 4 [ ’ i c e hockey ’ , ’ f i e l d hockey ’ , ’ b a s k e t b a l l ’ ] 5 > > > [ s f o r s i n s p o r t s i f ’ b a l l ’ i n s ] 6 [ ’ f o o t b a l l ’ , ’ b a s k e t b a l l ’ , ’ b a s e b a l l ’ ] 7 8 > > > s p o r t s l o c 9 [ ( ’ f o o t b a l l ’ , ’ out ’ ) , ( ’ t e n n i s ’ , ’ both ’ ) , ( ’ i c e hockey ’ , ’ i n ’ ) , ( ’ l a c r o s s e ’ , ’ out ’ ) , ( ’ f i e l d hockey ’ , ’ out ’ ) , ( ’ b a s k e t b a l l ’ , ’ i n ’ ) , ( ’ b a s e b a l l ’ , ’ out ’ ) , ( ’ swimming ’ , ’ i n ’ ) ] 10 > > > [ s [ 0 ] f o r s i n s p o r t s l o c ] 11 [ ’ f o o t b a l l ’ , ’ t e n n i s ’ , ’ i c e hockey ’ , ’ l a c r o s s e ’ , ’ f i e l d hockey ’ , ’ b a s k e t b a l l ’ , ’ b a s e b a l l ’ , ’ swimming ’ ] 12 > > > o u t d o o r s p o r t s =[ s [ 0 ] f o r s i n s p o r t s l o c i f s [1]== ’ out ’ ] 13 > > > o u t d o o r s p o r t s 14 [ ’ f o o t b a l l ’ , ’ l a c r o s s e ’ , ’ f i e l d hockey ’ , ’ b a s e b a l l ’ ]

19 / 34

List comprehension exercise

Write a function that squares each element of a list so long as the elements are positive using a list comprehension. Complete this code:

def s q u a r e L i s t ( l ) : return # f i l l i n l i s t comprehension

20 / 34

Notes Notes Notes Notes

Sorting and Searching

You should be comfortable with how differences in implementation can lead to different costs of operations (e.g., searching for an element in an unsorted array, sorted array, binary tree, balanced binary tree) You should be comfortable with how Quicksort and Mergesort work You should understand why randomizing inputs helps improve Quicksort’s expected performance

21 / 34

Recurrence Relations

Recurrence relations specify the cost of executing recursive functions. Consider mergesort

1

Linear-time cost to divide the lists

2

Two recursive calls are made, each given half the original input

3

Linear-time cost to merge the resulting lists together

Recurrence: T(n) = 2T( n

2) + Θ(n)

Great, but how does this help us estimate the running time?

22 / 34

Enter the Master Theorem!

Definition

The Master Theorem For any recurrence relation of the form T(n) = aT(n/b) + c · nk , T(1) = c, the following relationships hold: T(n) =      Θ(nlogb a) if a > bk Θ(nk log n) if a = bk Θ(nk) if a < bk. So what’s the complexity of Mergesort? Mergesort recurrence: T(n) = 2T( n

2) + Θ(n)

Since a = 2, b = 2, k = 1, 2 = 21. Thus T(n) = Θ(nk log n)

23 / 34

Apply the Master Theorem

Definition

The Master Theorem For any recurrence relation of the form T(n) = aT(n/b) + c · nk , T(1) = c, the following relationships hold: T(n) =      Θ(nlogb a) if a > bk Θ(nk log n) if a = bk Θ(nk) if a < bk. Let’s try another one: T(n) = 3T( n

5) + 8n2

Well a = 3, b = 5, c = 8, k = 2, and 3 < 52. Thus T(n) = Θ(n2)

24 / 34

Notes Notes Notes Notes

Apply the Master Theorem

Definition

The Master Theorem For any recurrence relation of the form T(n) = aT(n/b) + c · nk , T(1) = c, the following relationships hold: T(n) =      Θ(nlogb a) if a > bk Θ(nk log n) if a = bk Θ(nk) if a < bk. Now it’s your turn: T(n) = 4T( n

2) + 5n

25 / 34

What’s going on in the three cases?

Definition

The Master Theorem For any recurrence relation of the form T(n) = aT(n/b) + c · nk , T(1) = c, the following relationships hold: T(n) =      Θ(nlogb a) if a > bk Θ(nk log n) if a = bk Θ(nk) if a < bk.

1 Too many leaves: leaf nodes outweighs sum of divide and glue costs 2 Equal work per level: split between leaves matches divide and glue

costs

3 Too expensive a root: divide and glue costs dominate 26 / 34

Induction and Recursion (material from this slide on is NOT on the midterm)

Divide and conquer algorithms have shown that recursion can help improve efficiency by repeatedly dividing the problem into more manageable chunks In general, recursion leverages induction: assuming that a subset of the problem is true and then solving the bigger case using the results

• f the solved subset

Induction

1 n − 1

. . . n Assume Deduce Recursion

n − 1

n . . .

1

Delegate Extend

27 / 34

Recursive Insertion Sort

1 3 4 6 2 . . . i − 1 i . . . 1: Delegate to sort i − 1 1 2 3 4 6 . . . 2: Swap ith until bigger

1 def

i n s s o r t r e c ( seq , i ) :

2

i f i ==0: return #Base case − − do nothing

3

i n s s o r t r e c ( seq , i −1) #Sort 0 . . i −1

4

j = i #S t a r t ” walking ” down

5

while j > 0 and seq [ j −1] > seq [ j ] : #Look f o r OK spot

6

seq [ j −1] , seq [ j ] = seq [ j ] , seq [ j −1]#Keep moving seq [ j ] down

7

j −= 1 #Decrement j

28 / 34

Notes Notes Notes Notes

Equivalent Iterative Insertion Sort

1 def

i n s s o r t ( seq ) :

2

f o r i i n range (1 , l e n ( seq ) ) : #0 . . i −1 s o r t e d so f a r

3

j = i #S t a r t ” walking ” down

4

while j > 0 and seq [ j −1] > seq [ j ] : #Look f o r OK spot

5

seq [ j −1] , seq [ j ] = seq [ j ] , seq [ j −1] # Keep moving seq [ j ] down

6

j −= 1 #Decrement j

29 / 34

Recurrence

What’s the complexity of the recursive insertion sort? Recurrence: T(n) = T(n − 1) + n Complexity: Θ(n2)

30 / 34

Applying Induction: Finding a Maximum Permutation

Suppose 8 people have bought tickets to a concert. Some people are happy with their seats, while others prefer to swap with each other. People only prefer one seat (either their own or someone elses) Problem: present an algorithm that lets the most people sit in their preferred seat This so-called maximum permutation has an inductive solution

31 / 34

Example: see whiteboard

32 / 34

Notes Notes Notes Notes

Implementing the solution recursively

Data Structure: M = [2,2,0,5,3,5,7,4] Input: Mapping M, set of remaining people A Output: Maximal permutation of people Algorithm Procedure

1

Creates a set of seats B that are pointed to by A

2

Base case: only one person, so they get their choice

3

Find someone whose seat is not in B and remove them from A

4

Invoke the function recursively on the smaller A

33 / 34

Recursive Code: Maximum Permutation

1 def

naive max perm (M, A=None ) :

2

i f A i s None : #The e l t . s e t not s u p p l i e d ?

3

A = s e t ( range ( l e n (M) ) ) # A = {0 , 1 , . . . , n−1}

4

i f l e n (A) == 1 : return A #Base case − − s i n g l e −e l t . A

5

B = s e t (M[ i ] f o r i i n A) #The ” pointed to ” elements

6

C = A − B #”Not pointed to ” elements

7

i f C : #Any u s e l e s s elements ?

8

• A. remove (C. pop ( ) )

#Remove one

• f

them

9

return naive max perm (M, A)#Solve remaining problem

10

return A #A l l u s e f u l − − r e t u r n a l l Output

1 >

> > M = [ 2 , 2 , 0 , 5 , 3 , 5 , 7 , 4 ]

2 >

> > M[ 2 ]

3 0 4 >

> > s o r t e d ( naive max perm (M) )

5 [ 0 ,

2 , 5]

34 / 34

Notes Notes Notes Notes