12 A: Algorithm Design Techniques II CS1102S: Data Structures and - - PowerPoint PPT Presentation

▶
12 a algorithm design techniques ii
SMART_READER_LITE
LIVE PREVIEW

12 A: Algorithm Design Techniques II CS1102S: Data Structures and - - PowerPoint PPT Presentation

Jul 12, 2023 257 likes •841 views

Greedy Algorithms (brief review) Divide and Conquer (Example) Dynamic Programming Backtracking Algorithms Another Puzzler 12 A: Algorithm Design Techniques II CS1102S: Data Structures and Algorithms Martin Henz April 7, 2010 Generated on

slide-1
SLIDE 1

Greedy Algorithms (brief review) Divide and Conquer (Example) Dynamic Programming Backtracking Algorithms Another Puzzler

12 A: Algorithm Design Techniques II

CS1102S: Data Structures and Algorithms

Martin Henz

April 7, 2010

Generated on Tuesday 6th April, 2010, 14:42 CS1102S: Data Structures and Algorithms 12 A: Algorithm Design Techniques II 1

slide-2
SLIDE 2

Greedy Algorithms (brief review) Divide and Conquer (Example) Dynamic Programming Backtracking Algorithms Another Puzzler

1

Greedy Algorithms (brief review)

2

Divide and Conquer (Example)

3

Dynamic Programming

4

Backtracking Algorithms

5

Another Puzzler

CS1102S: Data Structures and Algorithms 12 A: Algorithm Design Techniques II 2

slide-3
SLIDE 3

Greedy Algorithms (brief review) Divide and Conquer (Example) Dynamic Programming Backtracking Algorithms Another Puzzler Scheduling Huffman Codes

1

Greedy Algorithms (brief review) Scheduling Huffman Codes

2

Divide and Conquer (Example)

3

Dynamic Programming

4

Backtracking Algorithms

5

Another Puzzler

CS1102S: Data Structures and Algorithms 12 A: Algorithm Design Techniques II 3

slide-4
SLIDE 4

Greedy Algorithms (brief review) Divide and Conquer (Example) Dynamic Programming Backtracking Algorithms Another Puzzler Scheduling Huffman Codes

Nonpreemptive Scheduling

Input A set of jobs with a running time for each Desired output A sequence for the jobs to execute on on single machine, minimizing the average completion time

CS1102S: Data Structures and Algorithms 12 A: Algorithm Design Techniques II 4

slide-5
SLIDE 5

Greedy Algorithms (brief review) Divide and Conquer (Example) Dynamic Programming Backtracking Algorithms Another Puzzler Scheduling Huffman Codes

Example

Some schedule: The optimal schedule:

CS1102S: Data Structures and Algorithms 12 A: Algorithm Design Techniques II 5

slide-6
SLIDE 6

Greedy Algorithms (brief review) Divide and Conquer (Example) Dynamic Programming Backtracking Algorithms Another Puzzler Scheduling Huffman Codes

The Multiprocessor Case

N processors Now we can run the jobs on N identical machines. What is a schedule that minimizes the average completion time?

CS1102S: Data Structures and Algorithms 12 A: Algorithm Design Techniques II 6

slide-7
SLIDE 7

Greedy Algorithms (brief review) Divide and Conquer (Example) Dynamic Programming Backtracking Algorithms Another Puzzler Scheduling Huffman Codes

Example

CS1102S: Data Structures and Algorithms 12 A: Algorithm Design Techniques II 7

slide-8
SLIDE 8

Greedy Algorithms (brief review) Divide and Conquer (Example) Dynamic Programming Backtracking Algorithms Another Puzzler Scheduling Huffman Codes

A “Slight” Variant

Miniming final completion time If we want to minimize the final completion time (completion time of the last task), the problem becomes NP-complete!

CS1102S: Data Structures and Algorithms 12 A: Algorithm Design Techniques II 8

slide-9
SLIDE 9

Greedy Algorithms (brief review) Divide and Conquer (Example) Dynamic Programming Backtracking Algorithms Another Puzzler Scheduling Huffman Codes

Optimal Prefix Code in Table Form

CS1102S: Data Structures and Algorithms 12 A: Algorithm Design Techniques II 9

slide-10
SLIDE 10

Greedy Algorithms (brief review) Divide and Conquer (Example) Dynamic Programming Backtracking Algorithms Another Puzzler Scheduling Huffman Codes

Optimal Prefix Code in Tree Form

CS1102S: Data Structures and Algorithms 12 A: Algorithm Design Techniques II 10

slide-11
SLIDE 11

Greedy Algorithms (brief review) Divide and Conquer (Example) Dynamic Programming Backtracking Algorithms Another Puzzler Scheduling Huffman Codes

Huffman’s Algorithm: An Example

CS1102S: Data Structures and Algorithms 12 A: Algorithm Design Techniques II 11

slide-12
SLIDE 12

Greedy Algorithms (brief review) Divide and Conquer (Example) Dynamic Programming Backtracking Algorithms Another Puzzler Scheduling Huffman Codes

Huffman’s Algorithm: An Example

CS1102S: Data Structures and Algorithms 12 A: Algorithm Design Techniques II 12

slide-13
SLIDE 13

Greedy Algorithms (brief review) Divide and Conquer (Example) Dynamic Programming Backtracking Algorithms Another Puzzler Scheduling Huffman Codes

Huffman’s Algorithm: An Example

CS1102S: Data Structures and Algorithms 12 A: Algorithm Design Techniques II 13

slide-14
SLIDE 14

Greedy Algorithms (brief review) Divide and Conquer (Example) Dynamic Programming Backtracking Algorithms Another Puzzler Scheduling Huffman Codes

Huffman’s Algorithm: An Example

CS1102S: Data Structures and Algorithms 12 A: Algorithm Design Techniques II 14

slide-15
SLIDE 15

Greedy Algorithms (brief review) Divide and Conquer (Example) Dynamic Programming Backtracking Algorithms Another Puzzler Scheduling Huffman Codes

Huffman’s Algorithm: An Example

CS1102S: Data Structures and Algorithms 12 A: Algorithm Design Techniques II 15

slide-16
SLIDE 16

Greedy Algorithms (brief review) Divide and Conquer (Example) Dynamic Programming Backtracking Algorithms Another Puzzler Scheduling Huffman Codes

Huffman’s Algorithm: An Example

CS1102S: Data Structures and Algorithms 12 A: Algorithm Design Techniques II 16

slide-17
SLIDE 17

Greedy Algorithms (brief review) Divide and Conquer (Example) Dynamic Programming Backtracking Algorithms Another Puzzler Scheduling Huffman Codes

Huffman’s Algorithm: An Example

CS1102S: Data Structures and Algorithms 12 A: Algorithm Design Techniques II 17

slide-18
SLIDE 18

Greedy Algorithms (brief review) Divide and Conquer (Example) Dynamic Programming Backtracking Algorithms Another Puzzler Scheduling Huffman Codes

Correctness of Huffman’s Algorithm

Observation 1 An optimal tree must be full; no node has only one child. Observation 2 The two least frequent characters α and β must be the two deepest nodes. Observation 3 Characters at the same level can be swapped without affecting

  • ptimality.

CS1102S: Data Structures and Algorithms 12 A: Algorithm Design Techniques II 18

slide-19
SLIDE 19

Greedy Algorithms (brief review) Divide and Conquer (Example) Dynamic Programming Backtracking Algorithms Another Puzzler Scheduling Huffman Codes

Correctness of Huffman’s Algorithm

Observation 3 Characters at the same level can be swapped without affecting

  • ptimality.

Initial Step of Huffman’s Algorithm An optimal tree can be found that contains the two least frequent symbols as siblings; the first step in Huffman’s algorithm is not a mistake. Observation 4 Every step of Huffman’s algorithm produces a simplified problem, resulting from treating two characters as indistinguishable. Each Step of Huffman’s Algorithm

CS1102S: Data Structures and Algorithms 12 A: Algorithm Design Techniques II 19

slide-20
SLIDE 20

Greedy Algorithms (brief review) Divide and Conquer (Example) Dynamic Programming Backtracking Algorithms Another Puzzler

1

Greedy Algorithms (brief review)

2

Divide and Conquer (Example)

3

Dynamic Programming

4

Backtracking Algorithms

5

Another Puzzler

CS1102S: Data Structures and Algorithms 12 A: Algorithm Design Techniques II 20

slide-21
SLIDE 21

Greedy Algorithms (brief review) Divide and Conquer (Example) Dynamic Programming Backtracking Algorithms Another Puzzler

Closest-Points Problem

Input Set of points in a plane Euclidean distance between p1 and p2 [(x1 − x2)2 + (y1 − y2)2]1/2 Required output Find the closest pair of points

CS1102S: Data Structures and Algorithms 12 A: Algorithm Design Techniques II 21

slide-22
SLIDE 22

Greedy Algorithms (brief review) Divide and Conquer (Example) Dynamic Programming Backtracking Algorithms Another Puzzler

Example

CS1102S: Data Structures and Algorithms 12 A: Algorithm Design Techniques II 22

slide-23
SLIDE 23

Greedy Algorithms (brief review) Divide and Conquer (Example) Dynamic Programming Backtracking Algorithms Another Puzzler

Naive Algorithm

Exhaustive search Compute the distance between each two points and keep the smallest Run time There are N2 pairs to check, thus O(N2)

CS1102S: Data Structures and Algorithms 12 A: Algorithm Design Techniques II 23

slide-24
SLIDE 24

Greedy Algorithms (brief review) Divide and Conquer (Example) Dynamic Programming Backtracking Algorithms Another Puzzler

Idea

Preparation Sort points by x coordinate; O(N log N) Divide and Conquer Split point set into two halves, PL and PR. Recursively find the smallest distance in each half. Find the smallest distance of pairs that cross the separation line.

CS1102S: Data Structures and Algorithms 12 A: Algorithm Design Techniques II 24

slide-25
SLIDE 25

Greedy Algorithms (brief review) Divide and Conquer (Example) Dynamic Programming Backtracking Algorithms Another Puzzler

Partitioning with Shortest Distances Shown

CS1102S: Data Structures and Algorithms 12 A: Algorithm Design Techniques II 25

slide-26
SLIDE 26

Greedy Algorithms (brief review) Divide and Conquer (Example) Dynamic Programming Backtracking Algorithms Another Puzzler

Two-lane Strip

CS1102S: Data Structures and Algorithms 12 A: Algorithm Design Techniques II 26

slide-27
SLIDE 27

Greedy Algorithms (brief review) Divide and Conquer (Example) Dynamic Programming Backtracking Algorithms Another Puzzler

Brute Force Calculation of min(δ, dC)

CS1102S: Data Structures and Algorithms 12 A: Algorithm Design Techniques II 27

slide-28
SLIDE 28

Greedy Algorithms (brief review) Divide and Conquer (Example) Dynamic Programming Backtracking Algorithms Another Puzzler

Better Idea

Sort points by y coordinate This allows a scan of the strip. Sort points by y coordinate This allows a scan of the strip.

CS1102S: Data Structures and Algorithms 12 A: Algorithm Design Techniques II 28

slide-29
SLIDE 29

Greedy Algorithms (brief review) Divide and Conquer (Example) Dynamic Programming Backtracking Algorithms Another Puzzler

Only p4 and p5 Need To Be Considered

CS1102S: Data Structures and Algorithms 12 A: Algorithm Design Techniques II 29

slide-30
SLIDE 30

Greedy Algorithms (brief review) Divide and Conquer (Example) Dynamic Programming Backtracking Algorithms Another Puzzler

Refined Calculation of min(δ, dC)

CS1102S: Data Structures and Algorithms 12 A: Algorithm Design Techniques II 30

slide-31
SLIDE 31

Greedy Algorithms (brief review) Divide and Conquer (Example) Dynamic Programming Backtracking Algorithms Another Puzzler

At Most Eight Points Fit in Rectangle

CS1102S: Data Structures and Algorithms 12 A: Algorithm Design Techniques II 31

slide-32
SLIDE 32

Greedy Algorithms (brief review) Divide and Conquer (Example) Dynamic Programming Backtracking Algorithms Another Puzzler Fibonacci Numbers Optimal Binary Search Tree All-pairs Shortest Path

1

Greedy Algorithms (brief review)

2

Divide and Conquer (Example)

3

Dynamic Programming Fibonacci Numbers Optimal Binary Search Tree All-pairs Shortest Path

4

Backtracking Algorithms

5

Another Puzzler

CS1102S: Data Structures and Algorithms 12 A: Algorithm Design Techniques II 32

slide-33
SLIDE 33

Greedy Algorithms (brief review) Divide and Conquer (Example) Dynamic Programming Backtracking Algorithms Another Puzzler Fibonacci Numbers Optimal Binary Search Tree All-pairs Shortest Path

Inefficient Algorithm

public static int f i b ( int n ) { i f ( n <= 1) return 1; else return f i b ( n − 1) + f i b ( n − 2 ) ; }

CS1102S: Data Structures and Algorithms 12 A: Algorithm Design Techniques II 33

slide-34
SLIDE 34

Greedy Algorithms (brief review) Divide and Conquer (Example) Dynamic Programming Backtracking Algorithms Another Puzzler Fibonacci Numbers Optimal Binary Search Tree All-pairs Shortest Path

Trace of Recursion

CS1102S: Data Structures and Algorithms 12 A: Algorithm Design Techniques II 34

slide-35
SLIDE 35

Greedy Algorithms (brief review) Divide and Conquer (Example) Dynamic Programming Backtracking Algorithms Another Puzzler Fibonacci Numbers Optimal Binary Search Tree All-pairs Shortest Path

Memoization

int [ ] f i b s = new int [ 1 0 0 ] ; public static int f i b ( int n ) { i f ( f i b s [ n ] ! = 0 ) return f i b s [ n ] ; i f ( n <= 1) return 1; int new fib = f i b ( n − 1) + f i b ( n − 2 ) ; f i b s [ n ] = new fib ; return new fib ; }

CS1102S: Data Structures and Algorithms 12 A: Algorithm Design Techniques II 35

slide-36
SLIDE 36

Greedy Algorithms (brief review) Divide and Conquer (Example) Dynamic Programming Backtracking Algorithms Another Puzzler Fibonacci Numbers Optimal Binary Search Tree All-pairs Shortest Path

A Simple Loop for Fibonacci Numbers

public static int f i b ( int n ) { i f ( n <= 1) return 1; int l a s t = 1 , nextToLast = 1; answer = 1; for ( int i = 2; i <= n ; i ++) { answer = l a s t + nextToLast ; nextToLast = l a s t ; l a s t = answer ; } return answer ; }

CS1102S: Data Structures and Algorithms 12 A: Algorithm Design Techniques II 36

slide-37
SLIDE 37

Greedy Algorithms (brief review) Divide and Conquer (Example) Dynamic Programming Backtracking Algorithms Another Puzzler Fibonacci Numbers Optimal Binary Search Tree All-pairs Shortest Path

Sample Input

CS1102S: Data Structures and Algorithms 12 A: Algorithm Design Techniques II 37

slide-38
SLIDE 38

Greedy Algorithms (brief review) Divide and Conquer (Example) Dynamic Programming Backtracking Algorithms Another Puzzler Fibonacci Numbers Optimal Binary Search Tree All-pairs Shortest Path

Three Possible Binary Search Trees

CS1102S: Data Structures and Algorithms 12 A: Algorithm Design Techniques II 38

slide-39
SLIDE 39

Greedy Algorithms (brief review) Divide and Conquer (Example) Dynamic Programming Backtracking Algorithms Another Puzzler Fibonacci Numbers Optimal Binary Search Tree All-pairs Shortest Path

Comparison of the Three Trees

CS1102S: Data Structures and Algorithms 12 A: Algorithm Design Techniques II 39

slide-40
SLIDE 40

Greedy Algorithms (brief review) Divide and Conquer (Example) Dynamic Programming Backtracking Algorithms Another Puzzler Fibonacci Numbers Optimal Binary Search Tree All-pairs Shortest Path

Structure of Optimal Binary Search Tree

CS1102S: Data Structures and Algorithms 12 A: Algorithm Design Techniques II 40

slide-41
SLIDE 41

Greedy Algorithms (brief review) Divide and Conquer (Example) Dynamic Programming Backtracking Algorithms Another Puzzler Fibonacci Numbers Optimal Binary Search Tree All-pairs Shortest Path

Idea

Proceed in order of growing tree size For each range of words, compute optimal tree Memoization For each range, store optimal tree for later retrieval

CS1102S: Data Structures and Algorithms 12 A: Algorithm Design Techniques II 41

slide-42
SLIDE 42

Greedy Algorithms (brief review) Divide and Conquer (Example) Dynamic Programming Backtracking Algorithms Another Puzzler Fibonacci Numbers Optimal Binary Search Tree All-pairs Shortest Path

Computation of Optimal Binary Search Tree

CS1102S: Data Structures and Algorithms 12 A: Algorithm Design Techniques II 42

slide-43
SLIDE 43

Greedy Algorithms (brief review) Divide and Conquer (Example) Dynamic Programming Backtracking Algorithms Another Puzzler Fibonacci Numbers Optimal Binary Search Tree All-pairs Shortest Path

Run Time

For each cell of table Consider all possible roots Overall runtime O(N3)

CS1102S: Data Structures and Algorithms 12 A: Algorithm Design Techniques II 43

slide-44
SLIDE 44

Greedy Algorithms (brief review) Divide and Conquer (Example) Dynamic Programming Backtracking Algorithms Another Puzzler Fibonacci Numbers Optimal Binary Search Tree All-pairs Shortest Path

Example

CS1102S: Data Structures and Algorithms 12 A: Algorithm Design Techniques II 44

slide-45
SLIDE 45

Greedy Algorithms (brief review) Divide and Conquer (Example) Dynamic Programming Backtracking Algorithms Another Puzzler

1

Greedy Algorithms (brief review)

2

Divide and Conquer (Example)

3

Dynamic Programming

4

Backtracking Algorithms

5

Another Puzzler

CS1102S: Data Structures and Algorithms 12 A: Algorithm Design Techniques II 45

slide-46
SLIDE 46

Greedy Algorithms (brief review) Divide and Conquer (Example) Dynamic Programming Backtracking Algorithms Another Puzzler

Example: The Turnpike Reconstruction Problem

From points to distances It is easy to calculate for a set of points on a line all distances between points. From distances to points Consider the reverse: given the set of all distances between the points, compute the positions of the points! Placement of first point By default, the first point is placed at position 0.

CS1102S: Data Structures and Algorithms 12 A: Algorithm Design Techniques II 46

slide-47
SLIDE 47

Greedy Algorithms (brief review) Divide and Conquer (Example) Dynamic Programming Backtracking Algorithms Another Puzzler

Example

Input set for 6 points D = {1, 2, 2, 2, 3, 3, 3, 4, 5, 5, 5, 6, 7, 8, 10} First and last point First point goes to 0, which means that the last point goes to 10. Placement of third point The third point can be placed either at position 2 or position 8. This choice is arbitrary; we can flip a solution to get from one to the other!

CS1102S: Data Structures and Algorithms 12 A: Algorithm Design Techniques II 47

slide-48
SLIDE 48

Greedy Algorithms (brief review) Divide and Conquer (Example) Dynamic Programming Backtracking Algorithms Another Puzzler

Decision Tree for the Example

CS1102S: Data Structures and Algorithms 12 A: Algorithm Design Techniques II 48

slide-49
SLIDE 49

Greedy Algorithms (brief review) Divide and Conquer (Example) Dynamic Programming Backtracking Algorithms Another Puzzler

Summary of Backtracking

Decision tree Backtracking explores decision tree; typically exponential in size Pruning At each node, reasoning is applied to avoid exploring the subtree

CS1102S: Data Structures and Algorithms 12 A: Algorithm Design Techniques II 49

slide-50
SLIDE 50

Greedy Algorithms (brief review) Divide and Conquer (Example) Dynamic Programming Backtracking Algorithms Another Puzzler

1

Greedy Algorithms (brief review)

2

Divide and Conquer (Example)

3

Dynamic Programming

4

Backtracking Algorithms

5

Another Puzzler

CS1102S: Data Structures and Algorithms 12 A: Algorithm Design Techniques II 50

slide-51
SLIDE 51

Greedy Algorithms (brief review) Divide and Conquer (Example) Dynamic Programming Backtracking Algorithms Another Puzzler

Remember Lecture 2 A: Parameter Passing

Java uses pass-by-value parameter passing. public static void tryChanging ( int a ) { a = 1; return ; } . . . int b = 2; tryChanging ( b ) ; System . out . p r i n t l n ( b ) ;

CS1102S: Data Structures and Algorithms 12 A: Algorithm Design Techniques II 51

slide-52
SLIDE 52

Greedy Algorithms (brief review) Divide and Conquer (Example) Dynamic Programming Backtracking Algorithms Another Puzzler

Remember Lecture 2 A: Parameter Passing with Objects

public static void tryChanging ( SomeObject obj ) {

  • bj . someField = 1;
  • bj = new SomeObject ( ) ;
  • bj . someField = 2;

return ; } . . . SomeObject someObj = new SomeObject ( ) ; tryChanging ( someObj ) ; System . out . p r i n t l n ( someObj . someField ) ;

CS1102S: Data Structures and Algorithms 12 A: Algorithm Design Techniques II 52

slide-53
SLIDE 53

Greedy Algorithms (brief review) Divide and Conquer (Example) Dynamic Programming Backtracking Algorithms Another Puzzler

Remember Lecture 7 A: Sorting

Input Unsorted array of elements Behavior Rearrange elements of array such that the smallest appears first, followed by the second smallest etc, finally followed by the largest element

CS1102S: Data Structures and Algorithms 12 A: Algorithm Design Techniques II 53

slide-54
SLIDE 54

Greedy Algorithms (brief review) Divide and Conquer (Example) Dynamic Programming Backtracking Algorithms Another Puzzler

Will This Work?

public static <AnyType extends Comparable<? super An void mergeSort ( AnyType [ ] a ) { AnyType [ ] r e t = . . . . ; / / declare helper array . . . . / / here goes a program that places . . . . / / the element

  • f

” a ” i n t o ” r e t ” so . . . . / / that ” r e t ” i s sorted a = r e t ; return ; } . . . Integer [ ] myArray = . . . ; IterativeMergeSort . mergeSort ( myArray ) ;

CS1102S: Data Structures and Algorithms 12 A: Algorithm Design Techniques II 54

slide-55
SLIDE 55

Greedy Algorithms (brief review) Divide and Conquer (Example) Dynamic Programming Backtracking Algorithms Another Puzzler

Will This Work?

public static <AnyType extends Comparable<? super An void mergeSort ( AnyType [ ] a ) { AnyType [ ] r e t = . . . . ; / / declare helper array . . . . / / here goes a program that places . . . . / / the element

  • f

” a ” i n t o ” r e t ” so . . . . / / that ” r e t ” i s sorted a = r e t ; return ; } . . . Integer [ ] myArray = . . . ; IterativeMergeSort . mergeSort ( myArray ) ; Answer: No! The assignment a = ret ; has no effect on myArray!

CS1102S: Data Structures and Algorithms 12 A: Algorithm Design Techniques II 55

slide-56
SLIDE 56

Greedy Algorithms (brief review) Divide and Conquer (Example) Dynamic Programming Backtracking Algorithms Another Puzzler

Next Lecture

Friday: Search trees with external storage (Section 4.7) Preview: Extendible hashing (Section 5.7) Preview: External sorting (Section 7.10)

CS1102S: Data Structures and Algorithms 12 A: Algorithm Design Techniques II 56