Dynamic tables Task: Store a dynamic set in a table/array. Elements - PowerPoint PPT Presentation

CS 5633 -- Spring 2008 Dynamic tables Task: Store a dynamic set in a table/array. Elements can only be inserted, and all inserted elements are stored in one contiguous part in the array. The table should be as small as possible, but large enough so that it won’t overflow. Problem: We may not know the proper size in advance! Amortized Analysis Solution: Dynamic tables. IDEA: Whenever the table overflows, “grow” it by Carola Wenk allocating (via malloc or new ) a new, larger table. Slides courtesy of Charles Leiserson with small Move all items from the old table into the new one, changes by Carola Wenk and free the storage for the old table. 3/13/08 CS 5633 Analysis of Algorithms 1 3/13/08 CS 5633 Analysis of Algorithms 2 Example of a dynamic table Example of a dynamic table 1. I NSERT 1. I NSERT 1 1 1 2. I NSERT 2. I NSERT overflow overflow 3/13/08 CS 5633 Analysis of Algorithms 3 3/13/08 CS 5633 Analysis of Algorithms 4 1

Example of a dynamic table Example of a dynamic table 1. I NSERT 1. I NSERT 1 1 1 1 2. I NSERT 2. I NSERT 2 2 2 3. I NSERT overflow 3/13/08 CS 5633 Analysis of Algorithms 5 3/13/08 CS 5633 Analysis of Algorithms 6 Example of a dynamic table Example of a dynamic table 1. I NSERT 1. I NSERT 1 1 2. I NSERT 2. I NSERT 2 2 3. I NSERT 3. I NSERT overflow 3/13/08 CS 5633 Analysis of Algorithms 7 3/13/08 CS 5633 Analysis of Algorithms 8 2

Example of a dynamic table Example of a dynamic table 1. I NSERT 1. I NSERT 1 1 2. I NSERT 2. I NSERT 2 2 3. I NSERT 3. I NSERT 3 3 4. I NSERT 4 4. I NSERT 4 5. I NSERT overflow 3/13/08 CS 5633 Analysis of Algorithms 9 3/13/08 CS 5633 Analysis of Algorithms 10 Example of a dynamic table Example of a dynamic table 1. I NSERT 1. I NSERT 1 1 2. I NSERT 2. I NSERT 2 2 3. I NSERT 3. I NSERT 3 3 4. I NSERT 4. I NSERT 4 4 5. I NSERT 5. I NSERT overflow 3/13/08 CS 5633 Analysis of Algorithms 11 3/13/08 CS 5633 Analysis of Algorithms 12 3

Example of a dynamic table Worst-case analysis Consider a sequence of n insertions. The 1. I NSERT 1 worst-case time to execute one insertion is 2. I NSERT 2 Ο ( n ). Therefore, the worst-case time for n 3. I NSERT 3 insertions is n · Ο ( n ) = Ο ( n 2 ). 4. I NSERT 4 WRONG! In fact, the worst-case cost for 5. I NSERT 5 n insertions is only Θ ( n ) ≪ Ο ( n 2 ). 6 6. I NSERT 7 7. I NSERT Let’s see why. 3/13/08 CS 5633 Analysis of Algorithms 13 3/13/08 CS 5633 Analysis of Algorithms 14 Tighter analysis Tighter analysis Let c i = the cost of the i th insertion Let c i = the cost of the i th insertion 1 + cost to double array size = i 1 2 3 4 5 6 7 8 9 10 i 1 2 3 4 5 6 7 8 9 10 size i 1 2 4 4 8 8 8 8 16 16 size i 1 2 4 4 8 8 8 8 16 16 1 1 1 1 1 1 1 1 1 1 c i c i ? ? ? ? ? ? ? ? ? ? 3/13/08 CS 5633 Analysis of Algorithms 15 3/13/08 CS 5633 Analysis of Algorithms 16 4

Tighter analysis Tighter analysis Let c i = the cost of the i th insertion Let c i = the cost of the i th insertion = 1 + cost to double array size = 1 + cost to double array size i 1 2 3 4 5 6 7 8 9 10 i 1 2 3 4 5 6 7 8 9 10 size i 1 2 4 4 8 8 8 8 16 16 size i 1 2 4 4 8 8 8 8 16 16 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 c i c i 1 2 3 1 5 1 1 1 9 1 0 1 2 0 4 0 0 0 8 0 0 1 2 0 4 0 0 0 8 0 3/13/08 CS 5633 Analysis of Algorithms 17 3/13/08 CS 5633 Analysis of Algorithms 18 Tighter analysis (continued) Amortized analysis An amortized analysis is any strategy for n ∑ = Cost of n insertions c analyzing a sequence of operations: i = i 1 • compute the total cost of the sequence, OR −  lg( n 1 )  ∑ j ≤ + n 2 • amortized cost of an operation = average = j 0 cost per operation, averaged over the number ≤ 3 n of operations in the sequence = Θ ( n ) . • amortized cost can be small, even though a Thus, the average cost of each dynamic-table single operation within the sequence might be operation is Θ ( n )/ n = Θ (1). expensive 3/13/08 CS 5633 Analysis of Algorithms 19 3/13/08 CS 5633 Analysis of Algorithms 20 5

Amortized analysis Types of amortized analyses Three common amortization arguments: Even though we’re taking averages, however, • the aggregate method, probability is not involved! • the accounting method, Won’t cover in class • the potential method. • An amortized analysis guarantees the We’ve just seen an aggregate analysis. average performance of each operation in the worst case . The aggregate method, though simple, lacks the precision of the other two methods. In particular, the accounting and potential methods allow a specific amortized cost to be allocated to each operation. 3/13/08 CS 5633 Analysis of Algorithms 21 3/13/08 CS 5633 Analysis of Algorithms 22 Accounting analysis of Accounting method dynamic tables • Charge i th operation a fictitious amortized cost ĉ i , Charge an amortized cost of ĉ i = $3 for the i th where $1 pays for 1 unit of work ( i.e. , time). insertion. • This fee is consumed to perform the operation, and • $1 pays for the immediate insertion. • any amount not immediately consumed is stored in • $2 is stored for later table doubling. the bank for use by subsequent operations. When the table doubles, $1 pays to move a • The bank balance must not go negative! We must recent item, and $1 pays to move an old item. ensure that n n Example: ∑ ∑ ≤ c c ˆ i i $0 $0 $0 $0 $0 $0 $0 $2 $2 $2 $2 $2 $2 = = overflow i 1 i 1 $0 for all n . • Thus, the total amortized costs provide an upper bound on the total true costs. 3/13/08 CS 5633 Analysis of Algorithms 23 3/13/08 CS 5633 Analysis of Algorithms 24 6

Accounting analysis of Accounting analysis of dynamic tables dynamic tables Charge an amortized cost of ĉ i = $3 for the i th Charge an amortized cost of ĉ i = $3 for the i th insertion. insertion. • $1 pays for the immediate insertion. • $1 pays for the immediate insertion. • $2 is stored for later table doubling. • $2 is stored for later table doubling. When the table doubles, $1 pays to move a When the table doubles, $1 pays to move a recent item, and $1 pays to move an old item. recent item, and $1 pays to move an old item. Example: Example: overflow $0 $0 $0 $0 $0 $0 $0 $0 $0 $0 $0 $0 $0 $0 $0 $0 $0 $0 $0 $0 $0 $0 $0 $0 $0 $0 $0 $0 $0 $0 $2 $2 $2 $0 $0 3/13/08 CS 5633 Analysis of Algorithms 25 3/13/08 CS 5633 Analysis of Algorithms 26 Accounting analysis Incrementing a Binary Counter (continued) Given: A k -bit binary counter A [0,1,…, k -1], initialized with Key invariant: Bank balance never drops below 0. 0,0,…,0. The counter supports the following I NREMENT Thus, the sum of the amortized costs provides an operation: upper bound on the sum of the true costs. I NCREMENT ( A ) // increases counter by 1 i ← 0 i 1 2 3 4 5 6 7 8 9 10 while i <length( A ) and A [ i ]=1 do A [ i ] ← 0 size i 1 2 4 4 8 8 8 8 16 16 i ++ if i <length( A ) then c i 1 2 3 1 5 1 1 1 9 1 A [ i ] ← 1 * ĉ i 2 3 3 3 3 3 3 3 3 3 • Question: In a sequence of n I NCREMENT operations, what is bank i 1 2 2 4 2 4 6 8 2 4 the amortized runtime of one I NCREMENT operation? *Okay, so I lied. The first operation costs only $2, not $3. 3/13/08 CS 5633 Analysis of Algorithms 27 3/13/08 CS 5633 Analysis of Algorithms 28 7

Binary Counter Example Accounting Method 1 → 0 0 → 1 • Charge $2 to set a bit to 1 (0 → 1 flip) Example for k =8 and n =9: flip flip Initial counter 0 0 0 0 0 0 0 0 � $1 pays for the actual flip $1 After 1 increment 0 0 0 0 0 0 0 1 $1 $1 � Store $1 on the bit as credit to be used later when this bit is After 2 increments 0 0 0 0 0 0 1 0 $1 After 3 increments 0 0 0 0 0 0 1 1 flipped back to 0 $2 $1 After 4 increments 0 0 0 0 0 1 0 0 $1 • Charge $0 to set a bit to 0 (1 → 0 flip) After 5 increments 0 0 0 0 0 1 0 1 $1 $1 After 6 increments 0 0 0 0 0 1 1 0 � Every 1 in the counter has $1 credit on it, which is used to $1 After 7 increments 0 0 0 0 0 1 1 1 pay for this flip $3 $1 After 8 increments 0 0 0 0 1 0 0 0 $1 After 9 increments 0 0 0 0 1 0 0 1 • The worst-case runtime of one I NCREMENT operation is O( k ) • For n operations the total is O( nk ) 3/13/08 CS 5633 Analysis of Algorithms 29 3/13/08 CS 5633 Analysis of Algorithms 30 Binary Counter Example Accounting Method 1 → 0 0 → 1 • Charge $2 to set a bit to 1 (0 → 1 flip) Example for k =8 and n =9: flip flip Initial counter 0 0 0 0 0 0 0 0 � $1 pays for the actual flip $1 After 1 increment 0 0 0 0 0 0 0 1 $1 $1 � Store $1 on the bit as credit to be used later when this bit is After 2 increments 0 0 0 0 0 0 1 0 $1 After 3 increments 0 0 0 0 0 0 1 1 flipped back to 0 $2 $1 After 4 increments 0 0 0 0 0 1 0 0 $1 • Charge $0 to set a bit to 0 (1 → 0 flip) After 5 increments 0 0 0 0 0 1 0 1 $1 $1 After 6 increments 0 0 0 0 0 1 1 0 � Every 1 in the counter has $1 credit on it, which is used to $1 After 7 increments 0 0 0 0 0 1 1 1 pay for this flip $3 $1 After 8 increments 0 0 0 0 1 0 0 0 $1 After 9 increments 0 0 0 0 1 0 0 1 ⇒ Since each I NCREMENT operation is composed of one 0 → 1 flip and possibly multiple 1 → 0 flips, the asymptotic runtime of one I NCREMENT operation is O(1). 3/13/08 CS 5633 Analysis of Algorithms 31 3/13/08 CS 5633 Analysis of Algorithms 32 8