CS294-1 On-line Computation & Net w ork Algorithms Spring - PDF document

CS294-1 On-line Computation & Net w ork Algorithms Spring 1997 Lecture 2: Jan uary 23 L e ctur er: Y air Bartal Scrib e: Keith V anderve en This lecture in tro duces amortized analysis tec hniques and demonstrates their use in analyzing the cost of t w o simple data structures problems. 2.1 Amorti ze d Anlysis Amortized analysis [T arjan85 , CLR90] is orginated in the study of dynamic data structures. Eac h op eration is regarded as starting with some initial amoun t of \money ." If the op eration costs less than this amoun t of money , the excess is stored as credit to w ard pa ying the costs of op erations whic h cost more than the initial amoun t of money . In Data Structures: Supp ose y ou carry out a sequence of op erations o ; o ; :::; o . Then 1 2 n w orst-case cost = max c ( o ) : i i X total cost = c ( o ) : i i total cost amortized cost = � c ~ n F or example, in dynamic hashing the w orst-case searc h cost is �(log n ) whereas the amortized cost is O (1). 2.2 Amorti zatri on T ec hniques Through Examples 2.2.1 The Binary Coun ter Initially , all the bits are set to 0. T o incremen t the coun ter b y 1, one go es righ t to left �ipping bits un til a bit is �ipp ed from 0 to 1. The cost of the i th up date, c , is the n um b er of bits whic h are i �ipp ed. w orst-case cost = k : 2-1

2-2 Lecture 2: Jan uary 23 Figure 2.1: k-bit binary coun ter total cost = c ( n ) n X = c = O ( nk ) : i i c ( n ) amortized cost c ~ = : n 0 0 0 0 0 � � � � � � � � � � � � Claim 2.1 ~ c � 2 . b b b b b k-1 k-2 k-3 1 0 Pro of: 3 tec hniques to pro v e this: � The Engineer's view � The Bank er's view � The Ph ysicist's view 2.2.1.1 The Engineer's view Idea : brute force calculation! Observ e: b �ips ev ery up date 0 b �ips ev ery 2 up dates 1 . . . i b �ips ev ery 2 up dates i b log n c X n c ( n ) = b c < 2 n: i 2 i =0 ~ c < 2 :

Lecture 2: Jan uary 23 2-3 2.2.1.2 The Bank er's View: (The Credit-Debit Metho d) Idea : Eac h op eration comes in with c ~ $. During execution, if the cost of an op eration is less than ~ c , store the excess ( c ~ � cost) as credit. If the cost of an op eration is greater than ~ c , withdra w previously stored credit to complete pa ymen t of (cost � ( ~ c ). Theorem 2.2 If system has non-ne gative cr e dit at al l times, then ~ c is a b ound on the amotize d c ost. Bac k to the Binary Coun ter problem, w e set c ~ = 2 and asso ciate credits with bit p ositions. Case 1: W e �ip a bit in to 1, pa y 1 unit and store 1 unit at that bit's p osition. Case 2: W e �ip a bit in to 0, withdra w 1 unit from credit at that bit's p osition. Figure 2.2: after �rst up date: 1 credit on bit 0 1 0 0 0 0 1 Figure 2.3: after 2nd up date: 1 credit on bit 1 (paid to �ip bit 0 with credit for that bit) � � � � � � � � � � � � It can b e sho wn b y an induction argumen t that ev ery time a bit i is �ipp ed to 1, it will ha v e a credit of 1 dep osited at its b p osition. b This is b b ecause all of the bits to its righ t m ust ha v e b een b b k-1 k-2 k-3 1 0 1 after the previous up date, hence they had credit to pa y for their �ips, and w e got t w o units of credit for the up date, one to �ip the bit i and the other to store as credit at i 's p osition. 2.2.1.3 The Ph ysicist's View: (The P oten tial F unction Metho d) 1 Idea : Store pre-paid credit as a "p oten tial energy" of the en tire system. Use p oten tial to pa y for op erations whic h cost more energy 0 than 0 they 0 bring in. 1 0 � � � � � � � � � � � � b b b b b k-1 k-2 k-3 1 0

2-4 Lecture 2: Jan uary 23 F ormally , let � denote the p oten tial at time i. Clearly , i c ~ = c + ��, i where �� = � � � , th us assume ~ c = b , the total amortized cost is i i � 1 i n n X X b � n = ~ c = ( c + � � � ) i i i i � 1 i =0 i =1 n X = c + � � � i n 0 i =1 c ( n ) Theorem 2.3 If � � � (usual ly � = 0 ), then c ~ ( n ) = � b . n 0 0 n Imp ortan t: W an t � � � for all i � n . i 0 T o pro v e ~ c b ounded, w e can sho w that c � ~ c � ��. i i i The p oten tial metho d is applied to the Binary Coun ter problem as follo ws. Let � = n um b er of i bits set to 1 at time i . Let � = 0. Then 8 i; � � 0. Consider the i th up date. There exists a j suc h 0 i that bit j is �ipp ed in to a 1 and all bits less than j are �ipp ed to 0. Therefore, � � � = 1 � j: i i � 1 W e w an t to sho w c ~ = 2. c = j + 1 = 2 � (1 � j ) = ~ c � �� i i 2.2.2 A Queue F rom Tw o Stac ks W e ma y use t w o stac ks A and B and need to implemen t a queue. The Queue op erations include: insert item x ( I ( x )) and delete item x ( D ( x )). An insert places a new item at the tail of the queue, and a delete remo v es an item from the head of the queue. W e are giv en a sequence of insert and delete op erations. Consider the follo wing algorithm. Newly inserted items are pushed in to stac k A. When a delete o ccurs w e �rst c hec k if there are items in stac k B. If so w e p op the item at the top of stac k B. Otherwise all items in stac k A are p opp ed o� the stac k and all except the last one are pushed in to stac k B. It is easy to c hec k the correctness of this algorithm. W e pro v e the follo wing:

Lecture 2: Jan uary 23 2-5 Claim 2.4 The 2-Stacks algorithm has amortize d c ost at most 3. Pro of: W e use the Credit-Debit metho d. Let eac h insert op eration come with 3 units and eac h delete op eration come with 1 unit. Then inserting an item in to stac k A tak es 1 unit, and p opping the item and inserting it in to stac k 2 tak es 2 units. Deleting the item from stac k 2 then tak es 1 unit. Then the cost (whic h is the total n um b er of pushes and p ops) ob eys total cost � 3 � # of Inserts + # of Deletes � 3 n; where n is the o v erall n um b er of op erations. References [CLR90] T.H. Cormen, C.E. Leiserson, and R.L. Riv est. Intr o duction to A lgorithms. The MIT Press, 1990. [T arjan85] R.E. T arjan. Amortized Computational Complexit y . SIAM Journal on A lgebr aic and Discr ete Metho ds , 6(2):306-318, 1985.