Cost of Sorts Cost of Sorts g is in O(f) iff there exist positive - PDF document

One-Slide Summary Cost of Sorts Cost of Sorts • g is in O(f) iff there exist positive constants c and n 0 such that g( n ) ≤ c f( n ) for all n ≥ n 0 . and and • If g is in O(f) we say that f is an upper bound Asymptotic Asymptotic for g. Growth Growth • We use Omega Ω for lower bounds and Theta Θ for tight bounds. • To prove that g is in O(f) you must find the (and addition!) (and addition!) constants c and n 0 . • We can add two numbers with electricity. #2 Outline Administrivia • Don't forget to turn in your fractal • Sorting: timing and costs – Separate form and button on server • Big Oh: upper bound • Late policy for PS3 Code • Big Omega: lower bound – about 10% per full day • Big Theta: tight bound • Reading – In Chapter 6 of the Course Book (“Machines”), • Time Permitting: there is a running example about computing and • Adding Two Numbers With Electricity implementing logic with a particular substance that you can pour. What was it? One word answer. – Write name, uvaID and word on paper. #3 #4 Recall “simple sorting”: Find best Timing Sort Sorting Cost card, take it out and set it aside, repeat. > (time (sort < (revintsto 100))) • What grows? cpu time: 20 real time: 20 gc time: 0 – n = the number of elements in lst > (time (sort < (revintsto 200))) cpu time: 80 real time: 80 gc time: 0 • How much work are the pieces? > (time (sort < (revintsto 400))) cpu time: 311 real time: 311 gc time: 0 find-best: work scales as n (increases by one) > (time (sort < (revintsto 800))) delete: work scales as n (increases by one) cpu time: 1362 real time: 1362 gc time: 0 > (time (sort < (revintsto 1600))) • How many times does sort evaluate cpu time: 6650 real time: 6650 gc time: 0 find-best and delete? n Cherry Blossom • Total cost: scales as n 2 by Ji Hyun Lee, Wei Wang #5 #6

Timing Sort Sorting Cost (define ( sort lst cf) 35000 (if (null? lst) lst measured (let ((best (find-best lst cf))) 30000 times (cons best (sort (delete lst best) cf))))) 25000 (define ( find-best lst cf) = n 2 /500 (if (= 1 (length lst)) (car lst) 20000 (pick-better cf (car lst) (find-best (cdr lst) cf)))) 15000 10000 If we double the length of the list, the amount of 5000 work approximately quadruples: there are twice as many applications of find-best, and each one takes 0 0 1000 2000 3000 twice as long! #7 #8 Growth Notations O Examples g is in O ( f ) iff there are positive constants • g ∈ O ( f ) (“Big-Oh”) c and n 0 such that g ( n ) ≤ cf ( n ) for all n ≥ n 0 . g grows no faster than f ( f is upper bound) • g ∈ Θ ( f ) (“Theta”) g grows as fast as f ( f is tight bound) Is n in O ( n 2 ) ? Yes, c = 1 and n 0 =1 works. • g ∈ Ω ( f ) (“Omega”) Is 10 n in O ( n ) ? Yes, c = 10 and n 0 =1 works. g grows no slower than f ( f is lower bound) No, no matter what c we pick, Is n 2 in O ( n ) ? cn 2 > n for big enough n ( n > c ) Which one would we most like to know? #9 #10 Ω (“Omega”): Lower Bound Proof Techniques g is in O ( f ) iff there are positive constants • Theorem: c and n 0 such that g ( n ) ≤ cf ( n ) for all n ≥ n 0 . There exists a polygon with four sides. g is in Ω ( f ) iff there are positive constants c and n 0 such that • Proof: g ( n ) ≥ cf ( n ) It is a polygon. It has four sides. for all n ≥ n 0 . QED. What kind of proof is this? #11 #12

Proof by Construction Dis-Proof by Construction • We can prove a “there exists an X with • To prove g is not in O ( f ) : property Y ” theorem, but showing an X that • O ( f ) means: there are positive constants c has property Y and n 0 such that g ( n ) ≤ cf ( n ) for all n ≥ n 0 • O ( f ) means “ there are positive constants c • So, to prove g is not in O ( f ) we need to and n 0 such that g ( n ) ≤ cf ( n ) for all n ≥ n 0 find a way given any c and n 0 , to find an n • So, to prove g is in O ( f ) we need to find c and ≥ n 0 such that g ( n ) > cf ( n ) . n 0 and show that g ( n ) ≤ cf ( n ) for all n ≥ n 0 Curious why :9 c. 9 n 0 .P(c,n 0 ) = 8 c. 8 n 0 . : P(c,n 0 ) ? Discrete math! #13 #14 Growth Notations Liberal Arts Trivia: Archaeology • g ∈ O ( f ) (“Big-Oh”) • In archaeology, this term is used to describe the exposure, processing and recording of g grows no faster than f (upper bound) archaeological remains. A related subconcept • g ∈ Θ ( f ) (“Theta”) is stratification: relationships exist between different events in the same location or g grows as fast as f (tight bound) context. Sedimentary layers are deposited in a • g ∈ Ω ( f ) (“Omega”) time sequence, with the oldest on the bottom and the youngest on top. g grows no slower than f (lower bound) #15 #16 Liberal Arts Trivia: The Sets O ( f ) and Ω( f ) Creative Writing • This technique is one of the four rhetorical Ω( f ) Functions modes of discourse, along with argumentation, O ( f ) Functions that grow description and narration. Its purpose is to that grow no slower f no faster than f inform, explain, analyze or define. When done than f Increasingly fast growing functions poorly, it is sometimes referred to as a “information dump” or “plot dump”. #17 #18

g is in O ( f ) iff there are positive g is in O ( f ) iff there are positive O and Ω O and Ω constants c and n 0 such that g ( n ) constants c and n 0 such that g ( n ) Examples Examples ≤ cf ( n ) for all n ≥ n 0 . ≤ cf ( n ) for all n ≥ n 0 . g is in Ω ( f ) iff there are positive g is in Ω ( f ) iff there are positive constants c and n 0 such that g ( n ) constants c and n 0 such that g ( n ) ≥ cf ( n ) for all n ≥ n 0 . ≥ cf ( n ) for all n ≥ n 0 . • n is in Ω ( n ) • n is in Ω ( n ) • n is in O ( n ) • n is in O ( n ) – ? – Yes, pick c = 1 – ? – Yes, pick c = 1 • 10 n is in O ( n ) • 10 n is in O ( n ) • 10 n is in Ω ( n ) • 10 n is in Ω ( n ) – ? – Yes, pick c = 10 – ? – Yes, pick c = 1 • n 2 is not in O ( n ) • n 2 is not in O ( n ) • Is n 2 in Ω ( n ) ? • Is n 2 in Ω ( n ) ? – ? – Pick n > c – ? – Yes! (pick c = 1 ) #19 #20 Θ (“Theta”): Tight Bound Example • Is n in Ω ( n 2 ) ? n ≥ cn 2 for all n ≥ n 0 g is Θ( f ) iff 1 ≥ cn for all n ≥ n 0 g is in O ( f ) No matter what c is, I can make this false by using n = (1/ c + 1) and g is in Ω ( f ) g is in Ω ( f ) iff there are positive constants c and n 0 such that g ( n ) ≥ cf ( n ) for all n ≥ n 0 . #21 Θ Examples The Sets O ( f ) , Ω( f ) , and Θ( f ) •Is 10 n in Θ( n ) ? Ω( f ) Functions O ( f ) •Is n 2 in Θ( n ) ? Functions that grow that grow no slower f no faster than f •Is n in Θ( n 2 ) ? than f Increasingly fast growing functions Θ( f ) How big are O ( f ) , Ω( f ) , and Θ( f ) ?

Θ Examples Recall: Sorting Cost • Is 10 n in Θ ( n ) ? • What grows? – Yes , since 10 n is Ω ( n ) and 10 n is in O ( n ) – n = the number of elements in lst •Doesn’t matter that you choose different c • How much work are the pieces? values for each part; they are independent • Is n 2 in Θ ( n ) ? find-best: work scales as n (increases by one) – No , since n 2 is not in O ( n ) delete: work scales as n (increases by one) • Is n in Θ ( n 2 ) ? • How many times does sort evaluate – No , since n 2 is not in Ω ( n ) find-best and delete? n • Total cost: scales as n 2 Sorting Cost Timing Sort (define (sort lst cf) 35000 (if (null? lst) lst measured (let ((best (find-best lst cf))) 30000 times (cons best (sort (delete lst best) cf))))) 25000 (define (find-best lst cf) = n 2 /500 (if (= 1 (length lst)) (car lst) 20000 (pick-better cf (car lst) (find-best (cdr lst) cf)))) 15000 Θ( n 2 ) If we double the length of the list, the amount of work 10000 approximately quadruples: there are twice as many applications 5000 of find-best, and each one takes twice as long. 0 0 1000 2000 3000 The running time of this sort procedure is in Θ( n 2 ) Is our sort good enough? Which of these is true? Takes over 1 second to sort 1000-length list. How long • Our sort procedure is too would it take to sort 1 million slow for VISA because its items? running time is in O ( n 2 ) 1s = time to sort 1000 4s ~ time to sort 2000 • Our sort procedure is too 1M is 1000 * 1000 slow for VISA because its running time is in Ω ( n 2 ) Sorting time is n 2 so, sorting 1000 times as many items will take • Our sort procedure is too 1000 2 times as long = 1 million seconds ~ 11 days Eyes by John Devor slow for VISA because its and Eric Montgomery Note: there are 800 Million VISA cards in circulation. running time is in Θ ( n 2 ) It would take 20,000 years to process a VISA transaction at this rate.