Video 1: Rounding errors -0 A number system can be represented as ! - PowerPoint PPT Presentation

Video 1: Rounding errors

-0 A number system can be represented as ! = ±1. & ! & " & # & $ ×2 % for ! ∈ [−6,6] and ( ! ∈ {0,1} . = Let’s say you want to represent the decimal number 19.625 using the binary number system above. Can you represent this number exactly? " ( 19.62540--40011.10112=4.00111014 × 2 1. 0011 × 24 19 = 1. 0100 × 24=20

Machine floating point number • Not all real numbers can be exactly represented as a machine floating-point bhtlbhtz / # number. - - - - I • Consider a real number in the normalized floating-point form: - = ±1. ( " ( # ( $ … ( % …× 2 & The real number - will be approximated by either - ' or - ( , the nearest two • mnauchminwer machine floating point numbers. DO g- ! & ! ! ' +∞ 0 X - = t.bibzbz-bnx2m-fasth.it Xt=IntO.O0OOj0 ② x2m ④ 2m=Emx2m

= q x 2M ! & ! ! ' +∞ 0 ! X - t Emt 2M Xt = = Em x 2M ) - E / Ht <gap=Em → larger gap larger #

× - flex ) =/ flex ) - Xl Rounding The process of replacing ! by a nearby machine number is called rounding, and the error involved is called roundoff error. Round Round Round Round towards towards towards towards zero + ∞ zero − ∞ De Ies ! ' ! ' −∞ ! ! & ! & ! +∞ 0 Round by chopping : s + is positive number + is negative number 00 ,- + = + ! ,- + = + " Round up (ceil) Rounding towards +∞ Rounding towards zero ↳ - OO ,- + = + " ,- + = + ! Round down (floor) Rounding towards −∞ Rounding towards zero Round to nearest : either round up or round down, whichever is closer

Rounding (roundoff) errors thx ) fl q Consider rounding by chopping: ! & ! ' - xkEm × Ig ¥ µ ¥FE¥ ! • Absolute error: Heat I • Relative error: EmX2Mw He¥gE:÷¥ → mA=¥:÷n tf HE

Rounding (roundoff) errors 1075 )*(!) − ! ≤ 0 % |!| The relative error due to rounding (the process of representing a real number as a machine number) is always bounded by machine epsilon. -

IEEE Single Precision IEEE Double Precision # I I :÷÷÷÷ O = )*(!) − ! )*(!) − ! ≤ 2 &"# ≈ 1.2×10 &/ ≤ 2 &0" ≈ 2.2×10 &!1 |!| |!| ' ' " ÷÷÷ : 7-

Gap between two machine numbers Erskine grneadmberrmamohimnwerfllx-XI.mx - I = fxx ) flcxto ) - few ) a- - gap 8sgap

Rule ofthlehnbs Gap between two machine numbers Decimal : x - qxlom Binary x=q × 2m ftp.?ngE%3fI#gap...osio' " × = 4.5 × 10 8 X=2 double -23 m " " lo 8510 ftlxto ) - flex ) → fllxto ) # G) - " → fffxto )=flG ) 's gelo @ L2 " → ftlxto ) # flex ) ) → felxto ) # flex ' 5 or > D- or > 2-

Gap between two machine numbers ¥¥*÷¥÷' 8 = gap 8 k → 1 m = ( II = I fe ( x ) = often HE fflxto ) = I = fl G) If 8 such that what is the smallest ⇒ 8L gap b flat 8) - felt -

practice :( Rule of Thumb ) In Show Ipython notebook demos ← Binary base x=qx2m Decimal base :÷÷. ⇒ * ⇒ §:÷÷÷ 'm + = of xD = fee ) fflxto ) Example x -4.5 × 104 n 8 SEMI :* . ¥ - ) - ' 5 → fffxt8)=feK 852 if otherwise feats ) # ffcx )

Video 2: Arithmetic with machine numbers

Mathematical properties of FP operations Not necessarily associative : For some ! , #, $ the result below is possible: ! + # + $ ≠ ! + (# + $) Not necessarily distributive : For some ! , #, $ the result below is possible: $ ! + # ≠ $ ! + $ # Not necessarily cumulative : Repeatedly adding a very small number to a large number may do nothing

Floating point arithmetic (basic idea) ! = (−1) ! 1. ( × 2 " • First compute the exact result • Then round the result to make it fit into the desired precision • ! + # = %& ! + # • ! × # = %& ! × #

Floating point arithmetic On Consider a number system such that ! = ±1. , ! , " , # ×2 % for ! ∈ [−4,4] and ( ! ∈ {0,1} . a Rough algorithm for addition and subtraction: , 1. Bring both numbers onto a common exponent 2. Do “grade-school” operation 3. Round result O too • Example 1: No rounding needed . - = 1.101 " ×2 ! , = 1.001 " ×2 ! O es / = - + , = 10.110 " ×2 ! = 1.011 " ×2 "

Floating point arithmetic Consider a number system such that ! = ±1. , ! , " , # ×2 % in for ! ∈ [−4,4] and ( ! ∈ {0,1} . • Example 2: Require rounding O { - = 1.101 " ×2 4 , = 1.000 " ×2 4 ' ✓ - 1.0101 × 2 / = - + , = 10.101 " ×2 4 tllatb ) -1.010 × 24 • Example 3: g { - = 1.100 " ×2 ! , = 1.100 " ×2 &! / = - + , = 1.100 " ×2 ! + 0.011 " ×2 ! = 1.111 " ×2 ! - -

Floating point arithmetic Consider a number system such that ! = ±1. , ! , " , # , $ ×2 % tF → p=5 for ! ∈ [−4,4] and ( ! ∈ {0,1} . • Example 4: I :i% :3 : } { - = 1.1011 " ×2 ! , = 1.1010 " ×2 ! - ' 0.0001 × 2 / = - − , = 0.0001 " ×2 ! - i. ¥42 not sign ' ' bits fffa - b) =L . 2

Floating point arithmetic Consider a number system such that ! = ±1. , ! , " , # , $ ×2 % - for ! ∈ [−4,4] and ( ! ∈ {0,1} . ✓ pill p=5 • Example 4: - = 1.1011 " ×2 ! , = 1.1010 " ×2 ! / = - − , = 0.0001 " ×2 ! / = 1. ? ? ? ? " ×2 &# Or after normalization: a • There is not data to indicate what the missing digits should be. • Machine fills them with its best guess, which is often not good (usually what is called spurious zeros). • Number of significant digits in the result is reduced. • This phenomenon is called Catastrophic Cancellation .

Loss of significance Assume - and , are real numbers with - ≫ , . For example - = 1. - ! - " - # - $ - 0 - 1 … - 5 …×2 4 , = 1. , ! , " , # , $ , 0 , 1 … , 5 …×2 &6 In Single Precision, compute - + , f- 23 frat 1. - ! - " - # - $ - 0 - 1 - / - 6 - 7 … - "" - "# ×2 4 0.00000001 bib . . - by b ,sx2° b= fllatb ) ⇒ b- bits of

Cancellation O Assume - and , are real numbers with - ≈ , . - = 1. - ! - " - # - $ - 0 - 1 … - 5 …×2 % , = 1. , ! , " , # , $ , 0 , 1 … , 5 …×2 % In single precision (without loss of generality), consider this example: = FE #% - = 1. - ! - " - # - $ - 0 - 1 … - "4 - "! 10- "$ - "0 - "1 - "/ … ×2 % , = 1. - ! - " - # - $ - 0 - 1 … - "4 - "! 11, "$ , "0 , "1 , "/ …×2 % , − - = 0.0000 … 0001×2 % 2-23 × 2 m tub - a) = I . not sig .

Examples: 1) 1) 2 and 3 are real numbers with same order of magnitude ( 4 ≈ 6) . They have the following representation in a decimal floating point system with 16 decimal digits of - accuracy: - g 78 4 = 3004.45 78 6 = 3004.46 O How many accurate digits does your answer have when you compute 6 − 4 ? 3¥ . lldigits - - 11 digits

Loss of Significance How can we avoid this loss of significance? For example, consider the ! " + 1 − 1 function 5 ! = If we want to evaluate the function for values ! near zero, there is a potential loss of significance in the subtraction. Assume you are performing this computation using a machine with 5 decimal accurate digits. Compute 5(10 &# ) 1.000000 f- (10-3)=110-7 - I to.QO.O.IO# 1.000001 - - I I =p

- E ' - b A - b) ( atb ) - ( Loss of Significance D ! " + 1 − 1 to avoid subtraction of two Re-write the function 5 ! = numbers with similar order of magnitude - C 15 ' e- GET fastnet - D N¥t HIT + I fH=n¥+ ⇒ = tho -3 ;o÷=o÷¢

round - down Example: If x = 0.3721448693 and y = 0.3720214371 what is the relative error in the computation of (x − y) in a computer with five decimal digits of accuracy? In 't :I¥HEEzoox " :÷¥:÷÷at¥ - = O . 0001234 322 X - y