Arithmetic Algorithms, Part 1 DPV Chapter 1 Jim Royer EECS - - PowerPoint PPT Presentation

Arithmetic Algorithms, Part 1

DPV Chapter 1

Jim Royer

EECS

January 18, 2019

Royer Arithmetic Algorithms, Part 1 1/ 15

Multiplication ` a la Franc ¸ais

function multiply(a, b) // input: two n-bit integers a and b with b ≥ 0 // output: a · b if b = 0 then return 0 c ← multiply(a, ⌊b/2⌋) if b is even then return (2 · c) else return (a + 2 · c)

Correctness

A proof by induction on b. Base Case: b = 0. Then multiply(a, b) = 0, which is correct.

Induction Step: b > 0. (IH = Induction Hypothesis) IH: multiply(a, b′) = a · b′ for b′ = 0, . . . , b − 1. By the IH, c = a · ⌊b/2⌋ Case: b is even. Then: (2 · c) = 2 · (a · (b/2)) = a · (2 · (b/2)). = a · b. Case: b is odd. Then: (a + 2 · c) = a + 2 · (a · ⌊b/2⌋) = a · (2⌊b/2⌋ + 1) = a · b.

Royer Arithmetic Algorithms, Part 1 2/ 15

Multiplication ` a la Franc ¸ais, Continued

function multiply(a, b) // input: two n-bit integers a and b with b ≥ 0 // output: a · b if b = 0 then return 0 c ← multiply(a, ⌊b/2⌋) if b is even then return (2 · c) else return (a + 2 · c)

Run-time analysis n recursive calls (b drops by 1-bit in each call). O(n) cost of each step on the recursion. (Why?) n · O(n) = O(n2).

Royer Arithmetic Algorithms, Part 1 3/ 15

Division

function divide(a,b) // input: two n-bit integers a and b with a ≥ 0 and b > 0 // output: (q, r) where a = q · b + r and 0 ≤ r < b if a = 0 then return (0, 0) (q′, r′) ← divide(⌊a/2⌋, b) q ← 2 · q′ r ← 2 · r′ if a is odd then r ← r + 1 if r ≥ b then r ← r − b; q ← q + 1 return (q, r)

Correctness Case a = 0: . . . On the board. Case a even and > 0: . . . On the board. Case a odd: . . . Exercise for the reader. Run-time analysis: Homework problem.

Royer Arithmetic Algorithms, Part 1 4/ 15

Division

function divide(a,b) // input: two n-bit integers a and b with a ≥ 0 and b > 0 // output: (q, r) where a = q · b + r and 0 ≤ r < b if a = 0 then return (0, 0) (q′, r′) ← divide(⌊a/2⌋, b) q ← 2 · q′ r ← 2 · r′ if a is odd then r ← r + 1 if r ≥ b then r ← r − b; q ← q + 1 return (q, r) Correctness Case a = 0: . . . On the board. Case a even and > 0: . . . On the board. Case a odd: . . . Exercise for the reader. Run-time analysis: Homework problem.

2019-01-18

Arithmetic Algorithms, Part 1 Division Case a = 0. Then q = r = 0 and a = 0 = 0 · b + 0 = q · b + r and 0 = r ≤ b. Case a > 0 and a is even. Then q = 2q′ and r = 2r′ where (q′, r′) = divide(⌊a/2⌋, b). IH: For a∗ ∈ { 0, . . . , a − 1 }, (q∗, r∗) = divide(a∗, b) is such that a∗ = q∗ · b + r∗ and 0 ≤ r∗ < b. Since ⌊a/2⌋ < a, the IH applies with a∗ = ⌊a/2⌋. Hence, ⌊a/2⌋ = q′ · b + r′ and 0 ≤ r′ < b. Since 2⌊a/2⌋ = a, a = 2⌊a/2⌋ = 2q′ · b + 2r′ and 0 ≤ 2r′ < 2b SUBCASE: 2r′ < b: Then q = 2q′ and r = 2r′ and we are done. SUBCASE: 2r′ ≥ b: Then q = 2q′ + 1 and r = 2r′ − b and we are done.

Modular Arithmetic

Definition

Suppose a, b, N ∈ N.

(i)

a|b ⇐ ⇒ def a divides b, i.e., b = k · a for some k ∈ N.

(ii)

a ≡ b (mod N) ⇐ ⇒ def N | (a − b) ⇐ ⇒ a − b = k · N for some integer k.

The substitution rule

Suppose a ≡ a′ (mod N) and b ≡ b′ (mod N). Then a + b ≡ a′ + b′ (mod N) and a · b ≡ a′ · b′ (mod N).

Modular addition, subtraction, and multiplication

Suppose N is n bits long and 0 ≤ a, b < N. Then computing (a + b) mod N and (a − b) mod N can be done in Θ(n) time. (a · b) mod N can be done in Θ(n2) time.

Royer Arithmetic Algorithms, Part 1 5/ 15

Modular Exponentiation

Exponentiation via repeated squaring

ab =      1, if b = 0; (a⌊b/2⌋)2, if b > 0 and even; a · (a⌊b/2⌋)2, if b is odd.

function modExp(a, b, N) // input: a, b, and N :: three n-bit integers // with 0 ≤ a, b and 1 < N // output: ab mod N if b = 0 then return 1 c ← modExp(a, ⌊b/2⌋, N) if b is even then return c2 mod N else return (a · c2) mod N

Example: x1000 via 15 multiplies

x1000 = (x500)2 x500 = (x250)2 x250 = x · (x125)2 x125 = x · (x62)2 x62 = (x31)2 x31 = x · (x15)2 x15 = x · (x7)2 x7 = x · (x3)2 x3 = x · (x)2

Royer Arithmetic Algorithms, Part 1 6/ 15

Modular Exponentiation, Continued

function modExp(a, b, N) // input: a, b, and N :: three n-bit integers with 0 ≤ a, b and 1 < N // output: ab mod N if b = 0 then return 1 c ← modExp(a, ⌊b/2⌋, N) if b is even then return c2 mod N else return (a · c2) mod N

Correctness:

Easy.

Runtime:

Let n = the number of bits in max(a, b, N). At most n-many recursive calls. Why? In each call, two or three n-bit numbers are multiplied at cost Θ(n2). Why? ∴ n × Θ(n2) = Θ(n3).

Royer Arithmetic Algorithms, Part 1 7/ 15

Euclid’s algorithm for greatest common divisor

Definition

The greatest common divisor of a and b ∈ N is the largest d ∈ N such that d divides both a and b. I.E.: gcd(a, b) = max { d d|a & d|b }.

Example

1035 = 32 · 5 · 23 & 759 = 3 · 11 · 23.

∴ gcd(1035, 759) = 3 · 23 = 69.

For a > 0, gcd(0, a) = a. gcd(0, 0) = 0 by convention.

Euclid’s Rule

Suppose a, b ∈ N+. Then gcd(a, b) = gcd(b, a mod b). Proof on next page

Royer Arithmetic Algorithms, Part 1 8/ 15

Euclid’s Rule: Suppose a, b ∈ N+. Then gcd(a, b) = gcd(b, a mod b).

Proof.

Recall: gcd(u, v) =def max({ d d|u & d|v }). Claim 1. If d|a & d|b, then (∀x, y ∈ Z)

• d|(x · a + y · b)
• .

[Proof on Board] Observe: (a) a = ⌊a b⌋ · b + 1 · (a mod b) (b) a mod b = 1 · a + (−⌊a b⌋) · b By (a) & Claim 1, gcd(b, a mod b)|a. Since gcd(b, a mod b)|b, we have: gcd(b, a mod b) ≤ gcd(a, b). (Why?) By (b) & Claim 1, gcd(a, b)|(a mod b). Since gcd(a, b)|b, we have: gcd(a, b) ≤ gcd(b, a mod b). (Why?)

∴

gcd(a, b) = gcd(b, a mod b).

Royer Arithmetic Algorithms, Part 1 9/ 15

Euclid’s algorithm, continued

Euclid’s Rule

Suppose a, b ∈ N+. Then gcd(a, b) = gcd(b, a mod b).

function Euclid(a, b) // Input: integers a and b with a ≥ b ≥ 0. // Output: the g.c.d. of a and b. if b = 0 then return a else return Euclid(b, a mod b).

• Correctness. Easy.

Royer Arithmetic Algorithms, Part 1 10/ 15

Euclid’s algorithm, Runtime analysis

function Euclid(a, b) // Input: integers a and b with a ≥ b ≥ 0. Output: the g.c.d. of a and b. if b = 0 then return a else return Euclid(b, a mod b).

Lemma

Suppose a ≥ b > 0. Then (a mod b) < a/2.

Proof.

Case: b ≤ a/2. Then: (a mod b) < b ≤ a/2. Case: b > a/2. Then: (a mod b) = (a − b) ≤ (a − a/2) = a/2.

Since Euclid(a, b) = Euclid(b, a mod b) = Euclid(a mod b, b mod (a mod b)) (generally), every two steps the a and b values are at least halved.

∴ On n-bit numbers, Euclid stops after 2n recursions.

On n-bit numbers, mod (i.e., a division) costs O(n2)

∴ 2n × O(n2) = O(n3).

Royer Arithmetic Algorithms, Part 1 11/ 15

The extended Euclid algorithm

Lemma

Suppose d|a & d|b & d = xa + yb for some x, y ∈ Z. Then d = gcd(a, b).

Proof.

Royer Arithmetic Algorithms, Part 1 12/ 15

The extended Euclid algorithm

Lemma

Suppose d|a & d|b & d = xa + yb for some x, y ∈ Z. Then d = gcd(a, b).

Proof.

Since d|a and d|b, then d ≤ gcd(a, b).

Royer Arithmetic Algorithms, Part 1 12/ 15

The extended Euclid algorithm

Lemma

Suppose d|a & d|b & d = xa + yb for some x, y ∈ Z. Then d = gcd(a, b).

Proof.

Since d|a and d|b, then d ≤ gcd(a, b). Since gcd(a, b)|a & gcd(a, b)|b,

Royer Arithmetic Algorithms, Part 1 12/ 15

The extended Euclid algorithm

Lemma

Suppose d|a & d|b & d = xa + yb for some x, y ∈ Z. Then d = gcd(a, b).

Proof.

Since d|a and d|b, then d ≤ gcd(a, b). Since gcd(a, b)|a & gcd(a, b)|b, then gcd(a, b)|(xa + yb),

Royer Arithmetic Algorithms, Part 1 12/ 15

The extended Euclid algorithm

Lemma

Suppose d|a & d|b & d = xa + yb for some x, y ∈ Z. Then d = gcd(a, b).

Proof.

Since d|a and d|b, then d ≤ gcd(a, b). Since gcd(a, b)|a & gcd(a, b)|b, then gcd(a, b)|(xa + yb), i.e., gcd(a, b)|d.

Royer Arithmetic Algorithms, Part 1 12/ 15

The extended Euclid algorithm

Lemma

Suppose d|a & d|b & d = xa + yb for some x, y ∈ Z. Then d = gcd(a, b).

Proof.

Since d|a and d|b, then d ≤ gcd(a, b). Since gcd(a, b)|a & gcd(a, b)|b, then gcd(a, b)|(xa + yb), i.e., gcd(a, b)|d. Therefore, gcd(a, b) ≤ d.

Royer Arithmetic Algorithms, Part 1 12/ 15

The extended Euclid algorithm

Lemma

Suppose d|a & d|b & d = xa + yb for some x, y ∈ Z. Then d = gcd(a, b).

Proof.

Since d|a and d|b, then d ≤ gcd(a, b). Since gcd(a, b)|a & gcd(a, b)|b, then gcd(a, b)|(xa + yb), i.e., gcd(a, b)|d. Therefore, gcd(a, b) ≤ d. Therefore, d = gcd(a, b).

Royer Arithmetic Algorithms, Part 1 12/ 15

The extended Euclid algorithm

Lemma

Suppose d|a & d|b & d = xa + yb for some x, y ∈ Z. Then d = gcd(a, b).

Proof.

Since d|a and d|b, then d ≤ gcd(a, b). Since gcd(a, b)|a & gcd(a, b)|b, then gcd(a, b)|(xa + yb), i.e., gcd(a, b)|d. Therefore, gcd(a, b) ≤ d. Therefore, d = gcd(a, b).

function extended-Euclid(a, b) // Input: integers a and b with a ≥ b ≥ 0. // Output: (x, y, d) where d = gcd(a, b) and d = xa + yb. if b = 0 then return (1, 0, a). (x′, y′, d) = extended-Euclid(b, a mod b) return (y′, x′ − ⌊a/b⌋y′, d)

Royer Arithmetic Algorithms, Part 1 12/ 15

The extended Euclid algorithm: Base case

function extended-Euclid(a, b) // Input: integers a and b with a ≥ b ≥ 0. // Output: (x, y, d) where d = gcd(a, b) and d = xa + yb. if b = 0 then return (1, 0, a). (x′, y′, d) = extended-Euclid(b, a mod b) return (y′, x′ − ⌊a/b⌋y′, d)

Proof of correctness, base case.

Base case: b = 0. gcd(a, b) = a & a = 1 · a + 0 · b. So (1, 0, a) is right.

Royer Arithmetic Algorithms, Part 1 13/ 15

The extended Euclid algorithm: Induction Step

function extended-Euclid(a, b) // Input: integers a and b with a ≥ b ≥ 0. // Output: (x, y, d) where d = gcd(a, b) and d = xa + yb. if b = 0 then return (1, 0, a). (x′, y′, d) = extended-Euclid(b, a mod b) return (y′, x′ − ⌊a/b⌋y′, d)

Proof of correctness, induction step.

Suppose b > 0. IH: extended-Euclid(a′, b′) is correct for all a′ and each b′ = 0, . . . , b − 1.

Royer Arithmetic Algorithms, Part 1 14/ 15

The extended Euclid algorithm: Induction Step

function extended-Euclid(a, b) // Input: integers a and b with a ≥ b ≥ 0. // Output: (x, y, d) where d = gcd(a, b) and d = xa + yb. if b = 0 then return (1, 0, a). (x′, y′, d) = extended-Euclid(b, a mod b) return (y′, x′ − ⌊a/b⌋y′, d)

Proof of correctness, induction step.

Suppose b > 0. IH: extended-Euclid(a′, b′) is correct for all a′ and each b′ = 0, . . . , b − 1. Let (x′, y′, d) = extended-Euclid(b, a mod b). Note: a mod b < b.

Royer Arithmetic Algorithms, Part 1 14/ 15

The extended Euclid algorithm: Induction Step

function extended-Euclid(a, b) // Input: integers a and b with a ≥ b ≥ 0. // Output: (x, y, d) where d = gcd(a, b) and d = xa + yb. if b = 0 then return (1, 0, a). (x′, y′, d) = extended-Euclid(b, a mod b) return (y′, x′ − ⌊a/b⌋y′, d)

Proof of correctness, induction step.

Suppose b > 0. IH: extended-Euclid(a′, b′) is correct for all a′ and each b′ = 0, . . . , b − 1. Let (x′, y′, d) = extended-Euclid(b, a mod b). Note: a mod b < b. So by the IH, gcd(b, a mod b) = d = x′ · b + y′ · (a mod b).

Royer Arithmetic Algorithms, Part 1 14/ 15

The extended Euclid algorithm: Induction Step

function extended-Euclid(a, b) // Input: integers a and b with a ≥ b ≥ 0. // Output: (x, y, d) where d = gcd(a, b) and d = xa + yb. if b = 0 then return (1, 0, a). (x′, y′, d) = extended-Euclid(b, a mod b) return (y′, x′ − ⌊a/b⌋y′, d)

Proof of correctness, induction step.

Suppose b > 0. IH: extended-Euclid(a′, b′) is correct for all a′ and each b′ = 0, . . . , b − 1. Let (x′, y′, d) = extended-Euclid(b, a mod b). Note: a mod b < b. So by the IH, gcd(b, a mod b) = d = x′ · b + y′ · (a mod b). So d = gcd(a, b). (Why?)

Royer Arithmetic Algorithms, Part 1 14/ 15

The extended Euclid algorithm: Induction Step

function extended-Euclid(a, b) // Input: integers a and b with a ≥ b ≥ 0. // Output: (x, y, d) where d = gcd(a, b) and d = xa + yb. if b = 0 then return (1, 0, a). (x′, y′, d) = extended-Euclid(b, a mod b) return (y′, x′ − ⌊a/b⌋y′, d)

Proof of correctness, induction step.

Suppose b > 0. IH: extended-Euclid(a′, b′) is correct for all a′ and each b′ = 0, . . . , b − 1. Let (x′, y′, d) = extended-Euclid(b, a mod b). Note: a mod b < b. So by the IH, gcd(b, a mod b) = d = x′ · b + y′ · (a mod b). So d = gcd(a, b). (Why?) . . . and d = x′ · b + y′ · (a mod b) = x′ · b + y′ · (a − ⌊a b⌋b) = y′ · a + (x′ − ⌊a b⌋y′) · b.

Royer Arithmetic Algorithms, Part 1 14/ 15

Modular division

Definition

x is the multiplicative inverse of a mod N when a · x ≡ 1 (mod N).

The inverse might not exist!

E.g., 2−1 mod 6 does not exist.

Theorem (Modular Division Theorem)

Suppose N > 2 and a ∈ { 1, . . . , N − 1 }.

(a)

a has an inverse mod N ⇐ ⇒ gcd(a, N) = 1.

(b)

When a−1 mod N exists, (a−1 mod N) = (x mod N), where (x, y, 1) = extended-Euclid(a, N) so that 1 = a · x + N · y.

Royer Arithmetic Algorithms, Part 1 15/ 15