Integer multiplication with generalized Fermat primes Svyatoslav - - PowerPoint PPT Presentation

Integer multiplication with generalized Fermat primes

Svyatoslav Covanov

CARAMEL Team, LORIA, University of Lorraine Supervised by: Emmanuel Thomé and Jérémie Detrey

Journées nationales du Calcul Formel 2015 (Cluny) November 4, 2015

Summary

1

Fast Fourier Transform Naive multiplication Multiplying integer using polynomials FFT Schönhage-Strassen Some remarks

2

Fürer Factorization of FFT A new ring and a new cut

3

Using generalized Fermat primes Number-theoretic transform A Fürer-like number theoretic transform Comparison of complexities

Svyatoslav Covanov Integer multiplication with generalized Fermat primes November 4, 2015 2 / 23

Fast Fourier Transform

1

Fast Fourier Transform Naive multiplication Multiplying integer using polynomials FFT Schönhage-Strassen Some remarks

2

Fürer Factorization of FFT A new ring and a new cut

3

Using generalized Fermat primes Number-theoretic transform A Fürer-like number theoretic transform Comparison of complexities

Svyatoslav Covanov Integer multiplication with generalized Fermat primes November 4, 2015 3 / 23

Fast Fourier Transform Naive multiplication

How to multiply two numbers a = (a0 · · · aN) and b = (b0 · · · bN) where a and b are given in binary representation?

Svyatoslav Covanov Integer multiplication with generalized Fermat primes November 4, 2015 4 / 23

Fast Fourier Transform Naive multiplication

How to multiply two numbers a = (a0 · · · aN) and b = (b0 · · · bN) where a and b are given in binary representation? First idea: Sum all ai ∗ b using 2-shift. This method has a O(N2) bit complexity.

Svyatoslav Covanov Integer multiplication with generalized Fermat primes November 4, 2015 4 / 23

Fast Fourier Transform Naive multiplication

How to multiply two numbers a = (a0 · · · aN) and b = (b0 · · · bN) where a and b are given in binary representation? First idea: Sum all ai ∗ b using 2-shift. This method has a O(N2) bit complexity. People believed long enough it was the best complexity we could reach (Kolmogorov). Karatsuba proved that it was wrong...

Svyatoslav Covanov Integer multiplication with generalized Fermat primes November 4, 2015 4 / 23

Fast Fourier Transform Multiplying integer using polynomials

The fastest known algorithm is based on the evaluation-interpolation paradigm. Input: 2 numbers a and b of N bits. We decompose the input into 2 polynomials A =

i aixi and

B =

i bixi (deg A = deg B = n, |ai| = |bi| = N/(2n) = k, and

ai = bi = 0 for i > n). A(2k) =a0 + 2k × a1 + · · · + a2n−1 × 2(2n−1)k= a B(2k) =b0 + 2k × b1 + · · · + b2n−1 × 2(2n−1)k= b

Svyatoslav Covanov Integer multiplication with generalized Fermat primes November 4, 2015 5 / 23

Fast Fourier Transform Multiplying integer using polynomials

The fastest known algorithm is based on the evaluation-interpolation paradigm. Input: 2 numbers a and b of N bits. We decompose the input into 2 polynomials A =

i aixi and

B =

i bixi (deg A = deg B = n, |ai| = |bi| = N/(2n) = k, and

ai = bi = 0 for i > n). A(2k) =a0 + 2k × a1 + · · · + a2n−1 × 2(2n−1)k= a B(2k) =b0 + 2k × b1 + · · · + b2n−1 × 2(2n−1)k= b We work in some ring R in which we have a 2n-th principal root of unity ω.

Svyatoslav Covanov Integer multiplication with generalized Fermat primes November 4, 2015 5 / 23

Fast Fourier Transform Multiplying integer using polynomials

The fastest known algorithm is based on the evaluation-interpolation paradigm. Input: 2 numbers a and b of N bits. We decompose the input into 2 polynomials A =

i aixi and

B =

i bixi (deg A = deg B = n, |ai| = |bi| = N/(2n) = k, and

ai = bi = 0 for i > n). A(2k) =a0 + 2k × a1 + · · · + a2n−1 × 2(2n−1)k= a B(2k) =b0 + 2k × b1 + · · · + b2n−1 × 2(2n−1)k= b We work in some ring R in which we have a 2n-th principal root of unity ω. We compute the A(ωi) and B(ωi).

Svyatoslav Covanov Integer multiplication with generalized Fermat primes November 4, 2015 5 / 23

Fast Fourier Transform Multiplying integer using polynomials

The fastest known algorithm is based on the evaluation-interpolation paradigm. Input: 2 numbers a and b of N bits. We decompose the input into 2 polynomials A =

i aixi and

B =

i bixi (deg A = deg B = n, |ai| = |bi| = N/(2n) = k, and

ai = bi = 0 for i > n). A(2k) =a0 + 2k × a1 + · · · + a2n−1 × 2(2n−1)k= a B(2k) =b0 + 2k × b1 + · · · + b2n−1 × 2(2n−1)k= b We work in some ring R in which we have a 2n-th principal root of unity ω. We compute the A(ωi) and B(ωi). We recover A · B from the points A(ωi) · B(ωi) with Lagrange interpolation for a polynomial of degree 2n − 1.

Svyatoslav Covanov Integer multiplication with generalized Fermat primes November 4, 2015 5 / 23

Fast Fourier Transform Multiplying integer using polynomials

The fastest known algorithm is based on the evaluation-interpolation paradigm. Input: 2 numbers a and b of N bits. We decompose the input into 2 polynomials A =

i aixi and

B =

i bixi (deg A = deg B = n, |ai| = |bi| = N/(2n) = k, and

ai = bi = 0 for i > n). A(2k) =a0 + 2k × a1 + · · · + a2n−1 × 2(2n−1)k= a B(2k) =b0 + 2k × b1 + · · · + b2n−1 × 2(2n−1)k= b We work in some ring R in which we have a 2n-th principal root of unity ω. We compute the A(ωi) and B(ωi). We recover A · B from the points A(ωi) · B(ωi) with Lagrange interpolation for a polynomial of degree 2n − 1. The DFT algorithm allows one to compute the A(ωi) and B(ωi).

Svyatoslav Covanov Integer multiplication with generalized Fermat primes November 4, 2015 5 / 23

Fast Fourier Transform Multiplying integer using polynomials [a0, . . . , a2n−1] [b0, . . . , b2n−1]

DFT DFT

[x0, . . . , x2n−1] [y0, . . . , y2n−1]

Component Multiply

[x0y0, . . . , x2n−1y2n−1]

inverse DFT

Svyatoslav Covanov Integer multiplication with generalized Fermat primes November 4, 2015 6 / 23

Fast Fourier Transform FFT

P is a polynomial of degree 2n − 1 (n is a power of 2) and ω is a 2n-th principal root of unity in R (C or Z/pZ for example), which means that

j∈[0,2n−1] ωij = 0 for i ∈ [1, 2n − 1].

Svyatoslav Covanov Integer multiplication with generalized Fermat primes November 4, 2015 7 / 23

Fast Fourier Transform FFT

P is a polynomial of degree 2n − 1 (n is a power of 2) and ω is a 2n-th principal root of unity in R (C or Z/pZ for example), which means that

j∈[0,2n−1] ωij = 0 for i ∈ [1, 2n − 1].

FFT(P, ω, 2n − 1)

if n = 1 then return P0 + P1 + X(P0 − P1) end if Peven ← (P2i)i Podd ← (P2i+1)i Qeven ← FFT(Peven, ω2, n − 1) Qodd ← FFT(Podd, ω2, n − 1) Q ← Qeven(X) + Qodd(ωX) + X n · (Qodd(X) − Qeven(ωX)) return Q

Svyatoslav Covanov Integer multiplication with generalized Fermat primes November 4, 2015 7 / 23

Fast Fourier Transform FFT

P is a polynomial of degree 2n − 1 (n is a power of 2) and ω is a 2n-th principal root of unity in R (C or Z/pZ for example), which means that

j∈[0,2n−1] ωij = 0 for i ∈ [1, 2n − 1].

FFT(P, ω, 2n − 1)

if n = 1 then return P0 + P1 + X(P0 − P1) end if Peven ← (P2i)i Podd ← (P2i+1)i Qeven ← FFT(Peven, ω2, n − 1) Qodd ← FFT(Podd, ω2, n − 1) Q ← Qeven(X) + Qodd(ωX) + X n · (Qodd(X) − Qeven(ωX)) return Q

Complexity: O(n log n) operations in R, among which multiplications by some powers of ω, additions and subtractions.

Svyatoslav Covanov Integer multiplication with generalized Fermat primes November 4, 2015 7 / 23

Fast Fourier Transform FFT

P0 P1 P2 P3 P4 P5 P6 P7 P8 P9 P10 P11 P12 P13 P14 P15

⊗ ⊗ ⊗ ⊗ ⊗ ⊗ ⊗ ⊗ ⊗ ⊗ ⊗ ⊗ ⊗ ⊗ ⊗ ⊗ ⊗ ⊗ ⊗ ⊗ ⊗ ⊗ ⊗ ⊗ ⊗ ⊗ ⊗ ⊗ ⊗ ⊗ ⊗ ⊗ ⊗ ⊗ ⊗ ⊗ ⊗ ⊗ ⊗ ⊗ ⊗ ⊗ ⊗ ⊗ ⊗ ⊗ ⊗ ⊗

Qbitrev(0) Qbitrev(1) Qbitrev(2) Qbitrev(3) Qbitrev(4) Qbitrev(5) Qbitrev(6) Qbitrev(7) Qbitrev(8) Qbitrev(9) Qbitrev(10) Qbitrev(11) Qbitrev(12) Qbitrev(13) Qbitrev(14) Qbitrev(15)

ω0 ω0 ω0 ω0 ω4 ω4 ω4 ω4 ω0 ω0 ω2 ω2 ω4 ω4 ω6 ω6 ω0 ω1 ω2 ω3 ω4 ω5 ω6 ω7 Svyatoslav Covanov Integer multiplication with generalized Fermat primes November 4, 2015 8 / 23

Fast Fourier Transform Schönhage-Strassen 1 N: # bits of the integers that we multiply. 2 2n: degree of the polynomials A and B used to represent a and

b.

3 k: # bits used to encode the coefficients of A and B: a = A(2k)

and b = B(2k).

Svyatoslav Covanov Integer multiplication with generalized Fermat primes November 4, 2015 9 / 23

Fast Fourier Transform Schönhage-Strassen 1 N: # bits of the integers that we multiply. 2 2n: degree of the polynomials A and B used to represent a and

b.

3 k: # bits used to encode the coefficients of A and B: a = A(2k)

and b = B(2k). Examples: If R = C, then ω = exp(iπ/n). k = O(log N) is the best choice. Thus, the recursive calls are manipulating O(log N)-sized data during convolution step.

Svyatoslav Covanov Integer multiplication with generalized Fermat primes November 4, 2015 9 / 23

Fast Fourier Transform Schönhage-Strassen 1 N: # bits of the integers that we multiply. 2 2n: degree of the polynomials A and B used to represent a and

b.

3 k: # bits used to encode the coefficients of A and B: a = A(2k)

and b = B(2k). Examples: If R = C, then ω = exp(iπ/n). k = O(log N) is the best choice. Thus, the recursive calls are manipulating O(log N)-sized data during convolution step. If R = Z/(2e + 1)Z, then, k ≃ e ≃ O( √ N) (smallest k possible). Then ω = 2j with j = e/n. Multiplications by powers of ω are negacyclic permutations.

Svyatoslav Covanov Integer multiplication with generalized Fermat primes November 4, 2015 9 / 23

Fast Fourier Transform Schönhage-Strassen [a0, . . . , a2n−1] [b0, . . . , b2n−1]

DFT DFT

[x0, . . . , x2n−1] [y0, . . . , y2n−1]

Component Multiply

[x0y0, . . . , x2n−1y2n−1]

inverse DFT

Svyatoslav Covanov Integer multiplication with generalized Fermat primes November 4, 2015 10 / 23

Fast Fourier Transform Schönhage-Strassen

Modular Case

[a0, . . . , a2n−1] [b0, . . . , b2n−1]

DFT DFT

[x0, . . . , x2n−1] [y0, . . . , y2n−1]

Component Multiply

[x0y0, . . . , x2n−1y2n−1]

inverse DFT

Recursion Svyatoslav Covanov Integer multiplication with generalized Fermat primes November 4, 2015 11 / 23

Fast Fourier Transform Schönhage-Strassen

Complex Case

[a0, . . . , a2n−1] [b0, . . . , b2n−1]

DFT DFT

[x0, . . . , x2n−1] [y0, . . . , y2n−1]

Component Multiply

[x0y0, . . . , x2n−1y2n−1]

inverse DFT

Recursion > < Svyatoslav Covanov Integer multiplication with generalized Fermat primes November 4, 2015 12 / 23

Fast Fourier Transform Some remarks

C Z/(2e + 1)Z O(log N) O( √ N) Coefficients size? Coefficients size? DFT, Convolution Convolution Expensive multiplications? Expensive multiplications?

Svyatoslav Covanov Integer multiplication with generalized Fermat primes November 4, 2015 13 / 23

Fast Fourier Transform Some remarks

C Z/(2e + 1)Z O(log N) O( √ N) Coefficients size? Coefficients size? DFT, Convolution Convolution Expensive multiplications? Expensive multiplications? Case Degree

• Mult. by a root

Recursion Complexity C O(N/ log N) expensive O(log N) N log N log log N · · · 2O(log∗ N) Z/(2e + 1)Z O( √ N) cheap O( √ N) N log N log log N

In C, computing an FFT in {1, −1, i, −i} is quite easy. But less

• bvious for superior orders...

Svyatoslav Covanov Integer multiplication with generalized Fermat primes November 4, 2015 13 / 23

Fürer

1

Fast Fourier Transform Naive multiplication Multiplying integer using polynomials FFT Schönhage-Strassen Some remarks

2

Fürer Factorization of FFT A new ring and a new cut

3

Using generalized Fermat primes Number-theoretic transform A Fürer-like number theoretic transform Comparison of complexities

Svyatoslav Covanov Integer multiplication with generalized Fermat primes November 4, 2015 14 / 23

Fürer Factorization of FFT

Here you can see the butterfly graph for 16-point transform.

a0 a4 a8 a12 a1 a5 a9 a13 a2 a6 a10 a14 a3 a7 a11 a15

Below, the matrix repre- sentation:     a0 a1 a2 a3 a4 a5 a6 a7 a8 a9 a10 a11 a12 a13 a14 a15    

Svyatoslav Covanov Integer multiplication with generalized Fermat primes November 4, 2015 15 / 23

Fürer Factorization of FFT

We start by inner transforms.

a0 a4 a8 a12 a1 a5 a9 a13 a2 a6 a10 a14 a3 a7 a11 a15

Below, the matrix repre- sentation:     a0 a1 a2 a3 a4 a5 a6 a7 a8 a9 a10 a11 a12 a13 a14 a15    

There are 4 4-points FFT.

Svyatoslav Covanov Integer multiplication with generalized Fermat primes November 4, 2015 16 / 23

Fürer Factorization of FFT

Then the outer ones.

a0 a4 a8 a12 a1 a5 a9 a13 a2 a6 a10 a14 a3 a7 a11 a15

Below, the matrix repre- sentation:     a0 a1 a2 a3 a4 a5 a6 a7 a8 a9 a10 a11 a12 a13 a14 a15    

There are 4 4-points FFT.

Svyatoslav Covanov Integer multiplication with generalized Fermat primes November 4, 2015 17 / 23

Fürer A new ring and a new cut

We use a polynomial ring R of the form R = C[x]/(xP + 1) or R = Z[x]/(qc, xP + 1)Z There exists a 2n-th root of unity ρ such that ρn/P = x (Lagrange interpolation) The computation of 2n-points FFT is factored into the computation of log2P 2n times n/P 2P-points FFT P = Θ(log N) and coefficients of elements of R are stored on Θ(log N) bits

Svyatoslav Covanov Integer multiplication with generalized Fermat primes November 4, 2015 18 / 23

Fürer A new ring and a new cut

We use a polynomial ring R of the form R = C[x]/(xP + 1) or R = Z[x]/(qc, xP + 1)Z There exists a 2n-th root of unity ρ such that ρn/P = x (Lagrange interpolation) The computation of 2n-points FFT is factored into the computation of log2P 2n times n/P 2P-points FFT P = Θ(log N) and coefficients of elements of R are stored on Θ(log N) bits

Case Degree

• Mult. by a root

Recursion Complexity C O(N/ log N) expensive O(log N) N log N log log N · · · 2O(log∗ N) Z/(2e + 1)Z O( √ N) cheap O( √ N) N log N log log N C[x]/(xP + 1) O(N/ log2 N) it depends O(log2 N) N log N 2O(log∗ N) Z[x]/(qc, xP + 1)Z O(N/ log2 N) it depends O(log2 N) N log N 2O(log∗ N)

Svyatoslav Covanov Integer multiplication with generalized Fermat primes November 4, 2015 18 / 23

Using generalized Fermat primes

1

Fast Fourier Transform Naive multiplication Multiplying integer using polynomials FFT Schönhage-Strassen Some remarks

2

Fürer Factorization of FFT A new ring and a new cut

3

Using generalized Fermat primes Number-theoretic transform A Fürer-like number theoretic transform Comparison of complexities

Svyatoslav Covanov Integer multiplication with generalized Fermat primes November 4, 2015 19 / 23

Using generalized Fermat primes Number-theoretic transform

Let us multiply integers by associating to them 2 polynomials of degree 2n − 1 for which the coefficients are embedded in a finite field R = Z/qZ.

Svyatoslav Covanov Integer multiplication with generalized Fermat primes November 4, 2015 20 / 23

Using generalized Fermat primes Number-theoretic transform

Let us multiply integers by associating to them 2 polynomials of degree 2n − 1 for which the coefficients are embedded in a finite field R = Z/qZ. The prime q must verify: 2n | q − 1. Thus, there exists a 2n-th principal root of unity.

Svyatoslav Covanov Integer multiplication with generalized Fermat primes November 4, 2015 20 / 23

Using generalized Fermat primes Number-theoretic transform

Let us multiply integers by associating to them 2 polynomials of degree 2n − 1 for which the coefficients are embedded in a finite field R = Z/qZ. The prime q must verify: 2n | q − 1. Thus, there exists a 2n-th principal root of unity.

1 N: # bits of the integers that we multiply. 2 2n: degree of the polynomials A and B used to represent a and

b.

3 k: # bits used to encode the coefficients of A and B: a = A(2k)

and b = B(2k); this number is given by roughly 1

2 log2 q.

Svyatoslav Covanov Integer multiplication with generalized Fermat primes November 4, 2015 20 / 23

Using generalized Fermat primes Number-theoretic transform

Let us multiply integers by associating to them 2 polynomials of degree 2n − 1 for which the coefficients are embedded in a finite field R = Z/qZ. The prime q must verify: 2n | q − 1. Thus, there exists a 2n-th principal root of unity.

1 N: # bits of the integers that we multiply. 2 2n: degree of the polynomials A and B used to represent a and

b.

3 k: # bits used to encode the coefficients of A and B: a = A(2k)

and b = B(2k); this number is given by roughly 1

2 log2 q.

A choice of q such that k = O(log N) is optimal.

Svyatoslav Covanov Integer multiplication with generalized Fermat primes November 4, 2015 20 / 23

Using generalized Fermat primes A Fürer-like number theoretic transform

q is chosen such that q = bP + 1. There exists b such that b < P · (log P)1+ǫ for any ǫ > 0. Thus, log2 q ≈ P log P. Let ρ be a 2n-th root of unity in Z/qZ such that ρn/P = b.

Svyatoslav Covanov Integer multiplication with generalized Fermat primes November 4, 2015 21 / 23

Using generalized Fermat primes A Fürer-like number theoretic transform

q is chosen such that q = bP + 1. There exists b such that b < P · (log P)1+ǫ for any ǫ > 0. Thus, log2 q ≈ P log P. Let ρ be a 2n-th root of unity in Z/qZ such that ρn/P = b. Working in radix b is like working with "polynomials" of degree P whose coefficients are bounded by b.

Svyatoslav Covanov Integer multiplication with generalized Fermat primes November 4, 2015 21 / 23

Using generalized Fermat primes A Fürer-like number theoretic transform

q is chosen such that q = bP + 1. There exists b such that b < P · (log P)1+ǫ for any ǫ > 0. Thus, log2 q ≈ P log P. Let ρ be a 2n-th root of unity in Z/qZ such that ρn/P = b. Working in radix b is like working with "polynomials" of degree P whose coefficients are bounded by b.

Naive Way New way x x X(b) = x0 + x1 · b + x2 · b2 · · · xP−1 · bP−1 y y Y (b) = y0 + y1 · b + y2 · b2 · · · yP−1 · bP−1 x ∗ y z = x · y and x ∗ y = z mod q Z = X · Y mod (X P + 1) and x ∗ y = Z(b)

Svyatoslav Covanov Integer multiplication with generalized Fermat primes November 4, 2015 21 / 23

Using generalized Fermat primes Comparison of complexities

Using Fürer’s algorithm, we got:

Case Degree

• Mult. by a root

Recursion Complexity C O(N/ log N) expensive O(log N) N log N log log N · · · 2O(log∗ N) Z/(2e + 1)Z O( √ N) cheap O( √ N) N log N log log N C[x]/(xP + 1) O(N/ log2 N) it depends O(log2 N) N log N 2O(log∗ N) Z[x]/(qc, xP + 1)Z O(N/ log2 N) it depends O(log2 N) N log N 2O(log∗ N)

Svyatoslav Covanov Integer multiplication with generalized Fermat primes November 4, 2015 22 / 23

Using generalized Fermat primes Comparison of complexities

Using Fürer’s algorithm, we got:

Case Degree

• Mult. by a root

Recursion Complexity C O(N/ log N) expensive O(log N) N log N log log N · · · 2O(log∗ N) Z/(2e + 1)Z O( √ N) cheap O( √ N) N log N log log N C[x]/(xP + 1) O(N/ log2 N) it depends O(log2 N) N log N 2O(log∗ N) Z[x]/(qc, xP + 1)Z O(N/ log2 N) it depends O(log2 N) N log N 2O(log∗ N)

Using the last trick, we get the following data:

Case Degree

• Mult. by a root

Recursion Complexity C O(N/ log N) expensive O(log N) N log N log log N · · · 2O(log∗ N) Z/(2e + 1)Z O( √ N) cheap O( √ N) N log N log log N C[x]/(xP + 1) O(N/ log2 N) it depends O(log2 N) N log N 2O(log∗ N) Z[x]/(qc, xP + 1)Z O(N/ log2 N) it depends O(log2 N) N log N 2O(log∗ N) Z/(bP + 1)Z O(N/(log N log log N)) it depends O(log N log log N) N log N 2O(log∗ N)

Svyatoslav Covanov Integer multiplication with generalized Fermat primes November 4, 2015 22 / 23

Conclusion

Removing the polynomial layer improves the complexity of the algorithm.

Svyatoslav Covanov Integer multiplication with generalized Fermat primes November 4, 2015 23 / 23

Conclusion

Removing the polynomial layer improves the complexity of the algorithm. With a careful complexity analysis, we get the following complexity estimate: N log N · 4log∗ N (the same as the one obtained by Harvey, Lecerf and Van der Hoeven with Mersenne primes).

Svyatoslav Covanov Integer multiplication with generalized Fermat primes November 4, 2015 23 / 23

Conclusion

Removing the polynomial layer improves the complexity of the algorithm. With a careful complexity analysis, we get the following complexity estimate: N log N · 4log∗ N (the same as the one obtained by Harvey, Lecerf and Van der Hoeven with Mersenne primes). An efficient implementation has to be done.

Svyatoslav Covanov Integer multiplication with generalized Fermat primes November 4, 2015 23 / 23

Conclusion

Removing the polynomial layer improves the complexity of the algorithm. With a careful complexity analysis, we get the following complexity estimate: N log N · 4log∗ N (the same as the one obtained by Harvey, Lecerf and Van der Hoeven with Mersenne primes). An efficient implementation has to be done. Some limitations: efficient multiplication of two polynomials modulo X P + 1 (bilinear rank), strategy for choosing a good prime, an algorithm for the decompositions...

Svyatoslav Covanov Integer multiplication with generalized Fermat primes November 4, 2015 23 / 23