Approximate Homomorphic Encryption and Privacy Preserving Machine - PowerPoint PPT Presentation

Approximate Homomorphic Encryption and Privacy Preserving Machine Learning Jung Hee Cheon (SNU, CryptoLab) Thanks to YongSoo Song, Kiwoo Lee, Andrey KIM

Outline 1. Homomorphic Encryption 2. HEAAN 3. Bootstrapping of HEAAN 4. Toolkit for Homomorphic Computation

Homomorphic Encryption  Integer-based HE scheme - RAD PH [1] • (Secret Key, Operation Key) = (a large prime 𝑞 , 𝑜 = 𝑞𝑟 0 ) • Encryption: 𝐹𝑜𝑑 𝑛 = 𝑛 + 𝑞𝑟 𝑛𝑝𝑒 𝑜 • Decryption: 𝐹𝑜𝑑 𝑛 𝑛𝑝𝑒 𝑞 = 𝑛 But, INSECURE! • 𝐹𝑜𝑑 𝑛 1 + 𝐹𝑜𝑑 𝑛 2 = 𝑛 1 + 𝑞𝑟 1 + 𝑛 2 + 𝑞𝑟 2 = 𝑛 1 + 𝑛 2 + 𝑞 𝑟 1 + 𝑟 2 = 𝐹𝑜𝑑 𝑛 1 + 𝑛 2 - DGHV HE scheme (on ℤ 2 ) [2] 2 100 𝑓 1 𝑛 1 X • 𝐹𝑜𝑑(𝑛 ∈ 𝑎 2 100 ) = 𝑛 + 2100𝑓 + 𝑞𝑟 2 100 𝑓 2 𝑛 2 • SECURE against quantum computing • Use a polynomial ring 𝑆 𝑟 = ℤ 𝑟 [𝑦]/(𝑦𝑜 + 1) 2 200 𝑓 1 𝑓 2 + 2 100 𝑛 1 𝑓 2 + 𝑛 2 𝑓 1 + 𝑛 1 𝑛 2

Fully Homomorphic Encryption - On Polynomials (RingLWE) • [Gen09] ideal lattice • NTRU: LTV12 • Ring-LWE: BV11b, GHS13, BLLN13, HEAAN etc - On Integers (AGCD) • [DGHV10] FHE over the Integers. Eurocrypt 2010 • CMNT11, CNT12, CCKLLTY13, CLT14, etc - On Matrices (LWE) • [BV11a] Efficient FHE from (Standard) LWE. FOCS11 • Bra12, BGV12, GSW13

Summary of Progress in HE 1. 2009~2012: Plausibility and Scalable for Large Circuits • [GH11] A single bit bootstrapping takes 30 minutes • [GHS12b] 120 blocks of AES-128 (30K gates) in 36 hours 2. 2012~2015: Depth-Linear Construction • [BGV12] Modulus/Key Switching • [Bra12] Scale Invariant Scheme • [HS14] IBM's open-source library Helib: AES evaluation in 4 minutes 3. 2015~Today: Usability • Various schemes with different advantages (HEAAN, TFHE) • Real-world tasks: Big data analysis, Machine learning • Competitions for Private Genome Computation (iDash, 2014~) • HE Standardization meetings (2017~) Continued on next page

Standardization: HomomorphicEncryption.org Mar 2018 in MIT Jul 2017 in Microsoft, Redmond Oct 2018 in Toronto 1 st Workshop (2017.7.13-14) 2 nd Workshop (2018.3.15-16) 3 rd Workshop (2018.10.20)

Homomorphic Encryption  Best Performing HE Schemes Type Classical HE Fast Bootstrapping Approximate Computation [BGV12] BGV [DM15] FHEW Scheme [CKKS17] HEAAN [Bra12, FV12] B/FV [CGGI16] TFHE Finite Field Real/Complex numbers Plaintext Binary string Packing Packing Operation Addition, Multiplication Look-up table & bootstrapping Fixed-point Arithmetic HElib (IBM) TFHE Library SEAL (Microsoft Research) HEAAN (SNU) (Inpher, Gemalto, etc.) Palisade (Duality inc.)

2. HEAAN: Approximate Homomorphic Encryption

Exact Multiplication 1.23 5.6088 4.56 3.98112624 0.78 0.7098 0.91 108.2456382397886496 2.34 13.2678 5.67 16 27.18970254 8.91 2.0493 0.23 8 4 2 - The plaintext size is doubled after a multiplication.

Approximate Multiplication 1.23 5.6188 4.56 3.98112624 0.78 0.7198 0.91 108.2556382397886496 2.34 13.2778 2 5.67 27.19970254 8.91 2.0593 0.23 2 HEAAN [CKKS17] 2 2 • Rescale after a multiplication • Tracing # of significands • Most data is processed approximately in Data analysis or ML

HEAAN = 慧眼 = Insightful Minds [CKKS, AC17] Homomorphic Encryption for Arithmetic of Approximate Numbers https://eprint.iacr.org/2016/421.pdf

Approximate Computation  Numerical Representation - Encode 𝑛 into an integer 𝑛 ≈ 𝑞𝑦 for a scaling factor 𝑞 : 2 ↦ 1412 ≈ 2 ⋅ 10 3 - Fixed – Point Multiplication Compute 𝑛 = 𝑛 1 𝑛 2 and extract its significant digits 𝑛 ′ ≈ 𝑞 −1 ⋅ 𝑛 • 5678 ⋅ 10 −3 = 7006652 ⋅ 10 −6 ↦ 7007 ⋅ 10 −3 = 7.007 : 1.234 × 5.678 = 1234 ⋅ 10 −3 ×  Previous HE on LWE problem (Regev, 2005) ct = Enc sk 𝑛 , ct, sk = 𝑟 • 𝑢 𝑛 + 𝑓 mod 𝑟 𝑓 • Modulo 𝑢 plaintext vs Rounding operation 𝑛 𝑢 𝑟/𝑢

HEAAN 𝜈 1 = 𝑞𝑛 1 + 𝑓 1  A New Message Encoding • ct = Enc sk 𝑛 , ct, sk = 𝑞𝑛 + 𝑓 mod 𝑟 𝑟 • Consider 𝑓 as part of approximation error 𝜈 2 = 𝑞𝑛 2 + 𝑓 2  Homomorphic Operations 𝜈 = 𝑞 2 𝑛 1 𝑛 2 + 𝑓 Input 𝜈 1 ≈ 𝑞𝑛 1 , 𝜈 2 ≈ 𝑞𝑛 2 Addition 𝜈 1 + 𝜈 2 ≈ 𝑞 ⋅ (𝑛 1 + 𝑛 2 ) 𝜈 = 𝜈 1 𝜈 2 ≈ 𝑞 2 ⋅ 𝑛 1 𝑛 2 Multiplication 𝜈 ′ = 𝑞 ⋅ 𝑛 1 𝑛 2 + 𝑓′ 𝜈 ′ ≈ 𝑞 −1 ⋅ 𝜈 ≈ 𝑞 ⋅ 𝑛 1 𝑛 2 Rounding 𝑟/𝑞  Support for the (approximate) fixed-point arithmetic - Leveled HE : 𝑟 = 𝑞 𝑀

HEAAN Packed Ciphertext  Construction over the ring • A single ctx can encrypt a vector of plaintext values 𝑨 = (𝑨 1 , 𝑨 2 , … , 𝑨 ℓ ) • Parallel computation in a SIMD manner 𝑨 ⊗ 𝑥 = (𝑨 1 𝑥 1 , 𝑨 2 𝑥 2 , … , 𝑨 ℓ 𝑥 ℓ ) Continued on next page

RLWE-based HEAAN 𝑌 𝑜 + 1 and 𝑆 𝑟 = 𝑆 mod 𝑟 = ℤ 𝑟 𝑌 𝑜 + 1  Let 𝑆 = ℤ 𝑌 𝑌 • A ciphertext can encrypt a polynomial 𝑛 𝑌 ∈ 𝑆 • Note (m 0 +m 1 X+ … ) ( m 0 ’ +m 1 ’ X + … )= m 0 m 0 ’ +(m 0 m 1 ’ +m 0 ’m 1 ) X+… • Decoding/Encoding function 𝑌 𝑜 + 1 ⊆ ℝ 𝑌 𝑜 + 1 → ℂ 𝑜/2 𝑆 = ℤ 𝑌 𝑌 𝑛 𝑌 ↦ 𝑨 = 𝑨 1 , … , 𝑨 𝑜∕2 , 𝑨 𝑗 = 𝜈 𝜂 𝑗 , w here 𝑌 𝑜 + 1 = 𝑌 − 𝜂 1 −1 ⋯ 𝑌 − 𝜂 −1 −1 𝑌 − 𝜂 1 𝑌 − 𝜂 2 𝑌 − 𝜂 2 𝑌 − 𝜂 𝑜 2 𝑜 2 • Example: 𝑜 = 4 , 𝜂 1 = 𝑓𝑦𝑞(𝜌𝑗/4) , 𝜂 2 = 𝑓𝑦𝑞(5𝜌𝑗/4) 𝑨 = 1 − 2𝑗, 3 + 4𝑗 ↦ 𝑛 𝑌 = 2 − 2 2 𝑌 + 𝑌 2 − 2 𝑌 3 • ↦ 𝜈 𝑌 = 2000 − 2828𝑌 + 1000𝑌 2 − 1414 𝑌 3 • • 𝜈 𝜂 1 ≈ 1000.15 − 1999.55 𝑗 , 𝜈 𝜂 2 ≈ 2999.85 + 3999.55 𝑗

3. Bootstrapping of HEAAN

Bootstrapping Input 𝐹𝑜𝑑(𝑑) • Old Ciphertext with large noise Safe Box • c = 𝐹𝑜𝑑 𝑙 (𝑛; 𝑠) Encrypted Secret Key Process 𝐹𝑜𝑑 𝑙 (𝐿) Decryption 𝐿 • Evaluate Decrypt circuit 𝑛 Output • New ciphertext with small noise 𝐹𝑜𝑑 𝑙 𝑛; 𝑠 ′ , 𝑠 ′ : small [1] Cheon-Han-Kim-Kim-Song: Bootstrapping for Approximate Homomorphic Encryption. EUROCRYPT 2018

Bootstrapping • Ciphertexts of a leveled HE have a limited lifespan • Refresh a ciphertext ct = Enc sk 𝑛 by evaluating the decryption circuit homomorphically : Dec sk ct = 𝑛 ⟺ 𝐺 ct sk = 𝑛 where 𝐺 ct ∗ = Dec ∗ ct • Bootstrapping key BK = Enc sk (sk) : 𝐺 ct BK = 𝐺 ct Enc sk sk = Enc sk 𝐺 ct sk = Enc sk (𝑛) • Homomorphic operations introduce errors  Fine : 𝐺 ct BK = 𝐺 ct Enc sk sk = Enc sk 𝐺 ct sk + 𝑓 = Enc sk (𝑛 + 𝑓) • How to evaluate the decryption circuit (efficiently)? : Dec sk ct = ct, sk (mod 𝑟)

Approximate Decryption 𝑢 𝑟 = 𝜈 , 𝐸𝑓𝑑 𝑡𝑙 𝑑𝑢 ↦ 𝑢 = 𝑑𝑢, 𝑡𝑙 ↦ 𝑢 = 𝑟𝐽 + 𝜈 for some 𝐽 < 𝐿 • Na ï ve solution: polynomial interpolation on [−𝐿𝑟, 𝐿𝑟] • Huge depth, complexity & inaccurate result

Approximate Decryption 𝑢 𝑟 = 𝜈 , 𝐸𝑓𝑑 𝑡𝑙 𝑑𝑢 ↦ 𝑢 = 𝑑𝑢, 𝑡𝑙 ↦ 𝑢 = 𝑟𝐽 + 𝜈 for some 𝐽 < 𝐿 • Idea1 : Restriction of domain 𝜈 ≪ 𝑟

Approximate Decryption 𝑢 𝑟 = 𝜈 , 𝐸𝑓𝑑 𝑡𝑙 𝑑𝑢 ↦ 𝑢 = 𝑑𝑢, 𝑡𝑙 ↦ 𝑢 = 𝑟𝐽 + 𝜈 for some 𝐽 < 𝐿 • Idea1 : Restriction of domain 𝜈 ≪ 𝑟 Idea 2 : Sine approximation 𝜈 ≈ 𝑟 2𝜌 • 2𝜌 sin 𝜄 for 𝜄 = 𝑟 𝑢 (period: 𝑟 , slope at 0=1) 1 𝑟

Bootstrapping of HEAAN  Sine Evaluation

Bootstrapping of HEAAN  Sine Evaluation - Direct Taylor approximation • Huge depth & complexity

Bootstrapping of HEAAN 0  Sine Evaluation - Direct Taylor approximation • Huge depth & complexity - Idea 1 : Low-degree approx. near 0 −1 𝑙 𝜄 2 𝑠 2𝑙 ≈ cos 𝑒 𝜄 2 𝑠 • 𝐷 0 𝜄 = 𝑙=0 2𝑙 ! −1 𝑙 𝜄 2 𝑠 2𝑙+1 ≈ sin( 𝑒 𝜄 2 𝑠 ) • 𝑇 0 𝜄 = 𝑙=0 2𝑙+1 !

Bootstrapping of HEAAN 1  Sine Evaluation - Direct Taylor approximation • Huge depth & complexity - Idea 1 : Low-degree approx. near 0 −1 𝑙 𝜄 2 𝑠 2𝑙 ≈ cos 𝑒 𝜄 2 𝑠 • 𝐷 0 𝜄 = 𝑙=0 2𝑙 ! −1 𝑙 𝜄 2 𝑠 2𝑙+1 ≈ sin( 𝑒 𝜄 2 𝑠 ) • 𝑇 0 𝜄 = 𝑙=0 2𝑙+1 ! - Idea 2: Iterate by double-angle formula 2 𝜄 − 𝑇 𝑙 2 𝜄 , 𝑇 𝑙+1 𝜄 = 2𝑇 𝑙 𝜄 ⋅ 𝐷 𝑙 𝜄 • 𝐷 𝑙+1 𝜄 = 𝐷 𝑙

Bootstrapping of HEAAN 2  Sine Evaluation - Direct Taylor approximation • Huge depth & complexity - Idea 1 : Low-degree approx. near 0 −1 𝑙 𝜄 2 𝑠 2𝑙 ≈ cos 𝑒 𝜄 2 𝑠 • 𝐷 0 𝜄 = 𝑙=0 2𝑙 ! −1 𝑙 𝜄 2 𝑠 2𝑙+1 ≈ sin( 𝑒 𝜄 2 𝑠 ) • 𝑇 0 𝜄 = 𝑙=0 2𝑙+1 ! - Idea 2: Iterate by double-angle formula 2 𝜄 − 𝑇 𝑙 2 𝜄 , 𝑇 𝑙+1 𝜄 = 2𝑇 𝑙 𝜄 ⋅ 𝐷 𝑙 𝜄 • 𝐷 𝑙+1 𝜄 = 𝐷 𝑙