BLAZE: BLAZING FAST PRIVACY-PRESERVING MACHINE LEARNING ARPITA PATRA - - PowerPoint PPT Presentation

▶
blaze blazing fast privacy preserving machine learning
SMART_READER_LITE
LIVE PREVIEW

BLAZE: BLAZING FAST PRIVACY-PRESERVING MACHINE LEARNING ARPITA PATRA - - PowerPoint PPT Presentation

Sep 02, 2022 1.85k likes •2.46k views

BLAZE: BLAZING FAST PRIVACY-PRESERVING MACHINE LEARNING ARPITA PATRA AND AJITH SURESH Ajith Suresh CrIS Lab, IISc https://www.csa.iisc.ac.in/~cris Outline q Secure Multi-party Computation (MPC) q MPC for small number of parties (3PC) q Our

slide-1
SLIDE 1

BLAZE: BLAZING FAST PRIVACY-PRESERVING MACHINE LEARNING

ARPITA PATRA AND AJITH SURESH

Ajith Suresh

CrIS Lab, IISc

https://www.csa.iisc.ac.in/~cris

slide-2
SLIDE 2

26-02-2020 AJITH SURESH | CRYPTOGRAPHY AND INFORMATION SECURITY LAB ,CSA, IISC

Outline

q Secure Multi-party Computation (MPC) q MPC for small number of parties (3PC) q Our Efficient BLAZE Protocol (Results) q Privacy Preserving Machine Learning (PPML)

slide-3
SLIDE 3

26-02-2020 AJITH SURESH | CRYPTOGRAPHY AND INFORMATION SECURITY LAB ,CSA, IISC

Secure Multi-party Computation (MPC) [Yao’82]

ü A set of parties with private inputs wish to compute some

joint function of their inputs.

ü Goals of MPC:

§

Correctness – Parties should correctly evaluate the function

  • utput.

§

Privacy – Nothing more than the function output should be revealed

slide-4
SLIDE 4

26-02-2020 AJITH SURESH | CRYPTOGRAPHY AND INFORMATION SECURITY LAB ,CSA, IISC

Secure Multi-party Computation (MPC) [Yao’82]

Trusted Third Party

ü A set of parties with private inputs wish to compute some

joint function of their inputs.

ü Goals of MPC:

§

Correctness – Parties should correctly evaluate the function

  • utput.

§

Privacy – Nothing more than the function output should be revealed

slide-5
SLIDE 5

26-02-2020 AJITH SURESH | CRYPTOGRAPHY AND INFORMATION SECURITY LAB ,CSA, IISC

Secure Multi-party Computation (MPC) [Yao’82]

Trusted Third Party

ü A set of parties with private inputs wish to compute some

joint function of their inputs.

ü Goals of MPC:

§

Correctness – Parties should correctly evaluate the function

  • utput.

§

Privacy – Nothing more than the function output should be revealed

slide-6
SLIDE 6

26-02-2020 AJITH SURESH | CRYPTOGRAPHY AND INFORMATION SECURITY LAB ,CSA, IISC

Secure Multi-party Computation (MPC) [Yao’82]

MPC

ü A set of parties with private inputs wish to compute some

joint function of their inputs.

ü Goals of MPC:

§

Correctness – Parties should correctly evaluate the function

  • utput.

§

Privacy – Nothing more than the function output should be revealed

slide-7
SLIDE 7

26-02-2020 AJITH SURESH | CRYPTOGRAPHY AND INFORMATION SECURITY LAB ,CSA, IISC

Secure Multi-party Computation (MPC) [Yao’82]

MPC

  • Semi – honest:
  • Follows the protocol but tries to learn more
  • Malicious:
  • Can arbitrarily deviate from the protocol

ADVERSARY

slide-8
SLIDE 8

26-02-2020 AJITH SURESH | CRYPTOGRAPHY AND INFORMATION SECURITY LAB ,CSA, IISC

Secure Multi-party Computation (MPC) [Yao’82]

MPC

  • Semi – honest:
  • Follows the protocol but tries to learn more
  • Malicious:
  • Can arbitrarily deviate from the protocol

ADVERSARY

Malicious Corruption

slide-9
SLIDE 9

26-02-2020 AJITH SURESH | CRYPTOGRAPHY AND INFORMATION SECURITY LAB ,CSA, IISC

MPC for small number of parties

Ø Efficiency and Simplicity [MRZ15,AFLNO16,FLNW17,CGMV17]

slide-10
SLIDE 10

26-02-2020 AJITH SURESH | CRYPTOGRAPHY AND INFORMATION SECURITY LAB ,CSA, IISC

MPC for small number of parties

Ø Efficiency and Simplicity [MRZ15,AFLNO16,FLNW17,CGMV17] Ø Our focus: MPC with 3 parties

slide-11
SLIDE 11

26-02-2020 AJITH SURESH | CRYPTOGRAPHY AND INFORMATION SECURITY LAB ,CSA, IISC

MPC for small number of parties

Ø Efficiency and Simplicity [MRZ15,AFLNO16,FLNW17,CGMV17] Ø Our focus: MPC with 3 parties Ø Corruption : honest majority

slide-12
SLIDE 12

26-02-2020 AJITH SURESH | CRYPTOGRAPHY AND INFORMATION SECURITY LAB ,CSA, IISC

MPC for small number of parties

Ø Efficiency and Simplicity [MRZ15,AFLNO16,FLNW17,CGMV17] Ø Our focus: MPC with 3 parties Ø Corruption : honest majority

q Majority of the parties are honest q 3PC – at most 1 corruption

slide-13
SLIDE 13

26-02-2020 AJITH SURESH | CRYPTOGRAPHY AND INFORMATION SECURITY LAB ,CSA, IISC

MPC for small number of parties

Ø Efficiency and Simplicity [MRZ15,AFLNO16,FLNW17,CGMV17] Ø Our focus: MPC with 3 parties Ø Corruption : honest majority Ø Outsourced Computation

slide-14
SLIDE 14

26-02-2020 AJITH SURESH | CRYPTOGRAPHY AND INFORMATION SECURITY LAB ,CSA, IISC

MPC for small number of parties

Ø Efficiency and Simplicity [MRZ15,AFLNO16,FLNW17,CGMV17] Ø Our focus: MPC with 3 parties Ø Corruption : honest majority Ø Outsourced Computation Ø Pre-processing Model

slide-15
SLIDE 15

26-02-2020 AJITH SURESH | CRYPTOGRAPHY AND INFORMATION SECURITY LAB ,CSA, IISC

MPC for small number of parties

Ø Efficiency and Simplicity [MRZ15,AFLNO16,FLNW17,CGMV17] Ø Our focus: MPC with 3 parties Ø Corruption : honest majority Ø Outsourced Computation Ø Pre-processing Model § Pre-processing phase

slide-16
SLIDE 16

26-02-2020 AJITH SURESH | CRYPTOGRAPHY AND INFORMATION SECURITY LAB ,CSA, IISC

MPC for small number of parties

Ø Efficiency and Simplicity [MRZ15,AFLNO16,FLNW17,CGMV17] Ø Our focus: MPC with 3 parties Ø Corruption : honest majority Ø Outsourced Computation Ø Pre-processing Model § Pre-processing phase

q Data-independent Computation q Relatively slow and expensive

slide-17
SLIDE 17

26-02-2020 AJITH SURESH | CRYPTOGRAPHY AND INFORMATION SECURITY LAB ,CSA, IISC

MPC for small number of parties

Ø Efficiency and Simplicity [MRZ15,AFLNO16,FLNW17,CGMV17] Ø Our focus: MPC with 3 parties Ø Corruption : honest majority Ø Outsourced Computation Ø Pre-processing Model § Pre-processing phase § Online Phase

slide-18
SLIDE 18

26-02-2020 AJITH SURESH | CRYPTOGRAPHY AND INFORMATION SECURITY LAB ,CSA, IISC

MPC for small number of parties

Ø Efficiency and Simplicity [MRZ15,AFLNO16,FLNW17,CGMV17] Ø Our focus: MPC with 3 parties Ø Corruption : honest majority Ø Outsourced Computation Ø Pre-processing Model § Pre-processing phase § Online Phase

q Minimized communication q Blazing fast

slide-19
SLIDE 19

BLAZE PROTOCOL

AJITH SURESH | CRYPTOGRAPHY AND INFORMATION SECURITY LAB ,CSA, IISC 26-02-2020

slide-20
SLIDE 20

26-02-2020 AJITH SURESH | CRYPTOGRAPHY AND INFORMATION SECURITY LAB ,CSA, IISC

BLAZE Protocol

S0 S1 S2

slide-21
SLIDE 21

26-02-2020 AJITH SURESH | CRYPTOGRAPHY AND INFORMATION SECURITY LAB ,CSA, IISC

BLAZE Protocol

S0 S1 S2

slide-22
SLIDE 22

26-02-2020 AJITH SURESH | CRYPTOGRAPHY AND INFORMATION SECURITY LAB ,CSA, IISC

BLAZE Protocol

S0 S1 S2

slide-23
SLIDE 23

26-02-2020 AJITH SURESH | CRYPTOGRAPHY AND INFORMATION SECURITY LAB ,CSA, IISC

BLAZE Protocol

S0 S1 S2

slide-24
SLIDE 24

26-02-2020 AJITH SURESH | CRYPTOGRAPHY AND INFORMATION SECURITY LAB ,CSA, IISC

BLAZE Protocol

S0 S1 S2

slide-25
SLIDE 25

26-02-2020 AJITH SURESH | CRYPTOGRAPHY AND INFORMATION SECURITY LAB ,CSA, IISC

BLAZE Protocol

S0 S1 S2

slide-26
SLIDE 26

26-02-2020 AJITH SURESH | CRYPTOGRAPHY AND INFORMATION SECURITY LAB ,CSA, IISC

BLAZE Protocol

Communication Cost per Multiplication Gate (malicious)

https://eprint.iacr.org/2020/042 BLAZE :

!"#$: &. (

slide-27
SLIDE 27

26-02-2020 AJITH SURESH | CRYPTOGRAPHY AND INFORMATION SECURITY LAB ,CSA, IISC

BLAZE Protocol

Ref Pre-processing

(#elements)

Online

(#elements)

Security

Araki et al’17 12 9 Abort

Communication Cost per Multiplication Gate (malicious)

https://eprint.iacr.org/2020/042 BLAZE :

!"#$: &. (

slide-28
SLIDE 28

26-02-2020 AJITH SURESH | CRYPTOGRAPHY AND INFORMATION SECURITY LAB ,CSA, IISC

BLAZE Protocol

Ref Pre-processing

(#elements)

Online

(#elements)

Security

Araki et al’17 ASTRA 12 21 9 4 Abort Fair

Communication Cost per Multiplication Gate (malicious)

https://eprint.iacr.org/2020/042 BLAZE :

!"#$: &. (

slide-29
SLIDE 29

26-02-2020 AJITH SURESH | CRYPTOGRAPHY AND INFORMATION SECURITY LAB ,CSA, IISC

BLAZE Protocol

Ref Pre-processing

(#elements)

Online

(#elements)

Security

Araki et al’17 ASTRA Boneh et al’19 12 21 9 4 3 Abort Fair Abort

Communication Cost per Multiplication Gate (malicious)

https://eprint.iacr.org/2020/042 BLAZE :

!"#$: &. (

slide-30
SLIDE 30

26-02-2020 AJITH SURESH | CRYPTOGRAPHY AND INFORMATION SECURITY LAB ,CSA, IISC

BLAZE Protocol

Ref Pre-processing

(#elements)

Online

(#elements)

Security

Araki et al’17 ASTRA Boneh et al’19

BLAZE

12 21

3

9 4 3

3

Abort Fair Abort

Fair Communication Cost per Multiplication Gate (malicious)

https://eprint.iacr.org/2020/042 BLAZE :

!"#$: &. (

slide-31
SLIDE 31

26-02-2020 AJITH SURESH | CRYPTOGRAPHY AND INFORMATION SECURITY LAB ,CSA, IISC

Privacy Preserving Machine Learning (PPML)

Alice (Model Owner) Model Parameters Bob (Client) Query Result ML Algorithm

Privacy ??

slide-32
SLIDE 32

26-02-2020 AJITH SURESH | CRYPTOGRAPHY AND INFORMATION SECURITY LAB ,CSA, IISC

Privacy Preserving Machine Learning (PPML)

Alice (Model Owner) Model Parameters Bob (Client) Query Result ML Algorithm Query

slide-33
SLIDE 33

26-02-2020 AJITH SURESH | CRYPTOGRAPHY AND INFORMATION SECURITY LAB ,CSA, IISC

Privacy Preserving Machine Learning (PPML)

Alice (Model Owner) Model Parameters Bob (Client) Query Result ML Algorithm Model Parameters

slide-34
SLIDE 34

26-02-2020 AJITH SURESH | CRYPTOGRAPHY AND INFORMATION SECURITY LAB ,CSA, IISC

Solution ??

ML MPC

slide-35
SLIDE 35

26-02-2020 AJITH SURESH | CRYPTOGRAPHY AND INFORMATION SECURITY LAB ,CSA, IISC

MPC MEETS ML

slide-36
SLIDE 36

26-02-2020 AJITH SURESH | CRYPTOGRAPHY AND INFORMATION SECURITY LAB ,CSA, IISC

Privacy Preserving Machine Learning (PPML)

Alice (Model Owner) Model Parameters Bob (Client) Query Result ML Algorithm

Use MPC to achieve privacy

MPC

slide-37
SLIDE 37

26-02-2020

AJITH SURESH | CRYPTOGRAPHY AND INFORMATION SECURITY LAB ,CSA, IISC

Alice (Model Owner) Bob (Client) MLaaS (3PC Servers) Model Parameters Query Result

SECURE OUTSOURCED SETTING (SOC)

slide-38
SLIDE 38

ML ALGORITHMS CONSIDERED

Linear Regression Logistic Regression Neural Networks

26-02-2020 AJITH SURESH | CRYPTOGRAPHY AND INFORMATION SECURITY LAB ,CSA, IISC

slide-39
SLIDE 39

26-02-2020 AJITH SURESH | CRYPTOGRAPHY AND INFORMATION SECURITY LAB ,CSA, IISC

Secure Dot Product

PPML using MPC: Hurdles to Clear

slide-40
SLIDE 40

26-02-2020 AJITH SURESH | CRYPTOGRAPHY AND INFORMATION SECURITY LAB ,CSA, IISC

Secure Dot Product Secure Comparison PPML using MPC: Hurdles to Clear

slide-41
SLIDE 41

26-02-2020 AJITH SURESH | CRYPTOGRAPHY AND INFORMATION SECURITY LAB ,CSA, IISC

Secure Dot Product Secure Comparison

Embedding Floating point Numbers

PPML using MPC: Hurdles to Clear

slide-42
SLIDE 42

26-02-2020 AJITH SURESH | CRYPTOGRAPHY AND INFORMATION SECURITY LAB ,CSA, IISC

Secure Dot Product Secure Comparison

Embedding Floating point Numbers

Single bit to Arithmetic Value

PPML using MPC: Hurdles to Clear

slide-43
SLIDE 43

26-02-2020 AJITH SURESH | CRYPTOGRAPHY AND INFORMATION SECURITY LAB ,CSA, IISC

Secure Dot Product Secure Comparison Embedding Floating point Numbers Single bit to Arithmetic Value Truncation

PPML using MPC: Hurdles to Clear

slide-44
SLIDE 44

26-02-2020 AJITH SURESH | CRYPTOGRAPHY AND INFORMATION SECURITY LAB ,CSA, IISC

Secure Dot Product Secure Comparison

Embedding Floating point Numbers

Single bit to Arithmetic Value Truncation Non-linear Activation Functions

PPML using MPC: Hurdles to Clear

slide-45
SLIDE 45

26-02-2020 AJITH SURESH | CRYPTOGRAPHY AND INFORMATION SECURITY LAB ,CSA, IISC

Secure Dot Product Secure Comparison

Embeddin g Floating point Numbers

Single bit to Arithmetic Value Truncation Non-linear Activation Functions and many more ...

PPML using MPC: Hurdles to Clear

slide-46
SLIDE 46

26-02-2020 AJITH SURESH | CRYPTOGRAPHY AND INFORMATION SECURITY LAB ,CSA, IISC

BLAZE Protocol

Ref Pre-processing

(#elements)

Online

(#elements)

Security

ABY3 12d 9d Abort

https://eprint.iacr.org/2020/042 BLAZE :

Communication Cost per Dot Product d – #elements in each vector !∎# = %

&'( )

*+ . -+

slide-47
SLIDE 47

26-02-2020 AJITH SURESH | CRYPTOGRAPHY AND INFORMATION SECURITY LAB ,CSA, IISC

BLAZE Protocol

Ref Pre-processing

(#elements)

Online

(#elements)

Security

ABY3 ASTRA 12d 21d 9d 2d+2 Abort Fair

https://eprint.iacr.org/2020/042 BLAZE :

Communication Cost per Dot Product d – #elements in each vector !∎# = %

&'( )

*+ . -+

slide-48
SLIDE 48

26-02-2020 AJITH SURESH | CRYPTOGRAPHY AND INFORMATION SECURITY LAB ,CSA, IISC

BLAZE Protocol

Ref Pre-processing

(#elements)

Online

(#elements)

Security

ABY3 ASTRA Boneh et al’19* 12d 21d 9d 2d+2 3d Abort Fair Abort

https://eprint.iacr.org/2020/042 BLAZE :

Communication Cost per Dot Product d – #elements in each vector !∎# = %

&'( )

*+ . -+

slide-49
SLIDE 49

26-02-2020 AJITH SURESH | CRYPTOGRAPHY AND INFORMATION SECURITY LAB ,CSA, IISC

BLAZE Protocol

Ref Pre-processing

(#elements)

Online

(#elements)

Security

ABY3 ASTRA Boneh et al’19*

BLAZE

12d 21d

3d

9d 2d+2 3d

3

Abort Fair Abort

Fair

https://eprint.iacr.org/2020/042 BLAZE :

Communication Cost per Dot Product d – #elements in each vector !∎# = %

&'( )

*+ . -+

slide-50
SLIDE 50

26-02-2020 AJITH SURESH | CRYPTOGRAPHY AND INFORMATION SECURITY LAB ,CSA, IISC

Summary of Our Benchmarking Results Algorithm

Improvement in terms of Online Throughput

  • ver State-of-the-art protocols over WAN

Training Prediction

Linear Regression 333.22 x 194.86 x Logistic Regression 53.19 x 27.52 x Neural Networks

  • 276.31x

*Throughput for Training - #iterations processed by servers / minute *Throughput for Prediction - #queries processed by servers / minute

slide-51
SLIDE 51

26-02-2020 AJITH SURESH | CRYPTOGRAPHY AND INFORMATION SECURITY LAB ,CSA, IISC

Summary of Our Benchmarking Results

slide-52
SLIDE 52

26-02-2020 AJITH SURESH | CRYPTOGRAPHY AND INFORMATION SECURITY LAB ,CSA, IISC

slide-53
SLIDE 53

26-02-2020 AJITH SURESH | CRYPTOGRAPHY AND INFORMATION SECURITY LAB ,CSA, IISC

References

1.

Andrew Chi-Chih Yao. Protocols for secure computations (extended abstract). In FOCS, pages 160-164, 1982.

2.

  • P. Mohassel, M. Rosulek, and Y. Zhang. Fast and Secure Three party Computation: Garbled Circuit Approach. In CCS, 2015.

3.

  • T. Araki, A. Barak, J. Furukawa, T. Lichter, Y. Lindell, A. Nof, K. Ohara, A. Watzman, and O. Weinstein. Optimized Honest-

Majority MPC for Malicious Adversaries - Breaking the 1 Billion-Gate Per Second Barrier. In IEEE S&P , 2017.

4.

  • J. Furukawa, Y. Lindell, A. Nof, and O. Weinstein. High-Throughput Secure Three-Party Computation for Malicious

Adversaries and an Honest Majority. In EUROCRYPT, 2017.

5.

  • K. Chida, D. Genkin, K. Hamada, D. Ikarashi, R. Kikuchi, Y. Lindell, and A. Nof. Fast Large-Scale Honest-Majority MPC for

Malicious Adversaries. In CRYPTO, 2018.

6.

  • P. Mohassel and P. Rindal, ABY3: A Mixed Protocol Framework for Machine Learning. In ACM CCS, 2018.

7.

  • H. Chaudhari, A. Choudhury, A. Patra and A. Suresh. ASTRA: High-throughput 3PC over Rings with Application to Secure

Prediction, In ACM CCSW, 2019.

8.

  • D. Boneh, E. Boyle, H. Corrigan{-}Gibbs, N. Gilboa and Y. Ishai. Zero-Knowledge Proofs on Secret-Shared Data via Fully

Linear PCPs. In CRYPTO, 2019.