Skip to main content

PINFER: Privacy-Preserving Inference

Logistic Regression, Support Vector Machines, and More, over Encrypted Data

Part of the Lecture Notes in Computer Science book series (LNSC,volume 11737)

Abstract

The foreseen growing role of outsourced machine learning services is raising concerns about the privacy of user data. This paper proposes a variety of protocols for privacy-preserving regression and classification that (i) only require additively homomorphic encryption algorithms, (ii) limit interactions to a mere request and response, and (iii) that can be used directly for important machine-learning algorithms such as logistic regression and SVM classification. The basic protocols are then extended and applied to simple feed-forward neural networks.

Keywords

  • Machine learning as a service
  • Linear regression
  • Logistic regression
  • Support vector machines
  • Feed-forward neural networks
  • Data privacy
  • Additively homomorphic encryption

This is a preview of subscription content, access via your institution.

Buying options

Chapter
USD   29.95
Price excludes VAT (USA)
  • DOI: 10.1007/978-3-030-31500-9_1
  • Chapter length: 19 pages
  • Instant PDF download
  • Readable on all devices
  • Own it forever
  • Exclusive offer for individuals only
  • Tax calculation will be finalised during checkout
eBook
USD   59.99
Price excludes VAT (USA)
  • ISBN: 978-3-030-31500-9
  • Instant PDF download
  • Readable on all devices
  • Own it forever
  • Exclusive offer for individuals only
  • Tax calculation will be finalised during checkout
Softcover Book
USD   79.99
Price excludes VAT (USA)
Fig. 1.
Fig. 2.
Fig. 3.
Fig. 4.
Fig. 5.
Fig. 6.
Fig. 7.
Fig. 8.
Fig. 9.

Notes

  1. 1.

    Given , the server obtains as if \(\eta _i = 0\), and as if \(\eta _i = 1\).

References

  1. Abu-Mostafa, Y.S., Magdon-Ismail, M., Lin, H.T.: Learning From Data: A Short Course. AMLbook.com, New York (2012). http://amlbook.com

    Google Scholar 

  2. Agrawal, R., Srikant, R.: Privacy-preserving data mining. ACM SIGMOD Rec. 29(2), 439–450 (2000). https://doi.org/10.1145/335191.335438

    CrossRef  Google Scholar 

  3. Barni, M., Orlandi, C., Piva, A.: A privacy-preserving protocol for neural-network-based computation. In: MM&Sec 2006, pp. 146–151. ACM (2006). https://doi.org/10.1145/1161366.1161393

  4. Bos, J.W., Lauter, K., Naehrig, M.: Private predictive analysis on encrypted medical data. J. Biomed. Inf. 50, 234–243 (2014). https://doi.org/10.1016/j.jbi.2014.04.003

    CrossRef  Google Scholar 

  5. Bost, R., Popa, R.A., Tu, S., Goldwasser, S.: Machine learning classification over encrypted data. In: NDSS 2015. The Internet Society (2015). https://doi.org/10.14722/ndss.2015.23241

  6. Damgård, I., Geisler, M., Krøigaard, M.: Homomorphic encryption and secure comparison. Int. J. Appl. Cryptogr. 1(1), 22–31 (2008). https://doi.org/10.1504/IJACT.2008.017048

    MathSciNet  CrossRef  MATH  Google Scholar 

  7. Damgård, I., Geisler, M., Krøigaard, M.: A correction to ‘efficient and secure comparison for on-line auctions’. Int. J. Appl. Cryptogr. 1(4), 323–324 (2009). https://doi.org/10.1504/IJACT.2009.028031

    MathSciNet  CrossRef  MATH  Google Scholar 

  8. Dwork, C., Feldman, V.: Privacy-preserving prediction. In: COLT 2018. PMLR, vol. 75, pp. 1693–1702. PMLR (2018). http://proceedings.mlr.press/v75/dwork18a/dwork18a.pdf

  9. Erkin, Z., Franz, M., Guajardo, J., Katzenbeisser, S., Lagendijk, I., Toft, T.: Privacy-preserving face recognition. In: Goldberg, I., Atallah, M.J. (eds.) PETS 2009. LNCS, vol. 5672, pp. 235–253. Springer, Heidelberg (2009). https://doi.org/10.1007/978-3-642-03168-7_14

    CrossRef  Google Scholar 

  10. Glorot, X., Bordes, A., Bengjio, Y.: Deep sparse rectifier neural networks. In: AISTAT 2011. PMLR, vol. 15, pp. 315–323. PMLR (2011). http://proceedings.mlr.press/v15/glorot11a/glorot11a.pdf

  11. Goethals, B., Laur, S., Lipmaa, H., Mielikäinen, T.: On private scalar product computation for privacy-preserving data mining. In: Park, C., Chee, S. (eds.) ICISC 2004. LNCS, vol. 3506, pp. 104–120. Springer, Heidelberg (2005). https://doi.org/10.1007/11496618_9

    CrossRef  Google Scholar 

  12. Goldwasser, S., Micali, S.: Probabilistic encryption. J. Comput. Syst. Sci. 28(2), 270–299 (1984). https://doi.org/10.1016/0022-0000(84)90070-9

    MathSciNet  CrossRef  MATH  Google Scholar 

  13. Hastie, T., Tibshirani, R., Friedman, J.: The Elements of Statistical Learning. Springer Series in Statistics, 2nd edn. Springer, New York (2009). https://doi.org/10.1007/978-0-387-84858-7

    CrossRef  MATH  Google Scholar 

  14. Hubara, I., Courbariaux, M., Soudry, D., El-Yaniv, R., Bengio, Y.: Binarized neural networks. In: NISP 2016, pp. 4107–4115. Curran Associates, Inc. http://papers.nips.cc/paper/6573-binarized-neural-networks.pdf

  15. Joye, M., Salehi, F.: Private yet efficient decision tree evaluation. In: Kerschbaum, F., Paraboschi, S. (eds.) DBSec 2018. LNCS, vol. 10980, pp. 243–259. Springer, Cham (2018). https://doi.org/10.1007/978-3-319-95729-6_16

    CrossRef  Google Scholar 

  16. Kim, M., Song, Y., Wang, S., Xia, Y., Jiang, X.: Secure logistic regression based on homomorphic encryption: design and evaluation. JMIR Med. Inform. 6(2), e19 (2018). https://doi.org/10.2196/medinform.8805

    CrossRef  Google Scholar 

  17. Lindell, Y., Pinkas, B.: Privacy preserving data mining. In: Bellare, M. (ed.) CRYPTO 2000. LNCS, vol. 1880, pp. 36–54. Springer, Heidelberg (2000). https://doi.org/10.1007/3-540-44598-6_3

    CrossRef  Google Scholar 

  18. Mohassel, P., Zhang, Y.: SecureML: A system for scalable privacy-preserving machine learning. In: IEEE S&P 2017, pp. 19–38. IEEE Computer Society (2017). https://doi.org/10.1109/SP.2017.12

  19. Tramèr, F., Zhang, F., Juels, A., Reiter, M.K., Ristenpart, T.: Stealing machine learning models via prediction APIs. In: USENIX Security 2016, pp. 601–618. USENIX Association (2016). https://www.usenix.org/system/files/conference/usenixsecurity16/sec16_paper_tramer.pdf

  20. Veugen, T.: Improving the DGK comparison protocol. In: WIFS 2012, pp. 49–54. IEEE (2012). https://doi.org/10.1109/WIFS.2012.6412624

  21. Zhang, J., Wang, X., Yiu, S.M., Jiang, Z.L., Li, J.: Secure dot product of outsourced encrypted vectors and its application to SVM. In: SCC@AsiaCCS 2017, pp. 75–82. ACM (2017). https://doi.org/10.1145/3055259.3055270

Download references

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Marc Joye .

Editor information

Editors and Affiliations

A A More Private Protocols

A A More Private Protocols

1.1 A.1 Linear Regression

As seen in Sect. 2.1, linear regression produces estimates using the identity map for g: . Since is linear, given an encryption of \({{\varvec{x}}}\), the value of can be homomorphically evaluated, in a provably secure way [11].

Therefore, the client encrypts its feature vector \({{\varvec{x}}}\) under its public key and sends to the server. Using \({\varvec{\theta }}\), the server then computes and returns it the client. Finally, the client decrypts and gets the output . This is straightforward and only requires one round of communication.

1.2 A.2 SVM Classification

A Naïve Protocol. A client holding a private feature vector \({{\varvec{x}}}\) wishes to evaluate where \({\varvec{\theta }}\) parametrises an SVM classification model. In the primal approach, the client can encrypt \({{\varvec{x}}}\) and send to the server. Next, the server computes for some random mask \(\mu \) and sends  to the client. The client decrypts  and recovers \(\eta \). Finally, the client and the server engage in a private comparison protocol with respective inputs \(\eta \) and \(\mu \), and the client deduces the sign of from the resulting comparison bit \([\mu \leqslant \eta ]\).

There are two issues. If we use the DGK+ protocol for the private comparison, at least one extra exchange from the server to the client is needed for the client to get \([\mu \leqslant \eta ]\). This can be fixed by considering the dual approach. A second, more problematic, issue is that the decryption of yields \(\eta \) as an element of \(\mathcal {M}\cong \mathbb {Z}/M\mathbb {Z}\), which is not necessarily equivalent to the integer . Note that if the inner product can take any value in \(\mathcal {M}\), selecting a smaller value for \(\mu \in \mathcal {M}\) to prevent the modular reduction does not solve the issue because the value of \(\eta \) may then leak information on .

A Heuristic Protocol (Dual Approach). The bandwidth usage with the heuristic comparison protocol (cf. Fig. 5) could be even reduced to one ciphertext and a single bit with the dual approach. From the published encrypted model , the client could homomorphically compute and send to the server for random \(\lambda , \mu \in \mathcal {B}\) with . The server would then decrypt \(\mathfrak {t}^*\), obtain \(t^*\), compute \({\smash {\delta }_{\scriptscriptstyle S}}= \frac{1}{2}(1-\mathrm{sign}(t^*))\), and return \({\smash {\delta }_{\scriptscriptstyle S}}\) to the client. Analogously to the primal approach, the output class is obtained by the client as \(\hat{y}= (-1)^{\smash {\delta }_{\scriptscriptstyle S}}\cdot \mathrm{sign}(\lambda )\). However, and contrarily to the primal approach, the potential information leakage resulting from \(t^*\)—in this case on \({{\varvec{x}}}\)—is now on the server’s side, which is in contradiction with our Requirement 1 (input confidentiality). We do not further discuss this variant.

Rights and permissions

Reprints and Permissions

Copyright information

© 2019 Springer Nature Switzerland AG

About this paper

Verify currency and authenticity via CrossMark

Cite this paper

Joye, M., Petitcolas, F. (2019). PINFER: Privacy-Preserving Inference. In: Pérez-Solà, C., Navarro-Arribas, G., Biryukov, A., Garcia-Alfaro, J. (eds) Data Privacy Management, Cryptocurrencies and Blockchain Technology. DPM CBT 2019 2019. Lecture Notes in Computer Science(), vol 11737. Springer, Cham. https://doi.org/10.1007/978-3-030-31500-9_1

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-31500-9_1

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-31499-6

  • Online ISBN: 978-3-030-31500-9

  • eBook Packages: Computer ScienceComputer Science (R0)