Secure Generalized Deduplication via Multi-Key Revealing Encryption

Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 12238)


Cloud Storage Providers (CSPs) offer solutions to relieve users from locally storing vast amounts of data, including personal and sensitive ones. While users may desire to retain some privacy on the data they outsource, CSPs are interested in reducing the total storage space by employing compression techniques such as deduplication. We propose a new cryptographic primitive that simultaneously realizes both requirements: Multi-Key Revealing Encryption (MKRE). The goal of MKRE is to disclose the result of a pre-defined function over multiple ciphertexts, even if the ciphertexts were generated using different keys, while revealing nothing else about the data. We present a formal model and a security definition for MKRE and provide a construction of MKRE for generalized deduplication that only uses symmetric key primitives in a black-box way. Our construction allows (a) cloud providers to reduce the storage space by using generalized deduplication to compress encrypted data across users, and (b) each user to maintain a certain privacy level for the outsourced information. Our scheme can be proven secure in the random oracle model (and we argue that this is a necessary evil). We develop a proof-of-concept implementation of our solution. For a test data set, our MKRE construction achieves secure generalized deduplication with a compression ratio of 87% for 1 KB file chunks and 82.2% for 8 KB chunks. Finally, our experiments show that, compared to generalized deduplication setup with un-encrypted files, adding privacy via MKRE introduces a compression overhead of less than \(3\%\) and reduces the storage throughput by at most \(6.9\%\).


Private cloud storage Secure deduplication Revealing encryption 



This work was partially financed by: the SCALE-IoT project (Grant No. DFF-7026-00042B) and FoCC (Grant No. DFF-6108-00169) granted by the Danish Council for Independent Research; the AUFF Starting Grant AUFF-2017-FLS-7-1; Aarhus University’s DIGIT Centre; the strategic research area ELLIIT; the Concordium Blockhain Research Center, Aarhus University, Denmark; the European Research Council (ERC) under the European Unions’s Horizon 2020 research and innovation programme under grant agreement No 803096 (SPEC).


  1. 1.
    Agrawal, S., Clear, M., Frieder, O., Garg, S., O’Neill, A., Thaler, J.: Ad hoc multi-input functional encryption (2019).
  2. 2.
    Agrawal, S., Gorbunov, S., Vaikuntanathan, V., Wee, H.: Functional encryption: new perspectives and lower bounds. In: Canetti, R., Garay, J.A. (eds.) CRYPTO 2013. LNCS, vol. 8043, pp. 500–518. Springer, Heidelberg (2013). Scholar
  3. 3.
    Bellare, M., Boldyreva, A., O’Neill, A.: Deterministic and efficiently searchable encryption. In: Menezes, A. (ed.) CRYPTO 2007. LNCS, vol. 4622, pp. 535–552. Springer, Heidelberg (2007). Scholar
  4. 4.
    Bellare, M., et al.: Hedged public-key encryption: how to protect against bad randomness. In: Matsui, M. (ed.) ASIACRYPT 2009. LNCS, vol. 5912, pp. 232–249. Springer, Heidelberg (2009). Scholar
  5. 5.
    Bellare, M., Keelveedhi, S., Ristenpart, T.: DupLESS: server-aided encryption for deduplicated storage. In: USENIX Security Symposium, pp. 179–194 (2013)Google Scholar
  6. 6.
    Bellare, M., Keelveedhi, S., Ristenpart, T.: Message-locked encryption and secure deduplication. In: Johansson, T., Nguyen, P.Q. (eds.) EUROCRYPT 2013. LNCS, vol. 7881, pp. 296–312. Springer, Heidelberg (2013). Scholar
  7. 7.
    Bendlin, R., Nielsen, J.B., Nordholt, P.S., Orlandi, C.: Lower and upper bounds for deniable public-key encryption. In: Lee, D.H., Wang, X. (eds.) ASIACRYPT 2011. LNCS, vol. 7073, pp. 125–142. Springer, Heidelberg (2011). Scholar
  8. 8.
    Boyd, C., Davies, G.T., Gjøsteen, K., Raddum, H., Toorani, M.: Security notions for cloud storage and deduplication. In: Baek, J., Susilo, W., Kim, J. (eds.) ProvSec 2018. LNCS, vol. 11192, pp. 347–365. Springer, Cham (2018). Scholar
  9. 9.
    Canetti, R., Feige, U., Goldreich, O., Naor, M.: Adaptively secure multi-party computation. In: ACM STOC, pp. 639–648 (1996)Google Scholar
  10. 10.
    Chenette, N., Lewi, K., Weis, S.A., Wu, D.J.: Practical order-revealing encryption with limited leakage. Fast Softw. Encryption 2016, 474–493 (2016)CrossRefGoogle Scholar
  11. 11.
    Chotard, J., Dufour Sans, E., Gay, R., Phan, D.H., Pointcheval, D.: Decentralized multi-client functional encryption for inner product. In: Peyrin, T., Galbraith, S. (eds.) ASIACRYPT 2018. LNCS, vol. 11273, pp. 703–732. Springer, Cham (2018). Scholar
  12. 12.
    Curtmola, R., Garay, J.A., Kamara, S., Ostrovsky, R.: Searchable symmetric encryption: improved definitions and efficient constructions. J. Comput. Secur. 19(5), 895–934 (2011)CrossRefGoogle Scholar
  13. 13.
    Douceur, J.R., Adya, A., Bolosky, W.J., Simon, D., Theimer, M., Simon, P.: Reclaiming space from duplicate files in a serverless distributed file system. ICDCS 2002, 617–624 (2002)Google Scholar
  14. 14.
    Goldwasser, S., et al.: Multi-input functional encryption. In: Nguyen, P.Q., Oswald, E. (eds.) EUROCRYPT 2014. LNCS, vol. 8441, pp. 578–602. Springer, Heidelberg (2014). Scholar
  15. 15.
    Haagh, H., Ji, Y., Li, C., Orlandi, C., Song, Y.: Revealing encryption for partial ordering. In: O’Neill, M. (ed.) IMACC 2017. LNCS, vol. 10655, pp. 3–22. Springer, Cham (2017). Scholar
  16. 16.
    Hamming, R.W.: Error detecting and error correcting codes. Bell Syst. Tech. J. 29(2), 147–160 (1950)MathSciNetCrossRefGoogle Scholar
  17. 17.
    Katz, J., Sahai, A., Waters, B.: Predicate encryption supporting disjunctions, polynomial equations, and inner products. J. Cryptol. 26, 191–224 (2013)MathSciNetCrossRefGoogle Scholar
  18. 18.
    Li, J., Chen, X., Li, M., Li, J., Lee, P.P., Lou, W.: Secure deduplication with efficient and reliable convergent key management. IEEE Trans. Parallel Distrib. Syst. 25(6), 1615–1625 (2013)CrossRefGoogle Scholar
  19. 19.
    Li, X., Li, J., Huang, F.: A secure cloud storage system supporting privacy-preserving fuzzy deduplication. Soft Comput. 20(4), 1437–1448 (2015). Scholar
  20. 20.
    Libert, B., Ţiţiu, R.: Multi-client functional encryption for linear functions in the standard model from LWE. In: Galbraith, S.D., Moriai, S. (eds.) ASIACRYPT 2019. LNCS, vol. 11923, pp. 520–551. Springer, Cham (2019). Scholar
  21. 21.
    Liu, J., Asokan, N., Pinkas, B.: secure deduplication of encrypted data without additional independent servers. In: ACM CCS, pp. 874–885 (2015)Google Scholar
  22. 22.
    Liu, J., Duan, L., Li, Y., Asokan, N.: Secure deduplication of encrypted data: refined model and new constructions. In: Smart, N.P. (ed.) CT-RSA 2018. LNCS, vol. 10808, pp. 374–393. Springer, Cham (2018). Scholar
  23. 23.
    Lucani, D.E., Nielsen, L., Orlandi, C., Pagnin, E., Vestergaard, R.: Secure generalized deduplication via multi-key revealing encryption. Cryptology ePrint Archive, Report 2020/799 (2020). (full version of this work)
  24. 24.
    Michalevsky, Y., Joye, M.: Decentralized policy-hiding ABE with receiver privacy. In: Lopez, J., Zhou, J., Soriano, M. (eds.) ESORICS 2018. LNCS, vol. 11099, pp. 548–567. Springer, Cham (2018). Scholar
  25. 25.
    Nielsen, J.B.: Separating random oracle proofs from complexity theoretic proofs: the non-committing encryption case. In: Yung, M. (ed.) CRYPTO 2002. LNCS, vol. 2442, pp. 111–126. Springer, Heidelberg (2002). Scholar
  26. 26.
    Nielsen, L., Vestergaard, R., Yazdani, N., Talasila, P., Lucani, D.E., Sipos, M.: Alexandria: a proof-of-concept implementation and evaluation of generalised data deduplication. In: IEEE GLOBECOM Workshop on Advances in Edge Computing (2019)Google Scholar
  27. 27.
    Oracle: What Is ZFS? (2019). Accessed 12 Oct 2019
  28. 28.
    Planet Labs Inc: download samples of our, high resolution imagery, for monitoring, tasking and large area mapping (2019). Accessed 17 Jun 2019
  29. 29.
    Stefanov, E., Papamanthou, C., Shi, E.: Practical dynamic searchable encryption with small leakage. NDSS 71, 72–75 (2014)Google Scholar
  30. 30.
    The OpenSSL Project: OpenSSL: the open source toolkit for SSL/TLS. Accessed 23 Sep 2019
  31. 31.
    Vestergaard, R., Lucani, D.E., Zhang, Q.: A randomly accessible lossless compression scheme for time-series data. In: IEEE INFOCOM (2020)Google Scholar
  32. 32.
    Vestergaard, R., Zhang, Q., Lucani, D.E.: Generalized deduplication: bounds, convergence, and asymptotic properties. In: IEEE GLOBECOM (2019)Google Scholar
  33. 33.
    Vestergaard, R., Zhang, Q., Lucani, D.E.: Lossless compression of time series data with generalized deduplication. In: IEEE GLOBECOM (2019)Google Scholar
  34. 34.
    Xia, W., et al.: A comprehensive study of the past, present, and future of data deduplication. Proc. IEEE 104(9), 1681–1710 (2016)CrossRefGoogle Scholar
  35. 35.
    Zhao, Y., Chow, S.S.M.: Updatable block-level message-locked encryption. IEEE Trans. Dependable Secure Comput. (2019)Google Scholar

Copyright information

© Springer Nature Switzerland AG 2020

Authors and Affiliations

  1. 1.Aarhus UniversityAarhusDenmark
  2. 2.Lund UniversityLundSweden

Personalised recommendations