In cloud computing, de-duplication plays an essential role in detecting the de-duplication of encoded data with minimal computation and cost. De-duplication cleans the cloud datacentre’s unwanted storage and helps to identify the right owner of the content in the cloud. Even if there is only one copy of each data file stored in the cloud, the cloud has huge quantity of cloud users who own such data file. The existing method discussed a convergent encryption technique to solve the de-duplication problem. It also developed a system which does not allow storing any duplicate data in the cloud. However, the method does not assure consistency, reliability and confidentiality in cloud. Similar or different cloud users could store duplicated file in the cloud server, where cloud storage utilises high volume of storage. To find a solution for the above problems, the paper introduces enhanced secure content de-duplication identification and prevention (ESCDIP) algorithm to enhance the file-level and content-level de-duplication detection of encoded data with reliability in cloud environment. Every cloud user’s files contain an independent master key for encryption using ESCDIP technique and outsourcing them into the cloud. It reduces the overheads that are associated with the interactive duplication detection and query processes. The proposed method identifies the unique data chunking to store in the cloud. Based on experimental result, the ESCDIP method reduces 2.3 data uploading time in seconds, 2.31 data downloading time in seconds and 32.66% communication cost compared to existing approaches.
This is a preview of subscription content, log in to check access.
Buy single article
Instant access to the full article PDF.
Price includes VAT for USA
Subscribe to journal
Immediate online access to all issues from 2019. Subscription will auto renew annually.
This is the net price. Taxes to be calculated in checkout.
Kaaniche N, Laurent M (2014) A secure client-side de-duplication scheme in cloud storage environments. In: 2014 6th international conference on new technologies, mobility and security (NTMS). IEEE, pp 1–7
Akhila K, Ganesh A, Sunitha C (2016) A study on de-duplication techniques over encrypted data. Procedia Comput Sci 87:38–43
Shobana R, Shalini KS, Leelavathy S, Sridevi V (2016) De-duplication of data in cloud. Int J Chem Sci 14(4):2933–2938
Stanek J, Sorniotti A, Androulaki E, Kencl L (2014) A secure data de-duplication scheme for cloud storage. In: International conference on financial cryptography and data security. Springer, Berlin, pp 99–118
Thakar MPD, Harkut DG (2015) Hybrid model for authorized de-duplication in cloud. Int J Emerg Trends Technol Comput Sci: IJETTCS 4(1):147–151
Harish B, Harshitha K (2017) Data de-duplication in cloud. Int J Pure Appl Math 115(8):353–358
Shieh F, Arani MG, Shamsi M (2015) De-duplication approaches in cloud computing environment: a survey. Int J Comput Appl 120(13):7–10
Kaur M, Singh J (2016) Data de-duplication approach based on hashing techniques for reducing time consumption over a cloud network. Int J Comput Appl 142(5):4–10
Puzio P, Molva R, Önen M, Loureiro S (2015) PerfectDedup: secure data deduplication. In: Data privacy management, and security assurance. DPM 2015, QASA 2015. Lecture notes in computer science, vol 9481. Springer, Cham, pp 150–166
Shashikala MK, Dhruva MS (2017) Secure de-duplication in cloud computing environment by managing ownership dynamically. Int J Eng Appl Comput Sci: IJEACS 2(6):196–201
Priyadharsini P, Dhamodran P, Kavitha MS (2014) A survey on de-duplication in cloud computing. IJCSMC 3(11):149–155
Harnik D, Pinkas B, Shulman-Peleg A (2010) Side channels in cloud services: de-duplication in cloud storage. IEEE Secur Priv 8(6):40–47
Xu J, Chang EC, Zhou J (2013) Weak leakage-resilient client-side de-duplication of encrypted data in cloud storage. In: Proceedings of the 8th ACM SIGSAC symposium on information, computer and communications security, pp 195–206
Puzio P, Molva R, Onen M, Loureiro S (2013) ClouDedup: secure de-duplication with encrypted data for cloud storage. In: 2013 IEEE 5th international conference on cloud computing technology and science (CloudCom), no 1, pp 363–370
Bharat S, Mandre BR (2015) A secured and authorized data de-duplication in the hybrid cloud with public auditing. Int J Comput Appl 120(16):19–24
Devi GU, Supriya G (2017) Encryption of big data in cloud using de-duplication technique. Res J Pharm Biol Chem Sci 8(3):1103–1108
Zhou B, Wen J (2014) Efficient file communication via deduplication over networks with manifest feedback. IEEE Commun Lett 18(1):94–97
De Carvalho MG, Laender AH, Gonçalves MA, da Silva AS (2012) A genetic programming approach to record deduplication. IEEE Trans Knowl Data Eng 24(3):399–412
Fu Y, Jiang H, Xiao N, Tian L, Liu F, Xu L (2014) Application-aware local-global source deduplication for cloud backup services of personal storage. IEEE Trans Parallel Distrib Syst 25(5):1155–1165
Kim SH, Jeong J, Lee J (2014) Selective memory deduplication for cost efficiency in mobile smart devices. IEEE Trans Consum Electron 60(2):276–284
Zhao X, Zhang Y, Wu Y, Chen K, Jiang J, Li K (2014) Liquid: a scalable deduplication file system for virtual machine images. IEEE Trans Parallel Distrib Syst 25(5):1257–1266
Hunashikatti L, Pujar PM (2016) Review on data deduplication and secured auditing of data on cloud. IEEE Trans Comput 65(8):2386–2396
Yu CM, Gochhayat SP, Conti M, Lu CS (2018) Privacy aware data deduplication for side channel in cloud storage. IEEE Trans Cloud Comput 17(1):1
Hur J, Koo D, Shin Y, Kang K (2016) Secure data de-duplication with dynamic ownership management in cloud storage. IEEE Trans Knowl Data Eng 28(11):3113–3125
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
About this article
Cite this article
Periasamy, J.K., Latha, B. An enhanced secure content de-duplication identification and prevention (ESCDIP) algorithm in cloud environment. Neural Comput & Applic 32, 485–494 (2020). https://doi.org/10.1007/s00521-019-04060-9
- Enhanced secure content de-duplication identification and prevention (ESCDIP)
- Cloud storage
- Data de-duplication
- Privacy preserving
- Data uploading time
- Data downloading time