Skip to main content

Distributed authentication framework for Hadoop based bigdata environment


Big data, the upcoming technology in the field of computing, refers to a large complex dataset. It deals with large complex datasets and yields great valued information when analysed properly. Data Security has become the greatest challenge in the minds of cyber experts and researchers in this scenario. Apache Hadoop frameworks that let distributed processing of these large datasets rely on Kerberos Authentication for mutual authentication and verification. The protocol comes with inherent challenges like Single point of failure, Dictionary Attacks, Replay Attacks, and Time Synchronization problems. This paper puts forward a one-off approach based on recent technologies like Blockchain Networks, Digital Signatures, and Elliptic ElGamal and Threshold Cryptosystem. The proposed scheme aims to mainly deal with the Single Point of Failure problem. Riverbed Modeller (AE) simulation is performed to do the comparative study of the proposed scheme with existing systems that use traditional encryption standards like RSA cryptosystems. Analysis of the simulation results proves that the proposed scheme is more efficient in terms of time and memory without compromising the level of security offered. The response time, network delay and traffic rates of the proposed system are compared with the existing RSA based system and the results strengthen the claims of this work. Lastly, the results from comparative analysis of security features and computational time cost indicate that the proposed method heightens the security level offered for big data systems with a nominal effect on performance.

This is a preview of subscription content, access via your institution.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Data availability

Data sharing not applicable to this article as no datasets were generated or analysed during the current study.


  1. Aazam M, Zeadally S, Harras KA (2018) Deploying fog computing in industrial internet of things and industry 4.0. IEEE Trans Ind Inform, 14: 4674–4682

  2. Abdullah N, Hakansson A, Moradian E (2017) Blockchain based approach to enhance big data authentication in distributed environment. In: 2017 Ninth international conference on ubiquitous and future networks (ICUFN), pages 887–892, 2017. 10.1109/ICUFN.2017.7993927

  3. Abidin Aysajan, Aly Abdelrahaman, Mustafa Mustafa A (2020) Collaborative authentication using threshold cryptography. Lecture Notes Comput Sci Emerging Technol Authorization Authentication 11967:122–137.

    Article  Google Scholar 

  4. Adarshpal S (2013) Sethi and Vasil Y Hnatyshin. CRC Press, Practical opnet user guide for computer network simulation

    Google Scholar 

  5. Adi Shamir. How to share a secret. Commun. ACM, 22 (11): 612–613, November 1979. ISSN 0001-0782. URL

  6. Cathy Sturges. How can blockchain improve data storage?, Jan 2020. URL

  7. Chao Wu and Yike Guo. Enhanced user data privacy with pay-by-data model. In 2013 IEEE International Conference on Big Data, pages 53–57, 2013. 10.1109/BigData.2013.6691688

  8. Chattaraj D, Sarma M, Kumar Das A, Kumar N, Rodrigues JJPC, Park Y (2018) Heap: An efficient and fault-tolerant authentication and key exchange protocol for hadoop-assisted big data platform. IEEE Access, 6: 75342–75382 10.1109/ACCESS.2018.2883105

  9. Chengqi Wang, Xiao Zhang, and Zhiming Zheng. Cryptanalysis and improvement of a biometric-based multi-server authentication and key agreement scheme. Plos One, 11 (2), 2016. 10.1371/journal.pone.0149173

  10. Christidis Konstantinos, Devetsikiotis Michael (2016) Blockchains and smart contracts for the internet of things. IEEE Access 4:2292–2303.

    Article  Google Scholar 

  11. Chundong Wang and Chaoran Feng. Security analysis and improvement for kerberos based on dynamic password and diffie-hellman algorithm. In 2013 Fourth International Conference on Emerging Intelligent Data and Web Technologies, pages 256–260, 2013. 10.1109/EIDWT.2013.49

  12. Desmedt Y, Frankel Y (1989) Threshold cryptosystems. In: Advances in cryptology—CRYPTO ’89, 9th annual international cryptology conference, Santa Barbara, California, USA, August 20-24, 1989, Proceedings, volume 435 of Lecture Notes in Computer Science, pages 307–315. Springer, 1989. 10.1007/0-387-34805-0\_28

  13. Dong Xinhua, Li Ruixuan, He Heng, Zhou Wanwan, Xue Zhengyuan, Hao Wu (2015) Secure sensitive data sharing on a big data platform. Tsinghua Sci Technol 20(1):72–80.

    MathSciNet  Article  Google Scholar 

  14. Dou Zuochao, Khalil Issa, Khreishah Abdallah, Al-Fuqaha Ala (2018) Robust insider attacks countermeasure for hadoop: design and implementation. IEEE Syst J 12(2):1874–1885.

    Article  Google Scholar 

  15. El-Emam E, Koutb M, Kelash H, Farag Allah OF (2009) An optimized kerberos authentication protocol. In: 2009 International Conference on Computer Engineering & Systems, pages 508–513

  16. Ertaul L, Chavan N (2007) Rsa and elliptic curve- elgamal threshold cryptography (ecceg-tc) implementations for secure data forwarding in manets. In: Proceedings of the 2007 international conference on security and management, SAM’07, pages 142–146, 01

  17. Gharib H, Belloulata K (2014) Authentication architecture using threshold cryptography in kerberos for mobile ad hoc networks. Adv Sci Technol Res J, 8 (22): 12–18, ISSN 2080-4075. URL

  18. GireeshKumar R, Dash S, Panigrahi. A Novel Authentication Framework for Hadoop, volume 324, page 333–340. Springer, 2014

  19. Hammi MT, Bellot P, Serhrouchni A (2018) Bctrust: A decentralized authentication blockchain-based mechanism. In 2018 IEEE wireless communications and networking conference (WCNC), pages 1–6, 10.1109/WCNC.2018.8376948

  20. He Debiao, Wang Ding (2015) Robust biometrics-based authentication scheme for multiserver environment. IEEE Syst J 9(3):816–823.

    Article  Google Scholar 

  21. Imine Y, Kouicem DE, Bouabdallah A, Ahmed L (2018) Masfog: an efficient mutual authentication scheme for fog computing architecture. In 2018 17th IEEE International Conference On Trust, Security And Privacy In Computing And Communications/ 12th IEEE International Conference On Big Data Science And Engineering (TrustCom/BigDataSE), pages 608–613, . 10.1109/TrustCom/BigDataSE.2018.00091

  22. Jegadeesan S, Azees M, Malarvizhi Kumar P, Manogaran G, Chilamkurti N, Varatharajan R, Hsu C-H (2019) An efficient anonymous mutual authentication technique for providing secure communication in mobile cloud computing for smart city applications. Sustain Cities Soc, 49: 101522, 2019. ISSN 2210-6707. URL

  23. Jeong Y-S, Shin S-S, Han K-H (2016) High-dimentional data authentication protocol based on hash chain for hadoop systems. Cluster Computing, 19 (1): 475–484, March 2016. ISSN 1386-7857. URL

  24. Kankal M (2019) Prof.Pramod Patil. An adaptive authentication based on blockchain for bigdata hadoop framework. Int J Eng Tech 5: 89–94

  25. Kashish A. Shakil, Farhana J. Zareen, Alam M, Jabin S (2020) Bamhealthcloud: A biometric authentication and data management system for healthcare data in cloud. J King Saud Univ - Comput Inform Sci, 32 (1): 57–64, . ISSN 1319-1578. URL

  26. Lein Harn and Ching-Fang Hsu. A novel threshold cryptography with membership authentication and key establishment. Wirel. Pers. Commun., 97 (3): 3943–3950, December 2017. ISSN 0929-6212. URL

  27. Liang K, Susilo W, Liu JK (2015) Privacy-preserving ciphertext multi-sharing control for big data storage. IEEE Trans Inform Forensics Security 10(8):1578–1589.

    Article  Google Scholar 

  28. Li R, Asaeda H, Li J, Fu X (2017a) A distributed authentication and authorization scheme for in-network big data sharing. Digital Communications and Networks, 3 (4): 226–235, 2017a. ISSN 2352-8648. URL Big Data Security and Privacy

  29. Li Y, Gai K, Qiu L, Qiu M, Zhao H (2017b) Intelligent cryptography approach for secure distributed big data storage in cloud computing. Inform Sci, 387: 103–115, 2017b. ISSN 0020-0255. URL

  30. Mukti Rani Sutradhar, N. Sultana, Himel Dey, and Hossain Arif. A new version of kerberos authentication protocol using ecc and threshold cryptography for cloud security. 2018 Joint 7th International Conference on Informatics, Electronics & Vision (ICIEV) and 2018 2nd International Conference on Imaging, Vision & Pattern Recognition (icIVPR), pages 239–244, 2018

  31. Omoniwa B, Hussain R, Javed M, Bouk S, Malik S (2019) Fog/edge computing-based iot (feciot): Architecture, applications, and research issues. IEEE Internet Things J, 6: 4118–4149

  32. Reddy AG, Yoon E-J, Kumar Das A, Odelu V, Yoo K-Y (2017) Design of mutually authenticated key agreement protocol resistant to impersonation attacks for multi-server environment. IEEE Access, 5: 3622–3639, 2017. 10.1109/access.2017.2666258

  33. Yoon-Su Jeong and Yong tae Kim (2014) A token-based authentication security scheme for hadoop distributed file system using elliptic curve cryptography. J Comput Virol Hacking Tech 11:137–142

    Google Scholar 

  34. Zhao Hu, Yuesheng Zhu, and Limin Ma. An improved kerberos protocol based on diffie-hellman-dsa key exchange. 2012 18th IEEE international conference on networks (ICON), pages 400–404, 2012

Download references


No funds, grants, or other support was received.

Author information



Corresponding author

Correspondence to M. Hena.

Ethics declarations

Conflict of interest:

The authors have no conflicts of interest to declare that are relevant to the content of this article.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Hena, M., Jeyanthi, N. Distributed authentication framework for Hadoop based bigdata environment. J Ambient Intell Human Comput (2021).

Download citation


  • Big data
  • Apache Hadoop
  • Kerberos
  • Blockchain
  • Digital signature
  • Threshold ElGamal cryptosystem