Deep Learning and Dempster-Shafer Theory Based Insider Threat Detection


Organizations’ own personnel now have a greater ability than ever before to misuse their access to critical organizational assets. Insider threat detection is a key component in identifying rare anomalies in context, which is a growing concern for many organizations. Existing perimeter security mechanisms are proving to be ineffective against insider threats. As a prospective filter for the human analysts, a new deep learning based insider threat detection method that uses the Dempster-Shafer theory is proposed to handle both accidental as well as intentional insider threats via organization’s channels of communication in real time. The long short-term memory (LSTM) architecture together with multi-head attention mechanism is applied in this work to detect anomalous network behavior patterns. Furthermore, belief is updated with Dempster’s conditional rule and utilized to fuse evidence to achieve enhanced prediction. The CERT Insider Threat Dataset v6.2 is used to train the behavior model. Through performance evaluation, our proposed method is proven to be effective as an insider threat detection technique.

This is a preview of subscription content, log in to check access.

Fig. 1
Fig. 2
Fig. 3.
Fig. 4
Fig. 5
Fig. 6
Fig. 7


  1. 1.

    Du X, Chen HH (2008) Security in wireless sensor networks. IEEE Wirel Commun Mag 15(4):60–66

    Article  Google Scholar 

  2. 2.

    Gupta BB, Agrawal DP, Yamaguchi S (2014) Call for chapters: handbook of research on modern cryptographic solutions for computer and cyber security

  3. 3.

    X. Xu, Z. Rong, Z.X. Wu, T. Zhou, and C. K. Tse. Phys Rev E, 95(5), id. 052302, 2017

  4. 4.

    Mao Y, Xu X, Rong Z, Wu Z-X (2018) The emergence of cooperation-extortion alliance on scale-free networks with normalized payoff. EPL Europhysics Lett 122(5)

  5. 5.

    Atat R, Liu L, Chen H et al (2017) Enabling cyber-physical communication in 5G cellular networks: challenges, spatial spectrum sensing, and cyber-security. IET Cyber-Phys Syst: Theory Appl 2(1):49–54

    Google Scholar 

  6. 6.

    Du X, Guizani M, Xiao Y, Chen HH (2009) Transactions papers, A Routing-Driven Elliptic Curve Cryptography based Key Management Scheme for Heterogeneous Sensor Networks. IEEE Trans Wireless Commun 8(3):1223–1229

    Article  Google Scholar 

  7. 7.

    Plageras A P, Stergiou C, Psannis K E, et al (2017) Efficient IoT-based sensor BIG data collection-processing and analysis in smart buildings. Futur Gener Comput Syst

  8. 8.

    Tian Z, Li M, Qiu M, Sun Y, Shen S (2019) Block-DES: a secure digital evidence system using Blockchain. Inf Sci 491:151–165.

    Article  Google Scholar 

  9. 9.

    Zhu L, Li M, Zhang Z, Qin Z (2018) ASAP: An Anonymous Smart-Parking and Payment Scheme in Vehicular Networks. IEEE Trans Dependable Secure Comput.

  10. 10.

    Tian Y, Guo J, Wu Y, Lin H (2019) Towards Attack and Defense Views of Rational Delegation of Computation. IEEE Access.

  11. 11.

    Tian Z, Shi W, Wang Y, Zhu C, Du X, Su S, Sun Y, Guizani N (2019) Real Time Lateral Movement Detection based on Evidence Reasoning Network for Edge Computing Environment. IEEE Trans Ind Inform.

  12. 12.

    Zhihong Tian, Shen Su, Wei Shi, Xiang Yu, Xiaojiang Du, and Mohsen Guizani. A Data-driven Model for Future Internet Route Decision Modeling. Future Gen Comput Syst. 2019.

  13. 13.

    Tian Z, Cui Y, An L, Su S, Yin X, Yin L, Cui X (2018) A Real-Time Correlation of Host-Level Events in Cyber Range Service for Smart Campus. IEEE Access 6:35355–35364.

    Article  Google Scholar 

  14. 14.

    Qingfeng Tan, Yue Gao, Jinqiao Shi, Xuebin Wang, Binxing Fang, and ZhiHong Tian. Towards a Comprehensive Insight into the Eclipse Attacks of Tor Hidden Services. IEEE Internet Things J. 2018

  15. 15.

    Zhu L, Tang X, Shen M, Xiaojiang D, Guizani M (2018) Privacy-preserving DDoS attack detection using cross-domain traffic in software defined networks. IEEE J Select Areas Commun 36(3):628–643

    Article  Google Scholar 

  16. 16.

    Tian Z, Wang Y, Sun Y, Qiu J (2019) Location privacy challenges in Mobile edge computing: classification and exploration. IEEE Netw

  17. 17.

    Xiao Y, Rayi V, Sun B, Du X, Hu F, Galloway M (2007) A survey of key management schemes in wireless sensor networks. J Comput Commun 30(11–12):2314–2341

    Article  Google Scholar 

  18. 18.

    Du X, Xiao Y, Guizani M, Chen HH (2007) An effective key management scheme for heterogeneous sensor networks. Ad Hoc Networks Elsevier 5(1):24–34

    Article  Google Scholar 

  19. 19.

    Ma Y, Wu Y, Li J, Ge J (2020) APCN: a scalable architecture for balancing accountability and privacy in large-scale content-based networks. Inf Sci., In Press

  20. 20.

    Cheng X, Wu Y, Min G, Zomaya AY (2018) Network function virtualization in dynamic networks: a stochastic perspective. IEEE J Select Areas Commun 36(10):2218–2232

    Article  Google Scholar 

  21. 21.

    Zuo Y, Wu Y, Min G, Cui L (2019) Learning-based network path planning for traffic engineering. Futur Gener Comput Syst 92:59–67.

    Article  Google Scholar 

  22. 22.

    Ma Y, Wu Y, Ge J, Li J (2018) An architecture for accountable anonymous access in the internet-of-things network. IEEE Access 6:14451–14461

    Article  Google Scholar 

  23. 23.

    Katz G, Elovici Y, Shapira B (2014) CoBAn: A context based model for data leakage prevention. J Inf Sci: Int J Arch 262:137–158

    MathSciNet  Article  Google Scholar 

  24. 24.

    Xiao Y, Du X, Zhang J, Guizani S (2007) Internet protocol television (IPTV): the killer application for the next generation internet. IEEE Commun Mag 45(11):126–134

    Article  Google Scholar 

  25. 25.

    Singh A, Patel SS (2014) Applying Modified K-Nearest Neighbor to Detect Insider Threat in Collaborative Information Systems. Int J Innovativ Res Sci Eng Technol 3(6)

  26. 26.

    Parveen P, Weger ZR, Thuraisingham B, Hamlen K, Khan L (2011) Supervised Learning for Insider Threat Detection Using Stream Mining. In: Proceeding of 23rd IEEE International Conference on Tools with Artificial Intelligence

  27. 27.

    Stolfo SJ, Apap F, Eskin E, Heller K, Hershkop S, Honig A, Svore K (2005) A comparative evaluation of two algorithms for Windows Registry Anomaly Detection. J Comput Secur 13 (4)

  28. 28.

    Hamedani K, Liu L, Rachad A, Wu J, Yi Y (2018) Reservoir Computing Meets Smart Grids: Attack Detection Using Delayed Feedback Networks. IEEE Trans Ind Inform 14:734–743.

    Article  Google Scholar 

  29. 29.

    Hodo E, Bellekens X, Hamilton A, Dubouilh P-L (2016) Threat analysis of IoT networks using artificial neural network intrusion detection system. Int Sympos Netw Comput Commun (ISNCC) 17

  30. 30.

    Panda, M., Abraham, A. & Patra, M. R. Discriminative multinomial naive bayes for network intrusion detection. 2010 6th international conference on information assurance and security, IAS 2010 5–10 (2010).

  31. 31.

    Panda M, Abraham A, Patra MR (2012) A hybrid intelligent approach for network intrusion detection. Procedia Eng 30(4):1–9

    Article  Google Scholar 

  32. 32.

    Heba FE, Darwish A, Hassanien AE, et al (2010) Principle components analysis and Support Vector Machine based Intrusion Detection System. In Proceedings of International Conference on Intelligent Systems Design and Applications. IEEE:363–367

  33. 33.

    Syarif I, Prugelbennett A, Wills G (2012) Unsupervised Clustering Approach for Network Anomaly Detection

  34. 34.

    Gogoi P, Bhuyan MH, Bhattacharyya DK et al (2012) Packet and Flow Based Network Intrusion Dataset. In: Proceedings of International Conference on Contemporary Computing. Springer, Berlin, pp 322–334

    Google Scholar 

  35. 35.

    Eid HF, Salama MA, Hassanien AE, et al. (2011) Bi-layer behavioral-based feature selection approach for network intrusion classification

  36. 36.

    Al-Ayyoub M, Nuseir A, Alsmearat K, et al. (2017) Deep learning for Arabic NLP: a survey. Journal of Computational Science

  37. 37.

    Elmisery AM, Sertovic M, Gupta BB (2017) Cognitive Privacy Middleware for Deep Learning Mashup in Environmental IoT. IEEE Access 99:1–1

    Google Scholar 

  38. 38.

    Al-Smadi M, Qawasmeh O, Al-Ayyoub M, et al. (2017) Deep Recurrent Neural Network vs. Support Vector Machine for Aspect-Based Sentiment Analysis of Arabic Hotels’ Reviews. J Comput Sci

  39. 39.

    Mozer MC (1995) A focused backpropagation algorithm for temporal pattern recognition. Backpropagation. L. Erlbaum Associates Inc.: 349–381

  40. 40.

    Werbos PJ (1990) Backpropagation through time: what it does and how to do it. Proc IEEE 78(10):1550–1560

    Article  Google Scholar 

  41. 41.

    Deutsch (2012) Supervised Sequence Labelling with Recurrent Neural Networks | Springer. Springer-Verlag, Berlin Heidelberg

    Google Scholar 

  42. 42.

    Qiu J, Chai Y, Liu Y, ZhaoQuan G, Li S, Tian Z (2018) Automatic Non-Taxonomic Relation Extraction from Big Data in Smart City. IEEE Access 6:74854–74864.

    Article  Google Scholar 

  43. 43.

    Javaid A, Niyaz Q, Sun W, et al. (2016) A Deep Learning Approach for Network Intrusion Detection System. In Proceedings of International Conference on Bio-Inspired Information and Communications Technologies. ICST Inst Comput Sci Social-Inform Telecommun Eng:21–26

  44. 44.

    Jiang F, Y Fu, Gupta BB, Lou F, Rho S (2018) Deep Learning based Multi-channel intelligent attack detection for Data Security. IEEE Trans Sustain Comput

  45. 45.

    Kim J, Kim J, Thu H L T, et al. (2016) Long Short Term Memory Recurrent Neural Network Classifier for Intrusion Detection. In: Proceedings of International Conference on Platform Technology and Service. IEEE:1–5

  46. 46.

    Vaswani A, Shazeer N, Parmar N, et al. (2017) Attention is all you need. Proc Adv Neural Inform Process Syst:5998–6008

  47. 47.

    Abadi M, Agarwal A (2015) Tensor Flow: Large-scale machine learning on heterogeneous systems. Software available from

  48. 48.

    Inglis J (1976) A mathematical theory of evidence. Technometrics 20(1):106–106

    Google Scholar 

  49. 49.

    Bengio Y, Simard P, Frasconi P (2002) Learning long-term dependencies with gradient descent is difficult. IEEE Trans Neural Netw 5(2):157–166

    Article  Google Scholar 

  50. 50.

    Chae H-s, Jo B-h, Choi S-H, Park T-k (2013) Feature Selection for Intrusion Detection using NSL-KDD. Recent Adv Comput Sci

Download references


This work is partially funded by the National Natural Science Foundation of China under Grant No. U1636215, 61871140, 61872100 and No. 61572153. And the National Key Research and Development Plan under Grant No. 2018YFB0803504. And the Guangdong Key Research and Development Plan under Grant No. 2019B010137004.

Author information



Corresponding authors

Correspondence to Jing Qiu or Yanbin Sun.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Tian, Z., Shi, W., Tan, Z. et al. Deep Learning and Dempster-Shafer Theory Based Insider Threat Detection. Mobile Netw Appl (2020).

Download citation


  • Deep learning
  • Insider threat
  • Network security
  • Recurrent neural networks