Detecting Anomalous Behaviour Using Heterogeneous Data

Conference paper
Part of the Advances in Intelligent Systems and Computing book series (AISC, volume 513)


In this paper, we propose a method to detect anomalous behaviour using heterogenous data. This method detects anomalies based on the recently introduced approach known as Recursive Density Estimation (RDE) and the so called eccentricity. This method does not require prior assumptions to be made on the type of the data distribution. A simplified form of the well-known Chebyshev condition (inequality) is used for the standardised eccentricity and it applies to any type of distribution. This method is applied to three datasets which include credit card, loyalty card and GPS data. Experimental results show that the proposed method may simplify the complex real cases of forensic investigation which require processing huge amount of heterogeneous data to find anomalies. The proposed method can simplify the tedious job of processing the data and assist the human expert in making important decisions. In our future research, more data will be applied such as natural language (e.g. email, Twitter, SMS) and images.


Heterogeneous data Anomaly detection RDE Eccentricity 



The first author would like to acknowledge the support from the Ministry of Education Malaysia and Universiti Teknologi MARA, Malaysia for the study grant. The second author would like to acknowledge the New Machine Learning Methods grant from The Royal Society (Grant number IE141329/2014).


  1. 1.
    Ernst & Young: Forensic Data Analytics (2013)Google Scholar
  2. 2.
    IDC: Where in the World is Storage: A Look at Byte Density Across the Globe (2013)Google Scholar
  3. 3.
    Jagadish, H.V., Gehrke, J., Labrinidis, A., Papakonstantinou, Y., Patel, J.M., Ramakrishnan, R., Shahabi, C.: Big data and its technical challenges. Commun. ACM 57(7), 86–94 (2014)CrossRefGoogle Scholar
  4. 4.
    Turcsany, D., Bargiela, A., Maul, T.: Local receptive field constrained deep networks. Inf. Sci. (Ny) 349–350, 229–247 (2016)CrossRefGoogle Scholar
  5. 5.
    Principe, B.J.C., Chalasani, R.: Cognitive architectures for sensory processing. In: Proceeding IEEE, vol. 102(4) (2014)Google Scholar
  6. 6.
    Maldonado, S., L’Huillier, G.: SVM-based feature selection and classification for email filtering. Pattern Recogn. Appl. Meth. 204, 1–11 (2013)CrossRefGoogle Scholar
  7. 7.
    Angelov, P., Sadeghi-Tehran, P.: A nested hierarchy of dynamically evolving clouds for big data structuring and searching. Procedia—Procedia Comput. Sci. 53, 1–8 (2015)CrossRefGoogle Scholar
  8. 8.
    Borgman, C.L.: Scholarship in the Digital Age: Information, Infrastructure and the Internet. The MIT Press, (2007)Google Scholar
  9. 9.
    Vincent, J.: Advent of electronic digital computing. IEEE Ann. Hist. Comput. 6(3), 229–282 (1984)MathSciNetCrossRefzbMATHGoogle Scholar
  10. 10.
    Mearian, L.: Data storage: then and now. Computerworld (2014)Google Scholar
  11. 11.
    Kitchin, R.: The Data Revolution: Big Data, Open Data. SAGE Publications Ltd, Data Infrastructures and Their Consequences (2014)CrossRefGoogle Scholar
  12. 12.
    Angelov, P.: Anomaly detection based on eccentricity analysis. In: 2014 IEEE Symposium Series on Computational Intelligence (2014)Google Scholar
  13. 13.
    Chandola, V., Banerjee, A., Kumar, V.: Anomaly detection. ACM Comput. Surv. 41(3), 1–58 (2009)CrossRefGoogle Scholar
  14. 14.
    Lughofer, E., Angelov, P.: Handling drifts and shifts in on-line data streams with evolving fuzzy systems. Appl. Soft Comput. J. 11(2), 2057–2068 (2011)CrossRefGoogle Scholar
  15. 15.
    Om, H., Kundu, A.: A hybrid system for reducing the false alarm rate of anomaly intrusion detection system. In: 2012 1st International Conference on Recent Advance in Information Technology RAIT-2012, pp. 131–136 (2012)Google Scholar
  16. 16.
    Kim, Y., Kogan, A.: Development of an anomaly detection model for a bank’s transitory account system. J. Inf. Syst. 28(1), 145–165 (2014)Google Scholar
  17. 17.
    Delgado, B., Tahboub, K., Delp, E.J.: Automatic Detection of Abnormal Human Events on Train Platforms no. 2009, pp. 169–173 (2014)Google Scholar
  18. 18.
    Wu, Y., Patterson, A., Santos, R.D.C., Vijaykumar, N.L.: Topology Preserving Mapping for Maritime Anomaly Detection, pp. 313–326 (2014)Google Scholar
  19. 19.
  20. 20.
    Kang, M., Islam, R., Kim, J., Kim, J., Pecht, M.: A Hybrid Feature Selection Scheme for Reducing Diagnostic Performance Deterioration Caused by Outliers in Data-Driven Diagnostics, vol. 63(5), pp. 3299–3310 (2016)Google Scholar
  21. 21.
    Hawkins, D.M.: Identification of Outliers. Chapman & Hall (1980)Google Scholar
  22. 22.
    Khaleghi, B., Khamis, A., Karray, F.O., Razavi, S.N.: Multisensor data fusion: a review of the state-of-the-art. Inf. Fusion 14(1), 28–44 (2013)CrossRefGoogle Scholar
  23. 23.
    Castanedo, F.: A review of data fusion techniques. Sci. World J. 2013 (2013)Google Scholar
  24. 24.
    Lahat, D., Adali, T., Jutten, C.: Multimodal data fusion: an overview of methods, challenges, and prospects. Proc. IEEE 103(9), 1449–1477 (2015)CrossRefGoogle Scholar
  25. 25.
    Angelov, P.: Evolving fuzzy systems. Comput. Complex. Theor. Tech. Appl. 2(2), 1053–1065 (2012)Google Scholar
  26. 26.
    Angelov, P., Ramezani, R., Zhou, X.: Autonomous Novelty Detection and Object Tracking in Video Streams using Evolving Clustering and Takagi-Sugeno type Neuro-Fuzzy System, pp. 1456–1463 (2008)Google Scholar
  27. 27.
    Costa, B.S.J., Angelov, P.P., Guedes, L.A.: Real-time fault detection using recursive density estimation. J. Control. Autom. Electr. Syst. 25(4), 428–437 (2014)CrossRefGoogle Scholar
  28. 28.
    Iglesias, J.A., Angelov, P., Ledezma, A., Sanchis, A.: Creating evolving user behavior profiles automatically. IEEE Trans. Knowl. Data Eng. 24(5), 854–867 (2012)CrossRefGoogle Scholar
  29. 29.
    Angelov, P.: Typicality distribution function—a new density—based data analytics tool. In: IJCNN 2015 International Joint Conference on Neural Networks (2015)Google Scholar
  30. 30.
    Angelov, P., Xiaowei, G., Kangin, D., Principe, J.: Empirical data analysis: a new tool for data analytics. In: IEEE International Conference on Systems, Man, and Cybernetics (2016)Google Scholar
  31. 31.
    Zheng, Y.U.: Trajectory Data Mining : An Overview, vol. 6(3), pp. 1–41 (2015)Google Scholar
  32. 32.
    Sadeghi-Tehran, P., Angelov, P.: A real-time approach for novelty detection and trajectories analysis for anomaly recognition in video surveillance systems. In: 2012 IEEE Conference on Evolving and Adaptive Intelligent Systems (EAIS), pp. 108–113 (2013)Google Scholar
  33. 33.
    Angelov, P.: Outside the box: an alternative data analytics frame-work. J. Autom. Mob. Robot. Intell. Syst. 8, 35–42 (2013)Google Scholar

Copyright information

© Springer International Publishing AG 2017

Authors and Affiliations

  1. 1.Data Science Group, School of Computing and CommunicationLancaster UniversityLancasterUK

Personalised recommendations