Skip to main content

Detecting Malicious Sessions Through Traffic Fingerprinting Using Hidden Markov Models

Part of the Lecture Notes of the Institute for Computer Sciences, Social Informatics and Telecommunications Engineering book series (LNICST,volume 152)

Abstract

Almost any malware attack involves data communication between the infected host and the attacker host/server allowing the latter to remotely control the infected host. The remote control is achieved through opening different types of sessions such as remote desktop, webcam video streaming, file transfer, etc. In this paper, we present a traffic analysis based malware detection technique using Hidden Markov Model (HMM). The main contribution is that the proposed system does not only detect malware infections but also identifies with precision the type of malicious session opened by the attacker. The empirical analysis shows that the proposed detection system has a stable identification precision of 90 % and that it allows to identify between 40 % and 75 % of all malicious sessions in typical network traffic.

Keywords

  • Malware detection
  • Hidden Markov Model (HMM)
  • Malicious sessions
  • Traffic analysis

This is a preview of subscription content, access via your institution.

Buying options

Chapter
USD   29.95
Price excludes VAT (USA)
  • DOI: 10.1007/978-3-319-23829-6_47
  • Chapter length: 9 pages
  • Instant PDF download
  • Readable on all devices
  • Own it forever
  • Exclusive offer for individuals only
  • Tax calculation will be finalised during checkout
eBook
USD   69.99
Price excludes VAT (USA)
  • ISBN: 978-3-319-23829-6
  • Instant PDF download
  • Readable on all devices
  • Own it forever
  • Exclusive offer for individuals only
  • Tax calculation will be finalised during checkout
Softcover Book
USD   89.99
Price excludes VAT (USA)
Fig. 1.
Fig. 2.
Fig. 3.
Fig. 4.
Fig. 5.
Fig. 6.

Notes

  1. 1.

    Malware and Bot will be used interchangeably.

  2. 2.

    For large scale systems, the honeypot machine can be replaced by a full honeyNet network.

  3. 3.

    All the experiments were carried out using virtual machines both for the infected host and the attacker/C&C server.

References

  1. Siroski, M., Honig, A.: Practical Malware Analysis: The Hands-On Guide to Dissecting Malicious Software. No Starch Press, San Francisco (2012)

    Google Scholar 

  2. Falliere, N., Murchu, L., Chien, E.: W32.stuxnet dossier. Technical report, Symantec Security Response, February 2011

    Google Scholar 

  3. Gostev, A.: The flame: Questions and answers. Technical report, Kaspersky, May 2012

    Google Scholar 

  4. Bencsáth, B., Pék, G., Buttyán, L., Félegyházi, M.: Duqu: analysis, detection, and lessons learned. In: ACM European Workshop on System Security (EuroSec). ACM (2012)

    Google Scholar 

  5. Leyden, J.: Hack on Saudi Aramco hit 30,000 workstations, oil firm admits (2012). http://www.theregister.co.uk/2012/08/29/

  6. Sun, Q., Simon, D.R., Wang, Y.M., Russell, W., Padmanabhan, V.N., Qiu, L.: Statistical identification of encrypted web browsing traffic. In: Proceedings of the 2002 IEEE Symposium on Security and Privacy, SP 2002, p. 19. IEEE Computer Society, Washington, DC (2002)

    Google Scholar 

  7. Liberatore, M., Levine, B.N.: Inferring the source of encrypted http connections. In: Proceedings of the 13th ACM conference on Computer and Communications Security, CCS 2006, pp. 255–263. ACM, New York (2006)

    Google Scholar 

  8. Herrmann, D., Wendolsky, R., Federrath, H.: Website fingerprinting: attacking popular privacy enhancing technologies with the multinomial naive-bayes classifier. In: Proceedings of the 2009 ACM Workshop on Cloud Computing Security, CCSW 2009, pp. 31–42. ACM, New York (2009)

    Google Scholar 

  9. Panchenko, A., Niessen, L., Zinnen, A., Engel, T.: Website fingerprinting in onion routing based anonymization networks. In: Proceedings of the 10th Annual ACM Workshop on Privacy in the Electronic Society, WPES 2011, pp. 103–114. ACM, New York (2011)

    Google Scholar 

  10. Cai, X., Zhang, X.C., Joshi, B., Johnson, R.: Touching from a distance: website fingerprinting attacks and defenses. In: Proceedings of the 2012 ACM Conference on Computer and Communications Security, CCS 2012, pp. 605–616. ACM, New York (2012)

    Google Scholar 

  11. Wang, T., Goldberg, I.: Improved website fingerprinting on tor. In: 12th ACM Workshop on Privacy in the Electronic Society, WPES 2013. ACM (2013)

    Google Scholar 

  12. Dingledine, R., Mathewson, N., Syverson, P.: Tor : the second-generation onion router. In: Proceedings of the 13th Usenix Security Symposium, August 2004

    Google Scholar 

  13. prorat trojan. http://en.wikipedia.org/wiki/ProRat

  14. Rabiner, L.: A tutorial on hidden Markov models and selected applications in speech recognition. Proc. IEEE 77(2), 257–286 (1989)

    CrossRef  Google Scholar 

  15. Durbin, R., Eddy, S.: Biological Sequence Analysis: Probabilistic Models of Proteins and Nucleic Acids. Cambridge University Press, Cambridge (1998)

    CrossRef  MATH  Google Scholar 

  16. Kohavi, R.: A study of cross-validation and bootstrap for accuracy estimation and model selection. In: Proceedings of the International Joint Conference on Artificial Intelligence (IJCAI), pp. 1137–1143 (1995)

    Google Scholar 

Download references

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Sami Zhioua .

Editor information

Editors and Affiliations

Rights and permissions

Reprints and Permissions

Copyright information

© 2015 Institute for Computer Sciences, Social Informatics and Telecommunications Engineering

About this paper

Cite this paper

Zhioua, S., Jabeur, A.B., Langar, M., Ilahi, W. (2015). Detecting Malicious Sessions Through Traffic Fingerprinting Using Hidden Markov Models. In: Tian, J., Jing, J., Srivatsa, M. (eds) International Conference on Security and Privacy in Communication Networks. SecureComm 2014. Lecture Notes of the Institute for Computer Sciences, Social Informatics and Telecommunications Engineering, vol 152. Springer, Cham. https://doi.org/10.1007/978-3-319-23829-6_47

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-23829-6_47

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-23828-9

  • Online ISBN: 978-3-319-23829-6

  • eBook Packages: Computer ScienceComputer Science (R0)