Skip to main content
SpringerLink
Log in

Detection of repetitive and irregular hypercall attacks from guest virtual machines to Xen hypervisor

  • Original Article
  • Published:
Iran Journal of Computer Science Aims and scope Submit manuscript

Abstract

Virtualization is critical to the infrastructure of cloud computing environment and other online services. Hypercall interface is provided by hypervisor to offer privileged requests by the guest domains. Attackers may use this interface to send malicious hypercalls. In the reported work, repetitive hypercall attacks and sending hypercalls within irregular sequences to Xen hypervisor were analyzed, and finally, an intrusion detection system (IDS) is proposed to detect these attacks. The proposed system is placed in the host domain (Dom0). Monitoring hypercalls traffic the system operates based on the identification of irregular behaviors in hypercalls sent from guest domains to hypervisor. Later on, the association rule algorithm is applied on the collected data within a fixed time window, and a set of thresholds for maximum number of all types of the hypercalls is extracted. The results from the implementation of the proposed system show 91% true positive rate.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

Similar content being viewed by others

Explore related subjects

Discover the latest articles, news and stories from top researchers in related subjects.

References

  1. Hashizume, K., Rosado, D., Fernández-Medina, E., Fernandez, E.: An analysis of security issues for cloud computing. J. Intern. Serv. Appl. 4, 1–13 (2013)

    Article  Google Scholar 

  2. Zheng, M.: Virtualization security in data centers and clouds. http://www.cse.wustl.edu/~jain/cse571-11/ftp/virtual/index.html (2011)

  3. von Hagen, W.: Professional Xen Virtualization. Wiley Publishing, Hoboken (2008)

    Google Scholar 

  4. Cully, B.: The virtual monkey monitor. Technical report (2006)

  5. Ormandy, T.: An empirical study into the security exposure to hosts of hostile virtualized environments. Technical report (2007)

  6. Chonka, A., Xiang, Y., Zhou, W., Bonti, A.: Cloud security defence to protect cloud computing against HTTP-DoS and XML-DoS attacks. J. Netw. Comput. Appl. 34, 1097–1107 (2011)

    Article  Google Scholar 

  7. Bacon, J., Eyers, D., Pasquier, T.F.J.M., Singh, J., Papagiannis, I., Pietzuch, P.: Information flow control for secure cloud computing. IEEE Trans. Netw. Serv. Manag. 11, 76–89 (2014)

    Article  Google Scholar 

  8. Patel, A., Taghavi, M., Bakhtiyari, K., Celestino Júnior, J.: An intrusion detection and prevention system in cloud computing: a systematic review. J. Netw. Comput. Appl. 36(1), 25–41 (2013)

    Article  Google Scholar 

  9. Chen, X., Garfinkel, T., Lewis, E.C., Subrahmanyam, P., Waldspurger, C.A., Boneh, D., et al.: Overshadow: a virtualization-based approach to retrofitting protection in commodity operating systems. ACM SIGARCH Comput. Archit. News 36, 2–13 (2008)

    Article  Google Scholar 

  10. Wang, Z., Jiang, X., Cui, W., Ning, P.: Countering kernel rootkits with lightweight hook protection. Presented at the Proceedings of the 16th ACM conference on Computer and communications security, Chicago, Illinois, USA (2009)

  11. Zhang, F., Chen, J., Chen, H., Zang, B.: CloudVisor: retrofitting protection of virtual machines in multi-tenant cloud with nested virtualization. Presented at the Proceedings of the Twenty-Third ACM Symposium on Operating Systems Principles, Cascais, Portugal (2011)

  12. Colp, P., Nanavati, M., Zhu, J., Aiello, W., Coker, G., Deegan, T., et al.: Breaking up is hard to do: security and functionality in a commodity hypervisor. Presented at the Proceedings of the Twenty-Third ACM Symposium on Operating Systems Principles, Cascais, Portugal (2011)

  13. Hoang, C.: Protecting Xen hypercalls: intrusion detection/prevention in a virtualized environment. MSc, Computer Science, University of British Columbia (2009)

  14. Jingzheng, W., Liping, D., Yuqi, L., Min-Allah, N., Wang, Y.: XenPump: a new method to mitigate timing channel in cloud computing. In: IEEE 5th International Conference on Cloud Computing (CLOUD) 2012, pp. 678–685 (2012)

  15. Bharadwaja, S., Weiqing, S., Niamat, M., Fangyang, S.: Collabra: a Xen hypervisor based collaborative intrusion detection system. In: Eighth International Conference on Information Technology: New Generations (ITNG) 2011, pp. 695–700 (2011)

  16. Wang, F., Chen, P., Mao, B., Xie, L.: RandHyp: preventing attacks via Xen hypercall interface. In: Gritzalis, D., Furnell, S., Theoharidou, M. (eds.) Information Security and Privacy Research, vol. 376, pp. 138–149. Springer, Berlin (2012)

    Chapter  Google Scholar 

  17. Yu, C., Li, L.X., Wang, K., Yu, W.T.: Protecting the security and privacy of the virtual machine through privilege separation. Appl. Mech. Mater. 347–350, 2488–2494 (2013)

    Article  Google Scholar 

  18. Li, C., Raghunathan, A. Jha, N.K.: Secure virtual machine execution under an untrusted management OS. In: IEEE 3rd International Conference on Cloud Computing, pp. 172–179 (2010)

  19. Milenkoski., Payne, B.D., Antunes, N., Vieira, M., Kounev, S.: HInjector: injecting hypercall attacks for evaluating vmi-based intrusion detection systems. Presented at the Annual Computer Security Applications Conference (ACSAC) (2013)

  20. National vulnerability database (NVD). CVE-2017-8903. https://nvd.nist.gov/vuln/detail/CVE-2017-8903 (2017)

  21. Milenkoski, Payne, B.D., Antunes, N., Vieira, M., Kounev, S.: Experience report: an analysis of hypercall handler vulnerabilities. In: IEEE 25th International Symposium on Software Reliability Engineering (ISSRE) 2014, pp. 100–111 (2014)

  22. National vulnerability database (NVD). CVE-2013-4494. https://web.nvd.nist.gov/view/vuln/detail?vulnId=CVE-2013-4494 (2013)

  23. National vulnerability database (NVD). CVE-2013-1920. https://web.nvd.nist.gov/view/vuln/detail?vulnId=CVE-2013-1920 (2013)

  24. Shropshire, J.: Hyperthreats: hypercall-based DoS attacks. In: IEEE SoutheastCon 2015, pp. 1–7 (2015)

  25. Milenkoski, Payne, B.D., Antunes, N., Vieira, M., Kounev, S., Avritzer, A., et al.: Evaluation of intrusion detection systems in virtualized environments using attack injection. In: Bos, H., Monrose, F., Blanc, G. (Eds.) Research in Attacks, Intrusions, and Defenses: 18th International Symposium, RAID 2015, Kyoto, Japan, November 2–4, 2015. Proceedings, pp. 471–492. Springer International Publishing, Cham (2015)

  26. Barham, P., Dragovic, B., Fraser, K., Hand, S., Harris, T., Ho, A., et al.: Xen and the art of virtualization. Presented at the Proceedings of the nineteenth ACM symposium on Operating systems principles, Bolton Landing, NY, USA (2003)

  27. Faggioli, D.: Tracing with Xentrace and Xenalyze. https://blog.xenproject.org/2012/09/27/tracing-with-xentrace-and-xenalyze/ (2012)

  28. Tan, P.-N., Steinbach, M., Kumar, V.: Introduction to Datamining, 1st edn. Pearson, London (2005)

    Google Scholar 

  29. Weka 3: data mining software in Java. http://www.cs.waikato.ac.nz/ml/weka/ (2013)

  30. Sedgewick, R., Wayne, K.: Algorithms, 4th edn. Addison-Wesley Professional, Boston (2011)

    Google Scholar 

  31. Rocha, F., Gross, T., Van Moorsel, A.: Defense-in-depth against malicious insiders in the cloud. In: IEEE International Conference on Cloud Engineering (IC2E) 2013, pp. 88–97 (2013)

  32. National vulnerability database (NVD). CVE-2015-7969. https://nvd.nist.gov/vuln/detail/CVE-2015-7969 (2015)

  33. National vulnerability database (NVD). CVE-2015-7971. https://nvd.nist.gov/vuln/detail/CVE-2015-7971 (2015)

Download references

Author information

Authors and Affiliations

  1. Iran University of Science and Technology (IUST), Tehran, 1684613114, Iran

    Mojtaba Mostafavi

  2. School of Computer Engineering, Iran University of Science and Technology (IUST), Tehran, 1684613114, Iran

    Peyman Kabiri

Authors
  1. Mojtaba Mostafavi
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Peyman Kabiri
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Peyman Kabiri.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Mostafavi, M., Kabiri, P. Detection of repetitive and irregular hypercall attacks from guest virtual machines to Xen hypervisor. Iran J Comput Sci 1, 89–97 (2018). https://doi.org/10.1007/s42044-018-0006-5

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s42044-018-0006-5

Keywords

Search

Navigation