Scotch: Combining Software Guard Extensions and System Management Mode to Monitor Cloud Resource Usage

Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10453)


The growing reliance on cloud-based services has led to increased focus on cloud security. Cloud providers must deal with concerns from customers about the overall security of their cloud infrastructures. In particular, an increasing number of cloud attacks target resource allocation in cloud environments. For example, vulnerabilities in a hypervisor scheduler can be exploited by attackers to effectively steal CPU time from other benign guests on the same hypervisor. In this paper, we present Scotch, a system for transparent and accurate resource consumption accounting in a hypervisor. By combining x86-based System Management Mode with Intel Software Guard Extensions, we can ensure the integrity of our accounting information, even when the hypervisor has been compromised by an escaped malicious guest. We show that we can account for resources at every task switch and I/O interrupt, giving us richly detailed resource consumption information for each guest running on the hypervisor. We show that using our system incurs small but manageable overhead—roughly 1 \(\upmu \)s every task switch or I/O interrupt. We further discuss performance improvements that can be made for our proposed system by performing accounting at random intervals. Finally, we discuss the viability of this approach against multiple types of cloud-based resource attacks.


Software Guard Extensions (SGX) System Management Module (SMM) Scotch Malicious Guest Resource Consumption Information 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Supplementary material

440190_1_En_18_MOESM1_ESM.txt (1 kb)
Supplementary material 1 (txt 1 KB)


  1. 1.
  2. 2.
    NSA’s ANT Division Catalog of Exploits for Nearly Every Major Software/Hardware/Firmware.
  3. 3.
  4. 4.
    Amazon AWS: Amazon CloudWatchamazon cloudwatch.
  5. 5.
    AMD: AMD RS800 ASIC family BIOS developer’s guide (2010)Google Scholar
  6. 6.
    AMD. AMD64 architecture programmer’s manual, Volume 2: System Programming (2013)Google Scholar
  7. 7.
    Arnautov, S., Trach, B., Gregor, F., Knauth, T., Martin, A., Priebe, C., Lind, J., Muthukumaran, D., O’Keeffe, D., Stillwell, M.L., et al.: SCONE: secure Linux containers with Intel SGX. In: 12th USENIX Symposium Operating Systems Design and Implementation (2016)Google Scholar
  8. 8.
    Azab, A.M., Ning, P., Wang, Z., Jiang, X., Zhang, X., Skalsky, N.C.: HyperSentry: enabling stealthy in-context measurement of hypervisor integrity. In: Proceedings of the 17th ACM Conference on Computer and Communications Security (CCS 2010) (2010)Google Scholar
  9. 9.
    Bates, A., Mood, B., Pletcher, J., Pruse, H., Valafar, M., Butler, K.: Detecting co-residency with active traffic analysis techniques. In: Proceedings of the 2012 ACM Workshop on Cloud computing security workshop, pp. 1–12. ACM (2012)Google Scholar
  10. 10.
    Baumann, A., Peinado, M., Hunt, G.: Shielding applications from an untrusted cloud with haven. ACM Trans. Comput. Syst. (TOCS) 33(3), 8 (2015)CrossRefGoogle Scholar
  11. 11.
    Bienia, C., Kumar, S., Singh, J.P., Li, K.: The PARSEC benchmark suite: characterization and architectural implications. In: Proceedings of the 17th International Conference on Parallel Architectures and Compilation Techniques, pp. 72–81. ACM (2008)Google Scholar
  12. 12.
    Chen, C., Maniatis, P., Perrig, A., Vasudevan, A., Sekar, V.: Towards verifiable resource accounting for outsourced computation. In: Proceedings of the 9th ACM SIGPLAN/SIGOPS International Conference on Virtual Execution Environments (VEE 2013) (2014)Google Scholar
  13. 13.
    Cherkasova, L., Gupta, D., Vahdat, A.: Comparison of the three CPU schedulers in Xen. SIGMnfluencingformance Eval. Rev. 35(2), 42–51 (2007)CrossRefGoogle Scholar
  14. 14.
    Columbus, L.: Roundup of cloud computing forecasts and market estimates (2016).
  15. 15.
    Common Vulnerability Database: VENOM: CVE-2015-3456, Xen 4.5 VM escape attack (2015)Google Scholar
  16. 16.
    Coreboot: Open-Source BIOS.
  17. 17.
    Domas, C.: The memory sinkhole. BlackHat, USA (2015)Google Scholar
  18. 18.
    Dragovic, B., Fraser, K., Hand, S., Harris, T., Ho, A., Pratt, I., Warfield, A., Barham, P., Neugebauer, R.: Xen and the art of virtualization. In: Proceedings of the ACM Symposium on Operating Systems Principles (2003)Google Scholar
  19. 19.
    Embleton, S., Sparks, S., Zou, C.: SMM rootkits: a new breed of OS independent malware. In: Proceedings of the 4th International Conference on Security and Privacy in Communication Networks (SecureComm 2008) (2008)Google Scholar
  20. 20.
    Garcia, A.: Target settles for $39 million over data breach (2015).
  21. 21.
    Hunt, T., Zhu, Z., Xu, Y., Peter, S., Witchel, E.: Ryoan: a distributed sandbox for untrusted computation on secret data. In: 12th USENIX Symposium on Operating Systems Design and Implementation (OSDI 2016), pp. 533–549. USENIX Association (2016)Google Scholar
  22. 22.
    Inci, M.S., Gulmezoglu, B., Irazoqui, G., Eisenbarth, T., Sunar, B.: Seriously, get off my cloud! Cross-VM RSA key recovery in a public cloud. Technical report, IACR Cryptology ePrint Archive (2015)Google Scholar
  23. 23.
    Intel: Intel software guard extensions programming reference (2014).
  24. 24.
    Jin, S., Seol, J., Huh, J., Maeng, S.: Hardware-assisted Secure Resource Accounting under a Vulnerable Hypervisor. In: Proceedings of the 11th ACM SIGPLAN/SIGOPS International Conference on Virtual Execution Environments (VEE 2015) (2015)Google Scholar
  25. 25.
    Kallenberg, C., Kovah, X.: How many million bioses would you like to infect? (2015).
  26. 26.
    Kelion, L.: Apple toughens iCloud security after celebrity breach (2014).
  27. 27.
    Kortchinsky, K.: CLOUDBURST: a VMware guest to host escape story. In: Black Hat USA (2009)Google Scholar
  28. 28.
    Leach, K., Spensky, C., Weimer, W., Zhang, F.: Towards transparent introspection. In: 23rd IEEE International Conference on Software Analysis, Evolution and Reengineering (2016)Google Scholar
  29. 29.
    National Institute of Standards, NIST: National vulnerability database. Accessed 10 May 2016
  30. 30.
    Prakash, A., Venkataramani, E., Yin, H., Lin. Z.: Manipulating semantic values in kernel data structures: attack assessments and implications. In: 2013 43rd Annual IEEE/IFIP International Conference on Dependable Systems and Networks (DSN), pp. 1–12. IEEE (2013)Google Scholar
  31. 31.
    Ren, G., Tune, E., Moseley, T., Shi, Y., Rus, S., Hundt, R., Profiling, G.-W.: A continuous profiling infrastructure for data centers. IEEE Micro (2010)Google Scholar
  32. 32.
    Rong, H., Xian, M., Wang, H., Shi, J.: Time-stealer: a stealthy threat for virtualization scheduler and its countermeasures. In: Qing, S., Zhou, J., Liu, D. (eds.) ICICS 2013. LNCS, vol. 8233, pp. 100–112. Springer, Cham (2013). doi: 10.1007/978-3-319-02726-5_8 CrossRefGoogle Scholar
  33. 33.
    Schiffman, J., Kaplan, D.: The SMM rootkit revisited: fun with USB. In: Proceedings of 9th International Conference on Availability, Reliability and Security (ARES 2014) (2014)Google Scholar
  34. 34.
    Schuster, F., Costa, M., Fournet, C., Gkantsidis, C., Peinado, M., Mainar-Ruiz, G., Russinovich, M.: Vc3: trustworthy data analytics in the cloud using SGX. In: 2015 IEEE Symposium on Security and Privacy (SP), pp. 38–54. IEEE (2015)Google Scholar
  35. 35.
    Varadarajan, V., Kooburat, T., Farley, B., Ristenpart, T., Swift, M.M.: Resource-freeing attacks: improve your cloud performance (at your neighbor’s expense). In: Proceedings of the 2012 ACM conference on Computer and communications security, pp. 281–292. ACM (2012)Google Scholar
  36. 36.
    VMware Inc.: vCenter chargeback manager.
  37. 37.
    Wang, H., Jing, Q., Chen, R., He, B., Qian, Z., Zhou, L.: Distributed systems meet economics: pricing in the cloud. HotCloud 10, 1–6 (2010)Google Scholar
  38. 38.
    Wang, J., Sun, K., Stavrou, A.: A dependability analysis of hardware-assisted polling integrity checking systems. In: Proceedings of the 42nd Annual IEEE/IFIP International Conference on Dependable Systems and Networks (DSN 2012) (2012)Google Scholar
  39. 39.
    Wang, L., Zhan, J., Luo, C., Zhu, Y., Yang, Q., He, Y., Gao, W., Jia, Z., Shi, Y., Zhang, S., et al.: Bigdatabench: a big data benchmark suite from internet services. In: 2014 IEEE 20th International Symposium on High Performance Computer Architecture (HPCA), pp. 488–499. IEEE (2014)Google Scholar
  40. 40.
    Weiser, S., Werner, M.: SGXIO: generic trusted I/O path for Intel SGX. In: Proceedings of the Seventh ACM on Conference on Data and Application Security and Privacy (CODASPY 2017), pp. 261–268, New York. ACM (2017)Google Scholar
  41. 41.
    Wojtczuk, R., Rutkowska, J.: Attacking Intel trust execution technologies (2009).
  42. 42.
    Wojtczuk, R., Rutkowska, J.: Attacking SMM memory via Intel CPU cache poisoning (2009)Google Scholar
  43. 43.
    Zhang, F., Leach, K., Sun, K., Stavrou, A.: SPECTRE: a dependable introspection framework via system management mode. In: Proceedings of the 43rd Annual IEEE/IFIP International Conference on Dependable Systems and Networks (DSN 2013) (2013)Google Scholar
  44. 44.
    Zhang, F., Leach, K., Wang, H., Stavrou, A.: Trustlogin: securing password-login on commodity operating systems. In: Proceedings of the 10th ACM Symposium on Information, Computer and Communications Security, pp. 333–344. ACM (2015)Google Scholar
  45. 45.
    Zhang, F., Leach, K., Wang, H., Stavrou, A., Sun, K.: Using hardware features for increased debugging transparency. In: Proceedings of the 36th IEEE Symposium on Security and Privacy (2015)Google Scholar
  46. 46.
    Zhang, F., Wang, H., Leach, K., Stavrou, A.: A framework to secure peripherals at runtime. In: Kutyłowski, M., Vaidya, J. (eds.) ESORICS 2014. LNCS, vol. 8712, pp. 219–238. Springer, Cham (2014). doi: 10.1007/978-3-319-11203-9_13 Google Scholar
  47. 47.
    Zhang, F., Wang, J., Sun, K., Stavrou, A.: HyperCheck: a hardware-assisted integrity monitor. In: IEEE Transactions on Dependable and Secure Computing (2013)Google Scholar
  48. 48.
    Zhang, T., Zhang, Y., Lee, R.B.: Memory dos attacks in multi-tenant clouds: Severity and mitigation. arXiv preprint arXiv:1603.03404 (2016)
  49. 49.
    Zhou, F., Goel, M., Desnoyers, P., Sundaram, R.: Scheduler vulnerabilities and coordinated attacks in cloud computing. J. Comput. Secur. 21(4), 533–559 (2013)CrossRefGoogle Scholar

Copyright information

© Springer International Publishing AG 2017

Authors and Affiliations

  1. 1.University of VirginiaCharlottesvilleUSA
  2. 2.Wayne State UniversityDetroitUSA
  3. 3.University of MichiganAnn ArborUSA

Personalised recommendations