Expanding Malware Defense by Securing Software Installations

  • Weiqing Sun
  • R. Sekar
  • Zhenkai Liang
  • V. N. Venkatakrishnan
Part of the Lecture Notes in Computer Science book series (LNCS, volume 5137)


Software installation provides an attractive entry vector for malware: since installations are performed with administrator privileges, malware can easily get the enhanced level of access needed to install backdoors, spyware, rootkits, or “bot” software, and to hide these installations from users. Previous research has been focused mainly on securing the execution phase of untrusted software, while largely ignoring the safety of installations. Even security-enhanced operating systems such as SELinux and Vista don’t usually impose restrictions during software installs, expecting the system administrator to “know what she is doing.” This paper addresses this “gap in armor” by securing software installations. Our technique can support a diversity of package managers and software installers. It is based on a framework that simplifies the development and enforcement of policies that govern safety of installations. We present a simple policy that can be used to prevent untrusted software from modifying any of the files used by benign software packages, thus blocking the most common mechanism used by malware to ensure that it is run automatically after each system reboot. While the scope of our technique is limited to the installation phase, it can be easily combined with approaches for secure execution, e.g., by ensuring that all future runs of an untrusted package will take place within an administrator-specified sandbox. Our experimental evaluation has considered over one hundred benign and untrusted software packages. Our technique was able to block malicious packages among these without breaking non-malicious ones.


Untrusted code Malicious code Software installation Sandboxing 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
  2. 2.
    Linux v-server,
  3. 3.
  4. 4.
    Acharya, A., Raje, M.: Mapbox: Using parameterized behavior classes to confine applications. In: USENIX Security Symposium (2000)Google Scholar
  5. 5.
  6. 6.
    Alexandrov, A., Kmiec, P., Schauser, K.: Consh: A confined execution environment for internet computations (1998)Google Scholar
  7. 7.
    Altiris. Software virtualization solution (2005),
  8. 8.
    Badger, L., Sterne, D.F., Sherman, D.L., Walker, K.M., Haghighat, S.A.: A domain and type enforcement unix prototype. In: USENIX Computing Systems, pp. 127–140 (1995)Google Scholar
  9. 9.
    Barham, P., Dragovic, B., Fraser, K., Hand, S., Harris, T., Ho, A., Neugebauer, R., Pratt, I., Warfield, A.: Xen and the art of virtualization. In: ACM Symposium on Operating systems principles, pp. 164–177 (2003)Google Scholar
  10. 10.
    Boebert, W.E., Kain, R.Y.: A practical alternative to hierarchical integrity policies. In: Proceedings of the 8th National Computer Security Conference, pp. 18–27 (1985)Google Scholar
  11. 11.
    Chien, E.: Techniques of adware and spyware. Symantec (April 2005)Google Scholar
  12. 12.
    Dan, A., Mohindra, A., Ramaswami, R., Sitaram, D.: Chakravyuha: A sandbox operating system for the controlled execution of alien code. Technical report, IBM T.J. Watson research center (1997)Google Scholar
  13. 13.
    Dike, J.: A User-Mode port of the linux kernel. In: Proceedings of the 4th Annual Showcase and Conference (LINUX 2000), Berkeley, CA, October 10–14, 2000, pp. 63–72 (2000)Google Scholar
  14. 14.
    Dolstra, E., de Jonge, M., Visser, E.: Nix: A safe and policy-free system for software deployment. In: LISA, pp. 79–92 (2004)Google Scholar
  15. 15.
    Eduardo, F.: Checkinstall (2004),
  16. 16.
  17. 17.
    Goldberg, I., Wagner, D., Thomas, R., Brewer, E.A.: A secure environment for untrusted helper applications: confining the wily hacker. In: USENIX Security Symposium (1996)Google Scholar
  18. 18.
    Hsu, F., Ristenpart, T., Chen, H.: Back to the future: A framework for automatic malware removal and system repair. In: Jesshope, C., Egan, C. (eds.) ACSAC 2006. LNCS, vol. 4186. Springer, Heidelberg (2006)CrossRefGoogle Scholar
  19. 19.
    Kamp, P.H., Watson, R.N.M.: Jails: Confining the omnipotent root. In: Proceedings of the 2nd International SANE Conference (2000)Google Scholar
  20. 20.
    Kato, K., Oyama, Y.: Softwarepot: An encapsulated transferable file system for secure software circulation. In: Okada, M., Pierce, B.C., Scedrov, A., Tokuda, H., Yonezawa, A. (eds.) ISSS 2002. LNCS, vol. 2609, pp. 112–132. Springer, Heidelberg (2003)CrossRefGoogle Scholar
  21. 21.
    Li, N., Mao, Z., Chen, H.: Usable mandatory integrity protection for operating systems. In: IEEE Symposium on Security and Privacy (2007)Google Scholar
  22. 22.
    Liang, Z., Venkatakrishnan, V.N., Sekar, R.: Isolated program execution: An application transparent approach for executing untrusted programs. In: Omondi, A.R., Sedukhin, S. (eds.) ACSAC 2003. LNCS, vol. 2823, pp. 182–191. Springer, Heidelberg (2003)Google Scholar
  23. 23.
    Loscocco, P., Smalley, S.: Integrating flexible support for security policies into the Linux o perating system. In: Proc. FREENIX track of the 2001 Usenix Annual Technical Conference (2001)Google Scholar
  24. 24.
    PHCN. Fedora-redhat fake security alert / trojan source code analysis (2004),
  25. 25.
    Prevelakis, V., Spinellis, D.: Sandboxing applications. In: Proceedings of Usenix Annual Technical Conference: FREENIX Track (2001)Google Scholar
  26. 26.
    Price, D., Tucker, A.: Solaris zones: Operating system support for consolidating commercial workloads. In: LISA, pp. 241–254. USENIX (2004)Google Scholar
  27. 27.
    Provos, N.: Improving host security with system call policies. In: Proceedings of the 11th USENIX Security Symposium, pp. 257–272 (2003)Google Scholar
  28. 28.
    Safford, D., Zohar, M.: A trusted linux client (tlc) (2005)Google Scholar
  29. 29.
    Schneider, F.B.: Enforceable security policies. ACM Transactions on Information and System Security 3(1), 30–50 (2000)CrossRefGoogle Scholar
  30. 30.
    Scott, K., Davidson, J.: Safe virtual execution using software dynamic translation. In: Proceedings of Annual Computer Security Applications Conference (2002)Google Scholar
  31. 31.
    Sekar, R., Venkatakrishnan, V.N., Basu, S., Bhatkar, S., DuVarney, D.C.: Model carrying code: a practical approach for safe execution of untrusted applications. In: Proceedings of 19th ACM symposium of Operating Systems Principles (SOSP), Bolton Landing, New York (October 2003)Google Scholar
  32. 32.
    Sun, W., Liang, Z., Venkatakrishnan, V.N., Sekar, R.: One-way isolation: An effective approach for realizing safe execution environments. In: NDSS (2005)Google Scholar
  33. 33.
    Sun, W., Sekar, R., Poothia, G., Karandikar, T.: Practical proactive integrity preservation: A basis for malware defense. In: IEEE Symposium on Security and Privacy (May 2008)Google Scholar
  34. 34.
    Venkatakrishnan, V.N., Sekar, R., Kamat, T., Tsipa, S., Liang, Z.: An approach for secure software installation. In: Proceedings of the 16th Systems Administration Conference (LISA 2002), Philadelphia, PA, November  3-8, 2002, pp. 219–226 (2002)Google Scholar
  35. 35.
    Walters, B.: VMware virtual platform. j-LINUX-J 63 (July 1999)Google Scholar
  36. 36.
    Young, W.D., Telega, P.A., Boebert, W.E., Kain, R.Y.: A verified labeler for the Secure Ada Target. In: Proc. National Computer Security Conference, pp. 55–61 (1986)Google Scholar
  37. 37.
    Yu, Y., Guo, F., Nanda, S., Lam, L.c., Chiueh, T.c.: A feather-weight virtual machine for windows applications. In: Proceedings of the 2nd ACM/USENIX Conference on Virtual Execution Environments (VEE 2006) (June 2006)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2008

Authors and Affiliations

  • Weiqing Sun
    • 1
  • R. Sekar
    • 1
  • Zhenkai Liang
    • 2
  • V. N. Venkatakrishnan
    • 3
  1. 1.Department of Computer ScienceStony Brook University 
  2. 2.Department of Computer ScienceCarnegie Mellon University 
  3. 3.Department of Computer ScienceUniversity of IllinoisChicago 

Personalised recommendations