Discovering Application-Level Insider Attacks Using Symbolic Execution

  • Karthik Pattabiraman
  • Nithin Nakka
  • Zbigniew Kalbarczyk
  • Ravishankar Iyer
Part of the IFIP Advances in Information and Communication Technology book series (IFIPAICT, volume 297)


This paper presents a technique to systematically discover insider attacks in applications. An attack model where the insider is in the same address space as the process and can corrupt arbitrary data is assumed. A formal technique based on symbolic execution and model-checking is developed to comprehensively enumerate all possible insider attacks corresponding to a given attack goal. The main advantage of the technique is that it operates directly on the program code in assembly language and no manual effort is necessary to translate the program into a formal model. We apply the technique to security-critical segments of the OpenSSH application.


  1. 1.
    Randazzo, M.R., et al.: Insider Threat Study: Illicit Cyber Activity in the Banking and Finance Sector, p. 25. ERT Coordination Center/Software Engineering Institute, Philadelphia, PA (2004)Google Scholar
  2. 2.
    Keeney, M.M., Kowalski, E.F.: Insider Threat Study: Computer System Sabotage in Critical Infrastructure Sectors. CERT/CC, Philadelphia, PA (2005)Google Scholar
  3. 3.
    Chinchani, R., et al.: Towards a Theory of Insider Threat Assessment. In: Proceedings of the 2005 International Conference on Dependable Systems and Networks. IEEE Computer Society, Los Alamitos (2005)Google Scholar
  4. 4.
    Probst, C.W., Hansen, R.R., Nielson, F.: Where Can an Insider Attack? In: Dimitrakos, T., Martinelli, F., Ryan, P.Y.A., Schneider, S. (eds.) FAST 2006. LNCS, vol. 4691, pp. 127–142. Springer, Heidelberg (2007)CrossRefGoogle Scholar
  5. 5.
    Pattabiraman, K., Nakka, N., Kalbarczyk, Z.: SymPLFIED: Symbolic Program Level Fault-Injection and Error-Detection Framework. In: International Conference on Dependable Systems and Networks (DSN) (2008)Google Scholar
  6. 6.
    OpenSSH Development Team., OpenSSH 4.21 (2004)Google Scholar
  7. 7.
    Clavel, M., et al.: The Maude 2.0 System. In: Rewriting Technologies and Applications. Springer, Heidelberg (2001)Google Scholar
  8. 8.
    Pattabiraman, K., et al.: Discovering Application-level Insider Attacks using Symbolic Execution, CRHC Technical Report, UIUC, Champaign, IL (2008)Google Scholar
  9. 9.
    Phillips, C., Swiler, L.P.: A graph-based system for network-vulnerability analysis. In: Proceedings of the 1998 workshop on New security paradigms. ACM, Charlottesville (1998)Google Scholar
  10. 10.
    Ammann, P., Wijesekera, D., Kaushik, S.: Scalable, graph-based network vulnerability analysis. In: Proceedings of the 9th ACM conference on Computer and communications security. ACM, Washington (2002)Google Scholar
  11. 11.
    Sheyner, O., et al.: Automated Generation and Analysis of Attack Graphs. In: Proceedings of the 2002 IEEE Symposium on Security and Privacy. IEEE Computer Society, Los Alamitos (2002)Google Scholar
  12. 12.
    King, J.C.: Symbolic execution and program testing. Commun. ACM 19(7), 385–394 (1976)MathSciNetCrossRefMATHGoogle Scholar
  13. 13.
    Costa, M., et al.: Bouncer: securing software by blocking bad input. In: Proceedings of twenty-first ACM SIGOPS symposium on Operating systems principles. ACM, Stevenson (2007)Google Scholar
  14. 14.
    Kruegel, C., et al.: Automating mimicry attacks using static binary analysis. In: Proceedings of the 14th conference on USENIX Security Symposium, vol. 14. USENIX, Baltimore (2005)Google Scholar
  15. 15.
    Molnar, D.A., Wagner, D.: Catchconv: Symbolic execution and run-time type inference for integer conversion errors, EECS Department, University of California, Berkeley (2007)Google Scholar
  16. 16.
    Cadar, C., et al.: EXE: automatically generating inputs of death. In: Proceedings of the 13th ACM conference on Computer and communications security. ACM, Virginia (2006)Google Scholar
  17. 17.
    Hsueh, M.-C., Tsai, T.K., Iyer, R.K.: Fault Injection Techniques and Tools. IEEE Computer 30(4), 75–82 (1997)CrossRefGoogle Scholar
  18. 18.
    Boneh, D., DeMillo, R., Lipton, R.J.: On the importance of checking cryptographic protocols for faults. In: Fumy, W. (ed.) EUROCRYPT 1997. LNCS, vol. 1233, pp. 37–51. Springer, Heidelberg (1997)CrossRefGoogle Scholar
  19. 19.
    Kocher, P.C., Jaffe, J., Jun, B.: Differential Power Analysis. In: Wiener, M. (ed.) CRYPTO 1999. LNCS, vol. 1666, p. 388. Springer, Heidelberg (1999)CrossRefGoogle Scholar
  20. 20.
    Xu, J., et al.: An Experimental Study of Security Vulnerabilities Caused by Errors. In: Proceedings of International Conference on Dependable Systems and Networks (DSN) (2001)Google Scholar
  21. 21.
    Govindavajhala, S., Appel, A.W.: Using Memory Errors to Attack a Virtual Machine. In: Proceedings of the 2003 IEEE Symposium on Security and Privacy. IEEE, Los Alamitos (2003)Google Scholar

Copyright information

© IFIP International Federation for Information Processing 2009

Authors and Affiliations

  • Karthik Pattabiraman
    • 1
  • Nithin Nakka
    • 1
  • Zbigniew Kalbarczyk
    • 1
  • Ravishankar Iyer
    • 1
  1. 1.Center for Reliable and High-Performance Computing (CRHC)University of Illinois at Urbana-Champaign (UIUC)UrbanaUSA

Personalised recommendations