Security and Software Engineering

  • Sam MalekEmail author
  • Hamid Bagheri
  • Joshua Garcia
  • Alireza Sadeghi


Software systems are permeating every facet of our society, making security breaches costlier than ever before. At the same time, as software systems grow in complexity, so does the difficulty of ensuring their security. As a result, the problem of securing software, in particular software that controls critical infrastructure, is growing in prominence. Software engineering community has developed numerous approaches for promoting and ensuring security of software. In fact, many security vulnerabilities are effectively avoidable through proper application of well-established software engineering principles and techniques. In this chapter, we first provide an introduction to the principles and concepts in software security from the standpoint of software engineering. We then provide an overview of four categories of approaches for achieving security in software systems, namely, static and dynamic analyses, formal methods, and adaptive mechanisms. We introduce the seminal work from each area and intuitively demonstrate their applications on several examples. We also enumerate on the strengths and shortcomings of each approach to help software engineers with making informed decisions when applying these approaches in their projects. Finally, the chapter provides an overview of the major research challenges from each approach, which we hope to shape the future research efforts in this area.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Andoni, A., Daniliuc, D., Khurshid, S.: Evaluating the small scope hypothesis. Technical report, MIT, 2003Google Scholar
  2. 2.
    Arzt, S., Rasthofer, S., Fritz, C., Bodden, E., Bartel, A., Klein, J., Le Traon, Y., Octeau, D., McDaniel, P.: Flowdroid: precise context, flow, field, object-sensitive and lifecycle-aware taint analysis for android apps. In: ACM SIGPLAN Notices, vol. 49, pp. 259–269. ACM, New York (2014)CrossRefGoogle Scholar
  3. 3.
    Avgerinos, T., Kil, C.S., Hao, B.L.T., David, B.: AEG: automatic exploit generation. In: Network and Distributed System Security Symposium (2011)Google Scholar
  4. 4.
    Bagheri, H., Sullivan, K.: Bottom-up model-driven development. In: Proceedings of the International Conference on Software Engineering (ICSE), pp. 1221–1224 (2013)Google Scholar
  5. 5.
    Bagheri, H., Sullivan, K.: Model-driven synthesis of formally precise stylized software architectures. Form. Asp. Comput. 28(3), 441–467 (2016)MathSciNetCrossRefGoogle Scholar
  6. 6.
    Bagheri, H., Kang, E., Malek, S., Jackson, D.: Detection of design flaws in the android permission protocol through bounded verification. In: FM 2015: Formal Methods. Lecture Notes in Computer Science, vol. 9109, pp. 73–89. Springer, Berlin (2015)CrossRefGoogle Scholar
  7. 7.
    Bagheri, H., Sadeghi, A., Garcia, J., Malek, S.: Covert: compositional analysis of android inter-app permission leakage. IEEE Trans. Softw. Eng. 41(9), 866–886 (2015)CrossRefGoogle Scholar
  8. 8.
    Bagheri, H., Sadeghi, A., Jabbarvand, R., Malek, S.: Practical, formal synthesis and automatic enforcement of security policies for android. In: Proceedings of the 46th IEEE/IFIP International Conference on Dependable Systems and Networks (DSN), pp. 514–525 (2016)Google Scholar
  9. 9.
    Ball, T., Levin, V., Rajamani, S.K.: A decade of software model checking with slam. Commun. ACM 54(7), 68–76 (2011)CrossRefGoogle Scholar
  10. 10.
    Barr, E., Harman, M., McMinn, P., Shahbaz, M., Yoo, S.: The Oracle problem in software testing: a survey. IEEE Trans. Softw. Eng. 41(5), 507–525 (2015)CrossRefGoogle Scholar
  11. 11.
    Beyer, D., Henzinger, T.A., Jhala, R., Majumdar, R.: The software model checker blast: applications to software engineering. Int. J. Softw. Tools Technol. Transf. 9(5), 505–525 (2007)CrossRefGoogle Scholar
  12. 12.
    Binkley, D.: Source code analysis: a road map. In: International Conference on Software Engineering, Minneapolis, May 2007, pp. 104–119Google Scholar
  13. 13.
    Brumley, D., Hartwig, C., Liang, Z., Newsome, J., Song, D., Yin, H.: Automatically identifying trigger-based behavior in Malware. In: Botnet Detection: Countering the Largest Security Threat, pp. 65–88. Springer, Boston (2008)Google Scholar
  14. 14.
    Brumley, D., Poosankam, P., Song, D., Zheng, J.: Automatic patch-based exploit generation is possible: techniques and implications. In: IEEE Symposium on Security and Privacy, SP 2008, pp. 143–157. IEEE, Piscataway (2008)Google Scholar
  15. 15.
    CanforaHarman, G., Di Penta, M.: New frontiers of reverse engineering. In: 2007 Future of Software Engineering, pp. 326–341. IEEE Computer Society, Los Alamitos (2007)Google Scholar
  16. 16.
    Cha, S.K., Avgerinos, T., Rebert, A., Brumley, D.: Unleashing mayhem on binary code. In: 2012 IEEE Symposium on Security and Privacy, May 2012, pp. 380–394Google Scholar
  17. 17.
    Cheng, S.-W., Garlan, D., Schmerl, B.: Evaluating the effectiveness of the rainbow self-adaptive system. In: ICSE Workshop on Software Engineering for Adaptive and Self-managing Systems, SEAMS ’09, May 2009, pp. 132–141Google Scholar
  18. 18.
    Cimatti, A., Clarke, E., Giunchiglia, E., Giunchiglia, F., Pistore, M., Roveri, M., Sebastiani, R., Tacchella, A.: Nusmv 2: an opensource tool for symbolic model checking. In: Computer Aided Verification. Lecture Notes in Computer Science, vol. 2404, pp. 359–364. Springer, Berlin (2002)CrossRefGoogle Scholar
  19. 19.
    Clarke, E., Emerson, E.: Design and synthesis of synchronisation skeletons using branching time temporal logic. In: Logic of Programs, Proceedings of Workshop. Lecture Notes in Computer Science, vol. 131, pp. 52–71. Springer, Berlin (1981)Google Scholar
  20. 20.
    Clarke, E., Emerson, E., Sistla, A.: Automatic verification of finite state concurrent system using temporal logic specifications: a practical approach. In: Proceedings of the 10th ACM SIGACT-SIGPLAN Symposium on Principles of Programming Languages (POPL’83), pp. 117–126. ACM Press, New York (1983)Google Scholar
  21. 21.
    Clarke, E., Grumberg, O., Peled, D.: Model Checking. MIT Press, Cambridge (1999)Google Scholar
  22. 22.
    Clarke, E., Kroening, D., Yorav, K.: Behavioral consistency of c and verilog programs using bounded model checking. In: DAC, pp. 368–371 (2003)Google Scholar
  23. 23.
    Coverity: Coverity code advisor.
  24. 24.
    De Moura, L., Bjørner, N.: Z3: an efficient SMT solver. In: Tools and Algorithms for the Construction and Analysis of Systems, pp. 337–340. Springer, Berlin (2008)Google Scholar
  25. 25.
    Dennis, G.: A relational framework for bounded program verification. PhD thesis, Massachusetts Institute of Technology (2009)Google Scholar
  26. 26.
    Dolby, J., Fink, S.J., Sridharan, M.: T.J. Watson Libraries for Analysis (WALA).
  27. 27.
    Dwyer, M.B., Avrunin, G.S., Corbett, J.C.: Patterns in property specifications for finite-state verification. In: Proceedings of the 21st International Conference on Software Engineering, ICSE ’99, pp. 411–420. ACM, New York (1999)Google Scholar
  28. 28.
    Enck, W., Octeau, D., McDaniel, P., Chaudhuri, S.: A study of android application security. In: USENIX Security Symposium, vol. 2, p. 2 (2011)Google Scholar
  29. 29.
    Enck, W., Gilbert, P., Han, S., Tendulkar, V., Chun, B.-G., Cox, L.P., Jung, J., McDaniel, P., Sheth, A.N.: Taintdroid: an information-flow tracking system for realtime privacy monitoring on smartphones. ACM Trans. Comput. Syst. 32(2), 5 (2014)CrossRefGoogle Scholar
  30. 30.
    Ernst, M.D.: Invited talk static and dynamic analysis: synergy and duality. In: Proceedings of the 5th ACM SIGPLAN-SIGSOFT Workshop on Program Analysis for Software Tools and Engineering, PASTE ’04, pp. 35–35. ACM, New York (2004)Google Scholar
  31. 31.
    Foo, B., Wu, Y.-S., Mao, Y.-C., Bagchi, S., Spafford, E.: ADEPTS: adaptive intrusion response using attack graphs in an e-commerce environment. In: International Conference on Dependable Systems and Networks, DSN 2005. Proceedings, July 2005, pp. 508–517Google Scholar
  32. 32.
    Fraser, G., Zeller, A.: Mutation-driven generation of unit tests and oracles. IEEE Trans. Softw. Eng. 38(2), 278–292 (2012)CrossRefGoogle Scholar
  33. 33.
    Garlan, D., Cheng, S.W., Huang, A.C., Schmerl, B., Steenkiste, P.: Rainbow: architecture-based self-adaptation with reusable infrastructure. Computer 37(10), 46–54 (2004)CrossRefGoogle Scholar
  34. 34.
    Gennari, J., Garlan, D.: Measuring attack surface in software architecture. Technical report CMU-ISR-11-121, Institute for Software Research, School of Computer Science, Carnegie Mellon University, 2011Google Scholar
  35. 35.
    Godefroid, P., Klarlund, N., Sen, K.: Dart: directed automated random testing. SIGPLAN Not. 40(6), 213–223 (2005)CrossRefGoogle Scholar
  36. 36.
    Godefroid, P., Levin, M.Y., Molnar, D.: Sage: Whitebox fuzzing for security testing. Queue 10(1), 20:20–20:27 (2012)CrossRefGoogle Scholar
  37. 37.
    Gupta, R., Harrold, M.J., Soffa, M.L.: An approach to regression testing using slicing. In: Conference on Software Maintenance. Proceedings, pp. 299–308. IEEE, Piscataway (1992)Google Scholar
  38. 38.
    Hoare, C.: An axiomatic basis for computer programming. Commun. ACM 12(10), 576–585 (1969)CrossRefGoogle Scholar
  39. 39.
    Holzmann, G.J.: The SPIN Model Checker: Primer and Reference Manual. Addison-Wesley, Boston (2003)Google Scholar
  40. 40.
    Hovemeyer, D., Pugh, W.: Finding bugs is easy. ACM Sigplan Not. 39(12), 92–106 (2004)CrossRefGoogle Scholar
  41. 41.
    HP Enterprise Security: Fortify static code analysis tool: static application security testing — micro focus.
  42. 42.
    Huang, Y., Kintala, C., Kolettis, N., Fulton, N.: Software rejuvenation: analysis, module and applications. In: Twenty-Fifth International Symposium on Fault-Tolerant Computing, FTCS-25. Digest of Papers, June 1995, pp. 381–390Google Scholar
  43. 43.
  44. 44.
    Jackson, D.: Software Abstractions, 2nd edn. MIT Press, Cambridge (2012)Google Scholar
  45. 45.
    Jlint: Find bugs in java programs.
  46. 46.
    Jones, J.A., Harrold, M.J.: Empirical evaluation of the tarantula automatic fault-localization technique. In: Proceedings of the 20th IEEE/ACM International Conference on Automated Software Engineering, pp. 273–282. ACM, New York (2005)Google Scholar
  47. 47.
    Kaufmann, M., Strother Moore, J.: ACL2: an industrial strength version of Nqthm. In: Proceedings of the Annual Conference on Computer Assurance (COMPASS), pp. 23–34 (1996)Google Scholar
  48. 48.
    Kephart, J.O., Chess, D.M.: The vision of autonomic computing. Computer 36(1), 41–50 (2003)MathSciNetCrossRefGoogle Scholar
  49. 49.
    Kremenek, T.: Finding Software Bugs with the Clang Static Analyzer. Apple Inc., California (2008)Google Scholar
  50. 50.
    Lint4j: Lint4j overview.
  51. 51.
    Livshits, V.B., Lam, M.S.: Finding security vulnerabilities in java applications with static analysis. In: Usenix Security, vol. 2013 (2005)Google Scholar
  52. 52.
    Marcus, A., Maletic, J.I.: Identification of high-level concept clones in source code. In: 16th Annual International Conference on Automated Software Engineering, ASE 2001. Proceedings, pp. 107–114. IEEE, Piscataway (2001)Google Scholar
  53. 53.
    McGraw, G.: Automated code review tools for security. Computer 41(12), 108–111 (2008)CrossRefGoogle Scholar
  54. 54.
    Meier, J., Mackman, A., Vasireddy, S., Dunner, M., Escamila, R., Murukan, A.: Improving Web Application Security: Threats and Countermeasures. Microsoft Corporation, Redmond (2003)Google Scholar
  55. 55.
    Morrisett, G., Tan, G., Tassarotti, J., Tristan, J.-B., Gan, E.: RockSalt: Better, faster, stronger SFI for the x86. In: Proceedings of the 33rd ACM SIGPLAN Conference on Programming Language Design and Implementation, PLDI ’12, pp. 395–404. ACM, New York (2012)Google Scholar
  56. 56.
    Moser, A., Kruegel, C., Kirda, E.: Exploring multiple execution paths for malware analysis. In: IEEE Symposium on Security and Privacy, SP’07, pp. 231–245. IEEE, Piscataway (2007)Google Scholar
  57. 57.
    Nagarajan, A., Nguyen, Q., Banks, R., Sood, A.: Combining intrusion detection and recovery for enhancing system dependability. In: 2011 IEEE/IFIP 41st International Conference on Dependable Systems and Networks Workshops (DSN-W), June 2011, pp. 25–30Google Scholar
  58. 58.
    National vulnerability database. Accessed 22 Apr 2016
  59. 59.
    Necula, G.C.: Proof-carrying code. In: Proceedings of the 24th ACM SIGPLAN-SIGACT Symposium on Principles of Programming Languages, POPL ’97, pp. 106–119. ACM, New York (1997)Google Scholar
  60. 60.
    Nethercote, N., Seward, J.: Valgrind: a framework for heavyweight dynamic binary instrumentation. In: ACM Sigplan Notices, vol. 42, pp. 89–100. ACM, New York (2007)CrossRefGoogle Scholar
  61. 61.
    Newsome, J., Song, D.: Dynamic taint analysis for automatic detection, analysis, and signature generation of exploits on commodity software. In: Network and Distributed System Security Symposium (2005)Google Scholar
  62. 62.
    Okhravi, H., Comella, A., Robinson, E., Haines, J.: Creating a cyber moving target for critical infrastructure applications using platform diversity. Int. J. Crit. Infrastruct. Prot. 5(1), 30–39 (2012)CrossRefGoogle Scholar
  63. 63.
    Oreizy, P., Medvidovic, N., Taylor, R.N.: Architecture-based runtime software evolution. In: Proceedings of the 20th International Conference on Software Engineering, ICSE ’98, pp. 177–186. IEEE Computer Society, Washington (1998)Google Scholar
  64. 64.
    Ouchani, S., Debbabi, M.: Specification, verification, and quantification of security in model-based systems. Computing 97, 691–711 (2015)MathSciNetCrossRefGoogle Scholar
  65. 65.
    Ouimet, M.: Formal software verification: model checking and theorem proving. Technical report ESL-TIK-00214, MIT, 2005Google Scholar
  66. 66. Cross-site scripting (XSS) - OWASP.
  67. 67.
  68. 68.
    Owre, S., Rushby, J.M., Shankar, N.: PVS: a prototype verification system. In: Kapur, D. (ed.) Automated DeductionCADE-11. Lecture Notes in Computer Science, vol. 607, pp. 748–752. Springer, Berlin (1992) Google Scholar
  69. 69.
    Pastore, F., Mariani, L., Fraser, G.: CrowdOracles: can the crowd solve the oracle problem? In: 2013 IEEE Sixth International Conference on Software Testing, Verification and Validation (ICST), March 2013, pp. 342–351Google Scholar
  70. 70.
    Paulson, L.: Isabelle: A Generic Theorem Prover. Lecture Notes in Computer Science, vol. 828. Springer, Berlin (1994)Google Scholar
  71. 71.
    PMD: Source code analyzer.
  72. 72.
    Pnueli, A.: The temporal logic of programs. In: Proceedings of the 18th Annual Symposium on Foundations of Computer Science (FOCS), pp. 46–57 (1977)Google Scholar
  73. 73.
    Ramananandro, T.: Mondex, an electronic purse: specification and refinement checks with the alloy model-finding method. Formal Asp. Comput. 20(1), 21–39 (2008)CrossRefGoogle Scholar
  74. 74.
    Ren, J.: A Connector-Centric Approach to Architectural Access Control. PhD thesis, University of California, Irvine (2006)Google Scholar
  75. 75.
    Ren, J., Taylor, R.: A secure software architecture description language. In: Workshop on Software Security Assurance Tools, Techniques, and Metrics, SSATTM’05 (2005)Google Scholar
  76. 76.
    Sen, K.: Concolic testing. In: Proceedings of the Twenty-Second IEEE/ACM International Conference on Automated Software Engineering, ASE ’07, pp. 571–572. ACM, New York (2007)Google Scholar
  77. 77.
    Sen, K., Marinov, D., Agha, G.: Cute: a concolic unit testing engine for c. In: Proceedings of the 10th European Software Engineering Conference Held Jointly with 13th ACM SIGSOFT International Symposium on Foundations of Software Engineering, ESEC/FSE-13, pp. 263–272. ACM, New York (2005)Google Scholar
  78. 78.
    Sousa, P., Bessani, A., Correia, M., Neves, N., Verissimo, P.: Highly available intrusion-tolerant services with proactive-reactive recovery. IEEE Trans. Parallel Distrib. Syst. 21(4), 452–465 (2010)CrossRefGoogle Scholar
  79. 79.
    Suryanarayana, G., Diallo, M., Erenkrantz, J., Taylor, R.N.: Architectural support for trust models in decentralized applications. In: 28th International Conference on Software Engineering, ICSE’06, May 2006Google Scholar
  80. 80.
    Takanen, A., DeMott, J., Miller, C.: Fuzzing for Software Security Testing and Quality Assurance, 1st edn. Artech House, Inc., Norwood (2008)zbMATHGoogle Scholar
  81. 81.
    Tam, K., Khan, S.J., Fattori, A., Cavallaro, L.: Copperdroid: automatic reconstruction of android malware behaviors. In: Network and Distributed System Security Symposium (2015)Google Scholar
  82. 82.
    Taylor, R.N., Medvidovic, N., Dashofy, E.M.: Software Architecture: Foundations, Theory, and Practice. Wiley, New York (2009)Google Scholar
  83. 83.
    The Coq Development Team: The Coq proof assistant reference manual. Technical report version 8.2, LogiCal Project, 2008Google Scholar
  84. 84.
    Vallée-Rai, R., Co, P., Gagnon, E., Hendren, L., Lam, P., Sundaresan, V.: Soot-a java bytecode optimization framework. In: Proceedings of the 1999 Conference of the Centre for Advanced Studies on Collaborative Research, p. 13. IBM Press, Toronto (1999)Google Scholar
  85. 85.
    Visser, W., Havelund, K., Brat, G., Park, S., Lerda, F.: Model checking programs. Autom. Softw. Eng. 10(2), 203–232 (2003)CrossRefGoogle Scholar
  86. 86.
    Wang, F., Jou, F., Gong, F., Sargor, C., Goseva-Popstojanova, K., Trivedi, K.: SITAR: a scalable intrusion-tolerant architecture for distributed services. In: Foundations of Intrusion Tolerant Systems, pp. 359–367. IEEE Computer Society, New York (2003)Google Scholar
  87. 87.
    Wang, T., Wei, T., Gu, G., Zou, W.: Taintscope: a checksum-aware directed fuzzing tool for automatic software vulnerability detection. In: 2010 IEEE Symposium on Security and Privacy, May 2010, pp. 497–512Google Scholar
  88. 88.
    Xie, Y., Aiken, A.: Scalable error detection using boolean satisfiability. In: Proceedings of the 32nd ACM SIGPLAN-SIGACT Symposium on Principles of Programming Languages (POPL), pp. 351–363 (2005)Google Scholar
  89. 89.
    Xie, T., Tillmann, N., de Halleux, J., Schulte, W.: Fitness-guided path exploration in dynamic symbolic execution. In: 2009 IEEE/IFIP International Conference on Dependable Systems Networks, June 2009, pp. 359–368Google Scholar
  90. 90.
    Yan, L.K., Yin, H.: Droidscope: seamlessly reconstructing the os and dalvik semantic views for dynamic android malware analysis. Presented as part of the 21st USENIX Security Symposium (USENIX Security 12), pp. 569–584 (2012)Google Scholar
  91. 91.
    Yuan, E., Malek, S., Schmerl, B., Garlan, D., Gennari, J.: Architecture-based self-protecting software systems. In: QoSA ’13 (2013)Google Scholar
  92. 92.
    Yuan, E., Esfahani, N., Malek, S.: A systematic survey of self-protecting software systems. ACM Trans. Auton. Adapt. Syst. 8(4), 17:1–17:41 (2014)CrossRefGoogle Scholar
  93. 93.
    Zaeem, R., Prasad, M., Khurshid, S.: Automated generation of oracles for testing user-interaction features of mobile apps. In: 2014 IEEE Seventh International Conference on Software Testing, Verification and Validation (ICST), March 2014, pp. 183–192Google Scholar
  94. 94.
    Zhu, M., Yu, M., Xia, M., Li, B., Yu, P., Gao, S., Qi, Z., Liu, L., Chen, Y., Guan, H.: VASP: virtualization assisted security monitor for cross-platform protection. In: Proceedings of the 2011 ACM Symposium on Applied Computing, pp. 554–559 (2011)Google Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  • Sam Malek
    • 1
    Email author
  • Hamid Bagheri
    • 2
  • Joshua Garcia
    • 1
  • Alireza Sadeghi
    • 1
  1. 1.University of California, IrvineIrvineUSA
  2. 2.University of Nebraska-LincolnLincolnUSA

Personalised recommendations