Message Confidentiality Testing of Security Protocols – Passive Monitoring and Active Checking

  • Guoqiang Shu
  • David Lee
Part of the Lecture Notes in Computer Science book series (LNCS, volume 3964)


Security protocols provide critical services for distributed communication infrastructures. However, it is a challenge to ensure the correct functioning of their implementations, particularly, in the presence of malicious parties. We study testing of message confidentiality – an essential security property. We formally model protocol systems with an intruder using Dolev-Yao model. We discuss both passive monitoring and active testing of message confidentiality. For adaptive testing, we apply a guided random walk that selects next input on-line based on transition coverage and intruder’s knowledge acquisition. For mutation testing, we investigate a class of monotonic security flaws, for which only a small number of mutants need to be tested for a complete checking. The well-known Needham-Schroeder-Lowe protocol is used to illustrate our approaches.


Test Sequence Message Secret Security Protocol Mutation Testing Mutation Function 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


  1. 1.
  2. 2.
    Chen, S., Kalbarczyk, Z., Xu, J., Iyer, R.K.: A Data-Driven Finite State Machine Model for Analyzing Security Vulnerabilities. In: International Conference on Dependable Systems and Networks (DSN 2003), p. 605 (2003)Google Scholar
  3. 3.
    DeMillo, R., Lipton, R., Sayward, F.: Hints on Test. Data Selection: Help For The Practicing Programmer. IEEE Computer 11(4), 34–41 (1978)CrossRefGoogle Scholar
  4. 4.
    Dolev, D., Yao, A.: On the security of public-key protocols. IEEE Transaction on Information Theory 29, 198–208 (1983)MathSciNetCrossRefMATHGoogle Scholar
  5. 5.
    Duale, A., Ümit Uyar, M.: A Method Enabling Feasible Conformance Test Sequence Generation for EFSM Models. IEEE Trans. Computers 53(5), 614–627 (2004)CrossRefGoogle Scholar
  6. 6.
    Fabbri, S., Maldonado, J., Sugeta, T., Masiero, P.: Mutation testing applied to validate specifications based on statecharts. In: International Symposium on Software Reliability Systems (ISSRE), pp. 210–219 (1999)Google Scholar
  7. 7.
    Geer, D., Harthorne, J.: Penetration Testing: A Duet. In: Proceedings of the 18th Annual Computer Security Applications Conference (ACSAC), pp. 185–198 (2002)Google Scholar
  8. 8.
    Jaiswal, S., Iannaccone, G., Kurose, J., Towlsey, D.: Formal Analysis of Passive Measurement Inference Techniques. In: Proceedings of IEEE Infocom 2006 (to appear, 2006)Google Scholar
  9. 9.
    Jurjens, J., Wimmel, G.: Formally Testing Fail-Safety of Electronic Purse Protocols. IEEE International Conference on Automated Software Engineering, 408 (2001)Google Scholar
  10. 10.
    Lee, D., Chen, D., Hao, R., Miller, R.E., Wu, J., Yin, X.: A formal approach for passive testing of protocol data portions. Proceedings of ICNP, 122–131 (2002)Google Scholar
  11. 11.
    Lee, D., Sabnani, K.K., Kristol, D.M., Paul, S.: Conformance Testing of Protocols Specified as Communicating Finite State Machines - a Guided Random Walk Based Approach. IEEE Trans. on Communications 44(5), 631–640 (1996)CrossRefGoogle Scholar
  12. 12.
    Lee, D., Yannakakis, M.: Principles and methods of testing finite state machines - A survey. Proceedings of the IEEE, 1090–1123 (August 1996)Google Scholar
  13. 13.
    Lee, D., Yannakakis, M.: Online minimization of transition systems. In: Proceedings of STOC, pp. 264–274 (1992)Google Scholar
  14. 14.
    Lowe, G.: Breaking and Fixing the Needham-Schroeder Public-Key Protocol Using FDR. In: Margaria, T., Steffen, B. (eds.) TACAS 1996. LNCS, vol. 1055. Springer, Heidelberg (1996)CrossRefGoogle Scholar
  15. 15.
    Marick, B.: The Weak Mutation Hypothesis. In: Proceedings of The ACM SIGSOFT Symposium on. Testing, Analysis, and Verification (October 1991)Google Scholar
  16. 16.
    Meadows, C.: Applying formal methods to the analysis of a key management protocol. J. Comput. Security 1, 5–53 (1992)CrossRefGoogle Scholar
  17. 17.
    Meadows, C.: Formal methods for cryptographic protocol analysis: emerging issues and trends. IEEE Journal on Selected Areas in Communications 21(1), 44–54 (2003)CrossRefGoogle Scholar
  18. 18.
    Needham, R., Schroeder, M.: Using encryption for authentication in large networks of computers. Communications of the ACM 21(12), 993–999 (1978)CrossRefMATHGoogle Scholar
  19. 19.
    Schneider, S.: Security Properties and CSP. In: Proceedings of the 1996 IEEE Symposium on Security and Privacy, p. 174 (1996)Google Scholar
  20. 20.
    Sheyner, O., Haines, J., Jha, S., Lippmann, R., Wing, J.: Automated Generation and Analysis of Attack Graphs. IEEE Symposium on Security and Privacy (2002)Google Scholar
  21. 21.
    Shu, G., Lee, D.: Network Protocol System Fingerprinting – A Formal Approach. In: Proceedings of IEEE Infocom 2006 (to appear, 2006)Google Scholar
  22. 22.
    Thompson, H.: Application Penetration Testing. IEEE Security & Privacy 3(1), 66–69 (2005)CrossRefGoogle Scholar
  23. 23.
    Thompson, H.: Why Security Testing Is Hard. IEEE Security and Privacy 1(4), 83–86 (2003)CrossRefGoogle Scholar
  24. 24.
    Wimmel, G., Jürjens, J.: Specification-Based Test Generation for Security-Critical Systems Using Mutations. In: George, C.W., Miao, H. (eds.) ICFEM 2002. LNCS, vol. 2495, pp. 471–482. Springer, Heidelberg (2002)CrossRefGoogle Scholar

Copyright information

© IFIP International Federation for Information Processing 2006

Authors and Affiliations

  • Guoqiang Shu
    • 1
  • David Lee
    • 1
  1. 1.Department of Computer Science and EngineeringThe Ohio State UniversityColumbusUSA

Personalised recommendations