A Tale of the OpenSSL State Machine: A Large-Scale Black-Box Analysis

Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10014)


State machine inference is a powerful black-box analysis technique that can be used to learn a state machine implemented in a system, i.e. by only exchanging valid messages with the implementation a state machine can be extracted. In this paper we perform a large scale analysis of the state machines as implemented over the last 14 years in OpenSSL, one of the most widely used implementations of TLS, and in LibreSSL, a fork of OpenSSL. By automating the learning process, the state machines were learned for 145 different versions of both the server-side and the client-side. For the server-side this resulted in 15 unique state machines for OpenSSL and 2 for LibreSSL. For the client-side, 9 unique state machines were learned for OpenSSL and one for LibreSSL. Analysing these state machines provides an interesting insight in the evolution of the state machine of OpenSSL. Security vulnerabilities and other bugs related to their implementation can be observed, together with the point at which these are fixed. We argue that these problems could have been detected and fixed earlier if the developers would have had the tools available to analyse the implemented state machines.


State Machine System Under Test Input Alphabet Transport Layer Security Alert Message 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


  1. 1.
    Angluin, D.: Learning regular sets from queries and counterexamples. Inf. Comput. 75(2), 87–106 (1987)MathSciNetCrossRefzbMATHGoogle Scholar
  2. 2.
    Aviram, N., Schinzel, S., Somorovsky, J., Heninger, N., Dankel, M., Steube, J., Valenta, L., Adrian, D., Halderman, J.A., Dukhovni, V., Käsper, E., Cohney, S., Engels, S., Paar, C., Shavitt, Y.: DROWN: breaking TLS using SSLv2. In: 25th USENIX Security Symposium (USENIX Security 2016), pp. 689–706. USENIX Association, Austin, August 2016Google Scholar
  3. 3.
    Beurdouche, B., Bhargavan, K., Delignat-Lavaud, A., Fournet, C., Kohlweiss, M., Pironti, A., Strub, P.Y., Zinzindohoue, J.K.: A messy state of the union: taming the composite state machines of TLS. In: 2015 IEEE Symposium on Security and Privacy, pp. 535–552 (2015)Google Scholar
  4. 4.
    Bhargavan, K., Fournet, C., Kohlweiss, M., Pironti, A., Strub, P.: Implementing TLS with verified cryptographic security. In: 2013 IEEE Symposium on Security and Privacy, pp. 445–459 (2013)Google Scholar
  5. 5.
    Chow, T.: Testing software design modeled by finite-state machines. IEEE Trans. Softw. Eng. 4(3), 178–187 (1978)CrossRefzbMATHGoogle Scholar
  6. 6.
    Díaz, G., Cuartero, F., Valero, V., Pelayo, F.: Automatic verification of the TLS handshake protocol. In: Proceedings of the 2004 ACM Symposium on Applied Computing, SAC 2004, pp. 789–794. ACM (2004)Google Scholar
  7. 7.
    Dierks, T., Allen, C.: The TLS protocol version 1.0. RFC 2246, Internet Engineering Task Force (1999)Google Scholar
  8. 8.
    Dierks, T., Rescorla, E.: The Transport Layer Security (TLS) protocol version 1.1. RFC 4346, Internet Engineering Task Force (2006)Google Scholar
  9. 9.
    Dierks, T., Rescorla, E.: The Transport Layer Security (TLS) protocol version 1.2. RFC 5246, Internet Engineering Task Force (2008)Google Scholar
  10. 10.
    Gajek, S., Manulis, M., Pereira, O., Sadeghi, A.-R., Schwenk, J.: Universally composable security analysis of TLS. In: Baek, J., Bao, F., Chen, K., Lai, X. (eds.) ProvSec 2008. LNCS, vol. 5324, pp. 313–327. Springer, Heidelberg (2008). doi: 10.1007/978-3-540-88733-1_22 CrossRefGoogle Scholar
  11. 11.
    He, C., Sundararajan, M., Datta, A., Derek, A., Mitchell, J.C.: A modular correctness proof of IEEE 802.11i and TLS. In: Proceedings of the 12th ACM Conference on Computer and Communications Security, CCS 2005, pp. 2–15. ACM (2005)Google Scholar
  12. 12.
    Jager, T., Kohlar, F., Schäge, S., Schwenk, J.: On the security of TLS-DHE in the standard model. In: Safavi-Naini, R., Canetti, R. (eds.) CRYPTO 2012. LNCS, vol. 7417, pp. 273–293. Springer, Heidelberg (2012). doi: 10.1007/978-3-642-32009-5_17 CrossRefGoogle Scholar
  13. 13.
    Kamil, A., Lowe, G.: Analysing TLS in the strand spaces model. J. Comput. Secur. 19(5), 975–1025 (2011)CrossRefGoogle Scholar
  14. 14.
    Krawczyk, H., Paterson, K.G., Wee, H.: On the security of the TLS protocol: a systematic analysis. In: Canetti, R., Garay, J.A. (eds.) CRYPTO 2013. LNCS, vol. 8042, pp. 429–448. Springer, Heidelberg (2013). doi: 10.1007/978-3-642-40041-4_24 CrossRefGoogle Scholar
  15. 15.
    Meyer, C., Schwenk, J.: SoK: lessons learned from SSL/TLS attacks. In: Kim, Y., Lee, H., Perrig, A. (eds.) WISA 2013. LNCS, vol. 8267, pp. 189–209. Springer, Heidelberg (2014). doi: 10.1007/978-3-319-05149-9_12 CrossRefGoogle Scholar
  16. 16.
    Morrissey, P., Smart, N.P., Warinschi, B.: A modular security analysis of the TLS handshake protocol. In: Pieprzyk, J. (ed.) ASIACRYPT 2008. LNCS, vol. 5350, pp. 55–73. Springer, Heidelberg (2008). doi: 10.1007/978-3-540-89255-7_5 CrossRefGoogle Scholar
  17. 17.
    Niese, O.: An integrated approach to testing complex systems. Ph.D. thesis, Dortmund University (2003)Google Scholar
  18. 18.
    Ogata, K., Futatsugi, K.: Equational approach to formal analysis of TLS. In: 2005 Proceedings of the 25th IEEE International Conference on Distributed Computing Systems, ICDCS 2005, pp. 795–804. IEEE (2005)Google Scholar
  19. 19.
    Paulson, L.C.: Inductive analysis of the internet protocol TLS. ACM Trans. Inf. Syst. Secur. 2(3), 332–351 (1999)CrossRefGoogle Scholar
  20. 20.
    Raffelt, H., Steffen, B., Berg, T.: LearnLib: a library for automata learning and experimentation. In: Formal methods for industrial critical systems (FMICS 2005), pp. 62–71. ACM (2005)Google Scholar
  21. 21.
    Rescorla, E., Ray, M., Dispensa, S., Oskov, N.: Transport Layer Security (TLS) renegotiation indication extension. RFC 5746, Internet Engineering Task Force (2010)Google Scholar
  22. 22.
    de Ruiter, J., Poll, E.: Protocol state fuzzing of TLS implementations. In: 24th USENIX Security Symposium (USENIX Security 2015). USENIX Association, Washington, D.C., August 2015Google Scholar
  23. 23.
    Turner, S., Polk, T.: Prohibiting Secure Sockets Layer (SSL) version 2.0. RFC 6176, Internet Engineering Task Force (2011)Google Scholar

Copyright information

© Springer International Publishing AG 2016

Authors and Affiliations

  1. 1.Institute for Computing and Information SciencesRadboud UniversityNijmegenThe Netherlands

Personalised recommendations