Identification of Bugs and Vulnerabilities in TLS Implementation for Windows Operating System Using State Machine Learning

  • Tarun YadavEmail author
  • Koustav Sadhukhan
Conference paper
Part of the Communications in Computer and Information Science book series (CCIS, volume 969)


TLS protocol is an essential part of secure Internet communication. In the past, many attacks have been identified on the protocol. Most of these attacks are not due to design flaws of the protocol, but due to flaws in specific implementation of protocol. One of the widely used implementation of TLS is SChannel which is used in Windows operating system since its inception. In this paper, we have used “protocol state fuzzing” to identify vulnerable and undesired state transitions in the state machine models of the protocol for various versions of SChannel. The technique of protocol state fuzzing has been implemented using query based state machine learning. The client as well as server components have been analyzed thoroughly using this technique and various flaws have been discovered in the implementation. Exploitation of these flaws under specific circumstances may lead to serious attacks which have potential to disrupt secure communication.


TLS protocol State machine SChannel Fuzzing 


  1. 1.
    Aviram, N., et al.: DROWN: breaking TLS using SSLv2. In: USENIX Security Symposium, pp. 689–706 (2016)Google Scholar
  2. 2.
    Green, M.: Attack of the week: FREAK (or ‘factoring the NSA for fun and profit’). A Few Thoughts on Cryptographic Engineering (2018).
  3. 3.
    Adrian, D., et al.: Imperfect forward secrecy: how Diffie-Hellman fails in practice. In: Proceedings of the 22nd ACM SIGSAC Conference on Computer and Communications Security, pp. 5–17. ACM (2015)Google Scholar
  4. 4.
    Bhargavan, K., Leurent, G.: Transcript collision attacks: breaking authentication in TLS, IKE, and SSH. In: Network and Distributed System Security Symposium-NDSS 2016 (2016)Google Scholar
  5. 5.
    Synopsys, S.: Heartbleed Bug. (2018).
  6. 6.
    De Ruiter, J., Poll, E.: Protocol state fuzzing of TLS implementations. In: USENIX Security Symposium, pp. 193–206 (2015)Google Scholar
  7. 7. The SSL 2.0 Protocol (1995).
  8. 8. RFC 6101 - The Secure Sockets Layer (SSL) Protocol Version 3.0 (2011).
  9. 9. RFC 2246 - The TLS Protocol Version 1.0 (1999).
  10. 10. RFC 4346 - The Transport Layer Security (TLS) Protocol Version 1.1 (2006).
  11. 11. RFC 5246 - The Transport Layer Security (TLS) Protocol Version 1.2 (2008).
  12. 12.
    Beurdouche, B., et al.: A messy state of the union: taming the composite state machines of TLS. In: 2015 IEEE Symposium on Security and Privacy (SP), pp. 535–552. IEEE (2015)Google Scholar
  13. 13. Secure Channel (Windows) (n.d.).
  14. 14.
    Yadav, T., Rao, A.M.: Technical aspects of cyber kill chain. In: Abawajy, J.H., Mukherjea, S., Thampi, S.M., Ruiz-Martínez, A. (eds.) SSCC 2015. CCIS, vol. 536, pp. 438–452. Springer, Cham (2015). Scholar
  15. 15.
    Koustav, S., Mallari, R.A., Yadav, T.: Cyber attack thread: a control-flow based approach to deconstruct and mitigate cyber threats. In: 2015 International Conference on Computing and Network Communications (CoCoNet), pp. 170–178. IEEE (2015)Google Scholar
  16. 16.
    Langley, A.: ImperialViolet - early changecipherspec attack. (2014).

Copyright information

© Springer Nature Singapore Pte Ltd. 2019

Authors and Affiliations

  1. 1.Defence Research and Development OrganisationDelhiIndia

Personalised recommendations