Automatic Classification of Transformed Protocols Using Deep Learning

  • Changmin JeongEmail author
  • Mirim Ahn
  • Haengho Lee
  • Younggiu Jung
Conference paper
Part of the Communications in Computer and Information Science book series (CCIS, volume 931)


Protocol reverse-engineering technique can be used to extract the specification of an unknown protocol. However, there is no standardized method and in most cases, the extracting process is done manually or semi-automatically. Since only frequently seen values are extracted as fields from the messages of a protocol, it is difficult to understand complete specification of the protocol. Therefore, if the information about the structure of the unknown protocol could be acquired in advance, it would be easy to conduct reverse engineering. This paper suggests a method of recognizing 8 commercial protocols and transformed protocols of their own using deep learning techniques. When the proposed method is conducted prior to APRE (Automatic Protocol Reverse Engineering) process, it is possible to obtain useful information beforehand when similarities exist between unknown protocols and learned protocols.


Automatic protocol reverse engineering Transformed protocol 


  1. 1.
    Weidong, C., Jayanthkumar, K., Wang, H.J.: Discoverer: automatic protocol reverse engineering from network traces. In: USENIX Security Symposium, pp. 199–212 (2007)Google Scholar
  2. 2.
    Caballero, J., Yin, H., Liang, Z., Song, D.: Polyglot: automatic extraction of protocol message format using dynamic binary analysis CCS 2007. In: Proceedings of the 14th ACM Conference on Computer and Communications Security, pp. 317–329. ACM, New York (2007)Google Scholar
  3. 3.
    Wondracek, G., Comparetti, P.M., Kruegel, C., Kirda, E.: Automatic network protocol analysis. In: Proceedings of the 15th Annual Network and Distributed System Security Symposium (NDSS 2008) (2008)Google Scholar
  4. 4.
    Cui, W., Peinado, M., Chen, K., Wang, H.J., Irun-Briz, L.: Tupni: automatic reverse engineering of input formats. In: Proceedings of the 15th ACM Conference on Computer and Communications Security, pp. 391–402 (2008)Google Scholar
  5. 5.
    Comparetti, P.M., Wondracek, G., Kruegel, C.: Prospex: protocol specification extraction. In: 30th IEEE Symposium on Security and Privacy, pp. 110–125 (2009)Google Scholar
  6. 6.
    Caballero, J., Poosankam, P., Kreibich, C., Song, D.: Dispatcher: enabling active Botnet infiltration using automatic protocol reverse-engineering. In: Proceedings of the 16th ACM Conference on Computer and Communications Security, Proceeding CCS 2009, pp. 621–634 (2009)Google Scholar
  7. 7.
    Caballero, J., Song, D.: Automatic protocol reverse-engineering: message format extraction and field semantics inference. Int. J. Comput. Telecommun. Netw. 57(2), 451–474 (2012)CrossRefGoogle Scholar
  8. 8.
    Lin, R., Li, O., Li, Q., Liu, Y.: Unknown network protocol classification method based on semi-supervised learning. In: Computer and Communications (ICCC), pp. 300–308 (2015)Google Scholar
  9. 9.
    McGregor, A., Hall, M., Lorier, P., Brunskill, J.: Flow clustering using machine learning techniques. In: Proceedings of the Passive and Active Measurement Workshop (PAM 2004), Antibes Juan-les-Pins, France, April 2004Google Scholar
  10. 10.
    Zander, S., Nguyen, T., Armitage, G.: Automated traffic classification and application identification using machine learning. In: IEEE 30th Conference on Local Computer Networks (LCN 2005), Sydney, Australia, November 2005 Google Scholar
  11. 11.
    Moore, A., Zuev, D.: Internet traffic classification using Bayesian analysis techniques. In: ACM International Conference on Measurement and Modeling of Computer Systems (SIGMETRICS) 2005, Banff, Alberta, Canada, June 2005Google Scholar
  12. 12.
    Auld, T., Moore, A.W., Gull, S.F.: Bayesian neural networks for Internet traffic classification. IEEE Trans. Neural Netw. 18(1), 223–239 (2007)CrossRefGoogle Scholar
  13. 13.
    Williams, N., Zander, S., Armitage, G.: A preliminary performance comparison of five machine learning algorithms for practical IP traffic flow classification. Special Interest Group on Data Communication (SIGCOMM) Comput. Commun. Rev., 36(5), 5–16 (2006)CrossRefGoogle Scholar
  14. 14.
    Erman, J., Mahanti, A., Arlitt, M.: Internet traffic identification using machine learning techniques. In: Proceedings of 49th IEEE Global Telecommunications Conference (GLOBECOM 2006), San Francisco, USA, December 2006Google Scholar

Copyright information

© Springer Nature Singapore Pte Ltd. 2019

Authors and Affiliations

  • Changmin Jeong
    • 1
    Email author
  • Mirim Ahn
    • 1
  • Haengho Lee
    • 1
  • Younggiu Jung
    • 2
  1. 1.Agency for Defense DevelopmentDaejeonRepublic of Korea
  2. 2.YM-NaeultechIncheonRepublic of Korea

Personalised recommendations