On the rewards of self-adaptive IoT honeypots


In an era of fully digitally interconnected people and machines, IoT devices become a real target for attackers. Recent incidents such as the well-known Mirai botnet, have shown that the risks incurred are huge and therefore a risk assessment is mandatory. In this paper we present a novel approach on collecting relevant data about IoT attacks. We detail a SSH/Telnet honeypot system that leverages reinforcement learning algorithms in order to interact with the attackers, and we present the results obtained in view of defining optimal reward functions to be used. One of the key issues regarding the performance of such algorithms is the direct dependence on the reward functions used. The main outcome of our study is a full implementation of an IoT honeypot system that leverages Apprenticeship Learning using Inverse Reinforcement Learning, in order to generate best suited reward functions.

This is a preview of subscription content, log in to check access.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13


  1. 1.

    Online dictionaries. (2017) Definition: “Internet of things (IoT)”. https://en.oxforddictionaries.com/definition/internet_of_things. Accessed on November 2018

  2. 2.

    Alper Erdal, “Cisco internet of things,” October 2015. Available: https://www.slideshare.net/Panduit/cisco-internet-of-things. Accessed on November 2018

  3. 3.

    Barcena, Mario Ballano, and Candid Wueest. Insecurity in the internet of things. Security response, symantec (2015)

  4. 4.

    Markowsky, Linda, and George Markowsky (2015) Scanning for vulnerable devices in the internet of things. In Intelligent data acquisition and advanced computing systems: technology and applications (IDAACS), 2015 IEEE 8th international conference on, vol. 1, pp. 463–467. IEEE

  5. 5.

    NSA (2016) The next wave - the internet of things: it’s a wonderfully integrated life, vol. 21, no. 2

  6. 6.

    Pa Y, Pa M, Suzuki S, Yoshioka K, Matsumoto T, Kasama T, Rossow C (2016) Iotpot: a novel honeypot for revealing current iot threats. J Inf Proc 24(3):522–533

    Google Scholar 

  7. 7.

    Alaba FA, Othman M, Hashem IAT, Alotaibi F (2017) Internet of things security: a survey. J Netw Comput Appl 88:10–28, ISSN 1084–8045. https://doi.org/10.1016/j.jnca.2017.04.002

    Article  Google Scholar 

  8. 8.

    Antonakakis M, April T, Bailey M, Bernhard M, Bursztein E, Cochran J, Durumeric Z et al. (2017) Understanding the mirai botnet. In USENIX security symposium, pp. 1092–1110

  9. 9.

    Aceto G, Botta A, Marchetta P, Persico V, Pescapé A (2018) A comprehensive survey on internet outages. J Netw Comput Appl 113:36–63, ISSN 1084-8045. https://doi.org/10.1016/j.jnca.2018.03.026

    Article  Google Scholar 

  10. 10.

    Cui A, and Stolfo SJ (2010) A quantitative analysis of the insecurity of embedded network devices: results of a wide-area scan. In Proceedings of the 26th annual computer security applications conference, pp. 97–106. ACM. https://doi.org/10.1145/1920261.1920276

  11. 11.

    He H, Maple C, Watson T, Tiwari A, Mehnen J, Jin Y, and Gabrys B (2016) The security challenges in the IoT enabled cyber-physical systems and opportunities for evolutionary computing & other computational intelligence. In Evolutionary computation (CEC), 2016 IEEE congress on, pp. 1015–1021. IEEE

  12. 12.

    Kumar P, Kunwar RS, and Sachan A (2016) A survey report on: security & challenges in internet of things. In Proc National Conference on ICT & IoT, pp. 35–39

  13. 13.

    Senpai A (2016) Mirai-source-code. Available: https://github.com/jgamblin/Mirai-Source-Code. Accessed on November 2018

  14. 14.

    Vlasenko D (2008) Busybox: The swiss army knife of embedded linux. Available: https://www.busybox.net/about.html, Accessed on November 2018

  15. 15.

    Catuogno L, Castiglione A, Palmieri F (2015) A honeypot system with honeyword-driven fake interactive sessions. In: 2015 international conference on high performance computing and simulation (HPCS), pp. 187–194. IEEE, July

  16. 16.

    Pauna A, and Bica I (2014) RASSH-reinforced adaptive SSH honeypot. In Communications (COMM), 2014 10th international conference on, pp. 1–6. IEEE. https://doi.org/10.1109/ICComm.2014.6866707

  17. 17.

    Pauna A, Iacob A-C, and Bica I (2018) QRASSH-A self-adaptive SSH honeypot driven by Q-learning. In 2018 international conference on communications (COMM), pp. 441–446. IEEE. https://doi.org/10.1109/ICComm.2018.8484261

  18. 18.

    Wagener G, State R, Engel T, and Dulaunoy A (2011) Adaptive and self-configurable honeypots. In 12th IFIP/IEEE international symposium on integrated network management (IM 2011) and workshops, pp. 345–352. IEEE. https://doi.org/10.1109/INM.2011.5990710

  19. 19.

    Abbeel P, and Ng AY (2004) Apprenticeship learning via inverse reinforcement learning. In Proceedings of the twenty-first international conference on machine learning, p. 1. ACM

  20. 20.

    Spitzner L: https://www.symantec.com/connect/articles/dynamic-honeypots, Accessed on November 2018

  21. 21.

    Cheswick B (1992) An evening with Berferd in which a cracker is lured, endured, and studied. In Proc. winter USENIX conference, San Francisco, pp. 20–24

  22. 22.

    Wagener G Thesis: self-adaptive honeypots coercing and assessing attacker behaviour http://hdl.handle.net/10993/15673. Accessed on November 2018

  23. 23.

    Pauna A (2012) Improved self adaptive honeypots capable of detecting rootkit malware. In Communications (COMM), 2012 9th international conference on, pp. 281–284. IEEE, https://doi.org/10.1109/ICComm.2012.6262612

  24. 24.

    Kippo: https://github.com/desaster/kippo, Accessed on November 2018

  25. 25.

    Sutton RS, Barto AG (2018) Reinforcement learning: An introduction. MIT press. https://www.amazon.com/Reinforcement-Learning-Introduction-Adaptive-Computation/dp/0262193981. Accessed 19 Dec 2018

  26. 26.

    Luo T, Xu Z, Jin X, Jia Y, and Ouyang X (2017) Iotcandyjar: towards an intelligent-interaction honeypot for iot devices. Black Hat

  27. 27.

    La QD, Quek TQS, and Lee J (2016) A game theoretic model for enabling honeypots in IoT networks. In Communications (ICC), 2016 IEEE international conference on, pp. 1–6. IEEE. https://doi.org/10.1109/ICC.2016.7510833

  28. 28.

    Yin Minn Pa Pa, Suzuki S, Yoshioka K, Matsumoto T, Kasama T, and Rossow C (2015) IoTPOT: analysing the rise of IoT compromises. In Proceedings of the 9th USENIX conference on offensive technologies (WOOT’15), Aurélien Francillon and Thomas Ptacek (Eds.). USENIX Association, Berkeley, CA, USA, 9–9

  29. 29.

    Bhangwar NH, Halepoto IA, Khokhar S, Laghari AA (2017) On routing protocols for high performance. Stud Inf Control 26(4):441–448

    Google Scholar 

  30. 30.

    Guarnizo JD, Tambe A, Bhunia SS, Ochoa M, Tippenhauer NO, Shabtai A, and Elovici Y (2017) SIPHON: towards scalable high-interaction physical honeypots. In Proceedings of the 3rd ACM workshop on cyber-physical system security (CPSS ‘17). ACM, New York, NY, USA, 57–68. https://doi.org/10.1145/3055186.3055192

  31. 31.

    Phype. Telnet IoT honeypot. https://github.com/Phype/telnet-iot-honeypot. Accessed on November 2018

  32. 32.

    Honeything. https://github.com/omererdem/honeything. Accessed on November 2018

  33. 33.

    Dowling S, Schukat M and Melvin H (2017) A ZigBee honeypot to assess IoT cyberattack behaviour, 28th Irish Signals and Systems Conference (ISSC), Killarney, 2017, pp. 1–6. https://doi.org/10.1109/ISSC.2017.7983603

  34. 34.

    Krishnaprasad P (2017) “Capturing attacks on IoT devices with a multi-purpose IoT honeypot”. PhD thesis, Indian Institute of Technology Kanpur

  35. 35.

    Krawetz N (2004) Anti-honeypot technology, in IEEE Security & Privacy, vol. 2, no. 1, pp. 76–79. https://doi.org/10.1109/MSECP.2004.1264861

  36. 36.

    T. Holz and F. Raynal (2005) Detecting honeypots and other suspicious environments, Proceedings from the Sixth Annual IEEE SMC Information Assurance Workshop, West Point, NY, USA, pp. 29–36. https://doi.org/10.1109/IAW.2005.1495930

  37. 37.

    Fioriti V, Chinnici M (2017) Node seniority ranking in networks. Stud Inf Control 26(4):397–402

    Google Scholar 

  38. 38.

    Năstase L, Sandu IE, Popescu N (2017) An experimental evaluation of application layer protocols for the internet of things. Stud Inf Control 26(4):403–412

    Google Scholar 

  39. 39.

    Bellman R (1957) A Markovian decision process. J Math Mech 6:679–684. https://www.jstor.org/stable/24900506?seq=1#page_scan_tab_contents. Accessed 19 Dec 2018

  40. 40.

    Ng AY, Russell SJ (2000) Algorithms for inverse reinforcement learning. In: Langley P (ed) Proceedings of the seventeenth international conference on machine learning (ICML ‘00). Morgan Kaufmann Publishers Inc., San Francisco, CA, pp 663–670

    Google Scholar 

  41. 41.

    Li H, Wei T, Ren AO, Qi Z, and Wang Y (2017) Deep reinforcement learning: framework, applications, and embedded implementations. In Computer-aided design (ICCAD), 2017 IEEE/ACM international conference on, pp. 847–854. IEEE. https://doi.org/10.1109/ICCAD.2017.8203866

  42. 42.

    Cowrie: https://github.com/micheloosterhof/cowrie, Accessed on November 2018

  43. 43.

    IRASSH-T: https://github.com/adpauna/irassh, Accessed on November 2018

  44. 44.

    Miguel Sousa Lobo, Lieven Vandenberghe, Stephen Boyd, Hervé Lebret, Applications of second-order cone programming, Linear Algebra Appl, volume 284, issues 1–3,1998, Pages 193–228, ISSN 0024-3795, https://doi.org/10.1016/S0024-3795(98)10032-0

  45. 45.

    Quadratic Programming in Python https://scaron.info/blog/quadratic-programming-in-python.html. Accessed on November 2018

  46. 46.

    Dulaunoy A, Wagener G, Mokaddem S, and Wagner C (2017) An extended analysis of an IoT malware from a blackhole network. TNC17

  47. 47.

    Alaa M, Zaidan AA, Zaidan BB, Talal M, Kiah MLM (2017) A review of smart home applications based on internet of things. J Netw Comput Appl 97:48–65. https://doi.org/10.1016/j.jnca.2017.08.017

    Article  Google Scholar 

  48. 48.

    Popa D, Pop F, Serbanescu C, and Castiglione A (2019) Deep learning model for home automation and energy reduction in a smart home environment platform. Neural Comput Applic (in press)

  49. 49.

    Mohd BJ, Hayajneh T, Vasilakos AV (2015) A survey on lightweight block ciphers for low-resource devices: comparative study and open issues. J Netw Comput Appl 58:73–93, ISSN 1084-8045. https://doi.org/10.1016/j.jnca.2015.09.00

    Article  Google Scholar 

  50. 50.

    Ishitaki T, Obukata R, Oda T, and Barolli L (2017) Application of deep recurrent neural networks for prediction of user behavior in Tor networks. In Advanced information networking and applications workshops (WAINA), 2017 31st international conference on, pp. 238–243. IEEE, https://doi.org/10.1109/WAINA.2017.63

  51. 51.

    Chifor B-C, Bica I, Patriciu V-V, Pop F (2018) A security authorization scheme for smart home internet of things devices. Futur Gener Comput Syst 86:740–749

    Article  Google Scholar 

  52. 52.

    Esposito C, Castiglione A, Palmieri F, Ficco M, Dobre C, Iordache GV, Pop F (2018) Event-based sensor data exchange and fusion in the internet of things environments. J Parallel Distrib Comput 118:328–343

    Article  Google Scholar 

  53. 53.

    Zhimin Yu, Chong-zhi Gao, Zhengjun Jing, Brij Bhooshan Gupta, Qiuru Cai. A practical public key encryption scheme based on learning parity with noise. IEEE Access, 2018. https://doi.org/10.1109/ACCESS.2018.2840119, 6, 31918, 31923

  54. 54.

    Yang L, Han Z, Huang Z, Ma J (2018) A remotely keyed file encryption scheme under mobile cloud computing. J Netw Comput Appl 106:90–99

    Article  Google Scholar 

  55. 55.

    Tan C, Li Y, and Cheng Y (2017) An inverse reinforcement learning algorithm for semi-Markov decision processes. In Computational intelligence (SSCI), 2017 IEEE symposium series on, pp. 1–6. IEEE. https://doi.org/10.1109/SSCI.2017.8280816

Download references


We express our thanks to all reviewers for their valuable comments and remarks.


The research presented in this paper is supported by the following projects: ATLAS (PN-III-P1-1.2-PCCDI-2017-0272) and ROBIN (PN-III-P1-1.2-PCCDI-2017-0734).

Author information



Corresponding author

Correspondence to Florin Pop.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Pauna, A., Bica, I., Pop, F. et al. On the rewards of self-adaptive IoT honeypots. Ann. Telecommun. 74, 501–515 (2019). https://doi.org/10.1007/s12243-018-0695-7

Download citation


  • Internet of things
  • Honeypot systems
  • Inverse reinforcement learning
  • Neural network
  • Self-adaptive honeypot systems
  • Reinforcement learning