Abstract
Attack vectors for adversaries have increased in organizations because of the growing use of less secure IoT devices. The risk of attacks on an organization’s network has also increased due to the bring your own device (BYOD) policy which permits employees to bring IoT devices onto the premises and attach them to the organization’s network. To tackle this threat and protect their networks, organizations generally implement security policies in which only white-listed IoT devices are allowed on the organization’s network. To monitor compliance with such policies, it has become essential to distinguish IoT devices permitted within an organization’s network from non-white-listed (unknown) IoT devices. In this research, deep learning is applied to network communication for the automated identification of IoT devices permitted on the network. In contrast to existing methods, the proposed approach does not require complex feature engineering of the network communication, because the ’communication behavior’ of IoT devices is represented as small images which are generated from the device’s network communication payload. The proposed approach is applicable for any IoT device, regardless of the protocol used for communication. As our approach relies on the network communication payload, it is also applicable for the IoT devices behind a network address translation (NAT) enabled router. In this study, we trained various classifiers on a publicly accessible dataset to identify IoT devices in different scenarios, including the identification of known and unknown IoT devices, achieving over 99% overall average detection accuracy.
This is a preview of subscription content, access via your institution.






Notes
This is an extended and revised version of the paper (Kotak and Elovici 2019) that was presented at the CISIS 2020 conference and was published in its proceedings.
References
Abomhara M, Køien GM (2015) Cyber security and the internet of things: vulnerabilities, threats, intruders and attacks. J Cyber Secur Mobil 4(1):65–88
Acar A, Fereidooni H, Abera T, Sikder AK, Miettinen M, Aksu H, Conti M, Sadeghi A-R and Uluagac S (2020) Peek-a-boo: I see your smart home activities, even encrypted! In Proceedings of the 13th ACM Conference on Security and Privacy in Wireless and Mobile Networks, pp 207–218
Aksoy A and Gunes MH (2019) Automated iot device identification using network traffic. In ICC 2019-2019 IEEE International Conference on Communications (ICC), pp 1–7. IEEE
Alexa (2022) Alexa ranking. http://www.alexa.com/topsites
Andrea I, Chrysostomou C and Hadjichristofi G (2015) Internet of things: security vulnerabilities and challenges. In 2015 IEEE symposium on computers and communication (ISCC), pp 180–187. IEEE
Anthraper JJ and Kotak J (2019) Security, privacy and forensic concern of mqtt protocol. In: Proceedings of International Conference on Sustainable Computing in Science, Technology and Management (SUSCOM), Amity University Rajasthan, Jaipur-India
Arenson S (2018) Security researchers find vulnerable iot devices and mongodb databases exposing corporate data. https://blog.shodan.io/security-researchers-find-vulnerable-iot-devices-and-mongodb-databases-exposing-corporate-data/
Celik ZB, Walls RJ, McDaniel P and Swami A (2015) Malware traffic detection using tamper resistant features. In MILCOM 2015-2015 IEEE Military Communications Conference, pp 330–335. IEEE
Geoip (2022) Geoip lookup tool. http://geoip.com/
Keras_Layer_Activation_functions (2022) Keras documentation: layer activation functions. https://keras.io/activations/
Keras_Layer_weight_initializers (2022) Keras documentation: layer weight initializers. https://keras.io/initializers/
Keras_Losses. Keras documentation: losses. https://keras.io/losses/
Keras_Metrics (2022) Keras documentation: Metrics. https://keras.io/metrics/
Keras_Optimizers (2022) Keras documentation: optimizers. https://keras.io/optimizers/
Kotak J and Elovici Y (2019) Iot device identification using deep learning. In Computational Intelligence in Security for Information Systems Conference, pp 76–86. Springer
Kotak J, Shah A and Rajdev P (2019) A comparative analysis on security of mqtt brokers
LeCun Y (2022) The mnist database. http://yann.lecun.com/exdb/mnist/
Ling Zhen, Luo Junzhou, Yiling Xu, Gao Chao, Kui Wu, Xinwen Fu (2017) Security vulnerabilities of internet of things: a case study of the smart plug system. IEEE Internet Things J 4(6):1899–1909
Lopez-Martin Manuel, Carro Belen, Sanchez-Esguevillas Antonio, Lloret Jaime (2017) Network traffic classifier with convolutional and recurrent neural networks for internet of things. IEEE Access 5:18042–18050
Meidan Y, Bohadana M, Shabtai A, Guarnizo JD, Ochoa M, Tippenhauer NO and Elovici Y (2017a) Profiliot: a machine learning approach for iot device identification based on network traffic analysis. In Proceedings of the symposium on applied computing, pp 506–509
Meidan Y, Bohadana Y, Shabtai A, Ochoa M, Tippenhauer NO, Guarnizo JD and Elovici Y (2017b) Detection of unauthorized IoT devices using machine learning techniques. arXiv preprint arXiv:1709.04647
Meidan Y, Sachidananda V, Elovici Y and Shabtai A (2019) Privacy-preserving detection of IoT devices connected behind a nat in a smart home setup. arXiv preprint arXiv:1905.13430
Miettinen M, Marchal S, Hafeez I, Asokan N, Sadeghi A-R and Tarkoma S (2017) Iot sentinel: Automated device-type identification for security enforcement in IoT. In 2017 IEEE 37th International Conference on Distributed Computing Systems (ICDCS), pp 2177–2184. IEEE
Nguyen Thuy TT, Armitage Grenville (2008) A survey of techniques for internet traffic classification using machine learning. IEEE Commun Surv Tutor 10(4):56–76
Olalere Morufu, Abdullah Mohd Taufik, Mahmod Ramlan, Abdullah Azizol (2015) A review of bring your own device on security issues. SAGE Open 5(2):2158244015580372
Sangaiah Arun Kumar, Medhane Darshan Vishwasrao, Tao Han M, Hossain Shamim, Muhammad Ghulam (2019) Enforcing position-based confidentiality with machine learning paradigm through mobile edge computing in real-time industrial informatics. IEEE Trans Industr Inf 15(7):4189–4196
Sangaiah Arun Kumar, Dhanaraj Jerline Sheebha Anni, Mohandas Prabu, Castiglione Aniello (2020) Cognitive IoT system with intelligence techniques in sustainable computing environment. Comput Commun 154:347–360
Sangaiah Arun Kumar, Hosseinabadi Ali Asghar Rahmani, Shareh Morteza Babazadeh, Rad Seyed Yaser Bozorgi, Zolfagharian Atekeh, Chilamkurti Naveen (2020) IoT resource allocation and optimization based on heuristic algorithm. Sensors 20(2):539
SCMagazine (2016) Interpol warns iot devices at risk. https://www.scmagazineuk.com/interpol-warns-iot-devices-risk/article/1473202
Shah A, Rajdev P and Kotak J (2019) Memory forensic analysis of mqtt devices. arXiv preprint arXiv:1908.07835
Shodan (2022) Shodan: Search engine for the internet of everything. https://www.shodan.io/
Sivanathan Arunan, Gharakheili Hassan Habibi, Loi Franco, Radford Adam, Wijenayake Chamith, Vishwanath Arun, Sivaraman Vijay (2018) Classifying IoT devices in smart environments using network traffic characteristics. IEEE Trans Mob Comput 18(8):1745–1759
Sivanathan A, Sherratt D, Gharakheili HH, Radford A, Wijenayake C, Vishwanath A and Sivaraman V (2017) Characterizing and classifying IoT traffic in smart cities and campuses. In 2017 IEEE Conference on Computer Communications Workshops (INFOCOM WKSHPS), pp 559–564. IEEE
SplitCap (2022) Splitcap—a fast pcap file splitter. https://www.netresec.com/?page=SplitCap
Sun Guanglu, Liang Lili, Chen Teng, Xiao Feng, Lang Fei (2018) Network traffic classification based on transfer learning. Comput Elect Eng 69:920–927
Vailshery LS (2016) IoT devices installed base worldwide 2015–2025. https://www.statista.com/statistics/471264/iot-number-of-connected-devices-worldwide/
Wang Zhanyi (2015) The applications of deep learning on traffic identification. BlackHat USA 24(11):1–10
Wang W, Zhu M, Zeng X, Ye X and Sheng Y (2017) Malware traffic classification using convolutional neural network for representation learning. In 2017 International conference on information networking (ICOIN), pp 712–717. IEEE
Xiao Liang, Wan Xiaoyue, Xiaozhen Lu, Zhang Yanyong, Di Wu (2018) IoT security techniques based on machine learning: How do IoT devices use AI to enhance security? IEEE Signal Process Mag 35(5):41–49
Yu L, Luo B, Ma J, Zhou Z and Liu Q (2020) You are what you broadcast: Identification of mobile and \(\{\)IoT\(\}\) devices from (public)\(\{\)WiFi\(\}\). In 29th USENIX security symposium (USENIX security 20), pp 55–72
Zhang Jun, Chen Xiao, Xiang Yang, Zhou Wanlei, Jie Wu (2014) Robust network traffic classification. IEEE/ACM Trans Netw 23(4):1257–1270
Acknowledgements
This project was partially funded by the European Union’s Horizon 2020 research and innovation programme under grant agreement No. 830927.
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of interest
The authors declare that they have no conflicts of interest.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Appendices
Appendix A: Performance evaluation results for the detection of a specific IoT device in the IoT device network communication
Actual IoT device/classified as | 1 | 2 | P | R | F1 |
---|---|---|---|---|---|
1- Samsung SmartCam | 813 | 0 | 1 | 1 | 1 |
2- Other IoT devices | 0 | 5647 | 1 | 1 | 1 |
Weighted Avg | 1 | 1 | 1 |
Actual IoT device/classified as | 1 | 2 | P | R | F1 |
---|---|---|---|---|---|
1- Withings Aura smart sleep sensor | 321 | 2 | 0.997 | 0.994 | 0.995 |
2- Other IoT devices | 1 | 6136 | 1 | 1 | 1 |
Weighted Avg | 1 | 1 | 1 |
Actual IoT device/classified as | 1 | 2 | P | R | F1 |
---|---|---|---|---|---|
1- Insteon camera | 364 | 1 | 1 | 0.997 | 0.999 |
2- Other IoT devices | 0 | 6095 | 1 | 1 | 1 |
Weighted Avg | 1 | 1 | 1 |
Actual IoT device/classified as | 1 | 2 | P | R | F1 |
---|---|---|---|---|---|
1- Amazon Echo | 305 | 2 | 1 | 0.993 | 0.997 |
2- Other IoT devices | 0 | 6153 | 1 | 1 | 1 |
Weighted Avg | 1 | 1 | 1 |
Actual IoT device/classified as | 1 | 2 | P | R | F1 |
---|---|---|---|---|---|
1- Netatmo weather station | 210 | 0 | 1 | 1 | 1 |
2- Other IoT devices | 0 | 6250 | 1 | 1 | 1 |
Weighted Avg | 1 | 1 | 1 |
Actual IoT device/classified as | 1 | 2 | P | R | F1 |
---|---|---|---|---|---|
1- Netatmo welcome | 239 | 3 | 1 | 0.988 | 0.994 |
2- Other IoT devices | 0 | 6218 | 1 | 1 | 1 |
Weighted Avg | 1 | 1 | 1 |
Actual IoT device/classified as | 1 | 2 | P | R | F1 |
---|---|---|---|---|---|
1- Pix-Star photo frame | 100 | 0 | 1 | 1 | 1 |
2- Other IoT devices | 0 | 6360 | 1 | 1 | 1 |
Weighted Avg | 1 | 1 | 1 |
Actual IoT device/classified as | 1 | 2 | P | R | F1 |
---|---|---|---|---|---|
1- Belkin Wemo light switch | 629 | 4 | 1 | 0.994 | 0.997 |
2- Other IoT devices | 0 | 5827 | 0.999 | 1 | 1 |
Weighted Avg | 0.999 | 0.999 | 0.999 |
Actual IoT device/classified as | 1 | 2 | P | R | F1 |
---|---|---|---|---|---|
1- Belkin Wemo motion sensor | 3467 | 0 | 0.998 | 1 | 0.999 |
2- Other IoT devices | 8 | 2985 | 1 | 0.997 | 0.999 |
Weighted Avg | 0.999 | 0.999 | 0.999 |
Appendix B: Performance evaluation results for the detection of a specific IoT device in the network communication
Actual IoT device/classified as | 1 | 2 | P | R | F1 |
---|---|---|---|---|---|
1- Samsung SmartCam | 813 | 0 | 1 | 1 | 1 |
2- Other devices | 0 | 7873 | 1 | 1 | 1 |
Weighted Avg | 1 | 1 | 1 |
Actual IoT device/classified as | 1 | 2 | P | R | F1 |
---|---|---|---|---|---|
1- Withings Aura smart sleep sensor | 322 | 1 | 1 | 0.997 | 0.998 |
2- Other devices | 0 | 8363 | 1 | 1 | 1 |
Weighted Avg | 1 | 1 | 1 |
Actual IoT device/classified as | 1 | 2 | P | R | F1 |
---|---|---|---|---|---|
1- Insteon camera | 364 | 1 | 1 | 0.997 | 0.999 |
2- Other devices | 0 | 8321 | 1 | 1 | 1 |
Weighted Avg | 1 | 1 | 1 |
Actual IoT device/classified as | 1 | 2 | P | R | F1 |
---|---|---|---|---|---|
1- Amazon Echo | 304 | 3 | 0.997 | 0.990 | 0.993 |
2- Other devices | 1 | 8378 | 1 | 1 | 1 |
Weighted Avg | 1 | 1 | 1 |
Actual IoT device/classified as | 1 | 2 | P | R | F1 |
---|---|---|---|---|---|
1- Netatmo weather station | 210 | 0 | 1 | 1 | 1 |
2- Other devices | 0 | 8476 | 1 | 1 | 1 |
Weighted Avg | 1 | 1 | 1 |
Actual IoT device/classified as | 1 | 2 | P | R | F1 |
---|---|---|---|---|---|
1- Netatmo welcome | 242 | 0 | 1 | 1 | 1 |
2- Other devices | 0 | 8444 | 1 | 1 | 1 |
Weighted Avg | 1 | 1 | 1 |
Actual IoT device/classified as | 1 | 2 | P | R | F1 |
---|---|---|---|---|---|
1- Pix-Star photo frame | 100 | 0 | 1 | 1 | 1 |
2- Other devices | 0 | 8586 | 1 | 1 | 1 |
Weighted Avg | 1 | 1 | 1 |
Actual IoT device/classified as | 1 | 2 | P | R | F1 |
---|---|---|---|---|---|
1- Belkin Wemo light switch | 631 | 2 | 1 | 0.997 | 0.998 |
2- Other devices | 0 | 8053 | 1 | 1 | 1 |
Weighted Avg | 1 | 1 | 1 |
Actual IoT device/classified as | 1 | 2 | P | R | F1 |
---|---|---|---|---|---|
1- Belkin Wemo motion sensor | 3465 | 2 | 0.998 | 0.999 | 0.999 |
2- Other devices | 7 | 5212 | 1 | 0.999 | 0.999 |
Weighted Avg | 0.999 | 0.999 | 0.999 |
Appendix C: Performance evaluation results for the detection of unauthorized IoT devices
Actual IoT device/classified as | 1 (U) | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | Precision | Recall | F1 score |
---|---|---|---|---|---|---|---|---|---|---|---|---|
1- Samsung SmartCam (unknown) | 802 | 4 | 0 | 0 | 0 | 0 | 0 | 0 | 7 | 0.93 | 0.986 | 0.958 |
2- Withings Aura smart sleep sensor | 0 | 323 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0.988 | 1 | 0.994 |
3- Insteon camera | 0 | 0 | 365 | 0 | 0 | 0 | 0 | 0 | 0 | 1 | 1 | 1 |
4- Amazon Echo | 1 | 0 | 0 | 306 | 0 | 0 | 0 | 0 | 0 | 1 | 0.997 | 0.998 |
5- Netatmo weather station | 0 | 0 | 0 | 0 | 210 | 0 | 0 | 0 | 0 | 1 | 1 | 1 |
6- Netatmo welcome | 0 | 0 | 0 | 0 | 0 | 242 | 0 | 0 | 0 | 1 | 1 | 1 |
7- Pix-Star photo frame | 0 | 0 | 0 | 0 | 0 | 0 | 100 | 0 | 0 | 1 | 1 | 1 |
8- Belkin Wemo light switch | 7 | 0 | 0 | 0 | 0 | 0 | 0 | 626 | 0 | 1 | 0.989 | 0.994 |
9- Belkin Wemo motion sensor | 52 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 3415 | 0.998 | 0.985 | 0.991 |
Weighted Avg | 0.990 | 0.989 | 0.989 |
Actual IoT device/classified as | 1 | 2 (U) | 3 | 4 | 5 | 6 | 7 | 8 | 9 | Precision | Recall | F1 score |
---|---|---|---|---|---|---|---|---|---|---|---|---|
1- Samsung SmartCam | 812 | 1 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 1 | 0.999 | 0.999 |
2- Withings Aura smart sleep sensor (unknown) | 0 | 250 | 0 | 0 | 0 | 0 | 0 | 0 | 73 | 0.804 | 0.774 | 0.789 |
3- Insteon camera | 0 | 0 | 365 | 0 | 0 | 0 | 0 | 0 | 0 | 1 | 1 | 1 |
4- Amazon Echo | 0 | 1 | 0 | 306 | 0 | 0 | 0 | 0 | 0 | 1 | 0.997 | 0.998 |
5- Netatmo weather station | 0 | 0 | 0 | 0 | 210 | 0 | 0 | 0 | 0 | 1 | 1 | 1 |
6- Netatmo welcome | 0 | 0 | 0 | 0 | 0 | 242 | 0 | 0 | 0 | 1 | 1 | 1 |
7- Pix-Star photo frame | 0 | 0 | 0 | 0 | 0 | 0 | 100 | 0 | 0 | 1 | 1 | 1 |
8- Belkin Wemo light switch | 0 | 7 | 0 | 0 | 0 | 0 | 0 | 626 | 0 | 1 | 0.989 | 0.994 |
9- Belkin Wemo motion sensor | 0 | 52 | 0 | 0 | 0 | 0 | 0 | 0 | 3415 | 0.979 | 0.985 | 0.982 |
Weighted Avg | 0.979 | 0.979 | 0.979 |
Actual IoT device/classified as | 1 | 2 | 3 (U) | 4 | 5 | 6 | 7 | 8 | 9 | Precision | Recall | F1 score |
---|---|---|---|---|---|---|---|---|---|---|---|---|
1- Samsung SmartCam | 811 | 0 | 1 | 0 | 0 | 0 | 0 | 0 | 1 | 1 | 0.998 | 0.999 |
2- Withings Aura smart sleep sensor | 0 | 322 | 1 | 0 | 0 | 0 | 0 | 0 | 0 | 0.994 | 0.997 | 0.995 |
3- Insteon camera (unknown) | 0 | 2 | 363 | 0 | 0 | 0 | 0 | 0 | 0 | 0.894 | 0.995 | 0.942 |
4- Amazon Echo | 0 | 0 | 1 | 306 | 0 | 0 | 0 | 0 | 0 | 1 | 0.997 | 0.998 |
5- Netatmo weather station | 0 | 0 | 0 | 0 | 210 | 0 | 0 | 0 | 0 | 1 | 1 | 1 |
6- Netatmo Welcome | 0 | 0 | 0 | 0 | 0 | 242 | 0 | 0 | 0 | 1 | 1 | 1 |
7- Pix-Star photo frame | 0 | 0 | 0 | 0 | 0 | 0 | 100 | 0 | 0 | 1 | 1 | 1 |
8- Belkin Wemo light switch | 0 | 0 | 7 | 0 | 0 | 0 | 0 | 626 | 0 | 1 | 0.989 | 0.994 |
9- Belkin Wemo motion sensor | 0 | 0 | 33 | 0 | 0 | 0 | 0 | 0 | 3434 | 1 | 0.990 | 0.995 |
Weighted Avg | 0.994 | 0.993 | 0.993 |
Actual IoT device/classified as | 1 | 2 | 3 | 4 (U) | 5 | 6 | 7 | 8 | 9 | Precision | Recall | F1 score |
---|---|---|---|---|---|---|---|---|---|---|---|---|
1- Samsung SmartCam | 813 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0.998 | 1 | 0.999 |
2- Withings Aura smart sleep sensor | 0 | 323 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0.883 | 1 | 0.938 |
3- Insteon camera | 0 | 0 | 365 | 0 | 0 | 0 | 0 | 0 | 0 | 1 | 1 | 1 |
4- Amazon Echo (unknown) | 2 | 43 | 0 | 259 | 0 | 1 | 0 | 0 | 2 | 0.814 | 0.844 | 0.829 |
5- Netatmo weather station | 0 | 0 | 0 | 0 | 210 | 0 | 0 | 0 | 0 | 1 | 1 | 1 |
6- Netatmo welcome | 0 | 0 | 0 | 0 | 0 | 242 | 0 | 0 | 0 | 0.996 | 1 | 0.998 |
7- Pix-Star photo frame | 0 | 0 | 0 | 0 | 0 | 0 | 100 | 0 | 0 | 1 | 1 | 1 |
8- Belkin Wemo light switch | 0 | 0 | 0 | 7 | 0 | 0 | 0 | 626 | 0 | 1 | 0.989 | 0.994 |
9- Belkin Wemo motion sensor | 0 | 0 | 0 | 52 | 0 | 0 | 0 | 0 | 3415 | 0.999 | 0.985 | 0.992 |
Weighted Avg | 0.985 | 0.983 | 0.984 |
Actual IoT device/classified as | 1 | 2 | 3 | 4 | 5 (U) | 6 | 7 | 8 | 9 | Precision | Recall | F1 score |
---|---|---|---|---|---|---|---|---|---|---|---|---|
1- Samsung SmartCam | 812 | 0 | 0 | 0 | 1 | 0 | 0 | 0 | 0 | 0.985 | 0.999 | 0.992 |
2- Withings Aura smart sleep sensor | 0 | 323 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0.956 | 1 | 0.977 |
3- Insteon camera | 0 | 0 | 365 | 0 | 0 | 0 | 0 | 0 | 0 | 1 | 1 | 1 |
4- Amazon Echo | 0 | 0 | 0 | 306 | 1 | 0 | 0 | 0 | 0 | 1 | 0.997 | 0.998 |
5- Netatmo weather station (unknown) | 12 | 15 | 0 | 0 | 144 | 0 | 0 | 37 | 2 | 0.966 | 0.686 | 0.802 |
6- Netatmo welcome | 0 | 0 | 0 | 0 | 0 | 242 | 0 | 0 | 0 | 1 | 1 | 1 |
7- Pix-Star photo frame | 0 | 0 | 0 | 0 | 0 | 0 | 100 | 0 | 0 | 1 | 1 | 1 |
8- Belkin Wemo light switch | 0 | 0 | 0 | 0 | 1 | 0 | 0 | 626 | 6 | 0.944 | 0.989 | 0.966 |
9- Belkin Wemo motion sensor | 0 | 0 | 0 | 0 | 2 | 0 | 0 | 0 | 3465 | 0.998 | 0.999 | 0.999 |
Weighted Avg | 0.988 | 0.988 | 0.987 |
Actual IoT device/classified as | 1 | 2 | 3 | 4 | 5 | 6 (U) | 7 | 8 | 9 | Precision | Recall | F1 score |
---|---|---|---|---|---|---|---|---|---|---|---|---|
1- Samsung SmartCam | 813 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 1 | 1 | 1 |
2- Withings Aura smart sleep sensor | 0 | 323 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 1 | 1 | 1 |
3- Insteon camera | 0 | 0 | 365 | 0 | 0 | 0 | 0 | 0 | 0 | 1 | 1 | 1 |
4- Amazon Echo | 0 | 0 | 0 | 307 | 0 | 0 | 0 | 0 | 0 | 1 | 1 | 1 |
5- Netatmo weather station | 0 | 0 | 0 | 0 | 210 | 0 | 0 | 0 | 0 | 1 | 1 | 1 |
6- Netatmo welcome (unknown) | 0 | 0 | 0 | 0 | 0 | 239 | 0 | 0 | 3 | 1 | 0.988 | 0.994 |
7- Pix-Star photo frame | 0 | 0 | 0 | 0 | 0 | 0 | 100 | 0 | 0 | 1 | 1 | 1 |
8- Belkin Wemo light switch | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 626 | 7 | 1 | 0.989 | 0.994 |
9- Belkin Wemo motion sensor | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 3467 | 0.997 | 1 | 0.999 |
Weighted Avg | 0.998 | 0.998 | 0.998 |
Actual IoT device/classified as | 1 | 2 | 3 | 4 | 5 | 6 | 7 (U) | 8 | 9 | Precision | Recall | F1 score |
---|---|---|---|---|---|---|---|---|---|---|---|---|
1- Samsung SmartCam | 813 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 1 | 1 | 1 |
2- Withings Aura smart sleep sensor | 0 | 323 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 1 | 1 | 1 |
3- Insteon camera | 0 | 0 | 365 | 0 | 0 | 0 | 0 | 0 | 0 | 1 | 1 | 1 |
4- Amazon Echo | 0 | 0 | 0 | 306 | 0 | 0 | 1 | 0 | 0 | 1 | 0.997 | 0.998 |
5- Netatmo weather station | 0 | 0 | 0 | 0 | 210 | 0 | 0 | 0 | 0 | 1 | 1 | 1 |
6- Netatmo welcome | 0 | 0 | 0 | 0 | 0 | 242 | 0 | 0 | 0 | 1 | 1 | 1 |
7- Pix-Star photo frame (unknown) | 0 | 0 | 0 | 0 | 0 | 0 | 100 | 0 | 0 | 0.952 | 1 | 0.976 |
8- Belkin Wemo light switch | 0 | 0 | 0 | 0 | 0 | 0 | 1 | 626 | 6 | 1 | 0.989 | 0.994 |
9- Belkin Wemo motion sensor | 0 | 0 | 0 | 0 | 0 | 0 | 3 | 0 | 3464 | 0.998 | 0.999 | 0.999 |
Weighted Avg | 0.998 | 0.998 | 0.998 |
Actual IoT device/classified as | 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 (U) | 9 | Precision | Recall | F1 score |
---|---|---|---|---|---|---|---|---|---|---|---|---|
1- Samsung SmartCam | 812 | 0 | 0 | 0 | 0 | 0 | 0 | 1 | 0 | 1 | 0.999 | 0.999 |
2- Withings Aura smart sleep sensor | 0 | 323 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 1 | 1 | 1 |
3- Insteon camera | 0 | 0 | 364 | 0 | 0 | 0 | 0 | 1 | 0 | 1 | 0.997 | 0.999 |
4- Amazon Echo | 0 | 0 | 0 | 306 | 0 | 0 | 0 | 1 | 0 | 1 | 0.997 | 0.998 |
5- Netatmo weather station | 0 | 0 | 0 | 0 | 210 | 0 | 0 | 0 | 0 | 1 | 1 | 1 |
6- Netatmo welcome | 0 | 0 | 0 | 0 | 0 | 242 | 0 | 0 | 0 | 1 | 1 | 1 |
7- Pix-Star photo frame | 0 | 0 | 0 | 0 | 0 | 0 | 100 | 0 | 0 | 1 | 1 | 1 |
8- Belkin Wemo light switch (unknown) | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 622 | 11 | 0.995 | 0.983 | 0.989 |
9- Belkin Wemo motion sensor | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 3467 | 0.997 | 1 | 0.998 |
Weighted Avg | 0.998 | 0.998 | 0.998 |
Actual IoT device/classified as | 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 (U) | Precision | Recall | F1 score |
---|---|---|---|---|---|---|---|---|---|---|---|---|
1- Samsung SmartCam | 810 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 3 | 1 | 0.996 | 0.998 |
2- Withings Aura smart sleep sensor | 0 | 322 | 0 | 0 | 0 | 0 | 0 | 0 | 1 | 1 | 0.997 | 0.998 |
3- Insteon camera | 0 | 0 | 364 | 0 | 0 | 0 | 0 | 0 | 1 | 1 | 0.997 | 0.999 |
4- Amazon Echo | 0 | 0 | 0 | 306 | 0 | 0 | 0 | 0 | 1 | 1 | 0.997 | 0.998 |
5- Netatmo weather station | 0 | 0 | 0 | 0 | 209 | 0 | 0 | 0 | 1 | 1 | 0.995 | 0.998 |
6- Netatmo welcome | 0 | 0 | 0 | 0 | 0 | 242 | 0 | 0 | 0 | 1 | 1 | 1 |
7- Pix-Star photo frame | 0 | 0 | 0 | 0 | 0 | 0 | 100 | 0 | 0 | 1 | 1 | 1 |
8- Belkin Wemo light switch | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 633 | 0 | 0.920 | 1 | 0.958 |
9- Belkin Wemo motion sensor (unknown) | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 55 | 3412 | 0.998 | 0.984 | 0.991 |
Weighted Avg | 0.991 | 0.990 | 0.991 |
Appendix D: K-fold cross-validation results of different classifiers for the first four experiments
Experiment name | IoT device classifier | 5-Fold cross-validation mean accuracy (%) | 10-Fold cross-validation mean accuracy (%) |
---|---|---|---|
Experiment 1 | IoT and non-IoT devices | 99.57 | 99.66 |
Experiment 2 | Samsung SmartCam | 99.90 | 99.88 |
Withings Aura smart sleep sensor | 99.82 | 99.87 | |
Insteon camera | 99.98 | 99.98 | |
Amazon Echo | 99.88 | 99.91 | |
Netatmo weather station | 99.99 | 99.99 | |
Netatmo Welcome | 99.88 | 99.90 | |
Pix-Star photo frame | 99.98 | 99.98 | |
Belkin Wemo light switch | 99.83 | 99.83 | |
Belkin Wemo motion sensor | 99.79 | 99.78 | |
Experiment 3 | Samsung SmartCam | 99.92 | 99.93 |
Withings Aura smart sleep sensor | 99.87 | 99.89 | |
Insteon camera | 99.98 | 99.98 | |
Amazon Echo | 99.78 | 99.82 | |
Netatmo weather station | 99.99 | 99.99 | |
Netatmo welcome | 99.92 | 99.93 | |
Pix-Star photo frame | 99.98 | 99.98 | |
Belkin Wemo light switch | 99.88 | 99.88 | |
Belkin Wemo motion sensor | 99.80 | 99.81 | |
Experiment 4 | Multiple IoT devices | 99.43 | 99.63 |
* the k-fold cross-validation results for the fifth experiment are not listed, as it requires deriving a threshold from the validation set which is not feasible in k-fold cross-validation.
Rights and permissions
Springer Nature or its licensor holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Kotak, J., Elovici, Y. IoT device identification based on network communication analysis using deep learning. J Ambient Intell Human Comput (2022). https://doi.org/10.1007/s12652-022-04415-6
Received:
Accepted:
Published:
DOI: https://doi.org/10.1007/s12652-022-04415-6
Keywords
- Internet of Things (IoT)
- Cyber security
- Deep learning
- IoT device identification