Abstract
Ambient assisted living (AAL) environments represent a key concept for dealing with the inevitable problem of population-ageing. Until recently, the use of computational intensive techniques, like Machine Learning (ML) or Computer Vision (CV), were not suitable for IoT end-nodes due to their limited resources. However, recent advances in edge intelligence have somehow changed this landscape for smart environments. This paper presents an AAL scenario where the use of ML is tested in kitchen appliances recognition using CV. The goal is to help users working with those appliances through Augmented Reality (AR) on a mobile device. Seven types of kitchen appliances were selected: blender, coffee machine, fridge, water kettle, microwave, stove, and toaster. A dataset with nearly 4900 images was organized. Three different deep learning (DL) models from the literature were selected, each with a total number of parameters and architecture compatibles with their execution on mobile devices. The results obtained in the training of these models reveal precision in the test set above 95% for the model with better results. The combination of edge AI and ML opens the application of CV in smart homes and AAL without compromising mandatory requirements as system privacy or latency.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
UN: Ageing. http://www.un.org/en/sections/issues-depth/ageing/. Accessed 21 Nov 2020
Miguez, A., Soares, C., Torres, J.M., Sobral, P., Moreira, R.S.: Improving ambient assisted living through artificial intelligence. In: Rocha, Á., Adeli, H., Reis, L.P., Costanzo, S. (eds.) New Knowledge in Information Systems and Technologies. WorldCIST 2019. Advances in Intelligent Systems and Computing, vol. 931, pp. 110–123. Springer, Cham (2019)
Moreiraa, R.S., Soaresa, C., Torresa, J.M., Sobrala, P.: Combining IoT architectures in next generation healthcare computing systems. In: Intelligent IoT Systems in Personalized Health Care, Chapter 1, pp. 1–29. Elsevier Inc. (2021)
Costa, P., Gomes, B., Melo, N., Rodrigues, R., Carvalho, C., Karmali, K., Soares, C., Torres, J.M., Sobral, P., Moreira, R.S.: Fog computing in real time resource limited IoT environments. In: Rocha, Á., Adeli, H., Reis, L.P., Costanzo, S., Orovic, I., Moreira, F. (eds.) Trends and Innovations in Information Systems and Technologies, pp. 102–112. Springer, Cham (2020)
Quintana, E., Favela, J.: Augmented reality annotations to assist persons with Alzheimers and their caregivers. Pers. Ubiquitous Comput. 17, 1105–1116 (2013)
Deng, Y.: Deep learning on mobile devices: a review. In: Agaian, S.S., Asari, V.K., DelMarco, S.P. (eds.) Mobile Multimedia/Image Processing. Security, and Applications 2019, vol. 10993, pp. 52–66. International Society for Optics and Photonics, SPIE (2019)
Chen, J., Ran, X.: Deep learning with edge computing: a review. Proc. IEEE 107(8), 1655–1674 (2019)
Qayyum, O., çah, M.: IoS mobile application for food and location image prediction using convolutional neural networks. In: 2018 IEEE 5th International Conference on Engineering Technologies and Applied Sciences (ICETAS), pp. 1–6 (2018)
Dai, X., Spasić, I., Meyer, B., Chapman, S., Andres, F.: Machine learning on mobile: an on-device inference app for skin cancer detection. In: 2019 Fourth International Conference on Fog and Mobile Edge Computing (FMEC), pp. 301–305 (2019)
Russakovsky, O., Deng, J., Hao, S., Krause, J., Satheesh, S., Ma, S., Huang, Z., Karpathy, A., Khosla, A., Bernstein, M., Berg, A.C., Fei-Fei, L.: ImageNet large scale visual recognition challenge. Int. J. Comput. Vision (IJCV) 115(3), 211–252 (2015)
Huang, G., Liu, Z., Van Der Maaten, L., Weinberger, K.Q.: Densely connected convolutional networks (2018)
Sandler, M., Howard, A., Zhu, M., Zhmoginov, A., Chen, L.-C.: Mobilenetv2: inverted residuals and linear bottlenecks (2019)
Zoph, B., Vasudevan, V., Shlens, J., Le, Q.V.: Learning transferable architectures for scalable image recognition (2018)
Chollet, F., et al.: Keras (2015). https://keras.io
Abadi, M., Agarwal, A., Barham, P., Brevdo, E., Chen, Z., Citro, C., Corrado, G.S., Davis, A., Dean, J., Devin, M., Ghemawat, S., Goodfellow, I., Harp, A., Irving, G., Isard, M., Jia, Y., Jozefowicz, R., Kaiser, L., Kudlur, M., Levenberg, J., Mané, D., Monga, R., Moore, S., Murray, D., Olah, C., Schuster, M., Shlens, J., Steiner, B., Sutskever, I., Talwar, K., Tucker, P., Vanhoucke, V., Vasudevan, V., Viégas, F., Vinyals, O., Warden, P., Wattenberg, M., Wicke, M., Yu, Y., Zheng, X.: TensorFlow: large-scale machine learning on heterogeneous systems (2015). tensorflow.org
Acknowledgements
This work was funded by Fundação Ensino e Cultura Fernando Pessoa (FECFP), represented here by its R&D group Intelligent Sensing and Ubiquitous Systems (ISUS).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Torres, J.M., Aguiar, L., Soares, C., Sobral, P., Moreira, R.S. (2021). Home Appliance Recognition Using Edge Intelligence. In: Rocha, Á., Adeli, H., Dzemyda, G., Moreira, F., Ramalho Correia, A.M. (eds) Trends and Applications in Information Systems and Technologies . WorldCIST 2021. Advances in Intelligent Systems and Computing, vol 1367. Springer, Cham. https://doi.org/10.1007/978-3-030-72660-7_59
Download citation
DOI: https://doi.org/10.1007/978-3-030-72660-7_59
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-72659-1
Online ISBN: 978-3-030-72660-7
eBook Packages: Intelligent Technologies and RoboticsIntelligent Technologies and Robotics (R0)