Abstract
The increase in computing power and integration of specialized hardware for Artificial Intelligence (AI) acceleration like Tensor Processing Units (TPU) enable complex machine learning at edge devices in the Internet of Things (IoT). However, wireless portable systems are limited in computing power and battery lifetime. To increase the battery lifetime of edge devices and accelerate inference of IoT systems, many developments focus on combining or outsourcing AI algorithms to a cloud via wireless links e.g. wireless LAN IEEE 802.11ac or mobile network 4G/5G. Due to limitations of restricted wireless transmissions in rural areas mainly below 50 MBit/s, resulting longer transfer times can significantly affect inference latency and energy consumption from the perspective of the IoT edge device and deteriorate the response time of the application. In this work, we provide a prototype setup for image processing via Convolutional Neural Networks (CNN) and investigate inference latency and energy consumption of an IoT edge device with a varying wireless link. The complexity of selected pre-trained CNN models is between 300 MFLOPs to 19.6 GFLOPs where FLOPs are Floating Point Operations. The first experiments address the latency and energy consumption by processing CNN models on the IoT device with and without TPU as edge AI accelerator. Following experiments address the latency and energy consumption on the IoT device in cloud processing mode with and without Graphics Processing Unit (GPU) as cloud AI accelerator. The edge device sends input data and receives the results via wireless link from 1 MBit/s to 50 MBit/s. For CNN models with \({\le }564\) MFLOPs edge processing with AI acceleration performs better than cloud processing regarding latency and energy efficiency. Even for complex CNN models with 7.6 GFLOPs edge processing can be useful at limited wireless link data rates up to 14 MBit/s. Edge processing without AI acceleration is only an option for low complexity (\({\le }300\) MFLOPs) and low expected wireless link data rates.
This work is founded by the Joachim Herz Stiftung.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Figueredo, K., Seed, D., Subotic, V.: Preparing for highly scalable and replicable IoT systems. IEEE Internet of Things Mag. 3, 94–98 (2020)
Liu, D., Kong, H., Luo, X., Liu, W., Subramaniam, R.: Bringing AI to Edge: from deep learning’s perspective. Neurocomputing (2021)
Cheng, C.-Y.: et al.: Design of a feeding system for cage aquaculture based on IoT and AI technology. In: 2021 International Symposium on Intelligent Signal Processing and Communication Systems (ISPACS). IEEE (2021)
Chin, T.-W., Ding, R., Zhang, C., Marculescu, D.: Towards efficient model compression via learned global ranking. In: 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). IEEE (2020)
Ahmad, R.W., Gani, A., Hamid, S.H.A., Xia, F., Shiraz, M.: A review on mobile application energy profiling: taxonomy, state-of-the-art, and open research issues. J. Netw. Comput. Appl. 58, 42–59 (2015)
Wang, S., Chen, M., Saad, W., Yin, C.: Federated learning for energy-efficient task computing in wireless networks. In: ICC 2020–2020 IEEE International Conference on Communications (ICC). IEEE (2020)
Yang, Z., Chen, M., Saad, W., Hong, C.S., Shikh-Bahaei, M.: Energy efficient federated learning over wireless communication networks. IEEE Trans. Wireless Commun. 20, 1935–1949 (2021)
Gaddam, A., Wilkin, T., Angelova, M., Gaddam, J.: Detecting sensor faults, anomalies and outliers in the Internet of Things: a survey on the challenges and solutions. Electronics 9, 511 (2020)
Lane, N.D., Bhattacharya, S., Georgiev, P., Forlivesi, C., Kawsar, F.: An early resource characterization of deep learning on wearables, smartphones and Internet-of-Things devices. In: Proceedings of the 2015 International Workshop on Internet of Things towards Applications. ACM (2015)
Rodrigues, C., Graham, R., Mikel, L.: SyNERGY: An energy measurement and prediction framework for convolutional neural networks on Jetson TX1. In: Proceedings of the International Conference on Parallel and Distributed Processing Techniques and Applications. PDPTA (2018)
Rodrigues, C.F., Riley, G., Lujan, M.: Energy predictive models for convolutional neural networks on mobile platforms (2020)
Joy-IT, JT-TC66C. Datasheet (2021). https://joy-it.net/de/products/JT-TC66C. Accessed 05 Apr 2022
keras, Keras applications - available models. Website (2022). https://keras.io/api/applications/. Accessed 17 Mar 2022
Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: ImageNet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition. IEEE (2009)
Tan, M., Le, Q.V., MixConv: mixed depthwise convolutional kernels BMVC. arXiv preprint arXiv:1907.09595 (2019)
He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). IEEE (2016)
Cao, K., Gao, J., Choi, K.-N., Duan, L.: Learning a hierarchical global attention for image classification. Future Internet 12, 178 (2020)
Jiang, H., Li, Q., Li, Y.: Post training quantization after neural network. In: 2022 14th International Conference on Computer Research and Development (ICCRD). IEEE (2022)
Ignatov, A., Malivenko, G., et al.: Fast and accurate quantized camera scene detection on smartphones, mobile AI 2021 challenge: Report. In: 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW). IEEE (2021)
Google, Tensorflow models on the edge TPU (2020). https://coral.ai/docs/edgetpu/models-intro/. Accessed 27 Jan 2022
Natarov, R., et al.: Artefacts in EEG signals epileptic seizure prediction using edge devices. In: 2020 9th Mediterranean Conference on Embedded Computing (MECO). IEEE (2020)
Cisco Systems, Cisco annual internet report. Statistic (2018). www.cisco.com/c/en/us/solutions/collateral/executive-perspectives/annual-internet-report/white-paper-c11-741490.pdf. Accessed 05 Apr 2022
Acknowledgements
This publication results from the research of the Center of Excellence CoSA at the Technische Hochschule Lübeck and is funded by the Joachim Herz Stiftung in the joint project PASBADIA, Germany. Horst Hellbrück is an adjunct professor at the Institute of Telematics of University of Lübeck.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Hauschild, S., Hellbrück, H. (2022). Latency and Energy Consumption of Convolutional Neural Network Models from IoT Edge Perspective. In: González-Vidal, A., Mohamed Abdelgawad, A., Sabir, E., Ziegler, S., Ladid, L. (eds) Internet of Things. GIoTS 2022. Lecture Notes in Computer Science, vol 13533. Springer, Cham. https://doi.org/10.1007/978-3-031-20936-9_31
Download citation
DOI: https://doi.org/10.1007/978-3-031-20936-9_31
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-20935-2
Online ISBN: 978-3-031-20936-9
eBook Packages: Computer ScienceComputer Science (R0)