Image classification on IoT edge devices: profiling and modeling

Abstract

With the advent of powerful, low-cost IoT systems, processing data closer to where the data originates, known as edge computing, has become an increasingly viable option. In addition to lowering the cost of networking infrastructures, edge computing reduces edge-cloud delay, which is essential for mission-critical applications. In this paper, we show the feasibility and study the performance of image classification using IoT devices. Specifically, we explore the relationships between various factors of image classification algorithms that may affect energy consumption, such as dataset size, image resolution, algorithm type, algorithm phase, and device hardware. In order to provide a means of predicting the energy consumption of an edge device performing image classification, we investigate the usage of three machine learning algorithms using the data generated from our experiments. The performance as well as the trade-offs for using linear regression, Gaussian process, and random forests are discussed and validated. Our results indicate that the random forest model outperforms the two former algorithms, with an R-squared value of 0.95 and 0.79 for two different validation datasets. The random forest also served as a feature extraction mechanism which enabled us to identify which predictor variables influenced our model the most.

This is a preview of subscription content, log in to check access.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

References

  1. 1.

    Hung, M.: Leading the IoT, Gartner insights on how to lead in a connected world. In: Gartner Reseach, pp. 1–29. https://www.gartner.com/imagesrv/books/iot/iotEbook_digital.pdf (2018)

  2. 2.

    Kumar, A., Goyal, S., Varma, M.: Resource-efficient machine learning in 2 kb ram for the internet of things. In: International Conference on Machine Learning, pp. 1935–1944 (2017)

  3. 3.

    Naha, R.K., Garg, S., Georgakopoulos, D., Jayaraman, P.P., Gao, L., Xiang, Y., Ranjan, R.: Fog computing: survey of trends, architectures, requirements, and research directions. IEEE Acess 6, 47980–48009 (2018)

    Article  Google Scholar 

  4. 4.

    Ni, J., Zhang, K., Lin, X., Shen, X.S.: Securing fog computing for internet of things applications: challenges and solutions. IEEE Commun. Surv. Tutor. 20(1), 601–628 (2017)

    Article  Google Scholar 

  5. 5.

    Lin, J., Yu, W., Zhang, N., Yang, X., Zhang, H., Zhao, W.: A survey on internet of things: architecture, enabling technologies, security and privacy, and applications. IEEE Internet Things J. 4(5), 1125–1142 (2017)

    Article  Google Scholar 

  6. 6.

    Wei, J.: How wearables intersect with the cloud and the internet of things: considerations for the developers of wearables. IEEE Consum. Electron. Mag. 3(3), 53–56 (2014)

    Article  Google Scholar 

  7. 7.

    Metcalf, D., Milliard, S.T.J., Gomez, M., Schwartz, M.: Wearables and the internet of things for health: wearable, interconnected devices promise more efficient and comprehensive health care. IEEE Pulse 7(5), 35–39 (2016)

    Article  Google Scholar 

  8. 8.

    Amirtharaj, I., Groot, T., Dezfouli, B.: Profiling and improving the duty-cycling performance of linux-based iot devices. J. Ambient Intell. Humaniz. Comput. 30, 1–29 (2018)

    Google Scholar 

  9. 9.

    Centre for Energy-Efficient Telecommunications: the power of wireless cloud. https://ceet.unimelb.edu.au/publications/ceet-white-paper-wireless-cloud.pdf (2013)

  10. 10.

    Krizhevsky, A., Sutskever, I., Hinton, G. E.: Imagenet Classification with deep convolutional neural networks. In: Advances in Neural Information Processing Systems, pp. 1097–1105 (2012)

  11. 11.

    Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V., Rabinovich, A.: Going deeper with convolutions. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1–9 (2015)

  12. 12.

    He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 770–778 (2016)

  13. 13.

    Shafique, M., Theocharides, T., Bouganis, C.-S., Hanif, M.A., Khalid, F., Hafız, R., Rehman, S.: An overview of next-generation architectures for machine learning: roadmap, opportunities and challenges in the IoT era. In: Design, Automation & Test in Europe Conference & Exhibition. IEEE, pp. 827–832 (2018)

  14. 14.

    ARM Ltd., Deploying neural networks on Android-based mobile and embedded devices. https://developer.arm.com/solutions/machine-learning-on-arm/developer-material/how-to-guides/optimizing-neural-networks-for-mobile-and-embedded-devices-with-tensorflow/deploying-neural-networks-on-android-based-mobile-and-embedded-devices-single-page

  15. 15.

    Cui, W., Kim, Y., Rosing, T.S.: Cross-platform machine learning characterization for task allocation in IoT ecosystems. In: IEEE 7th Annual Computing and Communication Workshop and Conference (CCWC), pp. 1–7 (2017)

  16. 16.

    Carbajales, R.J., Zennaro, M., Pietrosemoli, E., Freitag, F.: Energy-efficient internet of things monitoring with low-capacity devices. In: IEEE World Forum on Internet of Things: 14–16 December, Milan, Italy: Proceedings. Institute of Electrical and Electronics Engineers (IEEE), pp. 305–310 (2015)

  17. 17.

    Lane, N. D., Bhattacharya S., Georgiev, P., Forlivesi, C., Kawsar, F.: An early resource characterization of deep learning on wearables, smartphones and internet-of-things devices. In: Proceedings of the International Workshop on Internet of Things Towards Applications. ACM, pp. 7–12 (2015)

  18. 18.

    Top 10 Open Source Linux and Android SBCs, Aug 2014. https://www.linux.com/news/top-10-open-source-linux-and-android-sbcs

  19. 19.

    Top 10 Best Open-Spec Hacker SBCs, Jun 2016. https://www.linux.com/news/raspberry-pi-stays-top-survey-81-open-spec-sbcs

  20. 20.

    BCM2837 ARM Peripherals. https://web.stanford.edu/class/cs140e/docs/BCM2837-ARM-Peripherals.pdf

  21. 21.

    BeagleBone Black Wireless. https://beagleboard.org/black-wireless

  22. 22.

    Eclipse Foundation, Key Trends from the IoT Developer Survey, 2018. https://iot.eclipse.org/resources/iot-developer-survey/iot-developer-survey-2018.pdf

  23. 23.

    Dang, D.: Reducing the Cost, Power and Size of Connectivity in IoT Designs, Feb, 2018. http://www.ti.com/lit/wp/sway013/sway013.pdf

  24. 24.

    Dezfouli, B., Amirtharaj, I., Li, C.-C.: EMPIOT: an energy measurement platform for wireless IoT devices. J. Netwk. Comput. Appl. 121, 135–148 (2018)

    Article  Google Scholar 

  25. 25.

    Mahmud, M., Kaiser, M.S., Hussain, A., Vassanelli, S.: Applications of deep learning and reinforcement learning to biological data. IEEE Trans. Neural Netw. Learn. Syst. 29(6), 2063–2079 (2018)

    MathSciNet  Article  Google Scholar 

  26. 26.

    Manning, C., Raghavan, P., Schütze, H.: Introduction to Information Retrieval. Cambridge University Press, Cambridge (2008)

    Google Scholar 

  27. 27.

    Harrell, F.E.: Ordinal Logistic Regression, in Regression modeling strategies, pp. 311–325. Springer, Berlin (2015)

    Google Scholar 

  28. 28.

    Wang, L.: Support Vector Machines: Theory and Applications, vol. 177. Springer, Berlin (2005)

    Google Scholar 

  29. 29.

    Pedregosa, F., Varoquaux, G., Gramfort, A., Michel, V., Thirion, B., Grisel, O., Blondel, M., Prettenhofer, P., Weiss, R., Dubourg, V., Vanderplas, J., Passos, A., Cournapeau, D., Brucher, M., Perrot, M., Duchesnay, E.: Scikit-learn: machine learning in python. J. Mach. Learn. Res. 12, 2825–2830 (2011)

    MathSciNet  MATH  Google Scholar 

  30. 30.

    LeCun, Y., Bottou, L., Bengio, Y., Haffner, P., et al.: Gradient-based learning applied to document recognition. Proc. IEEE 86(11), 2278–2324 (1998)

    Article  Google Scholar 

  31. 31.

    Xiao, H., Rasul, K., Vollgraf, R.: Fashion-mnist: a novel image dataset for benchmarking machine learning algorithms, arXiv preprint arXiv:1708.07747 (2017)

  32. 32.

    Krizhevsky, A.: Learning multiple layers of features from tiny images. University of Toronto, Technical Report (2009)

  33. 33.

    Kermany, D.S., Goldbaum, M., Cai, W., Valentim, C.C., Liang, H., Baxter, S.L., McKeown, A., Yang, G., Wu, X., Yan, F., et al.: Identifying medical diagnoses and treatable diseases by image-based deep learning. Cell 172(5), 1122–1131 (2018)

    Article  Google Scholar 

  34. 34.

    Huang, G.B., Ramesh, M., Berg, T., Learned-Miller, E.: labeled faces in the wild: a database for studying face recognition in unconstrained environments. University of Massachusetts, Amherst. Technical Report, vol. 07–49 (October 2007)

  35. 35.

    Wasserman, L.: All of Nonparametric Statistics. Springer, New York (2006)

    Google Scholar 

  36. 36.

    Griffiths, E., Assana, S., Whitehouse, K.: Privacy-preserving image processing with binocular thermal cameras. Proc. ACM Interact. Mob. Wearable Ubiquitous Technol. 1(4), 133:1–133:25 (2018)

    Article  Google Scholar 

  37. 37.

    Suter, Y., Rummel, C., Wiest, R., Reyes, M.: Fast and uncertainty-aware cerebral cortex morphometry estimation using random forest regression. In: International Symposium on Biomedical Imaging (ISBI). IEEE, pp. 1052–1055 (2018)

  38. 38.

    Feng, Y., Wang, S.: A forecast for bicycle rental demand based on random forests and multiple linear regression. In: 16th International Conference Computer and Information Science (ICIS). IEEE, pp. 101–105 (2017)

  39. 39.

    Seber, G.A., Lee, A.J.: Linear Regression Analysis, vol. 329. Wiley, New York (2012)

    Google Scholar 

  40. 40.

    Griffin, G., Holub, A., Perona, P.: Caltech-256 object category dataset, California Institute of Technology, Technical Report 7694. (2007). http://authors.library.caltech.edu/7694

  41. 41.

    Mamaev, A.: Flowers Dataset, Reviewed Dataset from Kaggle. https://www.kaggle.com/alxmamaev/flowers-recognition

  42. 42.

    James, G., Witten, D., Hastie, T., Tibshirani, R.: An Introduction to Statistical Learning. Springer, New York (2013)

    Google Scholar 

Download references

Acknowledgements

This research has been partially supported by the Latimer Energy Lab and Santa Clara Valley Water District (Grant# SCWD02). This project involves the development of a flood monitoring system where Linux-based wireless systems, which rely on solar or battery power, capture images for analysis using ML to classify and report the debris carried by rivers and streams.

Author information

Affiliations

Authors

Corresponding author

Correspondence to Behnam Dezfouli.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Abdel Magid, S., Petrini, F. & Dezfouli, B. Image classification on IoT edge devices: profiling and modeling. Cluster Comput 23, 1025–1043 (2020). https://doi.org/10.1007/s10586-019-02971-9

Download citation

Keywords

  • Edge and fog computing
  • Machine learning
  • Energy efficiency
  • Accuracy