Abstract
Real-time video analytics on edge devices for changing scenes remains a difficult task. As edge devices are usually resource-constrained, edge deep neural networks (DNNs) have fewer weights and shallower architectures than general DNNs. As a result, they only perform well in limited scenarios and are sensitive to data drift. In this paper, we introduce EdgeMA, a practical and efficient video analytics system designed to adapt models to shifts in real-world video streams over time, addressing the data drift problem. EdgeMA extracts the gray level co-occurrence matrix based statistical texture feature and uses the Random Forest classifier to detect the domain shift. Moreover, we have incorporated a method of model adaptation based on importance weighting, specifically designed to update models to cope with the label distribution shift. Through rigorous evaluation of EdgeMA on a real-world dataset, our results illustrate that EdgeMA significantly improves inference accuracy.
L. Wang and N. Zhang—These two authors have contributed to this work equally.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Bhardwaj, R., et al.: Ekya: continuous learning of video analytics models on edge compute servers. In: 19th USENIX Symposium on Networked Systems Design and Implementation (NSDI 2022), pp. 119–135 (2022)
Bochkovskiy, A., Wang, C.Y., Liao, H.Y.M.: Yolov4: optimal speed and accuracy of object detection. arXiv preprint arXiv:2004.10934 (2020)
Cutler, A., Cutler, D.R., Stevens, J.R.: Random forests. In: Ensemble Machine Learning, pp. 157–175. Springer, Heidelberg (2012). https://doi.org/10.1007/978-1-4419-9326-7_5
Fang, T., Lu, N., Niu, G., Sugiyama, M.: Rethinking importance weighting for deep learning under distribution shift. Adv. Neural. Inf. Process. Syst. 33, 11996–12007 (2020)
Ghosh, A.M., Grolinger, K.: Edge-cloud computing for internet of things data analytics: embedding intelligence in the edge with deep learning. IEEE Trans. Ind. Inf. 17(3), 2191–2200 (2020)
Gu, Y., Du, Z., Zhang, H., Zhang, X.: An efficient joint training framework for robust small-footprint keyword spotting. In: Yang, H., Pasupa, K., Leung, A.C.-S., Kwok, J.T., Chan, J.H., King, I. (eds.) ICONIP 2020. LNCS, vol. 12532, pp. 12–23. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-63830-6_2
He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 770–778 (2016)
Howard, A.G., et al.: Mobilenets: efficient convolutional neural networks for mobile vision applications. arXiv preprint arXiv:1704.04861 (2017)
Hsieh, K., et al.: Focus: querying large video datasets with low latency and low cost. In: 13th \(\{\)USENIX\(\}\) Symposium on Operating Systems Design and Implementation (\(\{\)OSDI\(\}\) 2018), pp. 269–286 (2018)
Hua, H., Li, Y., Wang, T., Dong, N., Li, W., Cao, J.: Edge computing with artificial intelligence: a machine learning perspective. ACM Comput. Surv. 55(9), 1–35 (2023)
Ibrahim, M.R., Haworth, J., Cheng, T.: Weathernet: recognising weather and visual conditions from street-level images using deep residual learning. ISPRS Int. J. Geo Inf. 8(12), 549 (2019)
Jia, Y., Zhang, X., Lan, L., Luo, Z.: Counterfactual causal adversarial networks for domain adaptation. In: Neural Information Processing: 29th International Conference, ICONIP 2022, Virtual Event, 22–26 November 2022, Proceedings, Part VI, pp. 698–709. Springer, Heidelberg (2023). https://doi.org/10.1007/978-981-99-1645-0_58
Jiang, J., Ananthanarayanan, G., Bodik, P., Sen, S., Stoica, I.: Chameleon: scalable adaptation of video analytics. In: Proceedings of the 2018 Conference of the ACM Special Interest Group on Data Communication, pp. 253–266 (2018)
Kang, D., Emmons, J., Abuzaid, F., Bailis, P., Zaharia, M.: Noscope: optimizing neural network queries over video at scale. Proc. VLDB Endow. 10(11) (2017)
Khani, M., et al.: RECL: responsive resource-efficient continuous learning for video analytics. In: 20th USENIX Symposium on Networked Systems Design and Implementation (NSDI 2023), pp. 917–932 (2023)
Khani, M., Hamadanian, P., Nasr-Esfahany, A., Alizadeh, M.: Real-time video inference on edge devices via adaptive model streaming. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 4572–4582 (2021)
Liao, X.C., Qiu, W.J., Wei, F.F., Chen, W.N.: Combining traffic assignment and traffic signal control for online traffic flow optimization. In: Neural Information Processing: 29th International Conference, ICONIP 2022, Virtual Event, 22–26 November 2022, Proceedings, Part VI, pp. 150–163. Springer, Heidelberg (2023). https://doi.org/10.1007/978-981-99-1645-0_13
Lipton, Z., Wang, Y.X., Smola, A.: Detecting and correcting for label shift with black box predictors. In: International Conference on Machine Learning, pp. 3122–3130. PMLR (2018)
Liu, C., Qu, X., Wang, J., Xiao, J.: Fedet: a communication-efficient federated class-incremental learning framework based on enhanced transformer. arXiv preprint arXiv:2306.15347 (2023)
Maji, K., Sharma, R., Verma, S., Goel, T.: RVFL classifier based ensemble deep learning for early diagnosis of alzheimer’s disease. In: Neural Information Processing: 29th International Conference, ICONIP 2022, Virtual Event, 22–26 November 2022, Proceedings, Part III, pp. 616–626. Springer, Heidelberg (2023). https://doi.org/10.1007/978-3-031-30111-7_52
Moreno-Torres, J.G., Raeder, T., Alaiz-Rodríguez, R., Chawla, N.V., Herrera, F.: A unifying view on dataset shift in classification. Pattern Recogn. 45(1), 521–530 (2012)
Nesterov, Y.: Efficiency of coordinate descent methods on huge-scale optimization problems. SIAM J. Optim. 22(2), 341–362 (2012)
Nigade, V., Wang, L., Bal, H.: Clownfish: edge and cloud symbiosis for video stream analytics. In: 2020 IEEE/ACM Symposium on Edge Computing (SEC), pp. 55–69. IEEE (2020)
Qin, M., Chen, L., Zhao, N., Chen, Y., Yu, F.R., Wei, G.: Power-constrained edge computing with maximum processing capacity for iot networks. IEEE Internet Things J. 6(3), 4330–4343 (2018)
Qu, X., Wang, J., Xiao, J.: Quantization and knowledge distillation for efficient federated learning on edge devices. In: 2020 IEEE 22nd International Conference on High Performance Computing and Communications; IEEE 18th International Conference on Smart City; IEEE 6th International Conference on Data Science and Systems (HPCC/SmartCity/DSS), pp. 967–972. IEEE (2020)
Rojas, R., et al.: Adaboost and the super bowl of classifiers a tutorial introduction to adaptive boosting. Freie University, Berlin, Technical Report (2009)
Shi, W., Dustdar, S.: The promise of edge computing. Computer 49(5), 78–81 (2016)
Tan, M., Le, Q.: Efficientnet: rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114. PMLR (2019)
Wang, L., et al.: Shoggoth: towards efficient edge-cloud collaborative real-time video inference via adaptive online learning. arXiv preprint arXiv:2306.15333 (2023)
Wen, L., et al.: UA-DETRAC: a new benchmark and protocol for multi-object detection and tracking. Comput. Vision Image Underst. 193, 102907 (2020)
Wright, S.J.: Coordinate descent algorithms. Math. Program. 151(1), 3–34 (2015)
Xie, R., Yu, F., Wang, J., Wang, Y., Zhang, L.: Multi-level domain adaptive learning for cross-domain detection. In: Proceedings of the IEEE/CVF International Conference on Computer Vision Workshops, pp. 0–0 (2019)
Zhang, H., Ananthanarayanan, G., Bodik, P., Philipose, M., Bahl, P., Freedman, M.J.: Live video analytics at scale with approximation and delay-tolerance. In: 14th USENIX Symposium on Networked Systems Design and Implementation (2017)
Zhang, X., Zhou, X., Lin, M., Sun, J.: Shufflenet: an extremely efficient convolutional neural network for mobile devices. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 6848–6856 (2018)
Acknowledgements
This work is supported by the Key Research and Development Program of Guangdong Province under Grant No.2021B0101400003, the National Natural Science Foundation of China under Grant No.62072196, and the Creative Research Group Project of NSFC No.61821003.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2024 The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Wang, L. et al. (2024). EdgeMA: Model Adaptation System for Real-Time Video Analytics on Edge Devices. In: Luo, B., Cheng, L., Wu, ZG., Li, H., Li, C. (eds) Neural Information Processing. ICONIP 2023. Lecture Notes in Computer Science, vol 14447. Springer, Singapore. https://doi.org/10.1007/978-981-99-8079-6_23
Download citation
DOI: https://doi.org/10.1007/978-981-99-8079-6_23
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-99-8078-9
Online ISBN: 978-981-99-8079-6
eBook Packages: Computer ScienceComputer Science (R0)