Abstract
Innovative enhancement in embedded system platforms, specifically hardware accelerations, significantly influences the application of deep learning in real-world scenarios. These innovations translate human labor efforts into automated intelligent systems employed in various areas such as autonomous driving, robotics, Internet-of-Things (IoT), and numerous other impactful applications. NVIDIA’s Jetson platform is one of the pioneers in offering optimal performance regarding energy efficiency, desirable accuracy, and throughput in the execution of deep learning algorithms. Previously, most benchmarking analysis was based on 2D images with a single deep learning model for each comparison result. In this paper, we implement an end-to-end video-based crime-scene anomaly detection system inputting from surveillance videos and the system is deployed and completely operates on multiple Jetson edge devices (Nano, AGX Xavier, Orin Nano) for benchmarking purposes. The comparison analysis includes the integration of Torch-TensorRT as a software developer kit from NVIDIA for the model performance optimization. The system is built based on the PySlowfast open-source project from Facebook as the coding template. The end-to-end system process comprises the videos collection from camera, data preprocessing pipeline, feature extractor and the anomaly detection. We also provide the experience of an AI-based system deployment on various Jetson Edge devices with Docker technology. Regarding anomaly detectors, a weakly supervised video-based deep learning model called Robust Temporal Feature Magnitude Learning (RTFM) is applied in the system. The approach system reaches 47.56 frames per second (FPS) inference speed on a Jetson edge device with only 3.11 GB RAM usage total. We also discover the promising Jetson device that the AI system achieves 15% better performance than the previous version of Jetson devices while consuming 50% less energy power.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Tian, Y., Pang, G., Chen, Y., Singh, R., Verjans, J.W., Carneiro, G.: Weakly-supervised video anomaly detection with robust temporal feature magnitude learning. In: 2021 IEEE/CVF International Conference on Computer Vision (ICCV), Montreal, QC, Canada, pp. 4955–4966 (2021). https://doi.org/10.1109/ICCV48922.2021.00493
Haoqi, F., Yanghao, L., Bo, X., Wan-Yen, L., Christoph, F.: PySlowfast (2020). https://github.com/facebookresearch/slowfast
NVIDIA: Torch-TensorRT, 08 April 2020. https://github.com/NVIDIA/TensorRT
Mayur, R., Parate, K.M., Bhurchandi, A.K.: Anomaly detection in residential video surveillance on edge devices in IoT framework, July 2021
Piciarelli, C., Micheloni, C., Foresti, G.L.: Trajectory-based anomalous event detection. IEEE Trans. Circuits Syst. Video Technol. 18(11), 1544–1554 (2008). https://doi.org/10.1109/TCSVT.2008.2005599
Baller, S.P., Jindal, A., Chadha, M., Gerndt, M.: DeepEdgeBench: benchmarking deep neural networks on edge devices. In: 2021 IEEE International Conference on Cloud Engineering (IC2E), San Francisco, CA, USA, pp. 20–30 (2021). https://doi.org/10.1109/IC2E52221.2021.00016
NVIDIA: Jetson Nano, October 2020. https://developer.nvidia.com/embedded/jetson-nano-developer-kit
NVIDIA: Jetson AGX Xavier, September 2018. https://www.nvidia.com/en-us/autonomous-machines/embedded-systems/jetson-agx-xavier
NVIDIA: Jetson Orin Nano, January 2023. https://www.nvidia.com/en-us/autonomous-machines/embedded-systems/jetson-orin/
Logitech: C920 HD PRO Webcam (2012). https://www.logitech.com/en-gb/products/webcams/c920-pro-hd-webcam.960-001055.html
Docker, Inc: Docker (2013). https://www.docker.com/
Marc-André, C., Veronika, C., Eric, G., Ghyslain, G.: Multiple instance learning: a survey of problem characteristics and applications. Pattern Recognit. 77, 329–353 (2018). https://doi.org/10.1016/j.patcog.2017.10.009. ISSN 0031-3203
Huang, Y., Guo, Y., Gao, C.: Efficient parallel inflated 3D convolution architecture for action recognition. IEEE Access 8, 45753–45765 (2020). https://doi.org/10.1109/ACCESS.2020.2978223
Zhang, Q., Cui, Z., Niu, X., Geng, S., Qiao, Y.: Image segmentation with pyramid dilated convolution based on ResNet and U-Net. In: Liu, D., Xie, S., Li, Y., Zhao, D., El-Alfy, E.S. (eds.) Neural Information Processing. Lecture Notes in Computer Science, vol. 10635, pp. 364–372. Springer, Heidelberg (2017). https://doi.org/10.1007/978-3-319-70096-0_38
Yu, J., Hong, J.: SARNet: self-attention assisted ranking network for temporal action proposal generation. In: 2021 IEEE International Conference on Systems, Man, and Cybernetics (SMC), Melbourne, Australia, pp. 1062–1067 (2021). https://doi.org/10.1109/SMC52423.2021.9659016
Sultani, W., Chen, C., Shah, M.: Real-world anomaly detection in surveillance videos. In: 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA, pp. 6479–6488 (2018). https://doi.org/10.1109/CVPR.2018.00678
Vo, D.T.T., Tran, T.M., Vo, N.D., Nguyen, K.: UIT-anomaly: a modern vietnamese video dataset for anomaly detection. In: 2021 8th NAFOSTED Conference on Information and Computer Science (NICS), Hanoi, Vietnam, pp. 352–357 (2021). https://doi.org/10.1109/NICS54270.2021.9701556
NVIDIA NGC: NVIDIA L4T PyTorch, April 2020. https://catalog.ngc.nvidia.com/orgs/nvidia/containers/l4t-pytorch
Ullah, S., Kim, D.-H.: Benchmarking Jetson platform for 3D point-cloud and hyper-spectral image classification. In: 2020 IEEE International Conference on Big Data and Smart Computing (BigComp), Busan, Korea (South), pp. 477–482 (2020). https://doi.org/10.1109/BigComp48618.2020.00-21
Süzen, A.A., Duman, B., Şen, B.: Benchmark analysis of Jetson TX2, Jetson Nano and raspberry PI using deep-CNN. In: 2020 International Congress on Human-Computer Interaction, Optimization and Robotic Applications (HORA), Ankara, Turkey, pp. 1–5 (2020). https://doi.org/10.1109/HORA49412.2020.9152915
Acknowledgment
Grateful for Mr. Dat Thanh Vo, from University of Windsor, Canada, for the support on the process of designing a 3D cover box.
Grateful for Mr. Anh Duy Pham, from Hochschule Bonn-Rhein-Sieg Sankt Augustin, Germany for the constructive comments during the deployment process.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2024 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Pham, H.V., Tran, T.G., Le, C.D., Le, A.D., Vo, H.B. (2024). Benchmarking Jetson Edge Devices with an End-to-End Video-Based Anomaly Detection System. In: Arai, K. (eds) Advances in Information and Communication. FICC 2024. Lecture Notes in Networks and Systems, vol 920. Springer, Cham. https://doi.org/10.1007/978-3-031-53963-3_25
Download citation
DOI: https://doi.org/10.1007/978-3-031-53963-3_25
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-53962-6
Online ISBN: 978-3-031-53963-3
eBook Packages: Intelligent Technologies and RoboticsIntelligent Technologies and Robotics (R0)