Skip to main content

Advertisement

SpringerLink
Log in

An Edge Computing System for Fast Image Recognition Based on Convolutional Neural Network and Petri Net Model

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

As a computer system can precisely detect some target objects, many application scenarios will be developed. In this study, the customized object recognition model was trained by using a software tool, You Only Look Once (YOLO), and burned into one hardware component, Maix Bit, for an image edge computing system, so as to fast recognize humans or objects in the images. In the beginning of system development, a Petri net (PN) model was established to detect all possible abnormal processes and to verify the feasibility and completeness of an edge computing system by using Petri net software tool, Workflow Petri net Designer (WoPeD). Compared with many other development boards, Maix Bit is smaller, more flexible, more economical, and more adaptable to those needs of the edge side scenarios. When equipped with an Artificial Intelligence (AI) chip, Kendryte K210, it is suitable for object recognition due to its superior power consumption, film frames, and processors. YOLO, mainly characterized by its fast speed in using the convolutional neural network (CNN) to recognize objects, can be used to predict the positions and types of multiple objects concurrently. Also, CNN can be used to detect and recognize targets from end to end promptly, when configured to train an image recognition model. Finally, the experimental results have shown that the promising mean Average Precision (mAP), 78.6%, was obtained, which outperforms other existing systems.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13

Similar content being viewed by others

Data availability

Data sharing not applicable to this article as no datasets were generated or analysed during the current study.

References

  1. Abed R, Bahroun S, Zagrouba E (2021) KeyFrame extraction based on face quality measurement and convolutional neural network for efficient face recognition in videos. Multimed Tools Appli vol. 80, pp. 23157-23179 https://www.researchgate.net/publication/343482686_KeyFrame_extraction_based_on_face_quality_measurement_and_convolutional_neural_network_for_efficient_face_recognition_in_videos

  2. AlphaGo. [Online] Available: https://theinitium.com/article/20170105-opinion-AI-Go/. 2020.

  3. Architecture of Internet of Things. [Online] Available: http://www.csie.ntpu.edu.tw/~yschen/course/2012-1/WNMC/ch14.pdf?fbclid=IwAR0_wkVnI1MlPUB4-IOg8xDMtmqgQpinVSNnXbrfhXxSdFkVt4Ie8f3ECPw. 2020.

  4. Build your own YOLO recognition model: An example of citrus recognition. [Online] Available: https://chtseng.wordpress.com/2018/09/01/%E5%BB%BA%E7%AB%8B%E8%87%AA%E5%B7%B1%E7%9A%84yolo%E8%BE%A8%E8%AD%98%E6%A8%A1%E5%9E%8B-%E4%BB%A5%E6%9F%91%E6%A9%98%E8%BE%A8%E8%AD%98%E7%82%BA%E4%BE%8B/. 2020.

  5. Chung YH Probe into the efficiency and application of YOLO neural network. Master's Thesis, Department of Electronic Engineering, National Yunlin University of Science and Technology, 2020. https://www.google.com.tw/search?sa=N&sxsrf=ALiCzsbcwKpC3-WZeP7CXe4tI_Z-mLh5qw:1655967479518&source=univ&tbm=isch&q=Probe+into+the+efficiency+and+application+of+YOLO+neural+network.+Master%27s+Thesis,+Department+of+Electronic+Engineering,+National+Yunlin+University+of+Science+and+Technology,+2020.&fir=y-dJZsHS3kucGM%252Ci06ijvkJO_bqPM%252C_%253BXd2ZA4tP192ZTM%252Ci06ijvkJO_bqPM%252C_%253BZRd-g5b3BSiJSM%252Ci06ijvkJO_bqPM%252C_%253BPpBpMxIs3nya8M%252CLFC2TWgcKnrZeM%252C_%253BEa0xR8lb2PYwRM%252Ci06ijvkJO_bqPM%252C_%253BT5Z5Qr_zWp7SuM%252C2mxSM1dBNNnqxM%252C_%253BCxIh-LUtvr0geM%252CGsv-VD6-JJ__vM%252C_%253B_AxVuB6wskoETM%252Ci06ijvkJO_bqPM%252C_%253B5ie4bo6bHvfPJM%252Ci06ijvkJO_bqPM%252C_%253B2CXCh0vU7yi_XM%252Cdve2ggdfXU6bZM%252C_&usg=AI4_-kQRp3gXPGoBQgAGdmVny3CF5fxt1A&ved=2ahUKEwiB_YDL_8L4AhUcJqYKHdYSBpM4ChCMmQR6BAgMEAI&biw=1424&bih=726&dpr=1

  6. Computer vision object detection models: R-CNN, Fast R-CNN, Faster R-CNN, Mask R-CNN, YOLO [Online] Available: https://medium.com/cubo-ai/%E7%89%A9%E9%AB%94%E5%81%B5%E6%B8%AC-object-detection-740096ec4540. 2020.

  7. Convert .weights files to.pb files using darkflow. [Online] Available: https://www.twblogs.net/a/5c866140bd9eee35cd6a1abe. 2020.

  8. DFRobot M1 & M1W AI+IOT K210 Modules. [Online] Available: https://www.mouser.tw/new/dfrobot/dfrobot-m1-m1w-ai-iot-k210-modules/. 2020.

  9. Distribution of Foundry Revenue 2020-2021. [Online] Available: https://www.trendforce.com.tw/presscenter/news/20201229-10615.html. 2021.

  10. Fang J, Zhang M, Ye Z, Shi J, Wei J (2021) Smart collaborative optimizations strategy for mobile edge computing based on deep reinforcement learning. Computers and Electrical Engineering. vol. 96, part A. https://dl.acm.org/doi/abs/10.1016/j.compeleceng.2021.107539

  11. FDDB: Face Detection Data Set and Benchmark [Online] Available: http://vis-www.cs.umass.edu/fddb/index.html. 2020.

  12. GitHub AlexeyAB/darknet. [Online] Available: https://github.com/AlexeyAB/darknet#how-to-train-to-detect-your-custom-objects. 2020

  13. GitHub Isabek/XmlToTxt. [Online] Available: https://github.com/Isabek/XmlToTxt. 2020.

  14. GitHub kendryte/kflash.py [Online] Available: https://github.com/kendryte/kflash.py. 2020

  15. GitHub sipeed/Maix_Toolbox. [Online] Available: https://github.com/sipeed/Maix_Toolbox/blob/master/tflite2kmodel.sh. 2020.

  16. GitHub tfreytag/ WoPeD. [Online] Available: https://github.com/tfreytag/WoPeD. 2020.

  17. GitHub thtrieu/darkflow. [Online] Available: https://github.com/thtrieu/darkflow. 2020.

  18. GitHub tzutalin/labelImg. [Online] Available: https://github.com/tzutalin/labelImg. 2020.

  19. Hor N, Fekri-Ershad S (2019) Image retrieval approach based on local texture information derived from predefined patterns and spatial domain information. International Journal of Computer Science. Engineering. 8(6):246–254 https://arxiv.org/abs/1912.12978

    Google Scholar 

  20. Introduction MaixPy DOC. [Online] Available: https://maixpy.sipeed.com/en/. 2020

  21. Kayhan N, Fekri-Ershad S (2021) Content based image retrieval based on weighted fusion of texture and color features derived from modified local binary patterns and local neighborhood difference patterns. Multimedia Tools and Applications. 80:32763–32790. https://doi.org/10.1007/s11042-021-11217-z

    Article  Google Scholar 

  22. MAIX Bit AI K210 Pinout. [Online] Available: https://steward-fu.github.io/website/mcu/k210/pinout.htm. 2020

  23. Mean Average Precision (mAP) An Index to Evaluate Quality of Object Detection Model [Online] Available: https://medium.com/curiosity-and-exploration/mean-average-precision-map-%E8%A9%95%E4%BC%B0%E7%89%A9%E9%AB%94%E5%81%B5%E6%B8%AC%E6%A8%A1%E5%9E%8B%E5%A5%BD%E5%A3%9E%E7%9A%84%E6%8C%87%E6%A8%99-70a2d2872eb0. 2020.

  24. Petri net. Wikipedia. [Online] Available: https://en.wikipedia.org/wiki/Petri_net. 2020.

  25. Singh VK, Kolekar MH (2022) Deep learning empowered COVID-19 diagnosis using chest CT scan images for collaborative edge-cloud computing platform. Multimed Tools Appli 81:3–30. https://doi.org/10.1007/s11042-021-11158-7

    Article  Google Scholar 

  26. Singh RD, Mittal A, Bhatia R (2019) 3D convolutional neural network for object recognition: a review. Multimedia Tools and Applications. 78:15951–15995. https://doi.org/10.1007/s11042-018-6912-6

    Article  Google Scholar 

  27. Sipeed Maix Bit. [Online] Available: https://cn.maixpy.sipeed.com/en/hardware/bit.html. 2020.

  28. Understanding Webduino Blockly. [Online] Available: https://tutorials.webduino.io/zh-tw/docs/basic/blockly/blockly-information.html. 2020.

  29. Webduino Smart (Introduction). [Online] Available: https://tutorials.webduino.io/zh-tw/docs/basic/board/smart-information.html. 2020.

  30. WoPeD (Workflow Petri net Designer). [Online] Available: https://woped.dhbw-karlsruhe.de/. 2020.

  31. Workflow Petri Net Designer. [Online] Available: https://woped.dhbw-karlsruhe.de/. 2020.

  32. Yin H, Chen Y, Xiong J, Xia R, Xie J, Yang K (2021) An improved local binary pattern method for pollen image classification and recognition. Comput Electri Engin; vol. 90 https://www.semanticscholar.org/paper/An-improved-local-binary-pattern-method-for-pollen-Yin-Chen/cdef3474986044116ae5861a3a50bef1a60edb5d

  33. YOLO: Real-Time Object Detection. [Online] Available: https://pjreddie.com/darknet/yolo/. 2020.

Download references

Acknowledgments

The authors are grateful to the anonymous reviewers for their constructive comments which have improved the quality of this paper. During the second revision period, Mr. Frank H.C. Shen is highly appreciated for his great efforts in editing. Also, this work was supported by the Ministry of Science and Technology, Taiwan, under grants MOST 107-2221-E-845-001-MY3 and MOST 110-2221-E-845-002-.

Author information

Authors and Affiliations

  1. Department of Computer Science, University of Taipei, 1, Ai-Kao W. Rd., Taipei 100, Taipei, Taiwan

    Cheng-Ying Yang

  2. Department of Electronic Engineering, Ming Chi University of Technology, 84, Gongzhuan Rd., Taishan District, New Taipei City 243, New Taipei, Taiwan

    Yi-Nan Lin, Yi-Chih Tung & Jia-Fu Lin

  3. Department of Electrical Engineering, Ming Chi University of Technology, 84, Gongzhuan Rd., Taishan District, New Taipei City 243, New Taipei, Taiwan

    Sheng-Kuan Wang

  4. Department of Computer Science and Information Engineering, National Taipei University, 151, University Rd., Sanxia District, New Taipei City, 237, New Taipei, Taiwan

    Victor R.L. Shen

  5. Department of Information Management, Chaoyang University of Technology, Taichung, Taiwan

    Victor R.L. Shen

Authors
  1. Cheng-Ying Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yi-Nan Lin
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Sheng-Kuan Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Victor R.L. Shen
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Yi-Chih Tung
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Jia-Fu Lin
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Victor R.L. Shen.

Ethics declarations

Conflict of interests

The authors declare that there is no conflict of interests in this paper.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Yang, CY., Lin, YN., Wang, SK. et al. An Edge Computing System for Fast Image Recognition Based on Convolutional Neural Network and Petri Net Model. Multimed Tools Appl 83, 12849–12873 (2024). https://doi.org/10.1007/s11042-023-15388-9

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-023-15388-9

Keywords

Search

Navigation