Skip to main content
SpringerLink
Log in

Embedded Vision Systems: A Review of the Literature

  • Conference paper
  • First Online:
Applied Reconfigurable Computing. Architectures, Tools, and Applications (ARC 2018)

Part of the book series: Lecture Notes in Computer Science ((LNTCS,volume 10824))

Included in the following conference series:

Abstract

Over the past two decades, the use of low power Field Programmable Gate Arrays (FPGA) for the acceleration of various vision systems mainly on embedded devices have become widespread. The reconfigurable and parallel nature of the FPGA opens up new opportunities to speed-up computationally intensive vision and neural algorithms on embedded and portable devices. This paper presents a comprehensive review of embedded vision algorithms and applications over the past decade. The review will discuss vision based systems and approaches, and how they have been implemented on embedded devices. Topics covered include image acquisition, preprocessing, object detection and tracking, recognition as well as high-level classification. This is followed by an outline of the advantages and disadvantages of the various embedded implementations. Finally, an overview of the challenges in the field and future research trends are presented. This review is expected to serve as a tutorial and reference source for embedded computer vision systems.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    https://www.xilinx.com/products/silicon-devices/soc/zynq-ultrascale-mpsoc.html.

  2. 2.

    https://www.altera.com/products/soc/overview.html.

References

  1. Abeydeera, M., Karunaratne, M., Karunaratne, G., De Silva, K., Pasqual, A.: 4K real-time HEVC decoder on an FPGA. IEEE Trans. Circuit Systems Video Technol. 26(1), 236–249 (2016)

    Article  Google Scholar 

  2. Albo-Canals, J., Ortega, S., Perdices, S., Badalov, A., Vilasis-Cardona, X.: Embedded low-power low-cost camera sensor based on FPGA and its applications in mobile robots. In: 19th IEEE International Conference on Electronics, Circuits, and Systems, pp. 336–339 (2012)

    Google Scholar 

  3. Andargie, F.A., Rose, J., Austin, T., Bertacco, V.: Energy efficient object detection on the mobile GP-GPU. In: 2017 IEEE AFRICON, pp. 945–950, September 2017

    Google Scholar 

  4. Appiah, K., Hunter, A., Dickinson, P., Meng, H.: Implementation and applications of tri-state self-organizing maps on FPGA. IEEE Trans. Circuits Syst. Video Technol. 22(8), 1150–1160 (2012)

    Article  Google Scholar 

  5. Appiah, K., Hunter, A., Lotfi, A., Waltham, C., Dickinson, P.: Human behavioural analysis with self-organizing map for ambient assisted living. In: 2014 IEEE International Conference on Fuzzy Systems (FUZZ-IEEE), pp. 2430–2437, July 2014

    Google Scholar 

  6. Appiah, K., Hunter, A., Dickinson, P., Meng, H.: Accelerated hardware video object segmentation: from foreground detection to connected components labelling. Comput. Vis. Image Underst. 114(11), 1282–1291 (2010)

    Article  Google Scholar 

  7. Athi, M.V., Zekavat, S.R., Struthers, A.A.: Real-time signal processing of massive sensor arrays via a parallel fast converging SVD algorithm: latency, throughput, and resource analysis. IEEE Sens. J. 16(8), 2519–2526 (2016)

    Article  Google Scholar 

  8. Banz, C., Hesselbarth, S., Flatt, H., Blume, H., Pirsch, P.: Real-time stereo vision system using semi-global matching disparity estimation: architecture and FPGA-implementation. In: International Conference on Embedded Computer Systems (SAMOS), pp. 93–101 (2010)

    Google Scholar 

  9. Barbu, A., She, Y., Ding, L., Gramajo, G.: Feature selection with annealing for computer vision and big data learning. IEEE Trans. Pattern Anal. Mach. Intell. 39(2), 272–286 (2017)

    Article  Google Scholar 

  10. Basha, S.M., Kannan, M.: Design and implementation of low-power motion estimation based on modified full-search block motion estimation. J. Comput. Sci. 21, 327–332 (2017)

    Article  Google Scholar 

  11. Belbachir, A.N., Hofstatter, M., Litzenberger, M., Schon, P.: High-speed embedded-object analysis using a dual-line timed-address-event temporal-contrast vision sensor. IEEE Trans. Ind. Electron. 58(3), 770–783 (2011)

    Article  Google Scholar 

  12. Bettoni, M., Urgese, G., Kobayashi, Y., Macii, E., Acquaviva, A.: A convolutional neural network fully implemented on FPGA for embedded platforms. In: 2017 New Generation of CAS (NGCAS), pp. 49–52, September 2017

    Google Scholar 

  13. Bhowmik, D., Garcia, P., Wallace, A., Stewart, R., Michaelson, G.: Power efficient dataflow design for a heterogeneous smart camera architecture. In: Conference on Design and Architectures for Signal and Image Processing (DASIP 2017), August 2017

    Google Scholar 

  14. Camunas-Mesa, L., Acosta-Jimenez, A., Zamarreno-Ramos, C., Serrano-Gotarredona, T., Linares-Barranco, B.: A 32x32 pixel convolution processor chip for address event vision sensors with 155 ns event latency and 20 meps throughput. IEEE Trans. Circuits Syst. I: Regular Papers 58(4), 777–790 (2011)

    Article  MathSciNet  Google Scholar 

  15. Cesetti, A., Frontoni, E., Mancini, A., Zingaretti, P., Longhi, S.: A vision-based guidance system for UAV navigation and safe landing using natural landmarks. J. Intell. Robot. Syst. 57(1–4), 233 (2010)

    Article  Google Scholar 

  16. Chen, T., Du, Z., Sun, N., Wang, J., Wu, C., Chen, Y., Temam, O.: Diannao: a small-footprint high-throughput accelerator for ubiquitous machine-learning. In: Proceedings of 19th International Conference on Architectural Support for Programming Languages and Operating Systems, ASPLOS 2014, pp. 269–284. ACM, New York (2014)

    Google Scholar 

  17. Chen, Y.L., Wu, B.F., Huang, H.Y., Fan, C.J.: A real-time vision system for nighttime vehicle detection and traffic surveillance. IEEE Trans. Ind. Electron. 58(5), 2030–2044 (2011)

    Article  Google Scholar 

  18. Colangelo, P., Luebbers, E., Huang, R., Margala, M., Nealis, K.: Application of convolutional neural networks on Intel; Xeon; processor with integrated FPGA. In: 2017 IEEE High Performance Extreme Computing Conference (HPEC), pp. 1–7, September 2017

    Google Scholar 

  19. Courbariaux, M., Bengio, Y.: BinaryNet: training deep neural networks with weights and activations constrained to +1 or \(-\)1. CoRR abs/1602.02830 (2016)

    Google Scholar 

  20. Eker, J., Janneck, J.: CAL language report: specification of the CAL actor language (2003)

    Google Scholar 

  21. Fischer, T.: Model of all known spatial maps in primary visual cortex. Master’s thesis, University of Edinburghs (2014)

    Google Scholar 

  22. Flores-Delgado, J., Martínez-Santos, L., Lozano, R., Gonzalez-Hernandez, I., Mercado, D.: Embedded control using monocular vision: Face tracking. In: 2017 International Conference on Unmanned Aircraft Systems (ICUAS), pp. 1285–1291. IEEE (2017)

    Google Scholar 

  23. Frintrop, S., Rome, E., Christensen, H.I.: Computational visual attention systems and their cognitive foundations: a survey. ACM Trans. Appl. Percept. 7(1), 6:1–6:39 (2010)

    Article  Google Scholar 

  24. Gantala, A., Nehru, K., Telagam, N., Anjaneyulu, P., Swathi, D.: Human tracking system using beagle board-xM. Int. J. Appl. Eng. Res. 12(16), 5665–5669 (2017)

    Google Scholar 

  25. Ge, W., Guo, Z., Dong, Y., Chen, Y.: Dynamic background estimation and complementary learning for pixel-wise foreground/background segmentation. Pattern Recogn. 59(Suppl. C), 112–125 (2016)

    Article  Google Scholar 

  26. Gokhale, V., Jin, J., Dundar, A., Martini, B., Culurciello, E.: A 240 g-ops/s mobile coprocessor for deep neural networks. In: The IEEE Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, June 2014

    Google Scholar 

  27. Hashmi, M.F., Shukla, R.J., Keskar, A.G.: Platform independent real time copyright protection embedding and extraction algorithms on android and embedded framework. In: International Symposium on Signal Processing and Information Technology (ISSPIT), pp. 000189–000194 (2014)

    Google Scholar 

  28. He, G., Zhou, D., Li, Y., Chen, Z., Zhang, T., Goto, S.: High-throughput power-efficient VLSI architecture of fractional motion estimation for ultra-HD HEVC video encoding. IEEE Trans. Very Large Scale Integr. VLSI Syst. 23(12), 3138–3142 (2015)

    Article  Google Scholar 

  29. Ho, S.M.H., Hung, C.H.D., Ng, H.C., Wang, M., So, H.K.H.: A parameterizable activation function generator for FPGA-based neural network applications. In: IEEE 25th Annual International Symposium on Field-Programmable Custom Computing Machines (FCCM) (2017)

    Google Scholar 

  30. Horak, K., Zalud, L.: Image processing on raspberry PI for mobile robotics. Int. J. Sig. Process. Syst. 4(2), 1–5 (2016)

    Google Scholar 

  31. Humenberger, M., Schraml, S., Sulzbachner, C., Belbachir, A.N., Srp, A., Vajda, F.: Embedded fall detection with a neural network and bio-inspired stereo vision. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops (2012)

    Google Scholar 

  32. Humenberger, M., Zinner, C., Weber, M., Kubinger, W., Vincze, M.: A fast stereo matching algorithm suitable for embedded real-time systems. Comput. Vis. Image Underst. 114(11), 1180–1202 (2010)

    Article  Google Scholar 

  33. Islam, S.M.S., Rahman, S., Rahman, M.M., Dey, E.K., Shoyaib, M.: Application of deep learning to computer vision: a comprehensive study. In: 2016 5th International Conference on Informatics, Electronics and Vision (ICIEV), pp. 592–597, May 2016

    Google Scholar 

  34. Jain, V., Patel, D.: A GPU based implementation of robust face detection system. Procedia Comput. Sci. 87(Suppl. 1), 156–163 (2016). Fourth International Conference on Recent Trends in Computer Science & Engineering (ICRTCSE 2016)

    Article  Google Scholar 

  35. Jasani, B.A., Lam, S.K., Meher, P.K., Wu, M.: Threshold-guided design and optimization for Harris corner detector architecture. IEEE TCSVT PP(99), 1 (2017)

    Google Scholar 

  36. Jin, S., Cho, J., Dai Pham, X., Lee, K.M., Park, S.K., Kim, M., Jeon, J.W.: FPGA design and implementation of a real-time stereo vision system. IEEE Trans. Circuits Syst. Video Technol. 20(1), 15–26 (2010)

    Article  Google Scholar 

  37. Jing, X., Gong, C., Wang, Z., Li, X., Ma, Z.: Remote live-video security surveillance via mobile robot with raspberry Pi IP camera. In: Huang, Y.A., Wu, H., Liu, H., Yin, Z. (eds.) ICIRA 2017. LNCS (LNAI), vol. 10463, pp. 776–788. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-65292-4_67

    Chapter  Google Scholar 

  38. Kerr, D., McGinnity, T., Coleman, S., Clogenson, M.: A biologically inspired spiking model of visual processing for image feature detection. Neurocomputing 158(C), 268–280 (2015)

    Article  Google Scholar 

  39. Khan, M.U.K., Khan, A., Kyung, C.M.: EBSCAM: background subtraction for ubiquitous computing. IEEE Trans. Very Large Scale Integr. Syst. 25(1), 35–47 (2017)

    Article  Google Scholar 

  40. Krizhevsky, A., Sutskever, I., Hinton, G.E.: Imagenet classification with deep convolutional neural networks. Commun. ACM 60(6), 84–90 (2017)

    Article  Google Scholar 

  41. Li, Y., Chen, L., Benson, B., Kastner, R.: Determining the suitability of FPGAs for a low-cost, low-power underwater acoustic modem. In: Deng, W. (ed.) Future Control and Automation. LNEE, vol. 173, pp. 509–517. Springer, Heidelberg (2012). https://doi.org/10.1007/978-3-642-31003-4_65

    Chapter  Google Scholar 

  42. Lin, F., Dong, X., Chen, B.M., Lum, K.Y., Lee, T.H.: A robust real-time embedded vision system on an unmanned rotorcraft for ground target following. IEEE Trans. Ind. Electron. 59(2), 1038–1049 (2012)

    Article  Google Scholar 

  43. Liu, Z., Dou, Y., Jiang, J., Xu, J., Li, S., Zhou, Y., Xu, Y.: Throughput-optimized FPGA accelerator for deep convolutional neural networks. ACM Trans. Reconf. Technol. Syst. (TRETS) 10(3), 17 (2017)

    Google Scholar 

  44. Loureiro, R., Lopes, A., Carona, C., Almeida, D., Faria, F., Garrote, L., Premebida, C., Nunes, U.J.: ISR-RobotHead: robotic head with LCD-based emotional expressiveness. In: 2017 IEEE 5th Portuguese Meeting on Bioengineering (ENBENG), pp. 1–4, February 2017

    Google Scholar 

  45. Ma, X., Borbon, J.R., Najjar, W., Roy-Chowdhury, A.K.: Optimizing hardware design for human action recognition. In: 2016 26th International Conference on Field Programmable Logic and Applications (FPL), pp. 1–11, August 2016

    Google Scholar 

  46. Mazumdar, A., Moreau, T., Kim, S., Cowan, M., Alaghi, A., Ceze, L., Oskin, M., Sathe, V.: Exploring computation-communication tradeoffs in camera systems. In: 2017 IEEE International Symposium on Workload Characterization (IISWC) (2017)

    Google Scholar 

  47. Morison, G., Jenkins, M.D., Buggy, T., Barrie, P.: An implementation focused approach to teaching image processing and machine vision - from theory to beagleboard. In: European Embedded Design in Education and Research Conference (EDERC), pp. 274–277 (2014)

    Google Scholar 

  48. Nguyen, H.Q., Loan, T.T.K., Mao, B.D., Huh, E.N.: Low cost real-time system monitoring using raspberry PI. In: 2015 7th International Conference on Ubiquitous and Future Networks, pp. 857–859, July 2015

    Google Scholar 

  49. Nurvitadhi, E., Sheffield, D., Sim, J., Mishra, A., Venkatesh, G., Marr, D.: Accelerating binarized neural networks: comparison of FPGA, CPU, GPU, and ASIC. In: 2016 International Conference on Field-Programmable Technology (FPT), pp. 77–84, December 2016

    Google Scholar 

  50. Oleynikova, H., Honegger, D., Pollefeys, M.: Reactive avoidance using embedded stereo vision for MAV flight. In: International Conference on Robotics and Automation (ICRA), pp. 50–56 (2015)

    Google Scholar 

  51. Park, J.S., Kim, H.E., Kim, L.S.: A 182 mW 94.3 f/s in full HD pattern-matching based image recognition accelerator for an embedded vision system in 0.13-mm CMOS technology. IEEE Trans. Circuit Syst. Video Technol. 23(5), 832–845 (2013)

    Article  Google Scholar 

  52. Qiu, J., Wang, J., Yao, S., Guo, K., Li, B., Zhou, E., Yu, J., Tang, T., Xu, N., Song, S., Wang, Y., Yang, H.: Going deeper with embedded FPGA platform for convolutional neural network. In: Proceedings of 2016 ACM/SIGDA International Symposium on Field-Programmable Gate Arrays, FPGA 2016, pp. 26–35. ACM, New York (2016)

    Google Scholar 

  53. Rister, B., Wang, G., Wu, M., Cavallaro, J.R.: A fast and efficient sift detector using the mobile GPU. In: Proceedings of IEEE ICASSP, pp. 2674–2678 (2013)

    Google Scholar 

  54. Romoth, J., Porrmann, M., Ruckertr, U.: Survey of FPGA applications in the period 2000–2015. Technical report, Bielefeld University, Germany, March 2017

    Google Scholar 

  55. Sahani, M., Mohanty, M.N.: Realization of different algorithms using raspberry Pi for real-time image processing application. In: Jain, L.C., Patnaik, S., Ichalkaranje, N. (eds.) Intelligent Computing, Communication and Devices. AISC, vol. 309, pp. 473–479. Springer, New Delhi (2015). https://doi.org/10.1007/978-81-322-2009-1_53

    Chapter  Google Scholar 

  56. Sharma, G., Kumar, K.: Prototyping of image enhancement algorithms using beagle board for rural health monitoring. In: International Conference on Recent innovations in Science, Management, Education and Technology, pp. 346–358, August 2016

    Google Scholar 

  57. Singh, R., Ranasinghe, L.: Accelerating computer vision on mobile embedded platforms. In: 2016 IEEE Region 10 Conference (TENCON), pp. 3131–3134, November 2016

    Google Scholar 

  58. Solari, F., Chessa, M., Medathati, K., Kornprobst, P.: What can we expect from a classical V1-MT feedforward architecture for optical flow estimation? Sig. Process. Image Commun. 49(1), 250–257 (2015)

    Google Scholar 

  59. Stewart, R.J., Bhowmik, D., Wallace, A.M., Michaelson, G.: Profile guided dataflow transformation for FPGAs and CPUs. Sig. Process. Syst. 87(1), 3–20 (2017)

    Article  Google Scholar 

  60. Su, J., Liu, J., Thomas, D.B., Cheung, P.Y.: Neural network based reinforcement learning acceleration on FPGA platforms. SIGARCH Comput. Arch. News 44(4), 68–73 (2017)

    Article  Google Scholar 

  61. Sugiura, T., Yu, J., Takeuchi, Y., Imai, M.: A low-energy ASIP with flexible exponential Golomb codec for lossless data compression toward artificial vision systems. In: 2015 IEEE Biomedical Circuits and Systems Conference (BioCAS), pp. 1–4, October 2015

    Google Scholar 

  62. Szegedy, C., Vanhoucke, V., Ioffe, S., Shlens, J., Wojna, Z.: Rethinking the inception architecture for computer vision. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 2016

    Google Scholar 

  63. Tanabe, Y., Maruyama, T.: Fast and accurate optical flow estimation using FPGA. SIGARCH Comput. Arch. News 42(4), 27–32 (2014)

    Article  Google Scholar 

  64. Ttofis, C., Kyrkou, C., Theocharides, T.: A low-cost real-time embedded stereo vision system for accurate disparity estimation based on guided image filtering. IEEE Trans. Comput. 65(9), 2678–2693 (2016)

    Article  MathSciNet  Google Scholar 

  65. Velez, G., Cortés, A., Nieto, M., Vélez, I., Otaegui, O.: A reconfigurable embedded vision system for advanced driver assistance. J. Real-Time Image Process. 10(4), 725–739 (2015)

    Article  Google Scholar 

  66. Wang, G., Xiong, Y., Yun, J., Cavallaro, J.R.: Accelerating computer vision algorithms using OpenCL framework on the mobile GPU - a case study. In: 2013 IEEE International Conference on Acoustics, Speech and Signal Processing, pp. 2629–2633, May 2013

    Google Scholar 

  67. Wang, K., Yu, J.: An embedded vision system for robotic fish navigation. In: International Conference on Computer Application and System Modeling (ICCASM), vol. 4, pp. V4–333. IEEE (2010)

    Google Scholar 

  68. Xu, H., Shen, Y.: Target tracking control of mobile robot in diversified manoeuvre modes with a low cost embedded vision system. J. Ind. Robot 40(3), 275–287 (2013)

    Article  Google Scholar 

  69. Yang, M., Crenshaw, J., Augustine, B., Mareachen, R., Wu, Y.: AdaBoost-based face detection for embedded systems. Comput. Vis. Image Underst. 114(11), 1116–1125 (2010)

    Article  Google Scholar 

  70. Yang, X., Wu, Z., Yu, J.: Design and implementation of a robotic shark with a novel embedded vision system. In: 2016 IEEE International Conference on Robotics and Biomimetics (ROBIO), pp. 841–846. IEEE (2016)

    Google Scholar 

  71. Yi, S., Yoon, I., Oh, C., Yi, Y.: Real-time integrated face detection and recognition on embedded GPGPUs. In: 2014 IEEE 12th Symposium on Embedded Systems for Real-Time Multimedia (ESTIMedia), pp. 98–107, October 2014

    Google Scholar 

  72. Yun, K., Choi, J.Y.: Robust and fast moving object detection in a non-stationary camera via foreground probability based sampling. In: 2015 IEEE International Conference on Image Processing (ICIP), pp. 4897–4901, September 2015

    Google Scholar 

  73. Zhang, B., Zhao, C., Mei, K., Zheng, N., et al.: Hierarchical and parallel pipelined heterogeneous SoC for embedded vision processing. IEEE Trans. Circuit Syst. Video Technol. (2017)

    Google Scholar 

  74. Zhao, R., Niu, X., Wu, Y., Luk, W., Liu, Q.: Optimizing CNN-based object detection algorithms on embedded FPGA platforms. In: Wong, S., Beck, A.C., Bertels, K., Carro, L. (eds.) ARC 2017. LNCS, vol. 10216, pp. 255–267. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-56258-2_22

    Chapter  Google Scholar 

Download references

Acknowledgement

We acknowledge the support of two HEIF Impact fellowships at Sheffield Hallam University.

Author information

Authors and Affiliations

  1. Department of Computing, Sheffield Hallam University, Sheffield, S1 1WB, UK

    Deepayan Bhowmik & Kofi Appiah

Authors
  1. Deepayan Bhowmik
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Kofi Appiah
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Deepayan Bhowmik .

Editor information

Editors and Affiliations

  1. Technological Educational Institute of Western Greece, Antirrio, Greece

    Nikolaos Voros

  2. Ruhr-Universität Bochum, Bochum, Germany

    Michael Huebner

  3. Technological Educational Institute of Western Greece, Antirrio, Greece

    Georgios Keramidas

  4. Technische Universität Dresden, Dresden, Germany

    Diana Goehringer

  5. Technological Educational Institute of Western Greece, Antirio, Greece

    Christos Antonopoulos

  6. INESC-ID, Lisbon, Portugal

    Pedro C. Diniz

Rights and permissions

Reprints and permissions

Copyright information

© 2018 Springer International Publishing AG, part of Springer Nature

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Bhowmik, D., Appiah, K. (2018). Embedded Vision Systems: A Review of the Literature. In: Voros, N., Huebner, M., Keramidas, G., Goehringer, D., Antonopoulos, C., Diniz, P. (eds) Applied Reconfigurable Computing. Architectures, Tools, and Applications. ARC 2018. Lecture Notes in Computer Science(), vol 10824. Springer, Cham. https://doi.org/10.1007/978-3-319-78890-6_17

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-78890-6_17

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-78889-0

  • Online ISBN: 978-3-319-78890-6

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation