Skip to main content
SpringerLink
Log in

Real-time detection of colored objects in multiple camera streams with off-the-shelf hardware components

  • Special Issue
  • Published:
Journal of Real-Time Image Processing Aims and scope Submit manuscript

Abstract

We describe RTblob, a high speed vision system that detects objects in cluttered scenes based on their color and shape at a speed of over 800 frames/s. Because the system is available as open-source software and relies only on off-the-shelf PC hardware components, it can provide the basis for multiple application scenarios. As an illustrative example, we show how RTblob can be used in a robotic table tennis scenario to estimate ball trajectories through 3D space simultaneously from four cameras images at a speed of 200 Hz.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

Similar content being viewed by others

References

  1. Handmann, U., Kalinke, T., Tzomakas, C., Werner, M., Seelen, W.: An image processing system for driver assistance. Image Vis. Comput. 18(5), 367–376 (2000)

    Article  Google Scholar 

  2. Miura, J., Kanda, T., Shirai, Y.: An active vision system for real-time traffic sign recognition. In: IEEE Intelligent Transportation Systems, pp. 52–57 (2000)

  3. Tock D., Craw. I.: Tracking and measuring drivers’ eyes. Image Vis. Comput. 14(8), 541–547 (1996)

    Article  Google Scholar 

  4. Kirishima T., Sato K., Chihara, K.: Real-time gesture recognition by learning and selective control of visual interest points. IEEE Trans. Pattern Anal. Mach. Intell. 27(3), 351–364 (2005)

    Article  Google Scholar 

  5. Liang, R.H., Ouhyoung, M.: A real-time continuous gesture recognition system for sign language. In: International Conference on Automatic Face and Gesture Recognition, pp. 558–567 (1998)

  6. Stauffer, C., Grimson, W.E.L.: Learning patterns of activity using real-time tracking. IEEE Trans. Pattern Anal. Mach. Intell. 22(8), 747–757 (2000)

    Article  Google Scholar 

  7. Williams, B., Klein, G., Reid, I.: Real-time SLAM relocalisation. In: International Conference on Computer Vision (2007)

  8. Davison, A., Mayol, W., Murray, D.: Real-time localisation and mapping with wearable active vision. In: Proceedings of the IEEE International Symposium on Mixed and Augmented Reality, Tokyo (2003)

  9. Comport, A., Marchand, E., Chaumette, F.: Statistically robust 2d visual servoing. IEEE Trans. Robot. 22(2), 415–421 (2006)

    Article  Google Scholar 

  10. Frese, U., Bäuml, B., Haidacher, S., Schreiber, G., Schaefer, I., Hähnle, M., Hirzinger, G.: Off-the-shelf vision for a robotic ball catcher. In: Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems, 2001, vol. 3 (2001)

  11. Jähne, B.: Digital Image Processing: Concepts, Algorithms, and Scientific Applications. Springer, Berlin (1995)

    MATH  Google Scholar 

  12. Mizusawa, S., Namiki, A., Ishikawa, M.: Tweezers type tool manipulation by a multifingered hand using a high-speed visusal servoing. In: IEEE/RSJ International Conference on Intelligent Robots and Systems, 2008. IROS 2008, pp. 2709–2714 (2008)

  13. Viola, P., Jones, M.J.: Robust real-time face detection. Int. J. Comput. Vis. 57(2), 137–154 (2004)

    Article  Google Scholar 

  14. Avidan, S.: Support vector tracking. IEEE Trans. Pattern Anal. Mach. Intell. 26(8), 1064–1072 (2004)

    Article  Google Scholar 

  15. Shimizu, K., Hirai, S.: CMOS+FPGA vision system for visual feedback of mechanical systems. In: IEEE International Conference on Robotics and Automation, pp. 2060–2065 (2006)

  16. Watanabe, Y., Komuro, T., Ishikawa, M.: 955-fps real-time shape measurement of a moving/deforming object using high-speed vision for numerous-point analysis. In: IEEE International Conference on Robotics and Automation, pp. 3192–3197 (2007)

  17. Mühlmann, U., Ribo, M., Lang, P., Pinz, A.: A new high speed CMOS camera for real-time tracking applications. In: International Conference on Robotics and Automation, pp. 5195–5200 (2004)

  18. Nakabo, Y., Ishikawa, M., Toyoda, H., Mizuno, S.: 1 ms column parallel vision system and its application of high speed target tracking. In: IEEE International Conference on Robotics and Automation, pp. 650–655 (2000)

  19. Compute Unified Device Architecture Programming Guide, NVIDIA, Santa Clara, CA (2007)

  20. Camera link: specifications of the camera link interface standard for digital cameras and frame grabbers (2000). http://www.machinevisiononline.org/public/articles/index.cfm?cat=129

  21. GigE vision: camera interface standard for machine vision (2005). http://www.machinevisiononline.org/public/articles/index.cfm?cat=167

  22. Tong, X., Lu, H., Liu, Q.: An effective and fast soccer ball detection and tracking method. In: International Conference on Pattern Recognition, vol. 4, pp. 795–798 (2004)

  23. Pingali, G.S., Jean, Y., Carlbom, I.: Real time tracking for enhanced tennis broadcasts. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 260–265 (1998)

  24. d’Orazio, T., Guaragnella, C., Leo, M., Distante, A.: A new algorithm for ball recognition using circle Hough transform and neural classifer. Pattern Recognit. 37, 393–408 (2003)

    Article  Google Scholar 

  25. Kimmel, R.: Demosaicing: image reconstruction from color CCD samples. IEEE Trans. Image Process. 8(9), 1221–1228 (1999)

    Article  Google Scholar 

  26. Lewis, J.: Fast template matching. Vis. Interface 10, 120–123 (1995)

    Google Scholar 

  27. Brunelli, R., Poggio, T.: Template matching: matched spatial filters and beyond. Pattern Recognit. 30(5), 751–768 (1997)

    Article  Google Scholar 

  28. Crow, F.: Summed-area tables for texture mapping. Comput. Graph. 18(3), 207–212 (1984)

    Article  Google Scholar 

  29. Blelloch, G.E.: Prefix Sums and Their Applications. Morgan Kaufmann, Menlo Park (1991)

    Google Scholar 

  30. Harris, M., Sengupta, S., Owens, J.: Parallel prefix sum (scan) with CUDA. GPU Gems 3(39), 851–876 (2007)

    Google Scholar 

  31. Tian, Q., Huhns, M.: Algorithms for subpixel registration. Comput. Vis. Graph. Image Process. 35(2), 220–233 (1986)

    Article  Google Scholar 

  32. Hartley, R., Zisserman, A.: Multiple View Geometry, 2nd edn. Cambridge University Press, London (2000)

  33. Lindeberg, T.: Scale-Space Theory in Computer Vision. Kluwer, Norwell (1994)

  34. Hinz, S.: Fast and subpixel precise blob detection and attribution. In: IEEE International Conference on Image Processing, vol. 3, pp. 457–460 (2005)

  35. Turin, G.: An introduction to matched filters. IRE Trans. Inf. Theory 6(3), 311–329 (1960)

    Article  MathSciNet  Google Scholar 

  36. Zeidler, J., McCool, J., Widrow, B.: Adaptive correlator, 1982. US Patent 4,355,368 (1982)

  37. Flannery, D., Cartwright, S.: Optical adaptive correlator. In: Third Annual Aerospace Applications of Artificial Intelligence Conference, pp. 143–154 (1987)

  38. Sung, K., Poggio, T.: Example-based learning for view-based human face detection. IEEE Trans. Pattern Anal. Mach. Intell. 20(1), 39–51 (1998)

    Article  Google Scholar 

  39. Lampert, C.H., Peters, J.: Active structured learning for high-speed object detection. In: Symposium of the German Pattern Recognition Society, pp. 221–231 (2009)

  40. Triggs, B., McLauchlan, P., Hartley, R., Fitzgibbon, A.: Bundle adjustment—a modern synthesis. In: Vision Algorithms: Theory and Practice. LNCS, vol. 1883, pp. 298–372 (2000)

  41. Freeman, W.T., Adelson, E.H.: The design and use of steerable filters. IEEE Trans. Pattern Anal. Mach. Intell. 13(9), 891–906 (1991)

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Institute of Science and Technology Austria, Am Campus 1, 3400, Klosterneuburg, Austria

    Christoph H. Lampert

  2. Max-Planck Institute for Biological Cybernetics, Spemannstr. 38, 72076, Tübingen, Germany

    Jan Peters

Authors
  1. Christoph H. Lampert
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jan Peters
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Christoph H. Lampert.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Lampert, C.H., Peters, J. Real-time detection of colored objects in multiple camera streams with off-the-shelf hardware components. J Real-Time Image Proc 7, 31–41 (2012). https://doi.org/10.1007/s11554-010-0168-3

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11554-010-0168-3

Keywords

Search

Navigation