Skip to main content
SpringerLink
Log in

Real-time binocular smooth pursuit

  • Published:
International Journal of Computer Vision Aims and scope Submit manuscript

Abstract

This article examines the problem of a moving robot tracking a moving object with its cameras, without requiring the ability to recognize the target to distinguish it from distracting surroundings. A novel aspect of the approach taken is the use of controlled camera movements to simplify the visual processing necessary to keep the cameras locked on the target. A gaze-holding system implemented on a robot's binocular head demonstrates this approach. Even while the robot is moving, the cameras are able to track an object that rotates and moves in three dimensions.

The central idea is that localizing attention in 3-D space makes precategorical visual processing sufficient to hold gaze. Visual fixation can help separate the target object from distracting surroundings. Converged cameras produce a horopter (surface of zero stereo disparity) in the scene. Binocular features with no disparity can be located with a simple filter, showing the object's location in the image. Similarly, an object that is being tracked is imaged near the center of the field of view, so spatially localized processing helps concentrate visual attention on the target. Instead of requiring a way to recognize the target, the system relies on active control of camera movements and binocular fixation segmentation to locate the target.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  • Agre, P., and Chapman, D. 1987. Pengi: an implementation of a theory of activity,AAAI, pp. 268–272.

  • Allen, P. 1989. Real-time motion tracking using spatio-temporal filters,Proc. DARPA Image Understanding Workshop, Palo Alto, CA.

  • Bar-Shalom, Y., and Fortmann, T. 1988.Tracking and Data Association, Academic Press: San Diego, CA.

    Google Scholar 

  • Bogert, B., Healy, M., and Tukey, J. 1963. The quefrency analysis of time series for echoes: Cepstrum, pseudo-autocovariance, cross-cepstrum, and saphe cracking. In M. Rosenblatt, ed.,Proc. Symp. Time Series Analysis, Wiley: New York, pp. 209–243.

    Google Scholar 

  • Brown, C. 1990. Prediction and cooperation in gaze control,Biological Cybernetics, vol. 63, pp. 61–70.

    Google Scholar 

  • Brown, C., and Coombs, D. 1991. Notes on control with delay, Tech. Rept. 387, University of Rochester, Computer Science Department, Rochester, New York 14627 USA, August.

    Google Scholar 

  • Carpenter, R. 1988.Movements of the Eyes, 2nd ed., revised and enlarged, Pion: UK.

    Google Scholar 

  • Clark, J., and Ferrier, N. 1988. Modal control of an attentive vision system,Proc. 2nd Intern. Conf. Comput. Vis., Tampa, FL, December 5–8.

  • Coombs, D., and Brown, C. 1991. Cooperative gaze holding in binocular vision,IEEE Control Systems, June.

  • Coombs, D.,Horswill, I., and von Kaenel, P. 1992. Disparity filtering: Proximity detection and segmentation,Proc. 9th SPIE Conf. Intell. Robot. Comput. Vis., Algorithms, Techniques, and Active Vision, Boston, November 15–20.

  • Corke, P., and Paul, R. 1989. Video-rate visual servoing for robots, Tech. Rept. MS-CIS-89-18, Department of Computer and Information Science, University of Pennsylvania, GRASP Lab, Philadelphia, PA 19104, February.

    Google Scholar 

  • Dorf, R. 1980.Modern Control Systems, 3rd ed., Addison-Wesley: Reading, MA.

    Google Scholar 

  • Howard, I. 1982.Human Visual Orientation, Wiley: New York.

    Google Scholar 

  • Jenkin, M. 1991. Using stereomotion to track binocular targets,Proc. Conf. Comput. Vis. Patt. Recog., Maui, June 3–6.

  • Krauzlis, R., and Lisberger, S. 1989. A control systems model of smooth pursuit eye movements with realistic emergent properties,Neural Computation, 1.

  • Land, M. 1975. Similarities in the visual behavior of arthropods and men, In M. Gazzaniga and C. Blakemore, eds.,Handbook of Psychobiology, Academic Press: San Diego, CA, pp. 49–72.

    Google Scholar 

  • Lee, S.W., and Wohn, K. 1988. Tracking moving objects by a mobile camera, Tech. Rept. MS-CIS-8-97, University of Pennsylvania, Computer and Information Science Department, November.

  • Lisberger, S., Morris, E., and Tychsen, L. 1987. Visual motion processing and sensory-motor integration for smooth pursuit eye movements,Ann. Rev. Neurosci. 10:97–129.

    Google Scholar 

  • Matthies, L., Kanade, T., and Szeliski, R. 1989. Kalman filter-based algorithms for estimating depth from image sequences,Intern. J. Comput. Vis. 3:209–236.

    Google Scholar 

  • McDonald, J., Bahill, A., and Friedman, M. 1983. An adaptive control model for human head and eye movements while walking,IEEE Trans. Syst., Man, Cybern. 13 (3):167–174, April.

    Google Scholar 

  • Nishihara, H. 1984. Practical real-time imaging stereo matcher,Optical Engineering 23 (5):536–545, September/October.

    Google Scholar 

  • Olson, T. 1991. Stereopsis for fixating systems,Proc. IEEE Inter. Conf. Syst., Man, Cyb., Charlottesville, VA, October.

  • Olson, T., and Coombs, D. 1991. Real-time vergence control for binocular robots,Intern. J. Comput. Vis. 7 (1):67–89, November.

    Google Scholar 

  • Olson, T., and Lockwood, R. 1992. Fixation-based filtering,Proc. 11th SPIE Conf. Intell. Rob. Comput. Vis. Algorithms, Techniques, and Active Vision, Boston, MA, November 15–20.

  • Pahlavan, K.,Uhlin, T., and Eklundh, J-O. 1992. Integrating primary oculumotor processes,Proc. 2nd Europ. Conf. Comput. Vis., Santa Margherita Ligure, Italy, May 18–23.

  • Papanikolopoulos, N.,Khosla, P., and Kanade, T. 1991. Vision and control techniques for robotic visual tracking,Proc. Intern. IEEE Conf. Robot. Autom.

  • Reading, R. 1983.Binocular Vision: Foundations and Applications, Butterworth: Boston.

    Google Scholar 

  • Rojer, A., and Schwartz, E. 1990. Design considerations for a space-variant visual sensor with complex-logarithmic geometry,Proc. Intern. Conf. Patt. Recog., Philadelphia, PA, June.

  • Shinkman, P., Isley, M., and Rogers, D. 1985. Development of interocular relationships in visual cortex. In R. Aslin, ed.,Advances in Neural and Behavioral Development, Ablex: Norwood, NJ.

    Google Scholar 

  • Thorpe, C. 1983. An analysis of interest operators for FIDO, Tech. Rept. CMU-RI-TR-83-19, Carnegie-Mellon University, December.

  • Tilley, D. 1990. Zebra for MaxVideo: an application of object oriented microprogramming to register level devices, Tech. Rept. 315, University of Rochester, Computer Science Department, 1990.

  • Tölg, S. 1992. Gaze control for an active camera system by modeling human pursuit eye movements,Proc. 11th SPIE Conf. Intell. Robots Comput. Vis. Algorithms, Techniques, and Active Vision, Boston, MA, November 15–20, pp. 585–598.

  • Tsotsos, J. 1988. A ‘complexity level’ analysis of vision,Intern. Comput. Vis. 1(4), January.

  • Tyler, C., and Scott, A. 1979. Binocular vision. In R. Records, ed.,Physiology of the Human Eye and Visual System, pp. 634–674, Harper & Row: New York.

    Google Scholar 

  • Van der Spiegel, J., Kreider, G., Claeys, C., Debusschere, I., Sandini, G., Dario, P., Fantini, F., Bellutti, P., and Soncini, G. 1989. A foveated retina-like sensor using CCD technology. In C. Mead and M. Ismail, eds.,Analog VLSI Implementation of Neural Systems, Kluwer: Norwell, MA.

    Google Scholar 

  • Wavering, A.,Fiala, J.,Roberts, K., and Lumia, R. 1993. TRICLOPS: A high performance trinocular active vision system,Proc. IEEE Intern. Conf. Robotics Autom., Atlanta, GA, May 2–7.

  • Waxman, A., Wong, W., Goldenberg, R., Bayle, S., and Baloch, A. 1988. Robotic eye-head-neck motions and visual navigation reflex learning using adaptive linear neurons,Neural Networks Supplement: Abstracts of 1st INNS Meeting, 1: 365.

    Google Scholar 

  • Whitehead, S., and Ballard, D. 1990. Active perception and reinforcement learning,Neural Computation 2 (4), 1990. (Also inProc. 7th Intern. Conf. Mach. Learn., Morgan Kaufmann: San Mateo, CA, June.)

  • Young, L. 1971. Pursuit eye tracking movements, In P. Bach, C.C. Rita, and J. Hyde, eds.,Control of Eye Movements, Academic Press: San Diego, CA.

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Robot Systems Div., NIST, Building 220, Room B-124, 20899, Gaithersburg, MD

    David Coombs

  2. Computer Science Department, University of Rochester, 14627, Rochester, NY

    Christopher Brown

Authors
  1. David Coombs
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Christopher Brown
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to David Coombs.

Additional information

This work was largely completed while Coombs was affiliated with the University of Rochester.

Robot Systems Division, Manufacturing Engineering Laboratory, National Institute of Standards and Technology, Technology Administration, U.S. Department of Commerce. Product endorsement disclaimer: references to specific brands, equipment, or trade names in this document are made to facilitate understanding and do not imply endorsement by the National Institute of Standards and Technology.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Coombs, D., Brown, C. Real-time binocular smooth pursuit. Int J Comput Vision 11, 147–164 (1993). https://doi.org/10.1007/BF01469226

Download citation

  • Received:

  • Revised:

  • Issue Date:

  • DOI: https://doi.org/10.1007/BF01469226

Keywords

Search

Navigation