A Real-Time Vision Module for Interactive Perceptual Agents

  • Bruce A. Maxwell
  • Nathaniel Fairfield
  • Nikolas Johnson
  • Pukar Malla
  • Paul Dickson
  • Suor Kim
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 2095)


Interactive robotics demands real-time visual information about the environment. Real time vision processing, however, places a heavy load on the robot’s limited resources, and must accommodate other processes such as speech recognition, animated face displays, communication with other robots, navigation and control. For our entries in the 2000 American Association for Artificial Intelligence robot contest, we developed a vision module capable of providing real-time information about ten or more operators while maintaining at least a 20Hz frame rate and leaving sufficient processor time for the robot’s other capabilities. The vision module uses a probabilistic scheduling algorithm to ensure both timely information flow and a fast frame capture. The vision module makes its information available to other modules in the robot architecture through a shared memory structure. The information provided by the vision module includes the operator information along with a confidence measure and a time stamp. Because of this design, our robots are able to react in a timely manner to a wide variety of visual events.


Mobile Robot Face Detection Panoramic Image Facial Animation Event Loop 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. [1]
    E. Bizzi, S. Giszter, E. Loeb, F.A. Mussa-Ivaldi, and P. Saltiel, “Modular organization of motor behavior in the frog’s spinal cord”, Trends in Neuroscience, 18:442–446.Google Scholar
  2. [2]
    R. P. Bonasso, R. J. Firby, E. Gat, D. Kortenkamp, D. P. Miller, and M. G. Slack, “Experiments with an architecture for intelligent, reactive agents”, J. of Experimental & Theoretical Artificial Intelligence, 9(2/3):237–256, 1997.Google Scholar
  3. [3]
    R. A. Brooks, “A robust layered control system for a mobile robot”, IEEE J. of Robotics and Automation, vol. 2,no. 1, 1986.Google Scholar
  4. [4]
    J. Bryson, “Cross-Paradigm Analysis of Autonomous Agent Architecture”, J. of Experimental and Theoretical Artificial Intelligence, vol. 12,no. 2, pp 165–190, 2000.zbMATHCrossRefGoogle Scholar
  5. [5]
    D. R. Forsey and R. H. Bartels, “Hierarchical B-spline refinement”, in Computer Graphics (SIGGRAPH’ 88), 22(4):205–212, August, 1988.CrossRefGoogle Scholar
  6. [6]
    E. Gat, Reliable Goal-Directed Reactive Control of Autonomous Mobile Robots, Ph.D. thesis, Virginia Polytechnic Institute and State University, 1991.Google Scholar
  7. [7]
    IBM ViaVoiceTM Outloud API Reference Version 5.0, November 1999.Google Scholar
  8. [8]
    E. C. Ifeachor and B. W. Jervis, Digital Signal Processing. A Practical Approach, Addison Wesley Publishing Company, 1995.Google Scholar
  9. [9]
    D. Kortenkamp, R. P. Bonasso, and R. Murphy (ed.), Artificial Intelligence and Mobile Robots, AAAI Press/MIT Press, Cambridge, 1998.Google Scholar
  10. [10]
    B. A. Maxwell, L. A. Meeden, N. Addo, P. Dickson, N. Fairfield, N. Johnson, E. Jones, S. Kim, P. Malla, M. Murphy, B. Rutter, E. Silk, “REAPER: A Reflexive Architecture for Perceptive Agents”, AI Magazine, spring 2001.Google Scholar
  11. [11]
    B. A. Maxwell, L. A. Meeden, N. Addo, L. Brown, P. Dickson, J. Ng, S. Olshfski, E. Silk, and J. Wales, “Alfred: The Robot Waiter Who Remembers You,” in Proceedings of AAAI Workshop on Robotics, July, 1999. To appear in J. Autonomous Robots, 2001.Google Scholar
  12. [12]
    B. Maxwell, S. Anderson, D. Gomez-Ibanez, E. Gordon, B. Reese, M. Lafary, T. Thompson, M. Trosen, and A. Tomson, “Using Vision to Guide an Hors d’Oeuvres Serving Robot”, IEEE Workshop on Perception for Mobile Agents, June 1999.Google Scholar
  13. [13]
    H. P. Moravec, A. E. Elfes, “High Resolution Maps from Wide Angle Sonar”, Proceedings of IEEE Int’l Conf. on Robotics and Automation, March 1985, pp 116–21.Google Scholar
  14. [14]
    J. Neider, T. Davis, and M. Woo, OpenGL Programming Guide: The Official Guide to Learning OpenGL, Addison-Wesley, Reading, MA, 1993.Google Scholar
  15. [15]
    F. I. Parke and K. Waters, Computer Facial Animation, A. K. Peters, Wellesley, MA, 1996.Google Scholar
  16. [16]
    A. Rosenfeld and J. L. Pfaltz, “Sequential operations in digital picture processing”, ACM, 13:471–494, October 1966.zbMATHCrossRefGoogle Scholar
  17. [17]
    D. Scharstein and A. Briggs, “Fast Recognition of Self-Similar Landmarks”, IEEE Workshop on Perception for Mobile Agents, June 1999.Google Scholar
  18. [18]
    H. Wu, Q. Chen, and M. Yachida, “Face Detection From Color Images Using a Fuzzy Pattern Matching Method”, IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 21,no. 6, June 1999.Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2001

Authors and Affiliations

  • Bruce A. Maxwell
    • 1
  • Nathaniel Fairfield
    • 1
  • Nikolas Johnson
    • 1
  • Pukar Malla
    • 1
  • Paul Dickson
    • 1
  • Suor Kim
    • 1
  1. 1.Swarthmore CollegeSwarthmore

Personalised recommendations