International Journal of Computer Vision

, Volume 12, Issue 2–3, pp 209–230 | Cite as

Using intermediate objects to improve the efficiency of visual search

  • Lambert E. Wixson
  • Dana H. Ballard


When using a mobile camera to search for a target object, it is often important to maximize the efficiency of the search. We consider a method for increasing efficiency by searching only those subregions that are especially likely to contain the object. These subregions are identified via spatial relationships. Searches that use this method repeatedly find an “intermediate” object that commonly participates in a spatial relationship with the target object, and then look for the target in the restricted region specified by this relationship. Intuitively, such searches, calledindirect searches, seem likely to provide efficiency increases when the intermediate objects can be recognized at low resolutions and hence can be found with little extra overhead, and when they significantly restrict the area that must be searched for the target. But what is the magnitude of this increase, and upon what other factors does efficiency depend? Although the idea of exploiting spatial relationships has been used in vision systems before, few have quantitatively examined these questions.

We present a mathematical model of search efficiency that identifies the factors affecting efficiency and can be used to predict their effects. The model predicts that, in typical situations, indirect search provides up to an 8-fold increase in efficiency. Besides being useful as an analysis tool, the model is also suitable for use in an online system for selecting intermediate objects.


Image Processing Artificial Intelligence Computer Vision Vision System Analysis Tool 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. Ahlswede, R., and Wegener, I. 1987.Search Problems, Wiley: New York.Google Scholar
  2. Aloimonos, J. 1990. Purposive and qualitative active vision,AAAI Qualitative Vision Workshop, pp. 1–5.Google Scholar
  3. Bajcsy, R. 1988. Active perception,Proc. IEEE, 76: 996–1005, August.Google Scholar
  4. Ballard, D.H., and Brown, C.M. 1982.Computer Vision, Prentice-Hall: Englewood Cliffs, NJ.Google Scholar
  5. Ballard, D.H., and Brown, C.M. 1992. Principles of animate vision,Comput. Vis., Graph., Image Process., 56(1): 3–21.Google Scholar
  6. Barrow, H.G., and Tenenbaum, J.M. 1976. MSYS: a system for reasoning about scenes, Technical Note 121, AI Center, SRI International, March.Google Scholar
  7. Bolle, R.M., Califano, A., and Kjeldsen, R. 1989. Data and model driven foveation, Research report, Exploratory Computer Vision Group, IBM T.J. Watson Research Center.Google Scholar
  8. Bolles, R.C 1977. Verification vision for programmable assembly,Proc. 5th Intern. Joint Conf. Artif. Intell., Cambridge, MA.Google Scholar
  9. Burt, P.J. 1988. Smart sensing within a pyramid vision machine,Proc. IEEE, 76: 1006–1015, August.Google Scholar
  10. Garey, M.R., and Johnson, D.S. 1979.Computers and Intractability: A Guide to the Theory of NP-Completeness, Freeman: New York.Google Scholar
  11. Garvey, T.D. 1976. Perceptual strategies for purposive vision, Technical Note 117, SRI International, September.Google Scholar
  12. Johnson, D.T., and Schubert, L.K. 1982. A planning control strategy that allows for the cost of planning,6th European Meeting on Cybernetics and Systems Research, April.Google Scholar
  13. Larsen, R.J., and Marx, M.L. 1981.An Introduction to Mathematical Statistics and its Applications, Prentice-Hall: Englewood Cliffs, NJ.Google Scholar
  14. Maver, J., and Bajcsy, R. 1993. Occlusions as a guide for planning the next view,IEEE Trans. Patt. Anal. Mach. Intell., 15: 417–433, May.Google Scholar
  15. McKeown, Jr., D.M., Harvey, Jr., W.A., and McDermott, J. 1985. Rule-based interpretation of aerial imagery,IEEE Trans. Patt. Anal. Mach. Intell., 7: 570–585, September.Google Scholar
  16. Reece, D.A. 1992. Selective perception for robot driving, Tech. Rept. CMU-CS-92-139, Carnegie Mellon Computer Science, May.Google Scholar
  17. Reece, D.A., and Shafer, S. 1991. Using active vision to simplify perception for robot driving, Tech. Rept. CMU-CS-91-199, Carnegie Mellon Computer Science, November.Google Scholar
  18. Rimey, R.D., and Brown, C.M. 1992. Where to look next using a Bayes net: Incorporating geometric relations,Proc. 2nd Europ. Conf. Comput. Vis., Ligure, Italy.Google Scholar
  19. Rimey, R.D., and Brown, C.M. 1993. Control of selective perception using Bayes nets and decision theory,Intern. Comput. Vis., this issue.Google Scholar
  20. Russell, D.M. 1978. Constraint networks: modeling and inferring object locations by constraints, Tech. Rept. 38, University of Rochester Computer Science Dept., August.Google Scholar
  21. Sarachik, K.B., and Grimson, W.E.L. 1993. Gaussian error models for object recognition,Proc. Conf. Comput. Vis. Patt. Recog., June.Google Scholar
  22. Swain, M.J. 1990. Color indexing, Tech. Rept. 360, University of Rochester Computer Science Dept.Google Scholar
  23. Swain, M.J., Kahn, R.E., and Ballard, D.H. 1992. Low resolution cues for guiding saccadic eye movements,Proc. IEEE Conf. Comput. Vis. Patt. Recog., Urbana Champaign, IL, June.Google Scholar
  24. Tarabanis, K., Tsai, R.Y., and Allen, P.K. 1992. The MVP sensor planning system for robotic vision tasks, Tech. Rept., Columbia University Computer Science Department.Google Scholar
  25. Tsotsos, J.K. 1992. Active vs. passive visual search: Which is more efficient?,Intern. J. Comp. Vis., 7: 2.Google Scholar
  26. Van Trees, H.L. 1968.Detection, Estimation, and Modulation Theory, vol. 1, Wiley: New York.Google Scholar
  27. Wilkes, D., and Tsotsos, J.K. 1992. Active object recognition,Proc. IEEE Conf. Comput. Vis. Patt. Recog., Urbana Champaign, June.Google Scholar
  28. Wixson, L.E. 1992. Exploiting World Structure to Efficiently Search for Objects, Tech. Rept. 434, University of Rochester Computer Science Department, July.Google Scholar
  29. Wixson, L.E. 1994.Searching for Objects in 3D Space, Ph.D. thesis, University of Rochester Computer Science Dept., forthcoming.Google Scholar
  30. Wixson, L.E., and Ballard, D.H. 1989. Real-time detection of multicolored objects,SPIE Sensor Fusion II: Human and Machine Strategies, vol. 1198, November.Google Scholar

Copyright information

© Kluwer Academic Publishers 1994

Authors and Affiliations

  • Lambert E. Wixson
    • 1
  • Dana H. Ballard
    • 1
  1. 1.Computer Science Dept.University of RochesterRochester

Personalised recommendations