International Journal of Computer Vision

, Volume 1, Issue 1, pp 73–103 | Cite as

Interpreting line drawings of curved objects

  • Jitendra Malik


In this paper, we study the problem of interpreting line drawings of scenes composed of opaque regular solid objects bounded by piecewise smooth surfaces with no markings or texture on them. It is assumed that the line drawing has been formed by orthographic projection of such a scene under general viewpoint, that the line drawing is error free, and that there are no lines due to shadows or specularities. Our definition implicitly excludes laminae, wires, and the apices of cones.

A major component of the interpretation of line drawings is line labelling. By line labelling we mean (a) classification of each image curve as corresponding to either a depth or orientation discontinuity in the scene, and (b) further subclassification of each kind of discontinuity. For a depth discontinuity we determine whether it is a limb—a locus of points on the surface where the line of sight is tangent to the surface—or an occluding edge—a tangent plane discontinuity of the surface. For an orientation discontinuity, we determine whether it corresponds to a convex or concave edge. This paper presents the first mathematically rigorous scheme for labelling line drawings of the class of scenes described. Previous schemes for labelling line drawings of scenes containing curved objects were heuristic, incomplete, and lacked proper mathematical justification.

By analyzing the projection of the neighborhoods of different kinds of points on a piecewise smooth surface, we are able to catalog all local labelling possibilities for the different types of junctions in a line drawing. An algorithm is developed which utilizes this catalog to determine all legal labellings of the line drawing. A local minimum complexity rule—at each vertex select those labellings which correspond to the minimum number of faces meeting at the vertex—is used in order to prune highly counter-intuitive interpretations. The labelling scheme was implemented and tested on a number of line drawings. The labellings obtained are few and by and large in accordance with human interpretations.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    H.G. Barrow and J.M. Tenenbaum, “Interpreting line drawings as three-dimensional surfaces,” Artificial Intelligence 17, pp. 75–116, 1981.Google Scholar
  2. 2.
    T.O. Binford, “Inferring surfaces from images,” Artificial Intelligence 17, pp. 205–244, 1981.Google Scholar
  3. 3.
    J.M. Brady and A. Yuille, “An extremum principle for shape from contour,” in Proceedings of IJCAI-8, Karlsruhe, 1983, pp. 969–972.Google Scholar
  4. 4.
    I. Chakravarty, “A generalized line and junetion labelling scheme with applications to scene analysis,” IEEE Transactions on Pattern Analysis and Machine Intelligence 1, pp. 202–205, 1979.Google Scholar
  5. 5.
    M.B. Clowes, “On seeing things,” Artificial Intelligence 2, pp. 79–116, 1971.Google Scholar
  6. 6.
    H.S.M. Coxeter, Regular Polytopes, New York: Dover, 1973.Google Scholar
  7. 7.
    S.W. Draper, “Reasoning about depth in line-drawing interpretation,” PhD thesis, Sussex University, 1980.Google Scholar
  8. 8.
    S.W. Draper, “The use of gradient and dual space in line-drawing interpretation,” Artificial Intelligence 17, pp. 461–508, 1981.Google Scholar
  9. 9.
    G. Falk, “Interpretation of imperfect line data as a three-dimensional scene,” Artificial Intelligence 4, pp. 101–144, 1972.Google Scholar
  10. 10.
    A. Guzman, “Computer recognition of three-dimensional objects in a scene,” MIT Tech. Rept. MAC-TR-59, 1968.Google Scholar
  11. 11.
    D.A. Huffman, “Impossible objects as nonsense sentences,” Machine Intelligence 6, pp. 295–323, 1971.Google Scholar
  12. 12.
    D.A. Huffman, “Realizable configurations of lines in pictures of polyhedra,” Machine Intelligence 8, pp. 493–509, 1977.Google Scholar
  13. 13.
    T. Kanade, “A theory of the Origami World,” Tech. Rept. CMU-CS-144, Computer Science Department, Carnegie Mellon University, 1978.Google Scholar
  14. 14.
    T. Kanade, “Recovery of the three-dimensional shape of an object from a single view,” Artificial Intelligence 17, pp. 409–460, 1981.Google Scholar
  15. 15.
    L. Kirousis and C.H. Papadimitriou, “The complexity of recognizing polyhedral scenes,” Tech. Rept. STAN-CS-84-1015, Computer Science Department, Stanford University, 1984.Google Scholar
  16. 16.
    J.J. Koenderink and A.J.van Doorn, “The shape of smooth objects and the way contours end,” Perception 11, pp. 129–137, 1982.Google Scholar
  17. 17.
    J.J. Koenderink, “What does the occluding contour tell us about solid shape?” Perception 13, pp. 321–330, 1984.Google Scholar
  18. 18.
    S.J. Lee, R.M. Haralick, and M.C. Zhang, “Understanding objects with curved surfaces from a single perspective view of boundaries,” Artificial Intelligence 26, pp. 145–169, 1985.Google Scholar
  19. 19.
    A.K. Mackworth, “Interpreting pictures of polyhedral scenes,” Artificial Intelligence 4, pp. 121–137, 1973.Google Scholar
  20. 20.
    A.K. Mackworth, “Consistency in networks of relations,” Artificial Intelligence 8, pp. 99–118, 1977.Google Scholar
  21. 21.
    A.K. Mackworth and E.C. Freuder, “The complexity of some polynomial network consisteney algorithms for constraint satisfaction problems,” Artificial Intelligence 25, pp. 65–74, 1985.Google Scholar
  22. 22.
    A.K. Mackworth, J.A. Mulder, and W.S. Havens, “Hierarchical are consistency: exploiting structured domains in constraint satisfaction problems,” Computational Intelligence 1, pp. 118–126, 1985.Google Scholar
  23. 23.
    J. Malik, “Labelling line drawings of curved objects,” in Proceedings of the ARPA Image Understanding Workshop, Miami, 1985, pp. 209–218.Google Scholar
  24. 24.
    J. Malik, “Interpreting line drawings of curved objects,” Tech. Rept. STAN-CS-86-1099, Computer Science Department Stanford University, 1985.Google Scholar
  25. 25.
    D. Marr, Vision, San Francisco: W.H. Freeman, 1982.Google Scholar
  26. 26.
    R.S. Millman and G.D. Parker, Elements of Differential Geometry, New Jersey: Prentice-Hall, 1977.Google Scholar
  27. 27.
    V.S. Nalwa and E. Pauchon, “Edgel aggregation and edge description,” in Proceedings of the ARPA Image Understanding Workshop, Miami, 1985, pp. 176–185.Google Scholar
  28. 28.
    J.M. Rubin and W.A. Richards, “Color vision and image intensities: When are changes material?” Biological Cybernetics 45, pp. 215–226, 1982.Google Scholar
  29. 29.
    S. Shafer, “Using color to separate reflection components,” Tech. Rept. TR 136, Computer Science Department, University of Rochester, 1984.Google Scholar
  30. 30.
    S. Shafer, T. Kanade, and J. Kender, “Gradient space in orthography and perspective,” Computer Vision, Graphics and Image Proceeding 24, pp. 182–189, 1983.Google Scholar
  31. 31.
    R. Shapira and H. Freeman, “Computer description of bodies bounded by quadric surfaces from a set of imperfect profection,” IEEE Transactions on Computers, pp. 841–854, 1978.Google Scholar
  32. 32.
    Y. Shirai “Recognition of man-made objects using edge cues,” in Computer Vision Systems, A. Hanson and E. Riseman (eds) New York: Academic Press, 1978.Google Scholar
  33. 33.
    K.A. Stevens, “The visual interpretation of surface contours,” Artificial Intelligence 17, pp. 47–73, 1981.Google Scholar
  34. 34.
    K. Ikeuchi and B.K.P. Horn, “Numerical shape from shading and occluding boundaries,” Artificial Intelligence 17, pp. 141–185, 1981.Google Scholar
  35. 35.
    K. Sugihara, “Mathematical structure of line drawings of polyhedra—towards man-machine communication by means of line drawings,” IEEE Transactions on Pattern Analysis and Machine Intelligence 4, pp. 458–469, 1982.Google Scholar
  36. 36.
    K.J. Turner, “Computer perception of curved objects using a television camera.” PhD thesis, Edinburgh University, 1974.Google Scholar
  37. 37.
    D. Waltz, “Understanding line drawings of scenes with shadows,” in The Psychology of Computer Vision, P.H. Winston (ed.), New York: McGraw-Hill, 1975.Google Scholar
  38. 38.
    H. Whitney, “Singularities of mappings of Euclidean spaces. I: Mappings of the plane into the plane,” Ann. Math 62, pp. 374–410, 1955.Google Scholar
  39. 39.
    A.P. Witkin, “Intensity-based edge classification,” in Proceedings of AAAI-82, Pittsburgh, 1982, pp. 36–41.Google Scholar

Copyright information

© Kluwer Academic Publishers 1987

Authors and Affiliations

  • Jitendra Malik
    • 1
  1. 1.Computer Science Division, Department of EECSUniversity of CaliforniaBerkeley

Personalised recommendations