Skip to main content
SpringerLink
Log in

From 3D Point Clouds to Pose-Normalised Depth Maps

  • Published:
International Journal of Computer Vision Aims and scope Submit manuscript

Abstract

We consider the problem of generating either pairwise-aligned or pose-normalised depth maps from noisy 3D point clouds in a relatively unrestricted poses. Our system is deployed in a 3D face alignment application and consists of the following four stages: (i) data filtering, (ii) nose tip identification and sub-vertex localisation, (iii) computation of the (relative) face orientation, (iv) generation of either a pose aligned or a pose normalised depth map. We generate an implicit radial basis function (RBF) model of the facial surface and this is employed within all four stages of the process. For example, in stage (ii), construction of novel invariant features is based on sampling this RBF over a set of concentric spheres to give a spherically-sampled RBF (SSR) shape histogram. In stage (iii), a second novel descriptor, called an isoradius contour curvature signal, is defined, which allows rotational alignment to be determined using a simple process of 1D correlation. We test our system on both the University of York (UoY) 3D face dataset and the Face Recognition Grand Challenge (FRGC) 3D data. For the more challenging UoY data, our SSR descriptors significantly outperform three variants of spin images, successfully identifying nose vertices at a rate of 99.6%. Nose localisation performance on the higher quality FRGC data, which has only small pose variations, is 99.9%. Our best system successfully normalises the pose of 3D faces at rates of 99.1% (UoY data) and 99.6% (FRGC data).

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  • Ankerst, M., Kastenmüller, G., Kriegel, H.-P., & Seidl, T. (1999). 3d shape histograms for similarity search and classification in spatial databases. In 6th international symposium in advances in spatial databases (pp. 207–226).

  • Arun, K. S., Huang, T. S., & Blostein, S. D. (1987). Least-squares fitting of two 3d point sets. IEEE Transactions on Pattern Analysis and Machine Intelligence, 9(5), 698–700.

    Article  Google Scholar 

  • Assfalg, J., Bimbo, A. D., & Pala, P. (2004). Spin images for retrieval of 3d objects by local and global similarity. In Proc. 17th int. conf. on pattern recognition (ICPR’04) (vol. 3, pp. 906–909).

  • Belhumeur, P. N., Hespanha, J., & Kriegman, D. J. (1997). Eigenfaces vs. fisherfaces: recognition using class specific linear projection. IEEE Transactions on Pattern Analysis and Machine Intelligence, 19(7), 711–720.

    Article  Google Scholar 

  • Besl, P., & Jain, R. C. (1985). Three-dimensional object recognition. ACM Computing Surveys, 17(1), 75–145.

    Article  Google Scholar 

  • Besl, P., & McKay, N. D. (1992). A method for registration of 3D shapes. IEEE Transactions on Pattern Analysis and Machine Intelligence, 14(2), 239–256.

    Article  Google Scholar 

  • Blanz, V., & Vetter, T. (2003). Face recognition based on fitting a 3d morphable model. IEEE Transactions on Pattern Analysis and Machine Intelligence, 25(9), 1063–1074.

    Article  Google Scholar 

  • Bowyer, K. W., Chang, K. I., & Flynn, P. J. (2006). A survey of approaches and challenges in 3d and multi-modal 3d+2d face recognition. Computer Vision and Image Understanding, 101(1), 1–15.

    Article  Google Scholar 

  • Bronstein, A. M., Bronstein, M. M., & Kimmel, R. (2007). Expression-invariant representation of faces. IEEE Transactions on Image Processing, 16(1), 188–197.

    Article  MathSciNet  Google Scholar 

  • Carr, J., Fright, W.R., & Beatson, R. K. (1997). Surface interpolation with radial basis functions for medical imaging. IEEE Transactions on Medical Imaging, 16(1), 96–107.

    Article  Google Scholar 

  • Carr, J. C., Beatson, R. K., Cherrie, J. B., Mitchell, T. J., Fright, W. R., McCallum, B. C., & Evans, T. (2001). Reconstruction and representation of 3d objects with radial basis functions. In Proc. of ACM Siggraph 2001 (pp. 67–76).

  • Chang, K. I., Bowyer, K. W., & Flynn, P. J. (2005). An evaluation of multimodal 2d+3d face biometrics. IEEE Transactions on Pattern Analysis and Machine Intelligence, 27(4), 619–624.

    Article  Google Scholar 

  • Chang, K. I., Bowyer, K. W., & Flynn, P. J. (2006). Multiple nose region matching for 3d face recognition under varying facial expression. IEEE Transactions on Pattern Analysis and Machine Intelligence, 28(10), 1695–1700.

    Article  Google Scholar 

  • Chen, C., & Prakash, E. (2005). Face personalization: animated face modeling approach using radial basis function. In TENCON 2005 IEEE Region 10 (pp. 1–6) November 2005.

  • Chen, D.-Y., Tian, X.-P., Shen, Y.-T., & Ouhyoung, M. (2003). On visual similarity based 3d model retrieval. Eurographics, 22(3), 2003.

  • Chetverikov, D., Stepanov, D., & Krsek, P. (2005). Robust Euclidean alignment of 3d point sets: the trimmed iterative closest point algorithm. Image and Vision Computing, 23(3), 299–309.

    Article  Google Scholar 

  • Chin-Seng Chua, F. H., & Ho, Y.-K. (2001). 3d human face recognition using point signature. In 4th IEEE int. conf. on automatic face and gesture recognition 2000 (pp. 233–238).

  • Chua, C. S., & Jarvis, R. (1997). Point signatures: A new representation for 3D object recognition. International Journal of Computer Vision, 25(1), 63–85.

    Article  Google Scholar 

  • Colbry, D., Stockman, D., & Jain, A. (2005). Detection of anchor points for 3d face verification. In IEEE Computer Society conference on computer vision and pattern recognition. Workshops (p 118).

  • Conde, C., Cipolla, R., Rodriguez-Aragon, L. J., Serrano, A., & Cabello, E. (2005). 3d facial feature location with spin images. In IAPR conf. on machine vision applications (MVA’05) (pp. 418–421).

  • Dinh, H. Q., & Kropac, S. (2006). Multi-resolution spin-images. In Proc. IEEE conf. on computer vision and pattern recognition (CVPR’06) (pp. 863–870).

  • Dorai, C., & Jain, A. K. (1997). Cosmos-a representation scheme for 3d free-form objects. IEEE Transactions on Pattern Analysis and Machine Intelligence, 19(10), 1115–1130.

    Article  Google Scholar 

  • Faugeras, O. D., & Hebert, M. (1986). The representation, recognition and locating of 3d objects. International Journal of Robotics Research, 5(3), 27–52.

    Article  Google Scholar 

  • Franke, R. (1982). Scattered data interpolation: tests of some methods. Mathematics of Computation, 38(157), 181–200.

    Article  MATH  MathSciNet  Google Scholar 

  • Funkhouser, T., Min, P., Kazhdan, M., Chen, J., Halderman, A., Dobkin, D., & Jacobs, D. (2003). A search engine for 3d models. ACM Transactions on Graphics, 22, 83–105.

    Article  Google Scholar 

  • Gordon, G. G. (1992). Face recognition based on depth and curvature features. In Proc. IEEE computer society conf. on computer vision and pattern recognition (pp. 808–810).

  • Greengard, L., & Rokhlin, V. (1987). A fast algorithm for particle simulations. Journal of Computer Physics, 73, 325–348.

    Article  MATH  MathSciNet  Google Scholar 

  • Haralick, R. M., Joo, H., Lee, C.-N., Zhuang, X., Vaidya, V. G., & Kim, M. B. (1989). Pose estimation from corresponding point data. IEEE Transactions on Systems, Man and Cybernetics, 19(6), 1426–1446.

    Article  Google Scholar 

  • Heseltine, T., Pears, N. E., & Austin, J. (2004a). Three-dimensional face recognition: a fishersurface approach. In Lecture notes in computer science: Vol. 3212. Proc. of int. conf. on image analysis and recognition, part II (pp. 684–691). Berlin: Springer.

    Google Scholar 

  • Heseltine, T., Pears, N. E., & Austin, J. (2004b). Three-dimensional face recognition: an eigensurface approach. In Proc. IEEE int. conf. on image processing (pp. 1–2).

  • Heseltine, T., Pears, N. E., & Austin, J. (2008). Three-dimensional face recognition using combinations of surface feature map subspace components. Image and Vision Computing, 26(3), 382–396.

    Article  Google Scholar 

  • Horn, B. K. P. (1984). Extended Gaussian images. Proceedings of the IEEE, 72(2), 1671–1686.

    Article  Google Scholar 

  • Hou, Q., & Bai, L. (2005). Line feature detection from 3d point clouds via adaptive cs-rbfs shape reconstruction and multistep vertex normal manipulation. In: International conference on computer graphics, imaging and vision: new trends (pp. 79–83), July 2005.

  • Hu, X., Tang, Y., & Zhang, Z. (2008). Video object matching based on sift algorithm. In IEEE int. conference on neural networks and signal processing (pp. 412–415), June 2008.

  • Johnson, A. E., & Hebert, M. (1997). Using spin images for efficient object recognition in cluttered 3d scenes. IEEE Transactions on Pattern Analysis and Machine Intelligence, 21(5), 433–449.

    Article  Google Scholar 

  • Kakadiaris, I., Passalis, G., Toderici, G., Murtuza, N., & Theoharis, T. (2006). 3d face recognition. In British machine vision conference (BMVC’06).

  • Kazhdan, M. M., Funkhouser, T. A., & Rusinkiewicz, S. (2003). Rotation invariant spherical harmonic representation of 3d shape descriptors. In Symposium on geometry processing (pp. 156–165).

  • Kimmel, R., Bronstein, A. M., & Bronstein, M. M. (2005). Three-dimensional face recognition. International Journal of Computer Vision, 64(1), 5–30.

    Article  Google Scholar 

  • Lorensen, W. E., & Cline, H. E. (1987). Marching cubes: a high resolution 3d surface construction algorithm. Computer Graphics, 21(4), 163–169.

    Article  Google Scholar 

  • Lowe, D. G. (1999). Object recognition from local scale-invariant features. In 7th IEEE int. conf. on computer vision (Vol. 2, pp. 1150–1157), September 1999.

  • Lowe, D. G. (2004). Distinctive image features from scale-invariant keypoints. International Journal of Computer Vision, 60, 91–110.

    Article  Google Scholar 

  • Lu, X., Jain, A. K., & Colbry, D. (2006). Matching 2.5d face scans to 3d models. IEEE Transactions on Pattern Analysis and Machine Intelligence, 28(1), 31–43.

    Article  Google Scholar 

  • Mian, A. S., Bennamoun, M., & Owens, R. (2007). An efficient multimodal 2d–3d hybrid approach to automatic face recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence, 29(11), 1927–1943.

    Article  Google Scholar 

  • Mian, A. S., Bennamoun, M., & Owens, R. (2008). Keypoint detection and local feature matching for textured 3d face recognition. International Journal of Computer Vision, 79(1), 1–12.

    Article  Google Scholar 

  • Papadakis, P., Pratikakis, I., Perantonis, S., & Theoharis, T. (2007). Efficient 3d shape matching and retrieval using a concrete radialized spherical projection representation. Pattern Recognition, 40(9), 2437–2452.

    Article  MATH  Google Scholar 

  • Pears, N. E. (2008). Rbf shape histograms and their application to 3d face processing. In 8th IEEE int. conf. on automatic face and gesture recognition (FG’08), Amsterdam, Netherlands.

  • Pears, N. E., & Heseltine, T. D. (2006). Isoradius contours: New representations and techniques for 3d face matching and registration. In 3rd int. symposium on 3D data processing, visualization and transmission (3DPVT’06), University of North Carolina, USA (pp. 176–183).

  • Phillips, P. J., Flynn, P. J., Scruggs, T., Bowyer, K. W., Chang, J., Hoffman, K., Marques, J., Min, J., & Worek, W. (2005). Overview of the face recognition grand challenge. In IEEE conf. on computer vision and pattern recognition (pp. 947–954).

  • Rohling, R., Gee, A., Berman, L., & Treece, G. (1999). Radial basis function interpolation for freehand 3d ultrasound. In Lecture Notes in Computer Science : Vol. 1613. Information processing in medical imaging (pp. 478–483). Berlin: Springer.

    Chapter  Google Scholar 

  • Saupe, D., & Vranic, D. V. (2001). 3d model retrieval with spherical harmonics and moments. In Proceedings of the DAGM symposium on pattern recognition (pp. 392–397). Berlin: Springer.

    Google Scholar 

  • Savchenko, V. V., Pasko, A., Okunev, O. G., & Kunii, T. L. (1985). Function representation of solids reconstructed from scattered surface points and contours. Computer Graphics Forum, 14(4), 181–188.

    Article  Google Scholar 

  • Se, S., Lowe, D. G., & Little, J. (2002). Mobile robot localization mapping with uncertainty using scale-invariant visual landmarks. International Journal of Robotics Research, 21(8), 735–758.

    Article  Google Scholar 

  • Segundo, M., Queirolo, C., Bellon, O., & Silva, L. (2007). Automatic 3d facial segmentation and landmark detection. In Proc. of 14th int. conf. on image analysis and processing (pp. 431–436).

  • Shilane, P., Min, P., Kazhdan, M., & Funkhouser, T. (2004). The Princeton shape benchmark. In Shape modeling and applications (pp. 167–178).

  • Stein, F., & Medioni, G. (1992). Structural indexing: efficient 3-d object recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence, 14(2), 125–145.

    Article  Google Scholar 

  • Theoharis, T. (2008). 3d object retrieval, inter-class vs. intra-class. In Artificial intelligence techniques for computer graphics (pp. 55–66). Berlin: Springer.

    Google Scholar 

  • Theoharis, T., Passalis, G., Toderici, G., & Kakadiaris, I. A. (2008). Unified 3d face and ear recognition using wavelets on geometry images. Pattern Recognition, 41(3), 796–804.

    Article  MATH  Google Scholar 

  • Turk, G., & O’Brien, J. (1999). Shape transformation using variational implicit surfaces. In Proc. ACM on computer graphics SIGGRAPH 1999 (pp. 335–342).

  • Turk, M., & Pentland, A. (1991). Eigenfaces for recognition. Journal of Cognitive Neuroscience, 3(1), 71–86.

    Article  Google Scholar 

  • Wang, Y., Chua, C., & Ho, Y. (2002). Facial feature detection and face recognition from 2d and 3d images. Pattern Recognition Letters, 23(10), 1191–1202.

    Article  MATH  Google Scholar 

  • Whitmarsh, T., Veltkamp, R. C., Spagnuolo, M., Marini, S., & Harr, F. T. (2006). Landmark detection on 3d face scans by facial model registration. In 1st international symposium on shapes and semantics (pp. 71–75).

  • Xu, C., Tan, T., Wang, Y., & Quan, L. (2006). Combining local features for robust nose location in 3d facial data. Pattern Recognition Letters, 27, 1487–1494.

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Computer Science, University of York, York, UK

    Nick Pears & Marcelo Romero

  2. Aurora Computer Services Ltd, Northampton, UK

    Tom Heseltine

Authors
  1. Nick Pears
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Tom Heseltine
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Marcelo Romero
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Nick Pears.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Pears, N., Heseltine, T. & Romero, M. From 3D Point Clouds to Pose-Normalised Depth Maps. Int J Comput Vis 89, 152–176 (2010). https://doi.org/10.1007/s11263-009-0297-y

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11263-009-0297-y

Keywords

Search

Navigation