International Journal of Computer Vision

, Volume 89, Issue 2–3, pp 152–176 | Cite as

From 3D Point Clouds to Pose-Normalised Depth Maps

Article

Abstract

We consider the problem of generating either pairwise-aligned or pose-normalised depth maps from noisy 3D point clouds in a relatively unrestricted poses. Our system is deployed in a 3D face alignment application and consists of the following four stages: (i) data filtering, (ii) nose tip identification and sub-vertex localisation, (iii) computation of the (relative) face orientation, (iv) generation of either a pose aligned or a pose normalised depth map. We generate an implicit radial basis function (RBF) model of the facial surface and this is employed within all four stages of the process. For example, in stage (ii), construction of novel invariant features is based on sampling this RBF over a set of concentric spheres to give a spherically-sampled RBF (SSR) shape histogram. In stage (iii), a second novel descriptor, called an isoradius contour curvature signal, is defined, which allows rotational alignment to be determined using a simple process of 1D correlation. We test our system on both the University of York (UoY) 3D face dataset and the Face Recognition Grand Challenge (FRGC) 3D data. For the more challenging UoY data, our SSR descriptors significantly outperform three variants of spin images, successfully identifying nose vertices at a rate of 99.6%. Nose localisation performance on the higher quality FRGC data, which has only small pose variations, is 99.9%. Our best system successfully normalises the pose of 3D faces at rates of 99.1% (UoY data) and 99.6% (FRGC data).

Keywords

3D feature extraction Invariance 3D landmark localisation 3D pose normalisation 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Ankerst, M., Kastenmüller, G., Kriegel, H.-P., & Seidl, T. (1999). 3d shape histograms for similarity search and classification in spatial databases. In 6th international symposium in advances in spatial databases (pp. 207–226). Google Scholar
  2. Arun, K. S., Huang, T. S., & Blostein, S. D. (1987). Least-squares fitting of two 3d point sets. IEEE Transactions on Pattern Analysis and Machine Intelligence, 9(5), 698–700. CrossRefGoogle Scholar
  3. Assfalg, J., Bimbo, A. D., & Pala, P. (2004). Spin images for retrieval of 3d objects by local and global similarity. In Proc. 17th int. conf. on pattern recognition (ICPR’04) (vol. 3, pp. 906–909). Google Scholar
  4. Belhumeur, P. N., Hespanha, J., & Kriegman, D. J. (1997). Eigenfaces vs. fisherfaces: recognition using class specific linear projection. IEEE Transactions on Pattern Analysis and Machine Intelligence, 19(7), 711–720. CrossRefGoogle Scholar
  5. Besl, P., & Jain, R. C. (1985). Three-dimensional object recognition. ACM Computing Surveys, 17(1), 75–145. CrossRefGoogle Scholar
  6. Besl, P., & McKay, N. D. (1992). A method for registration of 3D shapes. IEEE Transactions on Pattern Analysis and Machine Intelligence, 14(2), 239–256. CrossRefGoogle Scholar
  7. Blanz, V., & Vetter, T. (2003). Face recognition based on fitting a 3d morphable model. IEEE Transactions on Pattern Analysis and Machine Intelligence, 25(9), 1063–1074. CrossRefGoogle Scholar
  8. Bowyer, K. W., Chang, K. I., & Flynn, P. J. (2006). A survey of approaches and challenges in 3d and multi-modal 3d+2d face recognition. Computer Vision and Image Understanding, 101(1), 1–15. CrossRefGoogle Scholar
  9. Bronstein, A. M., Bronstein, M. M., & Kimmel, R. (2007). Expression-invariant representation of faces. IEEE Transactions on Image Processing, 16(1), 188–197. CrossRefMathSciNetGoogle Scholar
  10. Carr, J., Fright, W.R., & Beatson, R. K. (1997). Surface interpolation with radial basis functions for medical imaging. IEEE Transactions on Medical Imaging, 16(1), 96–107. CrossRefGoogle Scholar
  11. Carr, J. C., Beatson, R. K., Cherrie, J. B., Mitchell, T. J., Fright, W. R., McCallum, B. C., & Evans, T. (2001). Reconstruction and representation of 3d objects with radial basis functions. In Proc. of ACM Siggraph 2001 (pp. 67–76). Google Scholar
  12. Chang, K. I., Bowyer, K. W., & Flynn, P. J. (2005). An evaluation of multimodal 2d+3d face biometrics. IEEE Transactions on Pattern Analysis and Machine Intelligence, 27(4), 619–624. CrossRefGoogle Scholar
  13. Chang, K. I., Bowyer, K. W., & Flynn, P. J. (2006). Multiple nose region matching for 3d face recognition under varying facial expression. IEEE Transactions on Pattern Analysis and Machine Intelligence, 28(10), 1695–1700. CrossRefGoogle Scholar
  14. Chen, C., & Prakash, E. (2005). Face personalization: animated face modeling approach using radial basis function. In TENCON 2005 IEEE Region 10 (pp. 1–6) November 2005. Google Scholar
  15. Chen, D.-Y., Tian, X.-P., Shen, Y.-T., & Ouhyoung, M. (2003). On visual similarity based 3d model retrieval. Eurographics, 22(3), 2003. Google Scholar
  16. Chetverikov, D., Stepanov, D., & Krsek, P. (2005). Robust Euclidean alignment of 3d point sets: the trimmed iterative closest point algorithm. Image and Vision Computing, 23(3), 299–309. CrossRefGoogle Scholar
  17. Chin-Seng Chua, F. H., & Ho, Y.-K. (2001). 3d human face recognition using point signature. In 4th IEEE int. conf. on automatic face and gesture recognition 2000 (pp. 233–238). Google Scholar
  18. Chua, C. S., & Jarvis, R. (1997). Point signatures: A new representation for 3D object recognition. International Journal of Computer Vision, 25(1), 63–85. CrossRefGoogle Scholar
  19. Colbry, D., Stockman, D., & Jain, A. (2005). Detection of anchor points for 3d face verification. In IEEE Computer Society conference on computer vision and pattern recognition. Workshops (p 118). Google Scholar
  20. Conde, C., Cipolla, R., Rodriguez-Aragon, L. J., Serrano, A., & Cabello, E. (2005). 3d facial feature location with spin images. In IAPR conf. on machine vision applications (MVA’05) (pp. 418–421). Google Scholar
  21. Dinh, H. Q., & Kropac, S. (2006). Multi-resolution spin-images. In Proc. IEEE conf. on computer vision and pattern recognition (CVPR’06) (pp. 863–870). Google Scholar
  22. Dorai, C., & Jain, A. K. (1997). Cosmos-a representation scheme for 3d free-form objects. IEEE Transactions on Pattern Analysis and Machine Intelligence, 19(10), 1115–1130. CrossRefGoogle Scholar
  23. Faugeras, O. D., & Hebert, M. (1986). The representation, recognition and locating of 3d objects. International Journal of Robotics Research, 5(3), 27–52. CrossRefGoogle Scholar
  24. Franke, R. (1982). Scattered data interpolation: tests of some methods. Mathematics of Computation, 38(157), 181–200. MATHCrossRefMathSciNetGoogle Scholar
  25. Funkhouser, T., Min, P., Kazhdan, M., Chen, J., Halderman, A., Dobkin, D., & Jacobs, D. (2003). A search engine for 3d models. ACM Transactions on Graphics, 22, 83–105. CrossRefGoogle Scholar
  26. Gordon, G. G. (1992). Face recognition based on depth and curvature features. In Proc. IEEE computer society conf. on computer vision and pattern recognition (pp. 808–810). Google Scholar
  27. Greengard, L., & Rokhlin, V. (1987). A fast algorithm for particle simulations. Journal of Computer Physics, 73, 325–348. MATHCrossRefMathSciNetGoogle Scholar
  28. Haralick, R. M., Joo, H., Lee, C.-N., Zhuang, X., Vaidya, V. G., & Kim, M. B. (1989). Pose estimation from corresponding point data. IEEE Transactions on Systems, Man and Cybernetics, 19(6), 1426–1446. CrossRefGoogle Scholar
  29. Heseltine, T., Pears, N. E., & Austin, J. (2004a). Three-dimensional face recognition: a fishersurface approach. In Lecture notes in computer science: Vol. 3212. Proc. of int. conf. on image analysis and recognition, part II (pp. 684–691). Berlin: Springer. Google Scholar
  30. Heseltine, T., Pears, N. E., & Austin, J. (2004b). Three-dimensional face recognition: an eigensurface approach. In Proc. IEEE int. conf. on image processing (pp. 1–2). Google Scholar
  31. Heseltine, T., Pears, N. E., & Austin, J. (2008). Three-dimensional face recognition using combinations of surface feature map subspace components. Image and Vision Computing, 26(3), 382–396. CrossRefGoogle Scholar
  32. Horn, B. K. P. (1984). Extended Gaussian images. Proceedings of the IEEE, 72(2), 1671–1686. CrossRefGoogle Scholar
  33. Hou, Q., & Bai, L. (2005). Line feature detection from 3d point clouds via adaptive cs-rbfs shape reconstruction and multistep vertex normal manipulation. In: International conference on computer graphics, imaging and vision: new trends (pp. 79–83), July 2005. Google Scholar
  34. Hu, X., Tang, Y., & Zhang, Z. (2008). Video object matching based on sift algorithm. In IEEE int. conference on neural networks and signal processing (pp. 412–415), June 2008. Google Scholar
  35. Johnson, A. E., & Hebert, M. (1997). Using spin images for efficient object recognition in cluttered 3d scenes. IEEE Transactions on Pattern Analysis and Machine Intelligence, 21(5), 433–449. CrossRefGoogle Scholar
  36. Kakadiaris, I., Passalis, G., Toderici, G., Murtuza, N., & Theoharis, T. (2006). 3d face recognition. In British machine vision conference (BMVC’06). Google Scholar
  37. Kazhdan, M. M., Funkhouser, T. A., & Rusinkiewicz, S. (2003). Rotation invariant spherical harmonic representation of 3d shape descriptors. In Symposium on geometry processing (pp. 156–165). Google Scholar
  38. Kimmel, R., Bronstein, A. M., & Bronstein, M. M. (2005). Three-dimensional face recognition. International Journal of Computer Vision, 64(1), 5–30. CrossRefGoogle Scholar
  39. Lorensen, W. E., & Cline, H. E. (1987). Marching cubes: a high resolution 3d surface construction algorithm. Computer Graphics, 21(4), 163–169. CrossRefGoogle Scholar
  40. Lowe, D. G. (1999). Object recognition from local scale-invariant features. In 7th IEEE int. conf. on computer vision (Vol. 2, pp. 1150–1157), September 1999. Google Scholar
  41. Lowe, D. G. (2004). Distinctive image features from scale-invariant keypoints. International Journal of Computer Vision, 60, 91–110. CrossRefGoogle Scholar
  42. Lu, X., Jain, A. K., & Colbry, D. (2006). Matching 2.5d face scans to 3d models. IEEE Transactions on Pattern Analysis and Machine Intelligence, 28(1), 31–43. CrossRefGoogle Scholar
  43. Mian, A. S., Bennamoun, M., & Owens, R. (2007). An efficient multimodal 2d–3d hybrid approach to automatic face recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence, 29(11), 1927–1943. CrossRefGoogle Scholar
  44. Mian, A. S., Bennamoun, M., & Owens, R. (2008). Keypoint detection and local feature matching for textured 3d face recognition. International Journal of Computer Vision, 79(1), 1–12. CrossRefGoogle Scholar
  45. Papadakis, P., Pratikakis, I., Perantonis, S., & Theoharis, T. (2007). Efficient 3d shape matching and retrieval using a concrete radialized spherical projection representation. Pattern Recognition, 40(9), 2437–2452. MATHCrossRefGoogle Scholar
  46. Pears, N. E. (2008). Rbf shape histograms and their application to 3d face processing. In 8th IEEE int. conf. on automatic face and gesture recognition (FG’08), Amsterdam, Netherlands. Google Scholar
  47. Pears, N. E., & Heseltine, T. D. (2006). Isoradius contours: New representations and techniques for 3d face matching and registration. In 3rd int. symposium on 3D data processing, visualization and transmission (3DPVT’06), University of North Carolina, USA (pp. 176–183). Google Scholar
  48. Phillips, P. J., Flynn, P. J., Scruggs, T., Bowyer, K. W., Chang, J., Hoffman, K., Marques, J., Min, J., & Worek, W. (2005). Overview of the face recognition grand challenge. In IEEE conf. on computer vision and pattern recognition (pp. 947–954). Google Scholar
  49. Rohling, R., Gee, A., Berman, L., & Treece, G. (1999). Radial basis function interpolation for freehand 3d ultrasound. In Lecture Notes in Computer Science : Vol. 1613. Information processing in medical imaging (pp. 478–483). Berlin: Springer. CrossRefGoogle Scholar
  50. Saupe, D., & Vranic, D. V. (2001). 3d model retrieval with spherical harmonics and moments. In Proceedings of the DAGM symposium on pattern recognition (pp. 392–397). Berlin: Springer. Google Scholar
  51. Savchenko, V. V., Pasko, A., Okunev, O. G., & Kunii, T. L. (1985). Function representation of solids reconstructed from scattered surface points and contours. Computer Graphics Forum, 14(4), 181–188. CrossRefGoogle Scholar
  52. Se, S., Lowe, D. G., & Little, J. (2002). Mobile robot localization mapping with uncertainty using scale-invariant visual landmarks. International Journal of Robotics Research, 21(8), 735–758. CrossRefGoogle Scholar
  53. Segundo, M., Queirolo, C., Bellon, O., & Silva, L. (2007). Automatic 3d facial segmentation and landmark detection. In Proc. of 14th int. conf. on image analysis and processing (pp. 431–436). Google Scholar
  54. Shilane, P., Min, P., Kazhdan, M., & Funkhouser, T. (2004). The Princeton shape benchmark. In Shape modeling and applications (pp. 167–178). Google Scholar
  55. Stein, F., & Medioni, G. (1992). Structural indexing: efficient 3-d object recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence, 14(2), 125–145. CrossRefGoogle Scholar
  56. Theoharis, T. (2008). 3d object retrieval, inter-class vs. intra-class. In Artificial intelligence techniques for computer graphics (pp. 55–66). Berlin: Springer. Google Scholar
  57. Theoharis, T., Passalis, G., Toderici, G., & Kakadiaris, I. A. (2008). Unified 3d face and ear recognition using wavelets on geometry images. Pattern Recognition, 41(3), 796–804. MATHCrossRefGoogle Scholar
  58. Turk, G., & O’Brien, J. (1999). Shape transformation using variational implicit surfaces. In Proc. ACM on computer graphics SIGGRAPH 1999 (pp. 335–342). Google Scholar
  59. Turk, M., & Pentland, A. (1991). Eigenfaces for recognition. Journal of Cognitive Neuroscience, 3(1), 71–86. CrossRefGoogle Scholar
  60. Wang, Y., Chua, C., & Ho, Y. (2002). Facial feature detection and face recognition from 2d and 3d images. Pattern Recognition Letters, 23(10), 1191–1202. MATHCrossRefGoogle Scholar
  61. Whitmarsh, T., Veltkamp, R. C., Spagnuolo, M., Marini, S., & Harr, F. T. (2006). Landmark detection on 3d face scans by facial model registration. In 1st international symposium on shapes and semantics (pp. 71–75). Google Scholar
  62. Xu, C., Tan, T., Wang, Y., & Quan, L. (2006). Combining local features for robust nose location in 3d facial data. Pattern Recognition Letters, 27, 1487–1494. CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC 2009

Authors and Affiliations

  1. 1.Department of Computer ScienceUniversity of YorkYorkUK
  2. 2.Aurora Computer Services LtdNorthamptonUK

Personalised recommendations