Advertisement

International Journal of Computer Vision

, Volume 89, Issue 2–3, pp 229–247 | Cite as

A 3D Shape Retrieval Framework Supporting Multimodal Queries

  • Petros DarasEmail author
  • Apostolos Axenopoulos
Article

Abstract

This paper presents a unified framework for 3D shape retrieval. The method supports multimodal queries (2D images, sketches, 3D objects) by introducing a novel view-based approach able to handle the different types of multimedia data. More specifically, a set of 2D images (multi-views) are automatically generated from a 3D object, by taking views from uniformly distributed viewpoints. For each image, a set of 2D rotation-invariant shape descriptors is produced. The global shape similarity between two 3D models is achieved by applying a novel matching scheme, which effectively combines the information extracted from the multi-view representation. The experimental results prove that the proposed method demonstrates superior performance over other well-known state-of-the-art approaches.

Keywords

3D object retrieval Multi-views Multimodal queries Image to 3D object Sketch to 3D object 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Akgul, C., Axenopoulos, A., Bustos, B., Chaouch, M., Daras, P., Dutagaci, H., Furuya, T., Godil, A., Kreft, S., Lian, Z., Napoleon, T., Mademlis, A., Ohbuchi, R., Rosin, P. L., Sankur, B., Schreck, T., Sun, X., Tezuka, M., Yemez, Y., Verroust-Blondet, A., & Walter, M. (2009). SHREC 2009—generic shape retrieval contest. In 30th international conference on EUROGRAPHICS 2009, workshop on 3D object retrieval. Munich, Germany, March 2009. Google Scholar
  2. Ankerst, M., Kastenmuller, G., Kriegel, H. P., & Seidl, T. (1999). 3D shape histograms for similarity search and classification in spatial databases. In Proc. of the 6th int. symp. spatial databases (SSD1999). Hong Kong, 1999. Google Scholar
  3. Ansary, T. F., Vandeborre, J.-P., & Daoudi, M. (2007). 3D-model search engine from photos. In Proc. ACM CIVR 2008 (pp. 89–92). Google Scholar
  4. Axenopoulos, A., Daras, P., Dutagaci, H., Furuya, T., Godil, A., & Ohbuchi, R. (2009). SHREC 2009—shape retrieval contest of partial 3D models. In 30th international conference on EUROGRAPHICS 2009, workshop on 3D object retrieval. Munich, Germany, March 2009. Google Scholar
  5. Belkasim, S. O., Shridhar, M., & Ahmadi, M. (1991). Pattern recognition with moment invariants: A comparative study and new results. Pattern Recognition, 24(12), 1117–1138. CrossRefGoogle Scholar
  6. Bustos, B., Keim, D. A., Saupe, D., Schreck, T., & Vranic, D. V. (2005). Feature-based similarity search in 3D object databases. ACM Computing Surveys, 37(4), 345–387. CrossRefGoogle Scholar
  7. Bustos, B., Keim, D., Saupe, D., & Schreck, T. (2007). Content-based 3d object retrieval. IEEE Computer Graphics and Applications, 27(4), 22–27. CrossRefGoogle Scholar
  8. Canterakis, N. (1999). 3D Zernike moments and Zernike affine invariants for 3D image analysis and recognition. In Proc. of the Scandinavian conference on image analysis, 1999. Google Scholar
  9. Chaouch, M., & Verroust-Blondet, A. (2006). Enhanced 2D/3D approaches based on relevance index for 3D-shape retrieval. In SMI’06. Matsushima, Japan, June 2006. Google Scholar
  10. Chen, D.-Y., & Ouhyoung, M. (2002). A 3D object retrieval system based on multi-resolution reeb graph. In Proc. of Computer Graphics Workshop. Tainan, ROC. Google Scholar
  11. Chen, D.-Y., Ouhyoung, M., Tian, X.-P., Shen, Y.-T., & Ouhyoung, M. (2003). On visual similarity based 3d model retrieval. In Proc. of Eurographics (pp. 223–232). Granada, Spain. Google Scholar
  12. Daras, P., Zarpalas, D., Axenopoulos, A., Tzovaras, D., & Strintzis, M. G. (2006a). Three-dimensional shape-structure comparison method for protein classification. IEEE/ACM Transactions on Computational Biology and Bioinformatics, 3(3), 193–207. CrossRefGoogle Scholar
  13. Daras, P., Zarpalas, D., Tzovaras, D., & Strintzis, M. G. (2006b). Efficient 3-d model search and retrieval using generalized 3-d radon transforms. IEEE Transactions on Multimedia, 8(1), 101–114. CrossRefGoogle Scholar
  14. Filali Ansary, T., Daoudi, M., & Vandeborre, J.-P. (2007). A Bayesian 3D search engine using adaptive views clustering. IEEE Transactions on Multimedia, 9(1), 78–88. CrossRefGoogle Scholar
  15. Fischer, K., & Gartner, B. (2004). The smallest enclosing ball of balls: Combinatorial structure and algorithms. International Journal of Computational Geometry and Applications (IJCGA), 14, 341–387. zbMATHCrossRefMathSciNetGoogle Scholar
  16. Fodor, J. A. (1983). The modularity of mind. Cambridge: Cambridge Bradford Books, MIT Press. Google Scholar
  17. Funkhouser, T., Min, P., Kazhdan, M., Chen, J., Halderman, A., Dobkin, D., & Jacobs, D. (2003). A search engine for 3D models. ACM Transactions on Graphics, 22(1), 83–105. CrossRefGoogle Scholar
  18. Goodall, S., Lewis, P. H., Martinez, K., Sinclair, P. A. S., Giorgini, F., Addis, M., Boniface, M. J., Lahanier, C., & Stevenson, J. (2004). SCULPTEUR: Multimedia retrieval for museums. In Proc. of the image and video retrieval: Third international conference (CIVR’04) (pp. 638–646). Google Scholar
  19. Hartveldt, J., Spagnuolo, M., Axenopoulos, A., Biasotti, S., Daras, P., Dutagaci, H., Furuya, T., Godil, A., Li, X., Mademlis, A., Marini, S., Napoleon, T., Ohbuchi, R., & Tezuka, M. (2009). SHREC 2009 track: Structural shape retrieval on watertight models. In 30th international conference on EUROGRAPHICS 2009, workshop on 3D object retrieval. Munich, Germany, March 2009. Google Scholar
  20. Hilaga, M., Shinagawa, Y., Kohmura, T., & Kunii, T. L. (2001). Topology matching for fully automatic similarity estimation of 3D shapes. In Proc. of ACM SIGGRAPH 2001 (pp. 203–212). Google Scholar
  21. Horn, B. (1984). Extended Gaussian images. Proceedings of the IEEE, 72(12), 1671–1686. CrossRefGoogle Scholar
  22. Hu, M. K. (1962). Visual pattern recognition by moment invariants. IRE Transactions on Information Theory, 8, 179–197. Google Scholar
  23. Huber, F., & Hebert, M. (2001). Fully automatic registration of multiple 3D data sets. In Proc. of IEEE workshop on computer vision beyond the visible spectrum: Methods and applications (CVBVS). Kauai, Hawaii, USA, Dec. 2001. Google Scholar
  24. Iyer, M., Jayanti, S., Lou, K., Kalyanaraman, Y., & Ramani, K. (2005). Three dimensional shape searching: State-of-the-art review and future trends. Computer Aided Design, 5(15), 509–530. CrossRefGoogle Scholar
  25. Jayanti, S., Kalyanaraman, K., Iyer, N., & Ramani, K. (2006). Developing an engineering shape benchmark for CAD models. Computer-Aided Design, 38(9), 939–953. CrossRefGoogle Scholar
  26. Kang, S. B., & Ikeuchi, K. (1993). The complex EGI: A new representation for 3D pose determination. IEEE Transactions on Pattern Analysis and Machine Intelligence, 15(7), 707–721. CrossRefGoogle Scholar
  27. Katz, S., & Tal, A. (2003). Hierarchical mesh decomposition using fuzzy clustering and cuts. ACM Transactions on Graphics 954–961. Google Scholar
  28. Kazhdan, M., Funkhouser, T., & Rusinkiewicz, S. (2003). Rotation invariant spherical harmonic representation of 3D shape descriptors. In Proc. of symposium on geometry processing, Jun. 2003. Google Scholar
  29. Khotanzad, A., & Hong, Y. H. (1990). Invariant image recognition by Zernike moments. IEEE Transactions on Pattern Analysis and Machine Intelligence, 12(5), 489–497. CrossRefGoogle Scholar
  30. Kriegel, H.-P., Kroeger, P., Mashael, Z., Pfeifle, M., Poetke, M., & Seidl, T. (2003). Effective similarity search on voxelized cad objects. In Proc. of IEEE eighth international conference on database systems for advanced applications. Washington, DC, USA. Google Scholar
  31. Lindstrom, P., & Turk, G. (2000). Image-driven simplification. ACM Transactions on Graphics, 19(3), 204–241. CrossRefGoogle Scholar
  32. Liu, Y., Zha, H., & Qin, H. (2006). The generalized shape distributions for shape matching and analysis. In Proc. of the IEEE international conference on shape modeling and applications (SMI2006). Matsushima, Japan. Google Scholar
  33. Lowe, D. G. (2004). Distinctive image features from scale-invariant keypoints. International Journal of Computer Vision, 60(2). Google Scholar
  34. Mademlis, A., Daras, P., Axenopoulos, A., Tzovaras, D., & Strintzis, M. G. (2008a). Combining topological and geometrical features for global and partial 3D shape retrieval. IEEE Transactions on Multimedia, 10(5), 819–831. CrossRefGoogle Scholar
  35. Mademlis, A., Daras, P., Tzovaras, D., & Strintzis, M. G. (2008b). 3D object retrieval based on resulting fields. In 29th international conference on EUROGRAPHICS 2008, workshop on 3D object retrieval. Crete, Greece, April 2008. Google Scholar
  36. Mahmoudi, S., & Daoudi, M. (2002). 3D models retrieval by using characteristic views. In ICPR’02 (pp. 11–15). Quebec, Canada, Aug. 2002. Google Scholar
  37. Napoleon, T., Adamek, T., Shmitt, F., & O’Connor, N. E. (2008). SHREC’08 entry: Multi-view 3D retrieval using multi-scale contour representation. In Proc. IEEE SMI 2009. Google Scholar
  38. Novotni, M., & Klein, R. (2003). 3d Zernike descriptors for content based shape retrieval. In Proc. of the eighth ACM symposium on solid modeling and applications (pp. 216–225). NY, USA, 2003. New York: ACM. CrossRefGoogle Scholar
  39. Ohbuchi, R., Otagiri, T., Ibato, M., & Takei, T. (2002). Shape-similarity search of three-dimensional models using parameterized statistics. In Oroc. of Pacific graphics (pp. 265–274). Beijng, China. Los Alamitos: IEEE Comput. Soc. Google Scholar
  40. Ohbuchi, R., Minamitani, T., & Takei, T. (2003a). Shape similarity search of 3D models by using enhanced shape functions. In Proc. TP.CG. 03 (pp. 97–104). Google Scholar
  41. Ohbuchi, R., Nakazawa, M., & Takei, T. (2003b). Retrieving 3D shapes based on their appearance. In Proc. ACM workshop on multimedia information retrieval (MIR) 2003 (pp. 39–46). Google Scholar
  42. Ohbuchi, R., Osada, K., Furuya, T., & Banno, T. (2008). Salient local visual features for shape-based 3D model retrieval. In Proc. of the IEEE international conference on shape modeling and applications (SMI 2008) (pp. 93–102). Google Scholar
  43. Osada, R., Funkhouser, T., Chazelle, B., & Dobkin, D. (2001). Matching 3D models with shape distributions. In Proc. of the IEEE international conference on shape modeling and applications (SMI2001) (pp. 154–166). Google Scholar
  44. Osada, R., Funkhouser, T., Chazelle, B., & Dobkin, D. (2002). Shape distributions. ACM Transactions on Graphics, 21(4), 807–832. CrossRefGoogle Scholar
  45. Papadakis, P., Pratikakis, I., Perantonis, S., & Theoharis, T. (2007). Efficient 3D shape matching and retrieval using a concrete radialized spherical projection representation. Pattern Recognition, 40(9), 2437–2452. zbMATHCrossRefGoogle Scholar
  46. Pu, J., & Ramani, K. (2005). An approach to drawing-like view generation from 3D models. In Proc. of IDETC/CIE 2005. ASME. Google Scholar
  47. Real-time 3D models, http://www.3drt.com/.
  48. Shih, J.-L., Lee, C.-H., & Wang, J. T. (2007). A new 3D model retrieval approach based on the elevation descriptor. Pattern Recognition, 40(1), 283–295. zbMATHCrossRefGoogle Scholar
  49. Shilane, P., Min, P., Kazhdan, M., & Funkhouser, T. (2004). The Princeton shape benchmark. In Proceedings of the shape modeling international (SMI ’04) (pp. 167–178). Genova, Italy, June 2004. Google Scholar
  50. Tal, A., & Zuckerberger, E. (2006). Mesh retrieval by components. International Conference on Computer Graphics Theory and Applications, 142–149. Google Scholar
  51. Tangelder, J., & Veltkamp, R. C. (2004). A survey of content based 3D shape retrieval methods. In Proc. of IEEE shape modelling international (pp. 145–156). Google Scholar
  52. Teague, M. R. (1979). Image analysis via the general theory of moments. Journal of Optical Society of America, 70, 920–930. CrossRefMathSciNetGoogle Scholar
  53. Tsatsaias, V., Daras, P., & Strintzis, M. G. (2007). 3D protein classification using topological, geometrical and biological information. In Proc. of IEEE international conference on image processing (ICIP 2007). San Antonio, Texas, USA, 2007. Google Scholar
  54. Tung, T., & Schmitt, F. (2004). Augmented reeb graphs for content-based retrieval of 3D mesh models. In Proc. of the IEEE international conference on shape modeling and applications (SMI2004) (pp. 157–166). Google Scholar
  55. Tung, T., & Schmitt, F. (2005). The augmented multiresolution Reeb graph approach for content-based retrieval of 3D shapes. International Journal of Shape Modeling (IJSM), 11(1). Google Scholar
  56. Vinanco, A. P., Ramirez, A. M., & Agustin, F. G. (2003). Digital image reconstruction by using Zernike moments. In Proc. of SPIE (pp. 281–289). Barcelona, Spain, Sept. 2003. Google Scholar
  57. Vranic, D. V. (2003). An improvement of rotation invariant 3d-shape based on functions on concentric spheres. In Proc. of IEEE ICIP (3) (pp. 757–760). Google Scholar
  58. Vranic, D. (2004). 3d model retrieval. Ph.D. Dissertation, University of Leipzig. Google Scholar
  59. Vranic, D., & Saupe, D. (2002). Description of 3d-shape using a complex function on the sphere. In Proc. of the IEEE international conference on multimedia and Expo (ICME2002) (Vol. 1, pp. 177–180). Google Scholar
  60. Vranic, D., Saupe, D., & Richter, J. (2001). Tools for 3d-object retrieval: Karhunen-loeve transform and spherical harmonics. In Proc. of the IEEE fourth workshop on multimedia signal processing (pp. 293–29). Google Scholar
  61. Wahl, E., Hillenbrand, U., & Hirzinger, G. (2003). Surflet-pair-relation histograms: A statistical 3D-shape representation for rapid classification. In Proc. 3DIM 2003 (pp. 474–481). Google Scholar
  62. Weyrich, T., Pauly, M., Keiser, R., Heinzle, S., Scandella, S., & Gross, M. (2004). Post-processing of scanned 3d surface data. In Proc. of Eurographics. Granada, Spain, 2004. Google Scholar
  63. Yap, P. T., Paramesran, R., & Ong, S. H. (2003). Image analysis by Krawtchouk moments. IEEE Transactions on Image Processing 12(11), 1367–1377. CrossRefMathSciNetGoogle Scholar
  64. Zarpalas, D., Daras, P., Axenopoulos, A., Tzovaras, D., & Strintzis, M. G. (2007). 3D model search and retrieval using the spherical trace transform. EURASIP Journal on Advances in Signal Processing. doi: 10.1155/2007/23912. Volume 2007, Article ID 23912, 14 pages.
  65. Zhang, D., & Lu, G. (2002). Shape-based image retrieval using generic Fourier descriptor. ELSEVIER Signal Processing: Image Communication, 17(10), 825–848. CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC 2009

Authors and Affiliations

  1. 1.Informatics and Telematics InstituteCentre for Research and Technology HellasThessalonikiGreece

Personalised recommendations