Skip to main content

TransHist: Occlusion-robust shape detection in cluttered images

Abstract

Shape matching plays an important role in various computer vision and graphics applications such as shape retrieval, object detection, image editing, image retrieval, etc. However, detecting shapes in cluttered images is still quite challenging due to the incomplete edges and changing perspective. In this paper, we propose a novel approach that can efficiently identify a queried shape in a cluttered image. The core idea is to acquire the transformation from the queried shape to the cluttered image by summarising all point-to-point transformations between the queried shape and the image. To do so, we adopt a point-based shape descriptor, the pyramid of arc-length descriptor (PAD), to identify point pairs between the queried shape and the image having similar local shapes. We further calculate the transformations between the identified point pairs based on PAD. Finally, we summarise all transformations in a 4D transformation histogram and search for the main cluster. Our method can handle both closed shapes and open curves, and is resistant to partial occlusions. Experiments show that our method can robustly detect shapes in images in the presence of partial occlusions, fragile edges, and cluttered backgrounds.

References

  1. [1]

    Jacobs, C. E.; Finkelstein, A.; Salesin, D. H. Fast multiresolution image querying. In: Proceedings of the 22nd Annual Conference on Computer Graphics and Interactive Techniques, 277–286, 1995.

    Google Scholar 

  2. [2]

    Granlund, G. H. Fourier preprocessing for hand print character recognition. IEEE Transactions on Computers Vol. C-21, No. 2, 195–201, 1972.

    MathSciNet  Article  MATH  Google Scholar 

  3. [3]

    Persoon, E.; Fu, K. S. Shape discrimination using Fourier descriptors. IEEE Transactions on Systems, Man, and Cybernetics Vol. 7, No. 3, 170–179, 1977.

    MathSciNet  Article  Google Scholar 

  4. [4]

    Zhang, D.; Lu, G. Enhanced generic Fourier descriptors for object-based image retrieval. In: Proceedings of the IEEE International Conference on Acoustics, Speech, and Signal Processing, IV-3668–IV-3671, 2002.

    Google Scholar 

  5. [5]

    Mokhtarian, F.; Mackworth, A. K. A theory of multiscale, curvature-based shape representation for planar curves. IEEE Transactions on Pattern Analysis and Machine Intelligence Vol. 14, No. 8, 789–805, 1992.

    Article  Google Scholar 

  6. [6]

    Mokhtarian, F.; Abbasi, S.; Kittler, J. Efficient and robust retrieval by shape content through curvature scale space. Image Databases and Multi-Media Search Vol. 8, 51–58, 1998.

    Article  Google Scholar 

  7. [7]

    Alajlan, N.; El Rube, I.; Kamel, M. S.; Freeman, G. Shape retrieval using triangle-area representation and dynamic space warping. Pattern Recognition Vol. 40, No. 7, 1911–1920, 2007.

    Article  MATH  Google Scholar 

  8. [8]

    Manay, S.; Cremers, D.; Hong, B.-W.; Yezzi, A. J.; Soatto, S. Integral invariants for shape matching. IEEE Transactions on Pattern Analysis and Machine Intelligence Vol. 28, No. 10, 1602–1618, 2006.

    Article  MATH  Google Scholar 

  9. [9]

    Hong, B. W.; Soatto, S. Shape matching using multiscale integral invariants. IEEE Transactions on Pattern Analysis and Machine Intelligence Vol. 37, No. 1, 151–160, 2015.

    Article  Google Scholar 

  10. [10]

    Belongie, S.; Malik, J.; Puzicha, J. Shape matching and object recognition using shape contexts. IEEE Transactions on Pattern Analysis and Machine Intelligence Vol. 24, No. 4, 509–522, 2002.

    Article  Google Scholar 

  11. [11]

    Ling, H.; Jacobs, D. W. Shape classification using the inner-distance. IEEE Transactions on Pattern Analysis and Machine Intelligence Vol. 29, No. 2, 286–299, 2007.

    Article  Google Scholar 

  12. [12]

    Kwan, K. C.; Sinn, L. T.; Han, C.; Wong, T.-T.; Fu, C.-W. Pyramid of arclength descriptor for generating collage of shapes. ACM Transactions on Graphics Vol. 35, No. 6, Article No. 229, 2016.

    Article  Google Scholar 

  13. [13]

    Chuang, G. C. H.; Kuo, C. C. J. Wavelet descriptor of planar curves: Theory and applications. IEEE Transactions on Image Processing Vol. 5, No. 1, 56–70, 1996.

    Article  Google Scholar 

  14. [14]

    Tabbone, S.; Wendling, L.; Salmon, J.-P. A new shape descriptor defined on the radon transform. Computer Vision and Image Understanding Vol. 102, No. 1, 42–51, 2006.

    Article  Google Scholar 

  15. [15]

    Lee, S.-M.; Abbott, A. L.; Clark, N. A.; Araman, P. A. A shape representation for planar curves by shape signature harmonic embedding. In: Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 1940–1947, 2006.

    Google Scholar 

  16. [16]

    Hu, M.-K. Visual pattern recognition by moment invariants. IRE Transactions on Information Theory Vol. 8, No. 2, 179–187, 1962.

    Article  MATH  Google Scholar 

  17. [17]

    Khotanzad, A.; Hong, Y. H. Invariant image recognition by Zernike moments. IEEE Transactions on Pattern Analysis and Machine Intelligence Vol. 12, No. 5, 489–497, 1990.

    Article  Google Scholar 

  18. [18]

    Belkasim, S. O.; Shridhar, M.; Ahmadi, M. Pattern recognition with moment invariants: A comparative study and new results. Pattern Recognition Vol. 24, No. 12, 1117–1138, 1991.

    Article  Google Scholar 

  19. [19]

    Sheng, Y.; Shen, L. Orthogonal Fourier–Mellin moments for invariant pattern recognition. Journal of the Optical Society of America A Vol. 11, No. 6, 1748–1757, 1994.

    Article  Google Scholar 

  20. [20]

    Bernier, T.; Landry, J.-A. A new method for representing and matching shapes of natural objects. Pattern Recognition Vol. 36, No. 8, 1711–1723, 2003.

    Article  Google Scholar 

  21. [21]

    Mori, G.; Belongie, S.; Malik, J. Shape contexts enable efficient retrieval of similar shapes. In: Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Vol. 1, I-723–I-730, 2001.

    Google Scholar 

  22. [22]

    Mori, G.; Belongie, S.; Malik, J. Efficient shape matching using shape contexts. IEEE Transactions on Pattern Analysis and Machine Intelligence Vol. 27, No. 11, 1832–1837, 2005.

    Article  MATH  Google Scholar 

  23. [23]

    Mori, G.; Malik, J. Recognizing objects in adversarial clutter: Breaking a visual CAPTCHA. In: Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Vol. 1, I-134–I-141, 2003.

    Google Scholar 

  24. [24]

    Tanase, M.; Veltkamp, R. C. Part-based shape retrieval. In: Proceedings of the 13th Annual ACM International Conference on Multimedia, 543–546, 2005

    Chapter  Google Scholar 

  25. [25]

    Tănase, M.; Veltkamp, R. C.; Haverkort, H. Multiple polyline to polygon matching. In: Algorithms and Computation. Lecture Notes in Computer Science, Vol. 3827. Deng, X.; Du, D. Z. Eds. Springer, Berlin, Heidelberg, 60–70, 2005.

    Google Scholar 

  26. [26]

    Pickup, D.; Sun, X.; Rosin, P. L.; Martin, R. R. Skeleton-based canonical forms for non-rigid 3D shape retrieval. Computational Visual Media Vol. 2, No. 3, 231–243, 2016.

    Article  Google Scholar 

  27. [27]

    Xu, K.; Chen, K.; Fu, H.; Sun, W.-L.; Hu, S.-M. Sketch2Scene: Sketch-based co-retrieval and co-placement of 3D models. ACM Transactions on Graphics Vol. 32, No. 4, Article No. 123, 2013.

    Article  Google Scholar 

  28. [28]

    Lian, W.; Zhang, L.; Zhang, D. Rotation-invariant nonrigid point set matching in cluttered scenes. IEEE Transactions on Image Processing Vol. 21, No. 5, 2786–2797, 2012.

    MathSciNet  Article  MATH  Google Scholar 

  29. [29]

    Thayananthan, A.; Stenger, B.; Torr, P. H. S.; Cipolla, R. Shape context and chamfer matching in cluttered scenes. In: Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Vol. 1, I-127–I-133, 2003.

    Google Scholar 

  30. [30]

    Riemenschneider, H.; Donoser, M.; Bischof, H. Using partial edge contour matches for efficient object category localization. In: Computer Vision–ECCV 2010. Lecture Notes in Computer Science, Vol. 6315. Daniilidis, K.; Maragos, P.; Paragios, N. Eds. Springer, Berlin, Heidelberg, 29–42, 2010.

    Google Scholar 

  31. [31]

    Bai, X.; Li, Q.; Latecki, L. J.; Liu, W.; Tu, Z. Shape band: A deformable object detection approach. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 1335–1342, 2009.

    Google Scholar 

  32. [32]

    Cheng, M.-M.; Zhang, F.-L.; Mitra, N. J.; Huang, X.; Hu, S.-M. RepFinder: Finding approximately repeated scene elements for image editing. ACM Transactions on Graphics Vol. 29, No. 4, Article No. 83, 2010.

    Article  Google Scholar 

  33. [33]

    Toshev, A.; Taskar, B.; Daniilidis, K. Shape-based object detection via boundary structure segmentation. International Journal of Computer Vision Vol. 99, No. 2, 123–146, 2012.

    MathSciNet  Article  MATH  Google Scholar 

  34. [34]

    Chi, Y.; Leung, M. K. H. Part-based object retrieval in cluttered environment. IEEE Transactions on Pattern Analysis and Machine Intelligence Vol. 29, No. 5, 890–895, 2007.

    Article  Google Scholar 

  35. [35]

    Ren, S.; He, K.; Girshick, R.; Sun, J. Faster RCNN: Towards real-time object detection with region proposal networks. IEEE Transactions on Pattern Analysis and Machine Intelligence Vol. 39, No. 6, 1137–1149, 2017.

    Article  Google Scholar 

  36. [36]

    Redmon, J.; Divvala, S.; Girshick, R.; Farhadi, A. You only look once: Unified, real-time object detection. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 779–788, 2016.

    Google Scholar 

  37. [37]

    Gidaris, S.; Komodakis, N. Object detection via a multi-region and semantic segmentation-aware CNN model. In: Proceedings of the IEEE International Conference on Computer Vision, 1134–1142, 2015.

    Google Scholar 

  38. [38]

    Canny, J. A computational approach to edge detection. IEEE Transactions on Pattern Analysis and Machine Intelligence Vol. PAMI-8, No. 6, 679–698, 1986.

    Article  Google Scholar 

  39. [39]

    Cui, M.; Femiani, J.; Hu, J.; Wonka, P.; Razdan, A. Curve matching for open 2D curves. Pattern Recognition Letters Vol. 30, No. 1, 1–10, 2009.

    Article  Google Scholar 

  40. [40]

    Ferrari, V.; Fevrier, L.; Jurie, F.; Schmid, C. Groups of adjacent contour segments for object detection. IEEE Transactions on Pattern Analysis and Machine Intelligence Vol. 30, No. 1, 36–51, 2008.

    Article  Google Scholar 

  41. [41]

    Jeannin, S.; Bober, M. Description of core experiments for MPEG-7 motion/shape. MPEG- 7, ISO/IEC/JTC1/SC29/WG11/MPEG99 N, 2690, 1999.

    Google Scholar 

Download references

Acknowledgements

This project was supported by the Research Grants Council of the Hong Kong Special Administrative Region, under the RGC General Research Fund (Project No. CUHK 14217516).

Author information

Affiliations

Authors

Corresponding author

Correspondence to Tien-Tsin Wong.

Additional information

This article is published with open access at Springerlink.com

Chu Han graduated from South China Agricultural University in 2011 with a B.Sc. degree in computer science. He received his M.Phil. degree in computer science from South China University of Technology in 2014, under the supervision of Prof. Xuemiao Xu. He is now pursuing his Ph.D. degree in the Department of Computer Science and Engineering of the Chinese University of Hong Kong, under the supervision of Prof. Tien-Tsin Wong. His current research interests include computer graphics, image processing, pattern recognition, and computer vision.

Xueting Liu received her B.Eng. degree from Tsinghua University and Ph.D. degree from the Chinese University of Hong Kong in 2009 and 2014 respectively. She is currently a postdoctoral research fellow in the Department of Computer Science and Engineering of the Chinese University of Hong Kong. Her research interests include computer graphics, computer vision, computational manga and anime, and non-photorealistic rendering.

Lok Tsun Sinn graduated from the Chinese University of Hong Kong with a B.Sc. degree in computer science, and is studying for his M.Phil. degree in computer science and engineering in the same department, under the supervision of Prof. Tien-Tsin Wong.

Tien-Tsin Wong received his B.Sc., M.Phil., and Ph.D. degrees in computer science from the Chinese University of Hong Kong in 1992, 1994, and 1998, respectively. He is currently a professor in the Department of Computer Science and Engineering of the Chinese University of Hong Kong. His main research interests include computer graphics, computational manga, precomputed lighting, image-based rendering, GPU techniques, medical visualization, multimedia compression, and computer vision. He received an IEEE Transactions on Multimedia Prize Paper Award in 2005 and a Young Researcher Award in 2004.

Rights and permissions

Open Access The articles published in this journal are distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.

Other papers from this open access journal are available free of charge from http://www.springer.com/journal/41095. To submit a manuscript, please go to https://www.editorialmanager.com/cvmj.

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Han, C., Liu, X., Sinn, L.T. et al. TransHist: Occlusion-robust shape detection in cluttered images. Comp. Visual Media 4, 161–172 (2018). https://doi.org/10.1007/s41095-018-0104-1

Download citation

Keywords

  • shape matching
  • shape detection
  • transformation histogram