Advertisement

The Visual Computer

, Volume 33, Issue 12, pp 1601–1612 | Cite as

Fast algorithm for 2D fragment assembly based on partial EMD

  • Meng Zhang
  • Shuangmin ChenEmail author
  • Zhenyu Shu
  • Shi-Qing Xin
  • Jieyu Zhao
  • Guang Jin
  • Rong Zhang
  • Jürgen Beyerer
Original Article

Abstract

2D Fragment assembly is an important research topic in computer vision and pattern recognition, and has a wide range of applications such as relic restoration and remote sensing image processing. The key to this problem lies in utilizing contour features or visual cues to find the optimal partial matching. Considering that previous algorithms are weak in predicting the best matching configuration of two neighboring fragments, we suggest using the earth mover’s distance, based on length/property correspondence, to measure the similarity, which potentially matches a point on the first contour to a desirable destination point on the second contour. We further propose a greedy algorithm for 2D fragment assembly by repeatedly assembling two neighboring fragments into a composite one. Experimental results on map-piece assembly and relic restoration show that our algorithm runs fast, is insensitive to noise, and provides a novel solution to the fragment assembly problem.

Keywords

Fragment assembly Partial EMD Contour features Lebesgue measure 

Notes

Acknowledgments

We are grateful to the editors and anonymous reviewers for their insightful comments and suggestions. This work is supported by NSF of China (61300168, 61571247, 11226328), NSF of Zhejiang (LZ16F030001, LY13F020018), the Open Research Fund of Zhejiang First-foremost Key Subject (XKXL1521, XKXL1406, XKXL1429), and the International Science and Technology Cooperation Project of Zhejiang (2013C24027).

References

  1. 1.
    Agathos, A., Pratikakis, I., Papadakis, P., Perantonis, S., Azariadis, P., Sapidis, N.S.: 3d articulated object retrieval using a graph-based representation. Visual Comput. 26(10), 1301–1319 (2009)CrossRefGoogle Scholar
  2. 2.
    Alt, H., Buchin, M.: Can we compute the similarity between surfaces? Discret. Comput. Geom. 43, 78–99 (2007)CrossRefzbMATHMathSciNetGoogle Scholar
  3. 3.
    Alt, H., Guibas, L.J.: Discrete geometric shapes: matching, interpolation, and approximation. Handb. Comput. Geom. 1, 121–153 (1999)Google Scholar
  4. 4.
    Alt, H., Knauer, C., Wenk, C.: Comparison of distance measures for planar curves. Algorithmica 38(1), 45–58 (2004)CrossRefzbMATHMathSciNetGoogle Scholar
  5. 5.
    Altantsetseg, E., Matsuyama, K., Konno, K.: Pairwise matching of 3d fragments using fast fourier transform. Visual Comput. 30(6–8), 929–938 (2014)CrossRefGoogle Scholar
  6. 6.
    Ancuti, C., Ancuti, C.O., Bekaert, P.: An efficient two steps algorithm for wide baseline image matching. Visual Comput. 25(5–7), 677–686 (2009)CrossRefGoogle Scholar
  7. 7.
    Andaló, F.A., Carneiro, G., Taubin, G., Goldenstein, S., Velho, L.: Automatic reconstruction of ancient portuguese tile panels. IEEE Comput. Graphics Appl. (2016) (accepted) Google Scholar
  8. 8.
    Baxter, L.A., Harche, F.: Note: on the greedy algorithm for optimal assembly. Naval Res. Logistics 39, 833–837 (1992)CrossRefzbMATHMathSciNetGoogle Scholar
  9. 9.
    Buchin, K., Buchin, M., Wang, Y.: Exact algorithms for partial curve matching via the fréchet distance. In: ACM-SIAM symposium on discrete algorithms, pp. 645–654 (2009)Google Scholar
  10. 10.
    Chen, B., Pan, X.: Geodesic Fourier descriptor for 2D shape matching. In: International Conference on Embedded Software and Systems Symposia, pp. 447–452 (2008)Google Scholar
  11. 11.
    Cui, M., Femiani, J., Hu, J., Wonka, P., Razdan, A.: Curve matching for open 2D curves. Pattern Recogn. Lett. 30(1), 1–10 (2009)CrossRefGoogle Scholar
  12. 12.
    Cui, M., Wonka, P., Razdan, A., Hu, J.: A new image registration scheme based on curvature scale space curve matching. Visual Comput. 23(8), 607–618 (2007)CrossRefGoogle Scholar
  13. 13.
    Domokos, C., Kato, Z.: Realigning 2d and 3d object fragments without correspondences. IEEE Trans. Pattern Anal. Mach. Intell. 38(1), 1–1 (2016)CrossRefGoogle Scholar
  14. 14.
    Driemel, A., Har-Peled, S.: Jaywalking your dog—computing the Fréchet distance with shortcuts. SIAM J. Comput. 42(5), 1830–1866 (2013)CrossRefzbMATHMathSciNetGoogle Scholar
  15. 15.
    Dyken, C., Dæhlen, M., Sevaldrud, T.: Simultaneous curve simplification. J. Geogr. Syst. 11(11), 273–289 (2009)CrossRefGoogle Scholar
  16. 16.
    Freeman, H., Garder, L.: Apictorial jigsaw puzzles: the computer solution of a problem in pattern recognition. IEEE Trans. Electron. Comput. 13(2), 118–127 (1964)CrossRefGoogle Scholar
  17. 17.
    da Gama Leito, H.C., Stolfi, J.: Automatic reassembly of irregular fragments. Univ. of Campinas, Tech. Rep. IC-98-06 (1998)Google Scholar
  18. 18.
    Giguere, M.: Three-dimensional puzzle assembly. US Patent 6015150Google Scholar
  19. 19.
    Goldberg, D., Malon, C., Bern, M.: A global approach to automatic solution of jigsaw puzzles. In: Conf Computational Geometry, pp. 82–87 (2002)Google Scholar
  20. 20.
    Grauman, K., Darrell, T.: Fast contour matching using approximate earth mover’s distance. In: Computer Vision and Pattern Recognition, 2004. CVPR 2004. Proceedings of the 2004 IEEE Computer Society Conference on, vol. 1, pp. I–220–I–227 (2004)Google Scholar
  21. 21.
    Gelfand, N., Pottmann, H. Flöry, S., Hofer, M.: Reassembling fractured objects by geometric matching, ACM Trans. Graphics. (3), 569–578 (2006)Google Scholar
  22. 22.
    Huang, Z., Cohen, F.S.: Affine-invariant B-spline moments for curve matching. IEEE Trans. Image Process. 5(10), 1473–1480 (1996)CrossRefGoogle Scholar
  23. 23.
    James, G.M.: Curve alignment by moments. Ann. Appl. Stat. 1(2), 2007 (2008)MathSciNetGoogle Scholar
  24. 24.
    Kanezaki, A., Harada, T., Kuniyoshi, Y.: Partial matching of real textured 3d objects using color cubic higher-order local auto-correlation features. Visual Comput. 26(10), 1269–1281 (2010)CrossRefGoogle Scholar
  25. 25.
    Khan, M.S., Ayob, A.F.M., Isaacs, A., Ray, T.: A novel evolutionary approach for 2D shape matching based on B-spline modeling. In: IEEE Congress on Evolutionary Computation (CEC), pp. 655–661 (2011)Google Scholar
  26. 26.
    Latecki, L.J., Megalooikonomou, V., Wang, Q., Yu, D.: An elastic partial shape matching technique. Pattern Recogn. 40(11), 3069–3080 (2007)CrossRefzbMATHGoogle Scholar
  27. 27.
    Liu, H., Latecki, L.J., Liu, W.: A unified curvature definition for regular, polygonal, and digital planar curves. Int. J. Comput. Vision 80(1), 104–124 (2008)CrossRefGoogle Scholar
  28. 28.
    Maheshwari, A., Sack, J.R., Shahbaz, K., Zarrabi-Zadeh, H.: Improved algorithms for partial curve matching. Algorithmica 69(3), 641–657 (2014)CrossRefzbMATHMathSciNetGoogle Scholar
  29. 29.
    Mccreath, E.: Partial matching of planar polygons under translation and rotation. In: Canadian Conference on Computational Geometry (2008)Google Scholar
  30. 30.
    Miller, J.M., Hoffman, R.L.: Automatic assembly planning with fasteners. In: IEEE International Conference on Robotics and Automation, pp. 69–74 (1989)Google Scholar
  31. 31.
    Min, G.C., Fleck, M.M., Forsyth, D.A.: Jigsaw puzzle solver using shape and color. In: The Fourth International Conference on Signal Processing Proceedings, pp. 877–880 (1998)Google Scholar
  32. 32.
    Gu, P., Yan, X.: CAD-directed automatic assembly sequence planning. Int. J. Prod. Res. 33(11), 3069–3100 (1995)CrossRefzbMATHGoogle Scholar
  33. 33.
    Pal, A., Shanmugasundaram, K., Memon, N.: Automated reassembly of fragmented images. In: International Conference on Multimedia and Expo, pp. 625–628 (2003)Google Scholar
  34. 34.
    Parikh, D., Sukthankar, R., Chen, T., Chen, M.: Feature-based part retrieval for interactive 3d reassembly. In: IEEE Winter Conference on Applications of Computer Vision, pp. 14–14 (2007)Google Scholar
  35. 35.
    Porrill, J., Pollard, S.: Curve matching and stereo calibration. Image Vis. Comput. 9(1), 45–50 (1991)CrossRefGoogle Scholar
  36. 36.
    Richter, F., Ries, C.X., Cebron, N., Lienhart, R.: Learning to reassemble shredded documents. IEEE Trans. Multimedia 15(3), 582–593 (2013)CrossRefGoogle Scholar
  37. 37.
    Rubner, Y., Tomasi, C.: Perceptual metrics for image database navigation. Springer International 594 (1999)Google Scholar
  38. 38.
    Rubner, Y., Tomasi, C., Guibas, L.J.: The earth mover’s distance as a metric for image retrieval. Int. J. Comput. Vis. 40(2), 99–121 (2000)CrossRefzbMATHGoogle Scholar
  39. 39.
    Shirdhonkar, S., Jacobs, D.W.: Approximate earth movers distance in linear time. In: Computer Vision and Pattern Recognition, 2008. CVPR 2008. IEEE Conference on, pp. 1–8. IEEE (2008)Google Scholar
  40. 40.
    Shu, X., Wu, X.J.: A novel contour descriptor for 2D shape matching and its application to image retrieval. Image Vis. Comput. 29(4), 286–294 (2011)CrossRefGoogle Scholar
  41. 41.
    Shuralyov, D., Stuerzlinger, W.: A 3D desktop puzzle assembly system. In: 2011 IEEE Symposium on 3D User Interfaces (3DUI), pp. 139–140 (2011)Google Scholar
  42. 42.
    Song, Y., Jin, S.: Matching sequences of salient contour points characterized by voronoi region features. Visual Comput. 28(5), 475–491 (2012)CrossRefGoogle Scholar
  43. 43.
    Wang, J., Yu, Z., Zhang, W., Wei, M., Tan, C., Dai, N., Zhang, X.: Robust reconstruction of 2D curves from scattered noisy point data. Comput. Aided Des. 50(3), 27–40 (2014)CrossRefGoogle Scholar
  44. 44.
    Wang, X., Hu, J., Zhang, D., Qin, H.: Efficient emd and hilbert spectra computation for 3d geometry processing and analysis via space-filling curve. Visual Comput. 31(6–8), 1135–1145 (2015)CrossRefGoogle Scholar
  45. 45.
    Webster, R.W., Lafollette, P.S., Stafford, R.L.: Isthmus critical points for solving jigsaw puzzles in computer vision. IEEE Trans. Syst. Man Cybern. 21(5), 1271–1278 (1991)CrossRefGoogle Scholar
  46. 46.
    Wei, G., Xiao-dong, S., Huan-ling, L.: Automatic assembly location method based on particle filter. Comput. Integr. Manuf. Syst. 20(7), 1615–1624 (2014)Google Scholar
  47. 47.
    Xu, C., Liu, J., Tang, X.: 2D shape matching by contour flexibility. IEEE Trans. Pattern Anal. Mach. Intell. 31(1), 180–186 (2009)CrossRefGoogle Scholar
  48. 48.
    Zheng, Y.F., Pei, R., Chen, C.: Strategies for automatic assembly of deformable objects. In: IEEE International Conference on Robotics and Automation, pp. 2598–2603 (1991)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2016

Authors and Affiliations

  • Meng Zhang
    • 1
  • Shuangmin Chen
    • 1
    Email author
  • Zhenyu Shu
    • 2
  • Shi-Qing Xin
    • 1
  • Jieyu Zhao
    • 1
  • Guang Jin
    • 1
  • Rong Zhang
    • 1
  • Jürgen Beyerer
    • 3
  1. 1.Faculty of Electrical Engineering and Computer ScienceNingbo UniversityNingboPeople’s Republic of China
  2. 2.School of Information Science and EngineeringNingbo Institute of Technology, Zhejiang UniversityNingboPeople’s Republic of China
  3. 3.Fraunhofer-Institute of OptronicsSystem Technologies and Image Exploitation IOSBKarlsruheGermany

Personalised recommendations