Evaluation of Discriminative Models for the Reconstruction of Hand-Torn Documents

  • Fabian RichterEmail author
  • Christian X. Ries
  • Rainer Lienhart
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 9005)


This work deals with the reconstruction of hand-torn documents from pairs of aligned fragments. In the first step we use a recent approach to estimate hypotheses for aligning pieces from a set of magazine pages. We then train a structural support vector machine to determine the compatibility of previously aligned pieces along their adjacent contour regions. Based on the output of this discriminative model we induce a ranking among all pairs of pieces, as high compatibility scores often correlate with spatial configurations found in the original document. To evaluate our system’s performance we provide a new baseline on a publicly available benchmark dataset in terms of mean average precision (mAP). With the (mean) average precision being widely recognized as de facto standard for evaluation of object detection and retrieval methods, our work is devoted to establish this performance measure for document reconstruction to enable a rigorous comparison of different methods.


Support Point Discriminative Model Outer Contour Orientation Estimate Foreground Region 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


  1. 1.
    Geller, T.: Darpa shredder challenge solved. Commun. ACM 55, 16–17 (2012)MathSciNetGoogle Scholar
  2. 2.
    Richter, F., Ries, C.X., Romberg, S., Lienhart, R.: Partial contour matching for document pieces with content-based prior. In: IEEE International Conference on Multimedia and Expo (2014)Google Scholar
  3. 3.
    Torr, P.H.S., Zisserman, A.: Mlesac: a new robust estimator with application to estimating image geometry. Comput. Vis. Image Underst. 78, 138–156 (2000)CrossRefGoogle Scholar
  4. 4.
    Richter, F., Ries, C.X., Cebron, N., Lienhart, R.: Learning to reassemble shredded documents. IEEE Trans. Multimedia 15, 582–593 (2012)CrossRefGoogle Scholar
  5. 5.
    Yao, F., Shao, G.: A shape and image merging technique to solve jigsaw puzzles. Pattern Recogn. Lett. 24, 1819–1835 (2003)CrossRefGoogle Scholar
  6. 6.
    Bunke, H., Bühler, U.: Applications of approximate string matching to 2D shape recognition. Pattern Recogn. 26, 1797–1812 (1993)CrossRefGoogle Scholar
  7. 7.
    Biswas, A., Bhowmick, P., Bhattacharya, B.B.: Reconstruction of torn documents using contour maps. In: IEEE International Conference on Image Processing, vol. 3, pp. 517–520 (2005)Google Scholar
  8. 8.
    Zhu, L., Zhou, Z., Zhang, J., Hu, D.: A partial curve matching method for automatic reassembly of 2d fragments. In: Huang, D.-S., Li, K., Irwin, G.W. (eds.) ICIC 2006. LNCIS, vol. 345, pp. 645–650. Springer, Heidelberg (2006)Google Scholar
  9. 9.
    Justino, E., Oliveira, L.S., Freitas, C.: Reconstructing shredded documents through feature matching. Forensic Sci. Int. 160, 140–147 (2006)CrossRefGoogle Scholar
  10. 10.
    Stieber, A., Schneider, J., Nickolay, B., Krüger, J.: A contour matching algorithm to reconstruct ruptured documents. In: Goesele, M., Roth, S., Kuijper, A., Schiele, B., Schindler, K. (eds.) Pattern Recognition. DAGM, vol. 6376, pp. 121–130. Springer, Heidelberg (2010) CrossRefGoogle Scholar
  11. 11.
    Zhu, L., Zhou, T., Hu, D.: Globally consistent reconstruction of ripped-up documents. IEEE Trans. Pattern Anal. Mach. Intell. 30, 1–13 (2008)CrossRefGoogle Scholar
  12. 12.
    Cao, S., Liu, H., Yan, S.: Automated assembly of shredded pieces from multiple photos. In: IEEE International Conference on Multimedia and Expo, pp. 358–363 (2010)Google Scholar
  13. 13.
    da Gama Leitão, H.C., Stolfi, J.: A multiscale method for the reassembly of two-dimensional fragmented objects. IEEE Trans. Pattern Anal. Mach. Intell. 24, 1239–1251 (2002)CrossRefGoogle Scholar
  14. 14.
    Papaodysseus, C., Panagopoulos, T., Exarhos, M., Triantafillou, C., Fragoulis, D., Doumas, C.: Contour-shape based reconstruction of fragmented, 1600 BC wall paintings. IEEE Trans. Sig. Process. 50, 1277–1288 (2002)CrossRefGoogle Scholar
  15. 15.
    Wolfson, H.J.: On curve matching. IEEE Trans. Pattern Anal. Mach. Intell. 12, 483–489 (1990)CrossRefGoogle Scholar
  16. 16.
    Chen, L., Feris, R., Turk, M.: Efficient partial shape matching using smith-waterman algorithm. In: CVPR Workshop (2008)Google Scholar
  17. 17.
    Smith, T.F., Waterman, M.S.: Identification of common molecular subsequences. J. Mol. Biol. 147, 195–197 (1981)CrossRefGoogle Scholar
  18. 18.
    Suzuki, S., Abe, K.: Topological structural analysis of digital binary images by border following. Comput. Vis. Graph. Image Process. 30, 32–46 (1985)CrossRefzbMATHGoogle Scholar
  19. 19.
    Douglas, D., Peucker, T.: Algorithms for the reduction of the number of points required to represent a digitized line or its caricature. Can. Cartographer 10, 112–122 (1973)CrossRefGoogle Scholar
  20. 20.
    Hollitt, C., Deeb, A.S.: Determining image orientation using the hough and fourier transforms. In: Proceedings of the 27th Conference on Image and Vision Computing, pp. 346–351 (2012)Google Scholar
  21. 21.
    Haralick, R.M., Shanmugam, K., Dinstein, I.: Textural features for image classification. IEEE Trans. Syst. Man Cybern. SMC–3, 610–621 (1973)CrossRefGoogle Scholar
  22. 22.
    Joachims, T.: Advances in Kernel Methods - Support Vector Learning. MIT Press, Cambridge (1999)Google Scholar
  23. 23.
    Yu, C.J., Joachims, T., Elber, R., Pillardy, J.: Support vector training of protein alignment models. J. Comput. Biol. 15, 867–880 (2008)CrossRefMathSciNetGoogle Scholar
  24. 24.
    Felzenszwalb, P.F., Girshick, R.B., McAllester, D., Ramanan, D.: Object detection with discriminatively trained part based models. IEEE Trans. Pattern Anal. Mach. Intell. 32, 1627–1645 (2010)CrossRefGoogle Scholar
  25. 25.
    Kruskal, J.: On the shortest spanning subtree and the traveling salesman problem. Proc. Am. Math. Soc. 7, 48–50 (1956)CrossRefzbMATHMathSciNetGoogle Scholar

Copyright information

© Springer International Publishing Switzerland 2015

Authors and Affiliations

  • Fabian Richter
    • 1
    Email author
  • Christian X. Ries
    • 1
  • Rainer Lienhart
    • 1
  1. 1.Multimedia Computing and Computer Vision LabUniversity of AugsburgAugsburgGermany

Personalised recommendations