DNN-Buddies: A Deep Neural Network-Based Estimation Metric for the Jigsaw Puzzle Problem

  • Dror Sholomon
  • Omid E. David
  • Nathan S. Netanyahu
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 9887)


This paper introduces the first deep neural network-based estimation metric for the jigsaw puzzle problem. Given two puzzle piece edges, the neural network predicts whether or not they should be adjacent in the correct assembly of the puzzle, using nothing but the pixels of each piece. The proposed metric exhibits an extremely high precision even though no manual feature extraction is performed. When incorporated into an existing puzzle solver, the solution’s accuracy increases significantly, achieving thereby a new state-of-the-art standard.


Jigsaw Puzzle Compatibility Measure Chromatic Information Imbalanced Nature Adjacent Piece 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


  1. 1.
    Altman, T.: Solving the jigsaw puzzle problem in linear time. Appl. Artif. Intell. Int. J. 3(4), 453–462 (1989)CrossRefGoogle Scholar
  2. 2.
    Brown, B., Toler-Franklin, C., Nehab, D., Burns, M., Dobkin, D., Vlachopoulos, A., Doumas, C., Rusinkiewicz, S., Weyrich, T.: A system for high-volume acquisition and matching of fresco fragments: Reassembling Theran wall paintings. ACM Trans. Graph. 27(3), 84 (2008)CrossRefGoogle Scholar
  3. 3.
    Cao, S., Liu, H., Yan, S.: Automated assembly of shredded pieces from multiple photos. In: IEEE International Conference on Multimedia and Expo, pp. 358–363 (2010)Google Scholar
  4. 4.
    Cho, T., Avidan, S., Freeman, W.: A probabilistic image jigsaw puzzle solver. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 183–190 (2010)Google Scholar
  5. 5.
    Cho, T., Butman, M., Avidan, S., Freeman, W.: The patch transform and its applications to image editing. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 1–8 (2008)Google Scholar
  6. 6.
    Collobert, R., Kavukcuoglu, K., Farabet, C.: Torch7: a matlab-like environment for machine learning. In: BigLearn, NIPS Workshop. No. EPFL-CONF-192376 (2011)Google Scholar
  7. 7.
    Deever, A., Gallagher, A.: Semi-automatic assembly of real cross-cut shredded documents. In: ICIP, pp. 233–236 (2012)Google Scholar
  8. 8.
    Demaine, E., Demaine, M.: Jigsaw puzzles, edge matching, and polyomino packing: Connections and complexity. Graphs Comb. 23, 195–208 (2007)MathSciNetCrossRefMATHGoogle Scholar
  9. 9.
    Freeman, H., Garder, L.: Apictorial jigsaw puzzles: The computer solution of a problem in pattern recognition. IEEE Trans. Electron. Comput. EC-13(2), 118–127 (1964)Google Scholar
  10. 10.
    Gallagher, A.: Jigsaw puzzles with pieces of unknown orientation. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 382–389 (2012)Google Scholar
  11. 11.
    Goldberg, D., Malon, C., Bern, M.: A global approach to automatic solution of jigsaw puzzles. Comput. Geom.: Theory Appl. 28(2–3), 165–174 (2004)MathSciNetCrossRefGoogle Scholar
  12. 12.
    Grubinger, M., Clough, P., Müller, H., Deselaers, T.: The IAPR TC-12 benchmark: a new evaluation resource for visual information systems. In: International Workshop OntoImage, vol. 5, p. 10 (2006)Google Scholar
  13. 13.
    He, H., Garcia, E.A.: Learning from imbalanced data. IEEE Trans. Knowl. Data Eng. 21(9), 1263–1284 (2009)CrossRefGoogle Scholar
  14. 14.
    Justino, E., Oliveira, L., Freitas, C.: Reconstructing shredded documents through feature matching. Forensic Sci. Int. 160(2), 140–147 (2006)CrossRefGoogle Scholar
  15. 15.
    Koller, D., Levoy, M.: Computer-aided reconstruction and new matches in the forma urbis romae. Bullettino Della Commissione Archeologica Comunale di Roma, 103–125 (2006)Google Scholar
  16. 16.
    Marande, W., Burger, G.: Mitochondrial DNA as a genomic jigsaw puzzle. Science 318(5849), 415–415 (2007)CrossRefGoogle Scholar
  17. 17.
    Marques, M., Freitas, C.: Reconstructing strip-shredded documents using color as feature matching. In: ACM Symposium on Applied Computing, pp. 893–894 (2009)Google Scholar
  18. 18.
    Morton, A.Q., Levison, M.: The computer in literary studies. In: IFIP Congress, pp. 1072–1081 (1968)Google Scholar
  19. 19.
    Paikin, G., Tal, A.: Solving multiple square jigsaw puzzles with missing pieces. In: 2015 IEEE Conference on Computer Vision and Pattern Recognition, pp. 4832–4839. IEEE (2015)Google Scholar
  20. 20.
    Pomeranz, D., Shemesh, M., Ben-Shahar, O.: A fully automated greedy square jigsaw puzzle solver. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 9–16 (2011)Google Scholar
  21. 21.
    Sholomon, D., David, O.E., Netanyahu, N.S.: A genetic algorithm-based solver for very large jigsaw puzzles. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 1767–1774 (2013)Google Scholar
  22. 22.
    Sholomon, D., David, O.E., Netanyahu, N.S.: A generalized genetic algorithm-based solver for very large jigsaw puzzles of complex types. In: AAAI Conference on Artificial Intelligence, pp. 2839–2845 (2014)Google Scholar
  23. 23.
    Sholomon, D., David, O.E., Netanyahu, N.S.: Genetic algorithm-based solver for very large multiple jigsaw puzzles of unknown dimensions and piece orientation. In: ACM Conference on Genetic and Evolutionary Computation, pp. 1191–1198 (2014)Google Scholar
  24. 24.
    Son, K., Hays, J., Cooper, D.B.: Solving square jigsaw puzzles with loop constraints. In: Fleet, D., Pajdla, T., Schiele, B., Tuytelaars, T. (eds.) ECCV 2014, Part VI. LNCS, vol. 8694, pp. 32–46. Springer, Heidelberg (2014)Google Scholar
  25. 25.
    Wang, C.S.E.: Determining molecular conformation from distance or density data. Ph.D. Thesis, Massachusetts Institute of Technology (2000)Google Scholar
  26. 26.
    Yang, X., Adluru, N., Latecki, L.J.: Particle filter with state permutations for solving image jigsaw puzzles. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 2873–2880 (2011)Google Scholar
  27. 27.
    Zhao, Y., Su, M., Chou, Z., Lee, J.: A puzzle solver and its application in speech descrambling. In: WSEAS International Conference on Computer Engineering and Applications, pp. 171–176 (2007)Google Scholar

Copyright information

© Springer International Publishing Switzerland 2016

Authors and Affiliations

  • Dror Sholomon
    • 1
  • Omid E. David
    • 1
  • Nathan S. Netanyahu
    • 1
    • 2
  1. 1.Department of Computer ScienceBar-Ilan UniversityRamat-GanIsrael
  2. 2.Center for Automation ResearchUniversity of MarylandCollege ParkUSA

Personalised recommendations