Photo Repair and 3D Structure from Flatbed Scanners Using 4- and 2-Source Photometric Stereo

  • Ruggero Pintus
  • Thomas Malzbender
  • Oliver Wang
  • Ruth Bergman
  • Hila Nachlieli
  • Gitit Ruckenstein
Conference paper
Part of the Communications in Computer and Information Science book series (CCIS, volume 68)


We recently introduced a technique that allows 3D information to be captured from a conventional flatbed scanner [22]. The technique requires no hardware modification and allows untrained users to easily capture 3D datasets. Once captured, these datasets can be used for interactive relighting and enhancement of surface detail on physical objects. We have also found that the method can be used to scan and repair damaged photographs. Since only the 3D structure on these photographs will typically be surface tears and creases, our method provides an accurate procedure for automatically detecting these flaws without any user intervention. Once detected, automatic techniques, such as infilling and texture synthesis, can be leveraged to seamlessly repair such damaged areas. We here provide a more thorough exposition and significant new material. We first present a method that is able to repair damaged photographs with minimal user interaction and then show how we can achieve similar results using a fully automatic process.


Scanners 3D reconstruction Photo repair Photometric stereo 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Barsky, S., Petrou, M.: The 4-Source Photometric Stereo Technique for Three-Dimensional Surfaces in the Presence of Highlights and Shadows. IEEE Transaction on Pattern Analysis and Machine Intelligence 25(10), 1239–1252 (2003)CrossRefGoogle Scholar
  2. 2.
    Bergman, R., Maurer, R., Nachlieli, H., Ruckenstein, G., Chase, P., Greig, D.: Comprehensive Solutions for Automatic Removal of Dust and Scratches from Images. Journal of Electronic Imaging (2007)Google Scholar
  3. 3.
    Bracewell, R.N.: The Fourier Transform and Its Applications. McGraw-Hill, New York (1965)zbMATHGoogle Scholar
  4. 4.
    Brown, B., Toler-Franklin, C., Nehad, D., Burns, M., Dobkin, D., Vlachopoulos, A., Doumas, C., Rusinkiewicz, S., Weyrich, T.: A System for High-Volume Acquisition and Matching of Fresco Fragments: Reassembling Theran Wall Paintings. ACM Transactions on Graphics 27(3), 83 (2008)CrossRefGoogle Scholar
  5. 5.
    Buades, A., Coll, B., Morel, J.: A Review of Image Denoising Algorithms, With a New One. Multiscale Modeling and Simulation (SIAM Interdisciplinary Journal) 4(2), 490–530 (2005)zbMATHCrossRefMathSciNetGoogle Scholar
  6. 6.
    Chantler, M., Spense, A.: Apparatus and Method for Obtaining Surface Texture Information, Patent GB 0424417.4 (2004)Google Scholar
  7. 7.
    DIGITAL ICETM, Eastman Kodak Company,
  8. 8.
    Efros, A., Leung, T.: Texture Synthesis by Non-parametric Sampling. In: Proceedings of IEEE Internation Conference on Computer Vision (September 1999)Google Scholar
  9. 9.
    Fattal, R., Agrawala, M., Rusinkiewicz, S.: Multiscale Shape and Detail Enhancement from Multiple-light Image Collections. ACM Transactions on Graphics 26(3) (2007)Google Scholar
  10. 10.
    Freeth, T., Bitsakis, Y., Moussas, X., Seiradakis, J., Tselikas, A., Mangou, H., Zafeiropoulou, M., Hadland, R., Bate, D., Ramsey, A., Allen, M., Crawley, A., Hockley, P., Malzbender, T., Gelb, D., Ambrisco, W., Edmunds, M.: Decoding the Ancient Greek Astronomical Calculator known as the Antikythera Mechanism. Nature 444, 587–591 (2006)CrossRefGoogle Scholar
  11. 11.
    Gardner, A., Tchou, C., Hawkins, T., Debevec, P.: Linear Light Source Reflectometry. ACM Transactions on Graphics 22(3), 749–758 (2003)CrossRefGoogle Scholar
  12. 12.
    Hammer, O., Bengston, S., Malzbender, T., Gelb, D.: Imaging Fossils Using Reflectance Transformation and Interactive Manipulation of Virtual Light Sources. Palaeontologia Electronica, August 23 (2002)Google Scholar
  13. 13.
    Hastie, T., Tibshirani, R., Friedman, J.: The Elements of Statistical Learning - Data Mining, Inference, and Prediction. Springer, Heidelberg (2001)zbMATHGoogle Scholar
  14. 14.
    Hewlett-Packard G4050 Photo Scanner (2007),
  15. 15.
    Horn, P.: Robot Vision. MIT Press, Cambridge (1986)Google Scholar
  16. 16.
    Klette, R., Schluns, K., Koschan, A.: Computer Vision: Three-Dimensional Data from Images. Springer, Heidelberg (1998)zbMATHGoogle Scholar
  17. 17.
    Kschischang, F.R., Frey, B.J., Loeliger, H.A.: Factor Graphs and the Sum-Product Algorithm. IEEE Transaction on Information Theory 47(2) (2001)Google Scholar
  18. 18.
    Malzbender, T., Gelb, D., Wolters, H.: Polynomial Texture Maps. In: Fiume, E. (ed.) Proceedings of ACM Siggraph 2001. Computer Graphics Proceedings, Annual Conference Series, pp. 519–528. ACM Press / ACM SIGGRAPH, New York (2001)Google Scholar
  19. 19.
    Medioni, G., Lee, M., Tang, C.: A Computational Framework for Segmentation and Grouping. Elsevier, Amsterdam (2000)zbMATHGoogle Scholar
  20. 20.
    Onn, R., Bruckstein, A.: Integrability Disambiguates Surface Recovery in Two-Image Photometric Stereo. International Journal of Computer Vision 5, 105–113 (1990)CrossRefGoogle Scholar
  21. 21.
    Petrovic, N., Cohen, I., Frey, B.J., Koetter, R., Huang, T.S.: Enforcing Integrability for Surface Reconstruction Algorithms Using Belief Propagation in Graphical Models. In: 2001 IEEE Conf. on Computer Vision and Pattern Recognition (CVPR), vol. 1, pp. 743–748 (2001)Google Scholar
  22. 22.
    Pintus, R., Malzbender, T., Wang, O., Bergman, R., Nachlieli, H., Ruckenstein, G.: Photo Repair and 3D Structure from Flatbed Scanners. In: VISAPP International Conference on Computer Vision Theory and Applications (February 2009)Google Scholar
  23. 23.
    Schubert, R.: Using a Flatbed Scanner as a Stereoscopic Near-Field Camera. IEEE Computer Graphics and Applications, 38–45 (2000)Google Scholar
  24. 24.
    Toler-Franklin, C., Finkelstein, A., Rusinkiewics, S.: Illustration of Complex Real-World Objects using Images with Normals. In: International Symposium on Non-Photorealistic Animation and Rendering, NPAR (2007)Google Scholar
  25. 25.
    Tu, P., Mendonca, P.R.S.: Surface Reconstruction via Helmholtz Reciprocity with a Single Image Pair. In: Proc. of 2003 IEEE Computer Society Conference on computer Vision and Pattern Recognition (CVPR 2003), pp. 541–547 (2003)Google Scholar
  26. 26.
    Yang, J., Ohnishi, N., Sugie, N.: Two Image Photometric Stero Method. In: Proc. SPIE, Intelligent Robots and Computer Vision XI, vol. 1826, pp. 452–463 (2003)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2010

Authors and Affiliations

  • Ruggero Pintus
    • 1
  • Thomas Malzbender
    • 2
  • Oliver Wang
    • 3
  • Ruth Bergman
    • 4
  • Hila Nachlieli
    • 4
  • Gitit Ruckenstein
    • 4
  1. 1.CRS4 (Center for Advanced Studies, Research and Development in Sardinia), Parco Scientifico e Tecnologico, POLARISPulaItaly
  2. 2.Hewlett-Packard LaboratoriesPalo AltoU.S.A.
  3. 3.University of CaliforniaSanta CruzU.S.A.
  4. 4.Hewlett-Packard LaboratoriesTechnion CityHaifaIsrael

Personalised recommendations