Dewarping Book Page Spreads Captured with a Mobile Phone Camera

Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 8357)

Abstract

Capturing book images is more convenient with a mobile phone camera than with more specialized flat-bed scanners or 3D capture devices. We built an application for the iPhone 4S that captures a sequence of hi-res (8 MP) images of a page spread as the user sweeps the device across the book. To do the 3D dewarping, we implemented two algorithms: optical flow (OF) and structure from motion (SfM). Making further use of the image sequence, we examined the potential of multi-frame OCR. Preliminary evaluation on a small set of data shows that OF and SfM had comparable OCR performance for both single-frame and multi-frame techniques, and that multi-frame was substantially better than single-frame. The computation time was much less for OF than for SfM.

Keywords

Document capture Document analysis Dewarping  Mobile phone camera Book scanning 

Notes

Acknowledgment

This work was done at FX Palo Alto Laboratory. We thank Michael Cutter and David Lee for helpful discussions.

References

  1. 1.
    Beardsley, P., Zisserman, A., Murray, D.: Sequential updating of projective and affine structure from motion. Intl. J. Comput. Vision 23(3), 235–259 (1997)CrossRefGoogle Scholar
  2. 2.
    Bukhari, S.S., Shafait, F., Breuel, T.M.: Border noise removal of camera-captured document images using page frame detection. In: Iwamura, M., Shafait, F. (eds.) CBDAR 2011. LNCS, vol. 7139, pp. 126–137. Springer, Heidelberg (2012)Google Scholar
  3. 3.
    Bradski, G.: The OpenCV Library. Dr. Dobb’s Journal of Software Tools (2000)Google Scholar
  4. 4.
    Brown, M., Seales, W.: Image restoration of arbitrarily warped documents. IEEE TPAMI 26, 1295–1306 (2004)CrossRefGoogle Scholar
  5. 5.
    Brown, M., Tsoi, Y.-C.: Geometric and shading correction for images of printed materials using boundary. IEEE Trans. Image Process. 15, 1544–1554 (2006)CrossRefGoogle Scholar
  6. 6.
    Cao, H., Ding, X., Liu, C.: Rectifying the bound document image captured by the camera: a model based approach. In: Proceedings of ICDAR 2003, pp. 71–75 (2003)Google Scholar
  7. 7.
    Cutter, M., Chiu, P.: Capture and dewarping of page spreads with a handheld compact 3D camera. In: Proceedings of DAS 2012, pp. 205–209 (2012)Google Scholar
  8. 8.
    Fu, B., Wu, M., Li, R., Li, W., Xu, Z., Yang, C.: A model-based book dewarping method using text line detection. In: Proceedings of CBDAR 2007, pp. 63–70 (2007)Google Scholar
  9. 9.
    Liang, J., DeMenthon, D., Doermann, D.: Geometric rectification of camera-captured document images. IEEE TPAMI 30, 591–605 (2008)CrossRefGoogle Scholar
  10. 10.
    Nakajima, N., Iketani, A., Sato, T., Ikeda, S., Kanbara, M., Yokoya, N.: Video mosaicing for document imaging. In: Proceedings of CBDAR 2007, pp. 171–178 (2007)Google Scholar
  11. 11.
    Newman, W., Dance, C., Taylor, A., Taylor, S., Taylor, M., Aldhous, T.: CamWorks: a video-based tool for efficient capture from paper source documents. In: Proceedings of International Conference on Multimedia Computing and Systems, ICMCS 1999, pp 647–653 (1999)Google Scholar
  12. 12.
    Peng, X., Cao, H., Subramanian, K., Prasad, R., Natarajan, P.: Automated image quality assessment for camera-captured OCR. In: Proceedings of ICIP 2011, pp. 2669–2672 (2011)Google Scholar
  13. 13.
    Rother, C., Kolmogorov, V., Blake, A.: Grabcut: interactive foreground extraction using iterated graph cuts. In: Proceedings of Siggraph 2004, pp. 309–314 (2004)Google Scholar
  14. 14.
    Shafait, F., Breuel, T.: Document image dewarping contest, CBDAR 2007Google Scholar
  15. 15.
    Shafait, F., Cutter, M., van Beusekom, J., Bukhari, S., Breuel, T.: Decapod: a flexible, low cost digitization solution for small and medium archives. In: Proceedings of CBDAR 2011, pp. 41–46 (2011)Google Scholar
  16. 16.
    Shi, J., Tomasi, C.: Good features to track. In: Proceedings of CVPR 1994, pp. 593–600 (1994)Google Scholar
  17. 17.
    Szeliski, R.: Computer Vision: Algorithms and Applications. Springer, New York (2010)Google Scholar
  18. 18.
    Taylor, M., Dance, C.: Enhancement of document images from cameras. In: SPIE Conference on Document Recognition V, vol. 3305, 230–241 (1998)Google Scholar
  19. 19.
  20. 20.
    Triggs, B., McLauchlan, P., Hartley, R., Fitzgibbon, A.: Bundle adjustment a modern synthesis. In: Proceedings of ICCV 1999, pp. 298–372 (1999)Google Scholar
  21. 21.
    Xu, Li, Jia, Jiaya: Two-phase kernel estimation for robust motion deblurring. In: Daniilidis, Kostas, Maragos, Petros, Paragios, Nikos (eds.) ECCV 2010, Part I. LNCS, vol. 6311, pp. 157–170. Springer, Heidelberg (2010)Google Scholar

Copyright information

© Springer International Publishing Switzerland 2014

Authors and Affiliations

  1. 1.Electrical Engineering DepartmentUniversity of CaliforniaSanta CruzUSA
  2. 2.FX Palo Alto LaboratoryPalo AltoUSA

Personalised recommendations