Advertisement

Exploiting Vector Fields for Geometric Rectification of Distorted Document Images

  • Gaofeng Meng
  • Yuanqi Su
  • Ying Wu
  • Shiming Xiang
  • Chunhong Pan
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11220)

Abstract

This paper proposes a segment-free method for geometric rectification of a distorted document image captured by a hand-held camera. The method can recover the 3D page shape by exploiting the intrinsic vector fields of the image. Based on the assumption that the curled page shape is a general cylindrical surface, we estimate the parameters related to the camera and the 3D shape model through weighted majority voting on the vector fields. Then the spatial directrix of the surface is recovered by solving an ordinary differential equation (ODE) through the Euler method. Finally, the geometric distortions in images can be rectified by flattening the estimated 3D page surface onto a plane. Our method can exploit diverse types of visual cues available in a distorted document image to estimate its vector fields for 3D page shape recovery. In comparison to the state-of-the-art methods, the great advantage is that it is a segment-free method and does not have to extract curved text lines or textual blocks, which is still a very challenging problem especially for a distorted document image. Our method can therefore be freely applied to document images with extremely complicated page layouts and severe image quality degradation. Extensive experiments are implemented to demonstrate the effectiveness of the proposed method.

Keywords

Document image processing Geometric rectification Vector fields 3D shape recovery OCR 

Notes

Acknowledgment

We thank the kind area chair and the anonymous reviewers for their valuable comments. This work was supported in part by the National Natural Science Foundation of China under Grants 91646207, National Science Foundation grant IIS-1217302, IIS-1619078, and the Army Research Office ARO W911NF-16-1-0138.

References

  1. 1.
    Liang, J., Doermann, D., Li, H.: Camera-based analysis of text and documents: a survey. Int. J. Doc. Anal. Recognit. 7(2–3), 84–104 (2005)CrossRefGoogle Scholar
  2. 2.
    Nagy, G.: Twenty years of document image analysis in pami. IEEE Trans. Pattern Anal. Mach. Intell. 22(1), 38–62 (2000)MathSciNetCrossRefGoogle Scholar
  3. 3.
    Brown, M.S., Tsoi, Y.C.: Geometric and shading correction for images of printed materials using boundary. IEEE Trans. Image Process. 15(6), 1544–1554 (2006)CrossRefGoogle Scholar
  4. 4.
    Stamatopoulos, N., Gatos, B., Pratikakis, I., Perantonis, S.J.: Goal-oriented rectification of camera-based document images. IEEE Trans. Image Process. 20(4), 910–920 (2011)MathSciNetCrossRefGoogle Scholar
  5. 5.
    Tsoi, Y.C., Brown, M.S.: Geometric and shading correction for images of printed materials: a unified approach using boundary. Proc. IEEE Conf. Comput. Vis. Pattern Recognit. 1, 240–246 (2004)Google Scholar
  6. 6.
    Ulges, A., Lampert, C.H., Breuel, T.M.: Document image dewarping using robust estimation of curled text lines. In: Proceedings of the 8th International Conference on Document Analysis and Recognition, pp. 1001–1005 (2005)Google Scholar
  7. 7.
    Zhang, Z., Tan, C.L.: Straightening warped text lines using polynomial regression. In: ICIP’02, vol. 3, pp. 977–980 (2002)Google Scholar
  8. 8.
    Zhang, Z., Tan, C.L.: Correcting document image warping based on regression of curved text lines. In: Proceedings of the 7th International Conference on Document Analysis and Recognition (ICDAR), pp. 589–593 (2003)Google Scholar
  9. 9.
    Brown, M.S., Sun, M., Yang, R., Yun, L., Seales, W.B.: Restoring 2d content from distorted documents. IEEE Trans. Pattern Anal. Mach. Intell. 29(11), 1904–1916 (2007)CrossRefGoogle Scholar
  10. 10.
    Cao, H., Ding, X., Liu, C.: A cylindrical surface model to rectify the bound document image. In: Proceedings of International Conference on Computer Vision (ICCV), pp. 228–233 (2003)Google Scholar
  11. 11.
    Hyung, I.K., Kim, J., Nam, I.C.: Composition of a dewarped and enhanced document image from two view images. IEEE Trans. Image Process. 18(7), 1551–1562 (2009)MathSciNetCrossRefGoogle Scholar
  12. 12.
    Liang, J., DeMenthon, D., Doermann, D.: Flattening curved documents in images. Proc. IEEE Conf. Comput. Vis. Pattern Recognit. (CVPR) 2, 338–345 (2005)Google Scholar
  13. 13.
    Liang, J., DeMenthon, D., Doermann, D.: Geometric rectification of camera-captured document images. IEEE Trans. Pattern Anal. Mach. Intell. 30(4), 591–605 (2008)CrossRefGoogle Scholar
  14. 14.
    Meng, G., Pan, C., Xiang, S., Duan, J., Zheng, N.: Metric rectification of curved document images. IEEE Trans. Pattern Anal. Mach. Intell. 34(4), 707–722 (2012)CrossRefGoogle Scholar
  15. 15.
    Meng, G., Xiang, S., Pan, C., Zheng, N.: Active rectification of curved document images using structured beams. Int. J. Comput. Vis. 122(1), 34–60 (2017)MathSciNetCrossRefGoogle Scholar
  16. 16.
    Tan, C.L., Zhang, L., Zhang, Z., Xia, T.: Restoring warped document images through 3d shape modeling. IEEE Trans. Pattern Anal. Mach. Intell. 28(2), 195–208 (2006)CrossRefGoogle Scholar
  17. 17.
    Tian, Y., Narasimhan, S.: Rectification and 3d reconstruction of curved document images. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 377–384 (June 2011)Google Scholar
  18. 18.
    You, S., Matsushita, Y., Sinha, S., Bou, Y., Ikeuchi, K.: Multiview rectification of folded documents. In: IEEE Transactions on Pattern Analysis and Machine Intelligence (2017)Google Scholar
  19. 19.
    Zhang, L., Yip, A.M., Brown, M.S., Tan, C.L.: A unified framework for document restoration using inpainting and shape-from-shading. Pattern Recognit. 42(11), 2961–2978 (2009)CrossRefGoogle Scholar
  20. 20.
    Meng, G., Wang, Y., Qu, S., Xiang, S., Pan, C.: Active flattening of curved document images via two structured beams. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3890–3897 (2014)Google Scholar
  21. 21.
    Meng, G., Huang, Z., Song, Y., Xiang, S., Pan, C.: Extraction of virtual baselines from distorted document images using curvilinear projection. In: IEEE International Conference on Computer Vision, pp. 3925–3933 (2015)Google Scholar
  22. 22.
    Schneider, D., Block, M., Rojas, R.: Robust document warping with interpolated vector fields. Proc. Ninth Int. Conf. Doc. Anal. Recognit. 1, 113–117 (2007)Google Scholar
  23. 23.
    Nikolova, M., Ng, M.K.: Analysis of half-quadratic minimization methods for signal and image recovery. SIAM J. Sci. Comput. 27(3), 937–966 (2005)MathSciNetCrossRefGoogle Scholar
  24. 24.
    Shafait, F., Breuel, T.M.: Document image dewarping contest. In: Proceedings of the 2nd Int. Workshop on Camera-Based Document Analysis and Recognition, Curitiba, Brazil, pp. 181–188 (Sep. 2007)Google Scholar
  25. 25.
    Kim, B.S., Koo, H.I., Cho, N.I.: Document dewarping via text-line based optimization. Pattern Recognit. 48(11), 3600–3614 (2015)CrossRefGoogle Scholar
  26. 26.
    Gatos, B., Pratikakis, I., Ntirogiannis, K.: Segmentation based recovery of arbitrarily warped document images. In: The 9th International Conference on Document Analysis and Recognition, Curitiba, Brazil, pp. 989–993 (Sep. 2007)Google Scholar
  27. 27.
    Masalovitch, A., Mestetskiy, L.: Usage of continuous skeletal image representation for document images dewarping. In: The 2nd International Workshop on Camera-Based Document Analysis and Recognition, Curitiba, Brazil, pp. 45–52 (Sep. 2007)Google Scholar
  28. 28.
    Fu, B., Wu, M., Li, R., Li, W., Xu, Z.: A model-based book dewarping method using text line detection. In: The 2nd International Workshop on Camera-Based Document Analysis and Recognition, Curitiba, Brazil, pp. 63–70 (Sep. 2007)Google Scholar
  29. 29.
    Bukhari, S.S., Shafait, F., Breuel, T.M.: Dewarping of document images using coupled-snakes. In: The 3rd International Workshop on Camera-Based Document Analysis and Recognition, Barcelona, Spain, pp. 34–41 (July 2009)Google Scholar

Copyright information

© Springer Nature Switzerland AG 2018

Authors and Affiliations

  • Gaofeng Meng
    • 1
    • 3
  • Yuanqi Su
    • 2
  • Ying Wu
    • 3
  • Shiming Xiang
    • 1
  • Chunhong Pan
    • 1
  1. 1.National Laboratory of Pattern Recognition, Institute of AutomationChinese Academy of SciencesBeijingChina
  2. 2.Xi’an Jiaotong UniversityXi’anChina
  3. 3.Northwestern UniversityEvanstonUSA

Personalised recommendations