Restoring camera-captured distorted document images

  • Changsong Liu
  • Yu Zhang
  • Baokang Wang
  • Xiaoqing Ding
Special Issue Paper


This article discusses restoration of camera-captured distorted document images. Without the assistance of 3D data or model, our algorithm estimates and rectifies document warping just from 2D image based on line segmentation. Warping shape of each text line is acquired by estimating baselines’ shape and characters’ slant angles after line segmentation. In order to get fluent recovery result, thin-plate splines are exploited whose key points are determined through the result of warping estimation. Such process can effectively depict document warping and successfully restore warped document images to be flat. Comparison of OCR recognition rate between original camera-captured images and restored images shows the effectiveness of the algorithm proposed. We also demonstrate evaluation on DFKI dewarping contest dataset with some related algorithms. Besides desirable restoration result, processing speed of the whole procedure is satisfactory as well. In conclusion, it is applicable to be performed in OCR application to achieve better understanding of camera-captured document images.


Baseline fitting Distorted document images Line segmentation Thin-plate splines 



This work is supported by National Natural Science Foundation of China (No. 61471214), 973 National Basic Research Program of China (No. 2014CB340506). The fourth author is also supported by National Natural Science Foundation of China (61032008).


  1. 1.
    Zhu, Y., Wang, C., Dai, R.: Document image binarization based on stroke enhancement. In: Proceedings of the 18th International Conference on Pattern Recognition, vol. 1, pp. 955–958 (2006)Google Scholar
  2. 2.
    Niblack, W.: An Introduction to Digital Image Processing, pp. 113–116. Prentice Hall, New Jersey (1986)Google Scholar
  3. 3.
    Meng, G., Xiang, S., Zheng, N., Pan, C.: Nonparametric illumination correction for scanned document images via convex Hulls. IEEE Trans. Pattern Anal. Mach. Intell. 35(7), 1730–1743 (2013)CrossRefGoogle Scholar
  4. 4.
    Cao, H., Ding, X., Liu, C.: A cylindrical surface model to rectify the bound document image. In: Proceedings of the IEEE International Conference on Computer Vision, vol. 2, pp. 228–233 (2003)Google Scholar
  5. 5.
    Zhang, L., Tan, CL.: Restoring warped document images using shape-from-shading and surface interpolation. In: Proceedings of the 18th International Conference on Pattern Recognition, vol. 1, pp. 642–645 (2006)Google Scholar
  6. 6.
    Brown, M.S.: Image restoration of arbitrarily warped documents. IEEE Trans. Pattern Anal. Mach. Intell. 26, 1295–1306 (2004)CrossRefGoogle Scholar
  7. 7.
    Liang, J., DeMenthon, D., Doermann, D.: Geometric rectification of camera-captured document images. IEEE Trans. Pattern Anal. Mach. Intell. 30, 591–605 (2008)CrossRefGoogle Scholar
  8. 8.
    Meng, G., Pan, C., Xiang, S., Duan, J.: Metric rectification of curved document images. IEEE Trans. Pattern Anal. Mach. Intell. 34(4), 707–722 (2012)CrossRefGoogle Scholar
  9. 9.
    He, Y., Pan, P., Xie, S., Sun, J., Naoi, S.: A book dewarping system by boundary-based 3D surface reconstruction. ICDAR, pp. 403–407. IEEE (2013)Google Scholar
  10. 10.
    Lu, S., Tan, CL.: Document flattening through grid modeling and regularization. In: Proceedings of the 18th International Conference on Pattern Recognition, vol. 1, pp. 971–974 (2006)Google Scholar
  11. 11.
    Gatos, B., Pratikakis, I., Ntirogiannis, K.: Segmentation based recovery of arbitrarily warped document image. In: Proceedings of the 9th International Conference on Document Analysis and Recognition, vol. 2, pp. 989–993 (2007)Google Scholar
  12. 12.
    Masalovitch, A., Mestetskiy, L.: Usage of continuous skeletal image representation for document images de-warping. In: Proceedings of International Workshop on Camera-Based Document Analysis and Recognition, Curitiba (2007)Google Scholar
  13. 13.
    Bukhari., S.S., Shafait, F., Breuel, T.M.: Dewarping of document images using coupled-snakes. In: Proceedings of Third International Workshop on Camera-Based Document Analysis and Recognition, Barcelona, Spain (2009)Google Scholar
  14. 14.
    Bukhari, Saqib, Syed, Shafait, Faisal, Breuel, Thomas M.: Coupled snakelets for curled text-line segmentation from warped document images. Int. J. Doc. Anal. Recogn. (IJDAR) 16.1, 33–53 (2013)CrossRefGoogle Scholar
  15. 15.
    Bin, Fu et al.: A model-based book dewarping method using text line detection. In: Proceedings of the international Workshop on Camera Based Document Analysis and Recognition, Curitiba, Brazil (2007)Google Scholar
  16. 16.
    Hsia, K., Lien, S., Su, J.: Fast restoration of warped document image based on text rectangle area segmentation. J. Softw. 8(5), 1162–1167 (2013)CrossRefGoogle Scholar
  17. 17.
    Stamatopoulos, N., Gatos, B., Pratikakis, I., Perantonis, S.J.: Goal-oriented rectification of camera-based document images. IEEE Trans. Image Process. 20(4), 910–920 (2011)Google Scholar
  18. 18.
    Cheng, P., Yan, H., Han, Z.: An algorithm for computing the minimum Area bounding rectangle of an arbitrary polygon. J. Eng. Graph. 29, 122–126 (2008)Google Scholar
  19. 19.
  20. 20.
  21. 21.
  22. 22.
    Rousseeuw, P.J., Leroy, A.M.: Robust Regression and Outlier Detection, pp. 1–18. Wiley, New York (1987)CrossRefzbMATHGoogle Scholar
  23. 23.
    Torr, P.H.S., Murray, D.W.: The development and comparison of Robust methods for estimating the fundamental matrix. Int. J. Comput. Vis. 24, 271–300 (1997)CrossRefGoogle Scholar
  24. 24.
    Lu, S., Chen, B.M., Ko, C.C.: Perspective rectification of document images using fuzzy set and morphological operations. Image Vis. Comput. 23, 541–553 (2005)CrossRefGoogle Scholar
  25. 25.
    Zhang, Y., Liu, C., Ding, X., Wang, K.: Restoring warped document image through segmentation and full page interpolation. In: Proceedings of SPIE (2009)Google Scholar
  26. 26.
    Bookstein, Fred L.: Principal warps: thin-plate splines and the decomposition of deformations. IEEE Trans. Pattern Anal. Mach. Intell. 11, 567–585 (1989)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2014

Authors and Affiliations

  • Changsong Liu
    • 1
  • Yu Zhang
    • 1
  • Baokang Wang
    • 1
  • Xiaoqing Ding
    • 1
  1. 1.The State Key Laboratory of Intelligent Technology and Systems, Tsinghua National Laboratory for Information Science and Technology, Department of Electronic EngineeringTsinghua UniversityBeijingChina

Personalised recommendations