Skip to main content
SpringerLink
Log in

Active Rectification of Curved Document Images Using Structured Beams

  • Published:
International Journal of Computer Vision Aims and scope Submit manuscript

Abstract

We propose an active method to rectify the geometric and photometric distortions in a document image. This method uses structured beams projected upon the curved document page to recover the spatial curves on it. A developable surface is interpolated to the curves by finding the correspondence between them. To correct the geometric distortion, the estimated developable surface is finally flattened onto a plane by explicitly solving a system of ordinary differential equations. The recovered 3D page shape, together with a Lambertian illumination model as guidance, is further used to rectify the non-uniform illumination in the images. Experimental results based on a variety of synthetic and real-captured document images demonstrate the effectiveness and efficiency of the proposed method.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18
Fig. 19
Fig. 20
Fig. 21
Fig. 22
Fig. 23
Fig. 24

Similar content being viewed by others

Notes

  1. The matlab codes can be downloaded from http://isit.u-clermont1.fr/~ab/Research/index.html.

  2. The software is freely available at http://ccwu.me/vsfm/.

  3. This matlab toolbox can be downloaded from http://www.cis.hut.fi/projects/somtoolbox/.

References

  • Bartoli, A., Perriollat, M., & Chambon, S. (2010). Generalized thin-plate spline warps. International Journal of Computer Vision, 88(1), 85–110.

    Article  Google Scholar 

  • Basu, S., Chaudhuri, C., Kundu, M., Nasipuri, M., & Basu, D. (2007). Text line extraction from multi-skewed handwritten documents. Pattern Recognition, 40(6), 1825–1839.

    Article  MATH  Google Scholar 

  • Blinn, J. F. (1977). Models of light reflection for computer synthesized pictures. SIGGRAPH Computer Graphics, 11(2), 192–198.

    Article  Google Scholar 

  • Brown, M. S., & Pisula, C. J. (2005). Conformal deskewing of non-planar documents. In Proceedings of computer vision and pattern recognition (pp. 998–1004), Washington, DC, USA.

  • Brown, M. S., & Seales, W. (2001). Document restoration using 3D shape: A general deskewing algorithm. In Proceedings of international conference on computer vision (Vol. 2, pp. 367–374).

  • Brown, M. S., & Seales, W. (2004). Image restoration of arbitrarily warped documents. IEEE Transactions on Pattern Analysis and Machine Intelligence, 26(10), 1295–1306.

    Article  Google Scholar 

  • Brown, M. S., Sun, M., Yang, R., Yun, L., & Seales, W. B. (2007). Restoring 2D content from distorted documents. IEEE Transactions on Pattern Analysis and Machine Intelligence, 27(11), 1904–1916.

    Article  Google Scholar 

  • Brown, M. S., & Tsoi, Y. C. (2006). Geometric and shading correction for images of printed materials using boundary. IEEE Transactions on Image Processing, 15(6), 1544–1554.

    Article  Google Scholar 

  • Bukhari, S. S., Breuel, T. M., & Shafait, F. (2009). Textline information extraction from grayscale camera-captured document images. In Proceedings of the international conference on image processing (pp. 1993–1996).

  • Cao, H., Ding, X., & Liu, C. (2003). A cylindrical surface model to rectify the bound document image. In Proceedings of the international conference on computer vision (pp. 228–233).

  • Cook, R. L., & Torrance, K. E. (1982). A reflectance model for computer graphics. ACM Transactions on Graphics, 1(1), 7–24.

    Article  Google Scholar 

  • Demartines, P., & Herault, J. (1997). Curvilinear component analysis: A self-organizing neural network for nonlinear mapping of data sets. IEEE Transactions on Neural Networks, 8(1), 148–154.

    Article  Google Scholar 

  • Dijkstra, E. W. (1959). A note on two problems in connexion with graphs. Numerical Mathematics, 1(1), 269–271.

    Article  MathSciNet  MATH  Google Scholar 

  • Doermann, D., Liang, J., & Li, H. (2003). Progress in camera-based document image analysis. In Proceedings of the international conference on document analysis and recognition (Vol. 1).

  • Fan, J. (2009). Robust color image enhancement of digitized books. In Proceedings of the international conference on document analysis and recognition (pp. 561–565).

  • Gatos, B., Pratikakis, I., & Perantonis, S. (2006). Adaptive degraded document image binarization. Pattern Recognition, 39(3), 317–327.

    Article  MATH  Google Scholar 

  • Gumerov, N., Zandifar, A., Duraiswami, R., & Davis, L. (2004). Structure of applicable surfaces from single views. In Proceedings of the European conference on computer vision (Vol. 3023, pp. 482–496).

  • Gumerov, N., Zandifar, A., Duraiswami, R., & Davis, L. (2006). 3D structure recovery and unwarping of surfaces applicable to planes. International Journal on Computer Vision, 66(3), 261–281.

    Article  Google Scholar 

  • He, Y., Pan, P., Xie, S., Sun, J., & Naoi, S. (2013). A book dewarping system by boundary-based 3D surface reconstruction. In Proceedings of the international conference on document analysis and recognition (pp. 403–407).

  • Hsia, S., Chen, M., & Chen, Y. (2006). A cost-effective line-based light-balancing technique using adaptive processing. IEEE Transactions on Image Processing, 15(9), 2719–2729.

    Article  Google Scholar 

  • Kim, C., Chiu, P., & Chandra, S. (2014). Dewarping book page spreads captured with a mobile phone camera. In Proceedings of the international workshop on camera-based document analysis and recognition (pp. 101–112).

  • Koo, H., & Cho, N. (2010). State estimation in a document image and its application in text block identification and text line extraction. In Proceedings of the European conference on computer vision (Vol. 6312, pp. 421–434).

  • Koo, H. I., Kim, J., & Cho, N. I. (2009). Composition of a dewarped and enhanced document image from two view images. IEEE Transactions on Image Processing, 18, 1551–1562.

    Article  MathSciNet  Google Scholar 

  • Lee, J. S., Chen, C. H., & Chang, C. C. (2009). A novel illumination-balance technique for improving the quality of degraded text-photo images. IEEE Transactions on Circuits and Systems for Video Technology, 19(6), 900–905.

    Article  Google Scholar 

  • Liang, J., DeMenthon, D., & Doermann, D. (2005a). Flattening curved documents in images. In Proceedings of computer vision and pattern recognition (Vol. 2, pp. 338–345).

  • Liang, J., DeMenthon, D., & Doermann, D. (2008). Geometric rectification of camera-captured document images. IEEE Transactions on Pattern Analysis and Machine Intelligence, 30(4), 591–605.

    Article  Google Scholar 

  • Liang, J., Doermann, D., & Li, H. (2005b). Camera-based analysis of text and documents: A survey. International Journal of Document Analysis and Recognition, 7(2), 84–104.

    Article  Google Scholar 

  • Lu, S., Chen, B. M., & Ko, C. C. (2006). A partition approach for the restoration of camera images of planar and curled document. Image and Vision Computing, 24, 837–848.

    Article  Google Scholar 

  • Meng, G., Wang, Y., Qu, S., Xiang, S., & Pan, C. (2014). Active flattening of curved document images via two structured beams. In Proceedings of computer vision and pattern recognition (pp. 1–8).

  • Meng, G., Pan, C., Xiang, S., Duan, J., & Zheng, N. (2012). Metric rectification of curved document images. IEEE Transactions on Pattern Analysis and Machine Intelligence, 34(4), 707–722.

    Article  Google Scholar 

  • Meng, G., Xiang, S., Zheng, N., & Pan, C. (2013). Nonparametric illumination correction for scanned document images via convex hulls. IEEE Transactions on Pattern Analysis and Machine Intelligence, 35(7), 1730–1743.

    Article  Google Scholar 

  • Nayar, S., & Mitsunaga, T. (2000). High dynamic range imaging: Spatially varying pixel exposures. In Proceedings of computer vision and pattern recognition (Vol. 1, pp. 472–479).

  • Perriollat, M., & Bartoli, A. (2013). A computational model of bounded developable surfaces with application to image-based 3D reconstruction. Computer Animation and Virtual Worlds, 24(5), 459–476.

    Article  Google Scholar 

  • Phong, B. T. (1975). Illumination for computer generated pictures. Communications of the ACM, 18(6), 311–317.

    Article  Google Scholar 

  • Pilu, M. (2001). Undoing paper curl distortion using applicable surfaces. In Proceedings of computer vision and pattern recognition (Vol. 1, pp. 67–72).

  • Schneider, D. C., Block, M., & Rojas, R. (2007). Robust document warping with interpolated vector fields. In Proceedings of the international conference on document analysis and recognition (pp. 113–117).

  • Stamatopoulos, N., Gatos, B., Pratikakis, I., & Perantonis, S. J. (2011). Goal-oriented rectification of camera-based document images. IEEE Transactions on Image Processing, 20(4), 910–920.

    Article  MathSciNet  Google Scholar 

  • Sun, M., & Fiume, E. (1996). A technique for constructing developable surfaces. In Proceedings of graphics interface (pp. 176–185).

  • Tan, C. L., Zhang, L., Zhang, Z., & Xia, T. (2006). Restoring warped document images through 3D shape modeling. IEEE Transactions on Pattern Analysis and Machine Intelligence, 28(2), 195–208.

    Article  Google Scholar 

  • Tian, Y., & Narasimhan, S. (2011). Rectification and 3D reconstruction of curved document images. In Proceedings of computer vision and pattern recognition (pp. 377–384).

  • Trier, O., & Taxt, T. (1995). Evaluation of binarization methods for document images. IEEE Transactions on Pattern Analysis and Machine Intelligence, 17(3), 312–315.

    Article  Google Scholar 

  • Tsoi, Y. C., & Brown, M. S. (2004). Geometric and shading correction for images of printed materials: A unified approach using boundary. In Proceedings of computer vision and pattern recognition (Vol. 1, pp. 240–246).

  • Tsoi, Y. C., & Brown, M. S. (2007). Multi-view document rectification using boundary. In Proceedings of computer vision and pattern recognition (pp. 1–8).

  • Ulges, A., Lampert, C. H., & Breuel, T. (2004). Document capture using stereo vision. In Proceedings of ACM symposium on document engineering (pp. 198–200).

  • Ulges, A., Lampert, C. H., & Breuel, T. M. (2005). Document image dewarping using robust estimation of curled text lines. In Proceedings of the international conference on document analysis and recognition (Vol. 2, pp. 1001–1005).

  • Wada, T., Ukida, H., & Matsuyama, T. (1997). Shape from shading with interreflections under a proximal light source: Distortion-free copying of an unfolded book. International Journal of Computer Vision, 24(2), 125–135.

    Article  Google Scholar 

  • Watanabe, Y., Itoyama, K., Yamada, M., & Ishikawa, M. (2012). Digitization of deformed documents using a high-speed multi-camera array. In Proceedings of the Asian conference on computer vision (pp. 394–407).

  • Zhang, Z., & Tan, C. L. (2002). Straightening warped text lines using polynomial regression. In Proceedings of the international conference on image processing (Vol. 3, pp. 977–980).

  • Zhang, Z., & Tan, C. L. (2003). Correcting document image warping based on regression of curved text lines. In Proceedings of the international conference on document analysis and recognition (Vol. 1, pp. 589–593).

  • Zhang, Z., Tan, C. L., & Fan, L. (2004). Restoration of curved document images through 3D shape modeling. In Proceedings of computer vision and pattern recognition (Vol. 1, pp. 10–15).

  • Zhang, L., Yip, A. M., & Tan, C. L. (2007). A restoration framework for correcting photometric and geometric distortions in camera-based document images. In Proceedings of the international conference on computer vision (pp. 1–8).

  • Zhang, L., Yip, A. M., Brown, M. S., & Tan, C. L. (2009). A unified framework for document restoration using inpainting and shape-from-shading. Pattern Recognition, 42(11), 2961–2978.

    Article  MATH  Google Scholar 

  • Zhang, Z., & Zhang, Z. (1998). A flexible new technique for camera calibration. IEEE Transactions on Pattern Analysis and Machine Intelligence, 22, 1330–1334.

    Article  Google Scholar 

  • Zhang, L., Zhang, Y., & Tan, C. L. (2008). An improved physically-based method for geometric restoration of distorted document images. IEEE Transactions on Pattern Analysis and Machine Intelligence, 30(4), 728–734.

    Article  Google Scholar 

Download references

Acknowledgments

Part of this work was done during the visit of Dr. Gaofeng Meng to Delft University of Technology. We are grateful to Dr. Wei Sui for his help on the use of VisualSFM. This work was supported in part by the National Natural Science Foundation of China (Grant No. 61370039, 61272331), the Beijing Nature Science Foundation (Grant No. 4162064) and the National 863 projects (Grant No. 2015AA042307).

Author information

Authors and Affiliations

  1. National Laboratory of Pattern Recognition, Institute of Automation, Chinese Academy of Sciences, Beijing, China

    Gaofeng Meng, Shiming Xiang & Chunhong Pan

  2. Institute of Artificial Intelligence and Robotics, Xi’an Jiaotong University, Xi’an, China

    Nanning Zheng

Authors
  1. Gaofeng Meng
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Shiming Xiang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Chunhong Pan
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Nanning Zheng
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Gaofeng Meng.

Additional information

Communicated by Yasuyuki Matsushita.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Meng, G., Xiang, S., Pan, C. et al. Active Rectification of Curved Document Images Using Structured Beams. Int J Comput Vis 122, 34–60 (2017). https://doi.org/10.1007/s11263-016-0952-z

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11263-016-0952-z

Keywords

Search

Navigation