Advertisement

Deep Vectorization of Technical Drawings

Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 12358)

Abstract

We present a new method for vectorization of technical line drawings, such as floor plans, architectural drawings, and 2D CAD images. Our method includes (1) a deep learning-based cleaning stage to eliminate the background and imperfections in the image and fill in missing parts, (2) a transformer-based network to estimate vector primitives, and (3) optimization procedure to obtain the final primitive configurations. We train the networks on synthetic data, renderings of vector line drawings, and manually vectorized scans of line drawings. Our method quantitatively and qualitatively outperforms a number of existing techniques on a collection of representative technical drawings.

Keywords

Transformer network Vectorization Floor plans Technical drawings 

Notes

Acknowledgements

We thank Milena Gazdieva and Natalia Soboleva for their valuable contributions in preparing real-world raster and vector datasets, as well as Maria Kolos and Alexey Bokhovkin for contributing parts of shared codebase used throughout this project. We acknowledge the usage of Skoltech CDISE HPC cluster Zhores for obtaining the presented results. The work was partially supported by Russian Science Foundation under Grant 19–41-04109.

Supplementary material

504454_1_En_35_MOESM1_ESM.pdf (6.1 mb)
Supplementary material 1 (pdf 6269 KB)

References

  1. 1.
    Open CASCADE Technology OCCT. https://www.opencascade.com/, Accessed 05 March 2005
  2. 2.
    PrecisionFloorplan. http://precisionfloorplan.com, Accessed 05 March 2020
  3. 3.
    Bessmeltsev, M., Solomon, J.: Vectorization of line drawings via polyvector fields. ACM Trans. Graph. (TOG) 38(1), 9 (2019)CrossRefGoogle Scholar
  4. 4.
    Chai, D., Forstner, W., Lafarge, F.: Recovering line-networks in images by junction-point processes. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. pp. 1894–1901 (2013)Google Scholar
  5. 5.
    Chen, J., Du, M., Qin, X., Miao, Y.: An improved topology extraction approach for vectorization of sketchy line drawings. The Visual Comput. 34(12), 1633–1644 (2018).  https://doi.org/10.1007/s00371-018-1549-zCrossRefGoogle Scholar
  6. 6.
    Chen, J.Z., Lei, Q., Miao, Y.W., Peng, Q.S.: Vectorization of line drawing image based on junction analysis. Sci. China Inf. Sci. 58(7), 1–14 (2015).  https://doi.org/10.1007/s11432-014-5246-xMathSciNetCrossRefGoogle Scholar
  7. 7.
    Chu, H., Wang, S., Urtasun, R., Fidler, S.: HouseCraft: building houses from rental ads and street views. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9910, pp. 500–516. Springer, Cham (2016).  https://doi.org/10.1007/978-3-319-46466-4_30CrossRefGoogle Scholar
  8. 8.
    Delalandre, M., Valveny, E., Pridmore, T., Karatzas, D.: Generation of synthetic documents for performance evaluation of symbol recognition & spotting systems. Int. J. Document Anal. Recogn. (IJDAR) 13(3), 187–207 (2010)CrossRefGoogle Scholar
  9. 9.
    Donati, L., Cesano, S., Prati, A.: A complete hand-drawn sketch vectorization framework. Multimed. Tools Appl. 78(14), 19083–19113 (2019).  https://doi.org/10.1007/s11042-019-7311-3CrossRefGoogle Scholar
  10. 10.
    Ellis, K., Ritchie, D., Solar-Lezama, A., Tenenbaum, J.: Learning to infer graphics programs from hand-drawn images. In: Advances in Neural Information Processing Systems. pp. 6059–6068 (2018)Google Scholar
  11. 11.
    Favreau, J.D., Lafarge, F., Bousseau, A.: Fidelity vs. simplicity: a global approach to line drawing vectorization. ACM Trans. Graph. (TOG) 35(4), 120 (2016)CrossRefGoogle Scholar
  12. 12.
    Gao, J., Tang, C., Ganapathi-Subramanian, V., Huang, J., Su, H., Guibas, L.J.: Deepspline: Data-driven reconstruction of parametric curves and surfaces. arXiv preprint arXiv:1901.03781 (2019)
  13. 13.
    Guo, Y., Zhang, Z., Han, C., Hu, W.B., Li, C., Wong, T.T.: Deep line drawing vectorization via line subdivision and topology reconstruction. Comput. Graph. Forum 38, 81–90 (2019)CrossRefGoogle Scholar
  14. 14.
    Ha, D., Eck, D.: A neural representation of sketch drawings. arXiv preprint arXiv:1704.03477 (2018)
  15. 15.
    He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. pp. 770–778 (2016)Google Scholar
  16. 16.
    de las Heras, L.-P., Terrades, O.R., Robles, S., Sánchez, G.: CVC-FP and SGT: a new database for structural floor plan analysis and its groundtruthing tool. Int. J. Document Anal. Recogn (IJDAR) 18(1), 15–30 (2015).  https://doi.org/10.1007/s10032-014-0236-5CrossRefGoogle Scholar
  17. 17.
    Hilaire, X., Tombre, K.: Robust and accurate vectorization of line drawings. IEEE Trans. Pattern Anal. Mach. Intell. 6, 890–904 (2006)CrossRefGoogle Scholar
  18. 18.
    Kaiyrbekov, K., Sezgin, M.: Stroke-based sketched symbol reconstruction and segmentation. arXiv preprint arXiv:1901.03427 (2019)
  19. 19.
    Kansal, R., Kumar, S.: A vectorization framework for constant and linear gradient filled regions. The Visual Comput. 31(5), 717–732 (2014).  https://doi.org/10.1007/s00371-014-0997-3CrossRefGoogle Scholar
  20. 20.
    Kanungo, T., Haralick, R.M., Baird, H.S., Stuezle, W., Madigan, D.: A statistical, nonparametric methodology for document degradation model validation. IEEE Trans. Pattern Anal. Mach. Intell. 22(11), 1209–1223 (2000)CrossRefGoogle Scholar
  21. 21.
    Kim, B., Wang, O., Öztireli, A.C., Gross, M.: Semantic segmentation for line drawing vectorization using neural networks. In: Computer Graphics Forum. vol. 37, pp. 329–338. Wiley Online Library (2018)Google Scholar
  22. 22.
    Koch, S., et al.: Abc: A big cad model dataset for geometric deep learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. pp. 9601–9611 (2019)Google Scholar
  23. 23.
    Li, C., Liu, X., Wong, T.T.: Deep extraction of manga structural lines. ACM Trans. Graph. (TOG) 36(4), 117 (2017)Google Scholar
  24. 24.
    Liu, C., Wu, J., Kohli, P., Furukawa, Y.: Raster-to-vector: revisiting floorplan transformation. In: Proceedings of the IEEE International Conference on Computer Vision. pp. 2195–2203 (2017)Google Scholar
  25. 25.
    Liu, C., Schwing, A.G., Kundu, K., Urtasun, R., Fidler, S.: Rent3d: Floor-plan priors for monocular layout estimation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. pp. 3413–3421 (2015)Google Scholar
  26. 26.
    Máttyus, G., Luo, W., Urtasun, R.: Deeproadmapper: Extracting road topology from aerial images. In: Proceedings of the IEEE International Conference on Computer Vision. pp. 3438–3446 (2017)Google Scholar
  27. 27.
    Munusamy Kabilan, V., Morris, B., Nguyen, A.: Vectordefense: Vectorization as a defense to adversarial examples. arXiv preprint arXiv:1804.08529 (2018)
  28. 28.
    Najgebauer, P., Scherer, R.: Inertia-based fast vectorization of line drawings. Comput. Graph. Forum 38, 203–213 (2019)CrossRefGoogle Scholar
  29. 29.
    Noris, G., Hornung, A., Sumner, R.W., Simmons, M., Gross, M.: Topology-driven vectorization of clean line drawings. ACM Trans. Graph. (TOG) 32(1), 4 (2013)CrossRefGoogle Scholar
  30. 30.
    Ronneberger, O., Fischer, P., Brox, T.: U-Net: convolutional networks for biomedical image segmentation. In: Navab, N., Hornegger, J., Wells, W.M., Frangi, A.F. (eds.) MICCAI 2015. LNCS, vol. 9351, pp. 234–241. Springer, Cham (2015).  https://doi.org/10.1007/978-3-319-24574-4_28CrossRefGoogle Scholar
  31. 31.
    Rusiñol, M., Borràs, A., Lladós, J.: Relational indexing of vectorial primitives for symbol spotting in line-drawing images. Pattern Recogn. Lett. 31(3), 188–201 (2010)CrossRefGoogle Scholar
  32. 32.
    Sasaki, K., Iizuka, S., SimoSerra, E., Ishikawa, H.: Learning to restore deteriorated line drawing. The Visual Comput. 34(6–8), 1077–1085 (2018)CrossRefGoogle Scholar
  33. 33.
    Selinger, P.: Potrace: a polygon-based tracing algorithm. Potrace. http://potrace.sourceforge.net/potrace.pdf (2003)
  34. 34.
    Sharma, D., Gupta, N., Chattopadhyay, C., Mehta, S.: Daniel: A deep architecture for automatic analysis and retrieval of building floor plans. In: 2017 14th IAPR International Conference on Document Analysis and Recognition (ICDAR). vol. 1, pp. 420–425. IEEE (2017)Google Scholar
  35. 35.
    Simo-Serra, E., Iizuka, S., Ishikawa, H.: Mastering sketching: adversarial augmentation for structured prediction. ACM Trans. Graph. (TOG) 37(1), 11 (2018)CrossRefGoogle Scholar
  36. 36.
    Vaswani, A., et al.: Attention is all you need. In: Advances in Neural Information Processing Systems. pp. 5998–6008 (2017)Google Scholar
  37. 37.
    Zhao, J., Feng, J., Zhou, B.: Image vectorization using blue-noise sampling. In: Imaging and Printing in a Web 2.0 World IV International Society for Optics and Photonics. vol. 8664, p. 86640H (2013)Google Scholar
  38. 38.
    Zheng, N., Jiang, Y., Huang, D.: Strokenet: Aneural painting environment. In: International Conference on Learning Representations (2018)Google Scholar
  39. 39.
    Zhou, T., et al.: Learning to doodle with stroke demonstrations and deep q-networks. In: BMVC. p. 13 (2018)Google Scholar

Copyright information

© Springer Nature Switzerland AG 2020

Authors and Affiliations

  1. 1.Skolkovo Institute of Science and TechnologySkolkovoRussian Federation
  2. 2.New York UniversityNew YorkUSA

Personalised recommendations