Advertisement

Learning Pairwise Inter-plane Relations for Piecewise Planar Reconstruction

Conference paper
  • 725 Downloads
Part of the Lecture Notes in Computer Science book series (LNCS, volume 12352)

Abstract

This paper proposes a novel single-image piecewise planar reconstruction technique that infers and enforces inter-plane relationships. Our approach takes a planar reconstruction result from an existing system, then utilizes convolutional neural network (CNN) to (1) classify if two planes are orthogonal or parallel; and 2) infer if two planes are touching and, if so, where in the image. We formulate an optimization problem to refine plane parameters and employ a message passing neural network to refine plane segmentation masks by enforcing the inter-plane relations. Our qualitative and quantitative evaluations demonstrate the effectiveness of the proposed approach in terms of plane parameters and segmentation accuracy.

Keywords

Piecewise planar Reconstruction Deep learning Single-view 

Notes

Acknowledgments

This research is partially supported by NSERC Discovery Grants, NSERC Discovery Grants Accelerator Supplements, and DND/NSERC Discovery Grant Supplement.

Supplementary material

Supplementary material 1 (mp4 72467 KB)

References

  1. 1.
    Bauchet, J.P., Lafarge, F.: KIPPI: kinetic polygonal partitioning of images. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3146–3154 (2018)Google Scholar
  2. 2.
    Chen, J., Liu, C., Wu, J., Furukawa, Y.: Floor-SP: inverse cad for floorplans by sequential room-wise shortest path. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2661–2670 (2019)Google Scholar
  3. 3.
    Dai, A., Chang, A.X., Savva, M., Halber, M., Funkhouser, T., Nießner, M.: ScanNet: richly-annotated 3d reconstructions of indoor scenes. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 5828–5839 (2017)Google Scholar
  4. 4.
    Deng, Z., Todorovic, S., Latecki, L.J.: Unsupervised object region proposals for RGB-D indoor scenes. Comput. Vis. Image Underst. 154, 127–136 (2017)CrossRefGoogle Scholar
  5. 5.
    Furukawa, Y., Curless, B., Seitz, S.M., Szeliski, R.: Manhattan-world stereo. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 1422–1429. IEEE (2009)Google Scholar
  6. 6.
    Gallup, D., Frahm, J.M., Pollefeys, M.: Piecewise planar and non-planar stereo for urban scene reconstruction. In: 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pp. 1418–1425. IEEE (2010)Google Scholar
  7. 7.
    Hedau, V., Hoiem, D., Forsyth, D.: Recovering the spatial layout of cluttered rooms. In: 2009 IEEE 12th International Conference on Computer Vision, pp. 1849–1856. IEEE (2009)Google Scholar
  8. 8.
    Kingma, D.P., Ba, J.: Adam: a method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014)
  9. 9.
    Kushal, A., Seitz, S.M.: Single view reconstruction of piecewise swept surfaces. In: 2013 International Conference on 3D Vision-3DV 2013, pp. 239–246. IEEE (2013)Google Scholar
  10. 10.
    Liu, C., Kim, K., Gu, J., Furukawa, Y., Kautz, J.: PlanerCNN: 3D plane detection and reconstruction from a single image. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 4450–4459 (2019)Google Scholar
  11. 11.
    Liu, C., Wu, J., Kohli, P., Furukawa, Y.: Raster-to-vector: revisiting floorplan transformation. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2195–2203 (2017)Google Scholar
  12. 12.
    Liu, C., Yang, J., Ceylan, D., Yumer, E., Furukawa, Y.: PlaneNet: piece-wise planar reconstruction from a single RGB image. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2579–2588 (2018)Google Scholar
  13. 13.
    Liu, C., Schwing, A.G., Kundu, K., Urtasun, R., Fidler, S.: Rent3D: floor-plan priors for monocular layout estimation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3413–3421 (2015)Google Scholar
  14. 14.
    Monszpart, A., Mellado, N., Brostow, G.J., Mitra, N.J.: RAPTER: rebuilding man-made scenes with regular arrangements of planes. ACM Trans. Graph. 34(4), 103–111 (2015)CrossRefGoogle Scholar
  15. 15.
    Nocedal, J., Wright, S.: Numerical Optimization. Springer, New York (2006).  https://doi.org/10.1007/978-0-387-40065-5CrossRefzbMATHGoogle Scholar
  16. 16.
    Russell, B.C., Torralba, A., Murphy, K.P., Freeman, W.T.: LabelMe: a database and web-based tool for image annotation. Int. J. Comput. Vis. 77(1–3), 157–173 (2008)CrossRefGoogle Scholar
  17. 17.
    Sharma, G., Goyal, R., Liu, D., Kalogerakis, E., Maji, S.: CSGNET: neural shape parser for constructive solid geometry. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 5515–5523 (2018)Google Scholar
  18. 18.
    Shi, Y., Xu, K., Nießner, M., Rusinkiewicz, S., Funkhouser, T.: PlaneMatch: patch coplanarity prediction for robust RGB-D reconstruction. In: Ferrari, V., Hebert, M., Sminchisescu, C., Weiss, Y. (eds.) ECCV 2018. LNCS, vol. 11212, pp. 767–784. Springer, Cham (2018).  https://doi.org/10.1007/978-3-030-01237-3_46CrossRefGoogle Scholar
  19. 19.
    Silberman, N., Hoiem, D., Kohli, P., Fergus, R.: Indoor segmentation and support inference from RGBD images. In: Fitzgibbon, A., Lazebnik, S., Perona, P., Sato, Y., Schmid, C. (eds.) ECCV 2012. LNCS, vol. 7576, pp. 746–760. Springer, Heidelberg (2012).  https://doi.org/10.1007/978-3-642-33715-4_54CrossRefGoogle Scholar
  20. 20.
    Sinha, S., Steedly, D., Szeliski, R.: Piecewise planar stereo for image-based rendering (2009)Google Scholar
  21. 21.
    Sun, C., Hsiao, C.W., Sun, M., Chen, H.T.: HorizonNet: learning room layout with 1D representation and pano stretch data augmentation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1047–1056 (2019)Google Scholar
  22. 22.
    Xiao, J., Furukawa, Y.: Reconstructing the world’s museums. Int. J. Comput. Vis. 110(3), 243–258 (2014)CrossRefGoogle Scholar
  23. 23.
    Yang, F., Zhou, Z.: Recovering 3D planes from a single image via convolutional neural networks. In: Ferrari, V., Hebert, M., Sminchisescu, C., Weiss, Y. (eds.) ECCV 2018. LNCS, vol. 11214, pp. 87–103. Springer, Cham (2018).  https://doi.org/10.1007/978-3-030-01249-6_6CrossRefGoogle Scholar
  24. 24.
    Yu, Z., Zheng, J., Lian, D., Zhou, Z., Gao, S.: Single-image piece-wise planar 3D reconstruction via associative embedding. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1029–1037 (2019)Google Scholar
  25. 25.
    Zebedin, L., Bauer, J., Karner, K., Bischof, H.: Fusion of Feature- and Area-Based Information for Urban Buildings Modeling from Aerial Imagery. In: Forsyth, D., Torr, P., Zisserman, A. (eds.) ECCV 2008. LNCS, vol. 5305, pp. 873–886. Springer, Heidelberg (2008).  https://doi.org/10.1007/978-3-540-88693-8_64CrossRefGoogle Scholar
  26. 26.
    Zhang, F., Nauata, N., Furukawa, Y.: Conv-mpn: Convolutional message passing neural network for structured outdoor architecture reconstruction. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (2020)Google Scholar
  27. 27.
    Zheng, J., Zhang, J., Li, J., Tang, R., Gao, S., Zhou, Z.: Structured3D: a large photo-realistic dataset for structured 3d modeling. arXiv preprint arXiv:1908.00222 (2019)

Copyright information

© Springer Nature Switzerland AG 2020

Authors and Affiliations

  1. 1.Simon Fraser UniversityBurnabyCanada

Personalised recommendations