Beyond Procedural Facade Parsing: Bidirectional Alignment via Linear Programming

  • Mateusz Koziński
  • Guillaume Obozinski
  • Renaud Marlet
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 9006)


We propose a novel formulation for parsing facade images with user-defined shape prior. Contrary to other state-of-the-art methods, we do not explore the procedural space of shapes derived from a grammar. Instead we formulate parsing as a linear binary program which we solve using Dual Decomposition. The algorithm produces plausible approximations of globally optimal segmentations without grammar sampling. It yields state-of-the-art performance on standard datasets.



This work was carried out in IMAGINE, a joint research project between Ecole des Ponts ParisTech (ENPC) and the Scientific and Technical Centre for Building (CSTB). It was partly supported by ANR project Semapolis ANR-13-CORD-0003.

Supplementary material (20 kb)
Supplementary material 1 (zip 20 KB)


  1. 1.
    Teboul, O., Simon, L., Koutsourakis, P., Paragios, N.: Segmentation of building facades using procedural shape priors. In: CVPR, pp. 3105–3112 (2010)Google Scholar
  2. 2.
    Teboul, O., Kokkinos, I., Simon, L., Koutsourakis, P., Paragios, N.: Shape grammar parsing via reinforcement learning. In: CVPR, pp. 2273–2280 (2011)Google Scholar
  3. 3.
    Riemenschneider, H., Krispel, U., Thaller, W., Donoser, M., Havemann, S., Fellner, D., Bischof, H.: Irregular lattices for complex shape grammar facade parsing. In: CVPR (2012)Google Scholar
  4. 4.
    Zhu, S.C., Mumford, D.: A stochastic grammar of images. Found. Trends Comput. Graph. Vis. 2, 259–362 (2006)CrossRefzbMATHGoogle Scholar
  5. 5.
    Stiny, G.N.: Pictorial and Formal Aspects of Shape and Shape Grammars and Aesthetic Systems. PhD thesis, University of California, Los Angeles (1975). AAI7526993Google Scholar
  6. 6.
    Ohta, Y., Kanade, T., Sakai, T.: An analysis system for scenes containing objects with substructures. In: Proceedings of the Fourth International Joint Conference on Pattern Recognitions, pp. 752–754 (1978)Google Scholar
  7. 7.
    Ohta, Y., Kanade, T., Sakai, T.: A production system for region analysis. In: Proceedings of the Sixth International Joint Conference on Artificial Intelligence, pp. 684–686 (1979)Google Scholar
  8. 8.
    Han, F., Zhu, S.C.: Bottom-up/top-down image parsing with attribute graph grammar. IEEE Trans. Pattern Anal. Mach. Intell. 31, 59–73 (2009)CrossRefGoogle Scholar
  9. 9.
    Wang, W., Pollak, I., Wong, T.S., Bouman, C.A., Harper, M.P.: Hierarchical stochastic image grammars for classification and segmentation. IEEE Trans. Image Process. 15, 3033–3052 (2006)CrossRefGoogle Scholar
  10. 10.
    Jin, Y., Geman, S.: Context and hierarchy in a probabilistic image model. In: CVPR (2), pp. 2145–2152 (2006)Google Scholar
  11. 11.
    Ahuja, N., Todorovic, S.: Connected segmentation tree - a joint representation of region layout and hierarchy. In: CVPR (2008)Google Scholar
  12. 12.
    Müller, P., Wonka, P., Haegler, S., Ulmer, A., Van Gool, L.: Procedural modeling of buildings. ACM Trans. Graph. 25, 614–623 (2006)CrossRefGoogle Scholar
  13. 13.
    Teboul, O.: Shape Grammar Parsing: Application to Image-based Modeling. PhD thesis, Ecole centrale Paris (2011)Google Scholar
  14. 14.
    Koziński, M., Marlet, R.: Image parsing with graph grammars and Markov random fields. In: WACV (2014)Google Scholar
  15. 15.
    Martinovic, A., Mathias, M., Weissenberg, J., Van Gool, L.: A three-layered approach to facade parsing. In: Fitzgibbon, A., Lazebnik, S., Perona, P., Sato, Y., Schmid, C. (eds.) ECCV 2012 Part VII. LNCS, vol. 7578, pp. 416–429. Springer, Heidelberg (2012)CrossRefGoogle Scholar
  16. 16.
    Cohen, A., Schwing, A., Pollefeys, M.: Efficient structured parsing of facades using dynamic programming. In: CVPR (2014)Google Scholar
  17. 17.
    Komodakis, N., Paragios, N., Tziritas, G.: MRF energy minimization and beyond via dual decomposition. IEEE Trans. PAMI 33, 531–552 (2011)CrossRefGoogle Scholar
  18. 18.
    Sontag, D., Globerson, A., Jaakkola, T.: Introduction to dual decomposition for inference. In: Sra, S., Nowozin, S., Wright, S.J. (eds.) Optimization for Machine Learning. MIT Press, Cambridge (2011)Google Scholar
  19. 19.
    Boyd, S., Parikh, N., Chu, E., Peleato, B., Eckstein, J.: Distributed optimization and statistical learning via the alternating direction method of multipliers. Found. Trends Mach. Learn. 3, 1–122 (2011)CrossRefGoogle Scholar
  20. 20.
    Ladický, L., Russell, C., Kohli, P., Torr, P.H.S.: Associative hierarchical random fields. IEEE Trans. Pattern Anal. Mach. Intell. 99, 1 (2013)Google Scholar
  21. 21.
    Shotton, J., Winn, J.M., Rother, C., Criminisi, A.: TextonBoost: joint appearance, shape and context modeling for multi-class object recognition and segmentation. In: Leonardis, A., Bischof, H., Pinz, A. (eds.) ECCV 2006, Part I. LNCS, vol. 3951, pp. 1–15. Springer, Heidelberg (2006)CrossRefGoogle Scholar
  22. 22.
    Komodakis, N.: Efficient training for pairwise or higher order CRFs via dual decomposition. In: CVPR, pp. 1841–1848 (2011)Google Scholar

Copyright information

© Springer International Publishing Switzerland 2015

Authors and Affiliations

  • Mateusz Koziński
    • 1
  • Guillaume Obozinski
    • 1
  • Renaud Marlet
    • 1
  1. 1.LIGM (UMR CNRS 8049)ENPC, Université Paris-EstMarne-la-ValléeFrance

Personalised recommendations