Graphical Model Parameter Learning by Inverse Linear Programming

  • Vera TrajkovskaEmail author
  • Paul Swoboda
  • Freddie Åström
  • Stefania Petra
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10302)


We introduce two novel methods for learning parameters of graphical models for image labelling. The following two tasks underline both methods: (i) perturb model parameters based on given features and ground truth labelings, so as to exactly reproduce these labelings as optima of the local polytope relaxation of the labelling problem; (ii) train a predictor for the perturbed model parameters so that improved model parameters can be applied to the labelling of novel data. Our first method implements task (i) by inverse linear programming and task (ii) using a regressor e.g. a Gaussian process. Our second approach simultaneously solves tasks (i) and (ii) in a joint manner, while being restricted to linearly parameterised predictors. Experiments demonstrate the merits of both approaches.


Learn Approach Subgradient Method Linear Support Vector Machine Ground Truth Labelings Pairwise Potential 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


  1. 1.
    Kappes, J., Andres, B., Hamprecht, F., Schnörr, C., Nowozin, S., Batra, D., Kim, S., Kausler, B., Kröger, T., Lellmann, J., Komodakis, N., Savchynskyy, B., Rother, C.: A comparative study of modern inference techniques for structured discrete energy minimization problems. Int. J. Comput. Vis. 115(2), 155–184 (2015)MathSciNetCrossRefGoogle Scholar
  2. 2.
    Wainwright, M.J., Jordan, M.I.: Graphical Models, Exponential Families, and Variational Inference. Now Publishers Inc., Breda (2008)zbMATHGoogle Scholar
  3. 3.
    Werner, T.: A linear programming approach to max-sum problem: a review. IEEE Trans. Pattern Anal. Mach. Intell. 29(7), 1165–1179 (2007)CrossRefGoogle Scholar
  4. 4.
    Zhang, J., Liu, Z.: Calculating some inverse linear programming problems. J. Comput. Appl. Math. 72(2), 261–273 (1996)MathSciNetCrossRefzbMATHGoogle Scholar
  5. 5.
    Ahuja, R., Orlin, J.: Inverse optimization. Oper. Res. 49(5), 771–783 (2001)MathSciNetCrossRefzbMATHGoogle Scholar
  6. 6.
    Kuhn, H.W., Tucker, A.W.: Nonlinear programming. In: 2nd Berkeley Symposium on Mathematical Statistics and Probability, pp. 481–492 (1951)Google Scholar
  7. 7.
    Rasmussen, C., Williams, C.: Gaussian Processes for Machine Learning. MIT Press, Cambridge (2006)zbMATHGoogle Scholar
  8. 8.
    Kolmogorov, V., Rother, C.: Minimizing non-submodular functions with graph cuts - a review. IEEE Trans. Pattern Anal. Mach. Intell. 29(7), 1274–1279 (2007)CrossRefGoogle Scholar
  9. 9.
    Finley, T., Joachims, T.: Training structural SVMs when exact inference is intractable. In: Proceedings of the ICML, pp. 304–311 (2008)Google Scholar
  10. 10.
    Tsochantaridis, I., Hofmann, T., Joachims, T., Altun, Y.: Support Vector Machine Learning for Interdependent and Structured Output Spaces. In: ICML, pp. 104–111 (2004)Google Scholar
  11. 11.
    d’Antonio, G., Frangioni, A.: Convergence analysis of deflected conditional approximate subgradient methods. SIAM J. Optim. 20(1), 357–386 (2009)MathSciNetCrossRefzbMATHGoogle Scholar
  12. 12.
    Guta, B.: Subgradient optimization methods in integer programming with an application to a radiation therapy problem. Technical report, TU Kaiserslautern (2003)Google Scholar
  13. 13.
    Camerini, P., Fratta, L., Maffioli, F.: On improving relaxation methods by modified gradient techniques. In: Balinski, M.L., Wolfe, P. (eds.) Nondifferentiable Optimization. Mathematical Programming Studies, vol. 3, pp. 26–34. Springer, Heidelberg (1975)CrossRefGoogle Scholar
  14. 14.
    Polyak, B.: Minimization of unsmooth functionals. U.S.S.R. Comput. Math. Math. Phys. 9, 14–29 (1969)CrossRefzbMATHGoogle Scholar
  15. 15.
    Bertsekas, D.: Convex Optimization Algorithms. Athena Scientific, Belmont (2015)zbMATHGoogle Scholar
  16. 16.
    Borenstein, E., Sharon, E., Ullman, S.: Combining top-down and bottom-up segmentation. In: CVPRW, p. 46 (2004)Google Scholar
  17. 17.
    Domke, J.: Learning graphical model parameters with approximate marginal inference. IEEE Trans. Pattern Anal. Mach. Intell. 35(10), 2454–2467 (2013)CrossRefGoogle Scholar
  18. 18.
    Achanta, R., Shaji, A., Smith, K., Lucchi, A., Fua, P., Susstrunk, S.: SLIC superpixels compared to state-of-the-art superpixel methods. IEEE Trans. Pattern Anal. Mach. Intell. 34(11), 2274–2282 (2012)CrossRefGoogle Scholar
  19. 19.
    Tuzel, O., Porikli, F., Meer, P.: Region covariance: a fast descriptor for detection and classification. In: Leonardis, A., Bischof, H., Pinz, A. (eds.) ECCV 2006. LNCS, vol. 3952, pp. 589–600. Springer, Heidelberg (2006). doi: 10.1007/11744047_45 CrossRefGoogle Scholar
  20. 20.
    Pele, O., Werman, M.: The quadratic-chi histogram distance family. In: Daniilidis, K., Maragos, P., Paragios, N. (eds.) ECCV 2010. LNCS, vol. 6312, pp. 749–762. Springer, Heidelberg (2010). doi: 10.1007/978-3-642-15552-9_54 CrossRefGoogle Scholar
  21. 21.
    Bertelli, L., Yu, T., Vu, D., Gokturk, B.: Kernelized structural SVM learning for supervised object segmentation. In: CVPR, pp. 2153–2160 (2011)Google Scholar
  22. 22.
    Suykens, J.A.K., Vandewalle, J.: Least squares support vector machine classifiers. Neural Process. Lett. 9(3), 293–300 (1999)CrossRefzbMATHGoogle Scholar

Copyright information

© Springer International Publishing AG 2017

Authors and Affiliations

  • Vera Trajkovska
    • 1
    Email author
  • Paul Swoboda
    • 2
  • Freddie Åström
    • 1
  • Stefania Petra
    • 3
  1. 1.Image and Pattern Analysis GroupHeidelberg UniversityHeidelbergGermany
  2. 2.Institute of Science and TechnologyKlosterneuburgAustria
  3. 3.Mathematical Imaging GroupHeidelberg UniversityHeidelbergGermany

Personalised recommendations