Optimizing Complex Loss Functions in Structured Prediction

  • Mani Ranjbar
  • Greg Mori
  • Yang Wang
Part of the Lecture Notes in Computer Science book series (LNCS, volume 6312)


In this paper we develop an algorithm for structured prediction that optimizes against complex performance measures, those which are a function of false positive and false negative counts. The approach can be directly applied to performance measures such as F β score (natural language processing), intersection over union (image segmentation), Precision/Recall at k (search engines) and ROC area (binary classifiers). We attack this optimization problem by approximating the loss function with a piecewise linear function and relaxing the obtained QP problem to a LP which we solve with an off-the-shelf LP solver. We present experiments on object class-specific segmentation and show significant improvement over baseline approaches that either use simple loss functions or simple compatibility functions on VOC 2009.


Loss Function Markov Random Field Piecewise Linear Approximation Compatibility Function Optimal Label 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


  1. 1.
    Hoiem, D., Efros, A.A., Hebert, M.: Closing the loop in scene interpretation. In: Proc. IEEE Comput. Soc. Conf. Comput. Vision and Pattern Recogn. (2008)Google Scholar
  2. 2.
    Blaschko, M.B., Lampert, C.H.: Learning to localize objects with structured output regression. In: Forsyth, D., Torr, P., Zisserman, A. (eds.) ECCV 2008, Part I. LNCS, vol. 5302, pp. 2–15. Springer, Heidelberg (2008)CrossRefGoogle Scholar
  3. 3.
    Desai, C., Ramanan, D., Fowlkes, C.: Discriminative models for multi-class object layout. In: ICCV (2009)Google Scholar
  4. 4.
    Malik, J., Belongie, S., Leung, T., Shi, J.: Contour and texture analysis for image segmentation. Int. Journal of Computer Vision 43, 7–27 (2001)zbMATHCrossRefGoogle Scholar
  5. 5.
    Boykov, Y., Veksler, O., Zabih, R.: Fast approximate energy minimization via graph cuts. IEEE Trans. PAMI 23, 1222–1239 (2001)Google Scholar
  6. 6.
    Szummer, M., Kohli, P., Hoiem, D.: Learning crfs using graph cuts. In: Forsyth, D., Torr, P., Zisserman, A. (eds.) ECCV 2008, Part II. LNCS, vol. 5303, pp. 582–595. Springer, Heidelberg (2008)CrossRefGoogle Scholar
  7. 7.
    Everingham, M., Van Gool, L., Williams, C.K.I., Winn, J., Zisserman, A.: The PASCAL Visual Object Classes Challenge (VOC2009) Results (2009),
  8. 8.
    Taskar, B., Chatalbashev, V., Koller, D., Guestrin, C.: Learning structured prediction models: a large margin approach. In: ICML 2005, pp. 896–903 (2005)Google Scholar
  9. 9.
    Tsochantaridis, I., Hofmann, T., Joachims, T., Altun, Y.: Support vector machine learning for interdependent and structured output spaces. In: ICML (2004)Google Scholar
  10. 10.
    Joachims, T.: A support vector method for multivariate performance measures. In: ICML 2005, pp. 377–384. ACM, New York (2005)CrossRefGoogle Scholar
  11. 11.
    Werner, T.: A linear programming approach to max-sum problem: A review. IEEE Trans. PAMI 29, 1165–1179 (2007)Google Scholar
  12. 12.
    Kohli, P., Torr, P.H.S.: Measuring uncertainty in graph cut solutions. Comput. Vis. Image Underst. 112, 30–38 (2008)CrossRefGoogle Scholar
  13. 13.
    Komodakis, N., Tziritas, G.: Approximate labeling via graph-cuts based on linear programming. IEEE Trans. PAMI 29 (2007)Google Scholar
  14. 14.
    Felzenszwalb, P.F., Huttenlocher, D.P.: Efficient graph-based image segmentation. Int. Journal of Computer Vision 59 (2004)Google Scholar
  15. 15.
    van de Sande, K.E.A., Gevers, T., Snoek, C.G.M.: Evaluating color descriptors for object and scene recognition. IEEE Trans. PAMI (2010)Google Scholar
  16. 16.
    Leibe, B., Leonardis, A., Schiele, B.: Combined object categorization and segmentation with an implicit shape model. In: ECCV Workshop on Statistical Learning in Computer Vision, pp. 17–32 (2004)Google Scholar
  17. 17.
    Felzenszwalb, P., Girshick, R., McAllester, D., Ramanan, D.: Object detection with discriminatively trained part based models. IEEE Trans. PAMI (2009)Google Scholar
  18. 18.
    Mosek: The mosek optimization software (2010),

Copyright information

© Springer-Verlag Berlin Heidelberg 2010

Authors and Affiliations

  • Mani Ranjbar
    • 1
  • Greg Mori
    • 1
  • Yang Wang
    • 1
  1. 1.School of Computing ScienceSimon Fraser UniversityCanada

Personalised recommendations