Abstract
Markov network (MN) structured output classifiers provide a transparent and powerful way to model dependencies between output labels. The MN classifiers can be learned using the M3N algorithm, which, however, is not statistically consistent and requires expensive fully annotated examples. We propose an algorithm to learn MN classifiers that is based on Fisher-consistent adversarial loss minimization. Learning is transformed into a tractable convex optimization that is amenable to standard gradient methods. We also extend the algorithm to learn from examples with missing labels. We show that the extended algorithm remains convex, tractable, and statistically consistent.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
We refer to the expectation of a loss \(\textrm{LOSS}\) as the \(\textrm{LOSS}\)-risk.
- 2.
To emphasize that \({\psi ^p_\textrm{lp}}\) is applicable only for the linear MN classifier, we use the notation \({\psi ^p_\textrm{lp}}(x,\boldsymbol{\theta },\boldsymbol{a})\) instead of \({\psi ^p_\textrm{lp}}(\boldsymbol{f},\boldsymbol{a})\) with \(\boldsymbol{f}(x)=\boldsymbol{\varPhi }(x)^T\boldsymbol{\theta }\).
References
Antoniuk, K., Franc, V., Hlaváč, V.: Consistency of structured output learning with missing labels. In: Asian Conference on Machine Learning (ACML) (2015)
Fathony, R., Liu, A., Asif, K., Ziebart, B.: Adversarial multiclass classification: A risk minimization perspective. In: NIPS (2016)
Fathony, R., et al.: Consistent robust adversarial prediction for general multiclass classification (2018). https://arxiv.org/abs/1812.07526
Franc, V., Laskov, P.: Learning maximal margin Markov networks via tractable convex optimization. Control Syst. Comput. 2, 25–34 (2011)
Franc, V., Savchynskyy, B.: Discriminative learning of max-sum classifiers. J. Mach. Learn. Res. 9(1), 67–104 (2008)
Franc, V., Yermakov, A.: Learning maximum margin Markov networks from examples with missing labels. In: Asian Conference on Machine Learning (2021)
Kingma, D.P., Ba, J.: Adam: a method for stochastic optimization. In: Proceedings of International Conference on Learning Representations (ICLR) (2015)
LeCun, Y., Cortes, C.: MNIST handwritten digit database (2010). http://yann.lecun.com/exdb/mnist/
Lin, Y.: A note on margin-based loss functions in classification. Stat. Probab. Lett. 68(1), 73–82 (2004)
Liu, Y.: Fisher consistency of multicategory support vector machines. In: International Conference on Artificial Intelligence and Statistics, pp. 291–298 (2007)
Lou, X., Hamprecht, F.A.: Structured learning from partial annotations. In: International Conference on Machine Learning (ICML), pp. 1519–1526 (2012)
Nowak, A., Bach, F., Rudi, A.: Consistent structured prediction with max-min margin Markov networks. In: International Conference on Machine Learning (2020)
Schlesinger, M.: Syntactic analysis of two-dimensional visual signals in noisy conditions. Kibernetika 4, 113–130 (1976). in Russian
Taskar, B., Chatalbashev, V., Koller, D.: Learning associative Markov networks. In: International Conference on Machine Learning (ICML) (2004)
Taskar, B., Guestrin, C., Koller, D.: Maximum-margin markov networks. In: Proceedings of Neural Information Processing Systems (NIPS) (2004)
Taskar, B., Guestrin, C., Koller, D.: Max-margin Markov networks. In: NIPS (2003)
Tsochantaridis, I., Joachims, T., Hofmann, T., Altun, Y.: Large margin methods for structured and interdependent output variables. J. Mach. Learn. Res. 6(50), 1453–1484 (2005)
Wang, P., Donti, P., Wilder, B., Kolter, J.: SATnet: bridging deep learning and logical reasoning using a differential satisfiability solver. In: ICML (2019)
Werner, T.: A linear programming approach to max-sum problem: a review. IEEE Trans. Pattern Anal. Mach. Intell. 29(7), 1165–1179 (2007)
Yhang, T.: Statistical analysis of some multi-category large margin classification methods. J. Mach. Learn. Res. 5 (2004)
Acknowledgments
The research was supported by the Czech Science Foundation project GACR GA19-21198S and OP VVV project CZ.02.1.01\(\backslash \)0.0\(\backslash \)0.0\(\backslash \)16 019\(\backslash \)0000765 Research Center for Informatics.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
1 Electronic supplementary material
Below is the link to the electronic supplementary material.
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Franc, V., Prusa, D., Yermakov, A. (2023). Consistent and Tractable Algorithm for Markov Network Learning. In: Amini, MR., Canu, S., Fischer, A., Guns, T., Kralj Novak, P., Tsoumakas, G. (eds) Machine Learning and Knowledge Discovery in Databases. ECML PKDD 2022. Lecture Notes in Computer Science(), vol 13716. Springer, Cham. https://doi.org/10.1007/978-3-031-26412-2_27
Download citation
DOI: https://doi.org/10.1007/978-3-031-26412-2_27
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-26411-5
Online ISBN: 978-3-031-26412-2
eBook Packages: Computer ScienceComputer Science (R0)
