Abstract
Perceptron like large margin algorithms are introduced for the experiments with various margin selections. Compared to the previous perceptron reranking algorithms, the new algorithms use full pairwise samples and allow us to search for margins in a larger space. Our experimental results on the data set of [1] show that a perceptron like ordinal regression algorithm with uneven margins can achieve Recall/Precision of 89.5/90.0 on section 23 of Penn Treebank. Our result on margin selection can be employed in other large margin machine learning algorithms as well as in other NLP tasks.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Collins, M.: Discriminative reranking for natural language parsing. In: ICML 2000 (2000)
Rosenblatt, F.: The perceptron: A probabilistic model for information storage and organization in the brain. Psychological Review 65, 386–408 (1958)
Novikoff, A.B.J.: On convergence proofs on perceptrons. In: The Symposium on the Mathematical Theory of Automata, vol. 12 (1962)
Collins, M., Duffy, N.: New ranking algorithms for parsing and tagging: Kernels over discrete structures, and the voted perceptron. In: Proceedings of ACL 2002 (2002)
Ratnaparkhi, A.: A linear observed time statistical parser based on maximum entropy models. In: The Second Conference on Empirical Methods in Natural Language Processing (1997)
Charniak, E.: A maximum-entropy-inspired parser. In: Proceedings of NAACL 2000 (2000)
Schapire, R.E., Freund, Y., Bartlett, P., Lee, W.S.: Boosting the margin: a new explanation for the effectiveness of voting methods. In: Proc. 14th International Conference on Machine Learning, pp. 322–330 (1997)
Shen, L., Joshi, A.K.: An SVM based voting algorithm with application to parse reranking. In: Proc. of CoNLL 2003 (2003)
Crammer, K., Singer, Y.: PRanking with Ranking. In: NIPS 2001 (2001)
Herbrich, R., Graepel, T., Obermayer, K.: Large margin rank boundaries for ordinal regression. In: Advances in Large Margin Classifiers, pp. 115–132. MIT Press, Cambridge (2000)
Vapnik, V.N.: Statistical Learning Theory. John Wiley and Sons, Chichester (1998)
Zhang, T.: Large Margin Winnow Methods for Text Categorization. In: KDD 2000 Workshop on Text Mining (2000)
Krauth, W., Mezard, M.: Learning algorithms with optimal stability in neural networks. Journal of Physics A 20, 745–752 (1987)
Marcus, M.P., Santorini, B., Marcinkiewicz, M.A.: Building a large annotated corpus of English: The Penn Treebank. Computational Linguistics 19, 313–330 (1994)
Freund, Y., Schapire, R.E.: Large margin classification using the perceptron algorithm. Machine Learning 37, 277–296 (1999)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2005 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Shen, L., Joshi, A.K. (2005). Flexible Margin Selection for Reranking with Full Pairwise Samples. In: Su, KY., Tsujii, J., Lee, JH., Kwong, O.Y. (eds) Natural Language Processing – IJCNLP 2004. IJCNLP 2004. Lecture Notes in Computer Science(), vol 3248. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-30211-7_47
Download citation
DOI: https://doi.org/10.1007/978-3-540-30211-7_47
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-24475-2
Online ISBN: 978-3-540-30211-7
eBook Packages: Computer ScienceComputer Science (R0)