Abstract
Sequence-to-Sequence (Seq2Seq) learning has immense interest in recent years. The prosperous approach of end-to-end training fashion using encoder-decoder neural networks like machine translation has sprouted active research in transduction tasks such as abstractive summarization or especially Paraphrase Generation (PG). Dealing with paraphrase generation problem, one of the most intrinsic obstruction of existing solutions do not pay enough attention to the fact that words and sentences in particular context own differential importance. Consequently, the loss of crucial information probably occurs and irrelevant paraphrasing components are generated. To overcome these barriers, an emerging Hierarchical Conditional Attention-based Neural Networks (HCANN) architecture to construct end-to-end text generation framework is proposed. More specifically, included method in that represents hierarchy of document along with conditional decoder for paraphrase generation processes. Quantitative evaluation of the method on several benchmark paraphrase datasets demonstrates its efficiency and performance capability by a significant margin.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Bahdanau, D., Cho, K., Bengio, Y.: Neural machine translation by jointly learning to align and translate. arXiv preprint arXiv:1409.0473 (2014)
Bannard, C., Callison-Burch, C.: Paraphrasing with bilingual parallel corpora. In: Proceedings of the 43rd Annual Meeting on Association for Computational Linguistics, pp. 597–604. Association for Computational Linguistics (2005)
Cho, K., Van Merriënboer, B., Bahdanau, D., Bengio, Y.: On the properties of neural machine translation: Encoder-decoder approaches. arXiv preprint arXiv:1409.1259 (2014)
Chorowski, J.K., Bahdanau, D., Serdyuk, D., Cho, K., Bengio, Y.: Attention-based models for speech recognition. In: Advances in Neural Information Processing Systems, pp. 577–585 (2015)
Fader, A., Zettlemoyer, L., Etzioni, O.: Paraphrase-driven learning for open question answering. In: Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), vol. 1, pp. 1608–1618 (2013)
Goodfellow, I.J., Warde-Farley, D., Mirza, M., Courville, A., Bengio, Y.: Maxout networks. arXiv preprint arXiv:1302.4389 (2013)
Graves, A., Wayne, G., Danihelka, I.: Neural turing machines. arXiv preprint arXiv:1410.5401 (2014)
Gupta, A., Agarwal, A., Singh, P., Rai, P.: A deep generative framework for paraphrase generation. arXiv preprint arXiv:1709.05074 (2017)
Hasan, S.A., et al.: Neural clinical paraphrase generation with attention. In: ClinicalNLP 2016, p. 42 (2016)
Huang, G., Liu, Z., Weinberger, K.Q., van der Maaten, L.: Densely connected convolutional networks. arXiv preprint arXiv:1608.06993 (2016)
Kohavi, R.: A Study of Cross-validation and Bootstrap for Accuracy Estimation and Model Selection, pp. 1137–1143. Morgan Kaufmann (1995)
Kolesnyk, V., Rocktäschel, T., Riedel, S.: Generating natural language inference chains. arXiv preprint arXiv:1606.01404 (2016)
Kozlowski, R., McCoy, K.F., Vijay-Shanker, K.: Generation of single-sentence paraphrases from predicate/argument structure using lexico-grammatical resources. In: Proceedings of the Second International Workshop on Paraphrasing, vol. 16, pp. 1–8. Association for Computational Linguistics (2003)
Lavie, A., Agarwal, A.: Meteor: An automatic metric for MT evaluation with high levels of correlation with human judgments. In: Proceedings of the Second Workshop on Statistical Machine Translation, pp. 228–231. Association for Computational Linguistics (2007)
Li, X., Wu, X.: Constructing long short-term memory based deep recurrent neural networks for large vocabulary speech recognition. In: IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) 2015, pp. 4520–4524. IEEE (2015)
Li, Z., Jiang, X., Shang, L., Li, H.: Paraphrase generation with deep reinforcement learning. arXiv preprint arXiv:1711.00279 (2017)
Liu, C., Dahlmeier, D., Ng, H.T.: PEM: a paraphrase evaluation metric exploiting parallel texts. In: Proceedings of the 2010 Conference on Empirical Methods in Natural Language Processing, pp. 923–932. Association for Computational Linguistics (2010)
Liu, P., Qiu, X., Chen, X., Wu, S., Huang, X.: Multi-timescale long short-term memory neural network for modelling sentences and documents. In: EMNLP, pp. 2326–2335 (2015)
Madnani, N., Dorr, B.J.: Generating phrasal and sentential paraphrases: a survey of data-driven methods. Comput. Linguist. 36(3), 341–387 (2010)
McKeown, K.R.: Paraphrasing questions using given and new information. Comput. Linguist. 9(1), 1–10 (1983)
Manning, C.D., Luong, M.-T., Pham, H.: Effective approaches to attention-based neural machine translation. CoRR abs/1508.0402 (2017)
Nguyen-Ngoc, K., Le, A.-C., Nguyen, V.-H.: An attention-based long-short-term-memory model for paraphrase generation. In: Huynh, V.-N., Inuiguchi, M., Tran, D.H., Denoeux, T. (eds.) IUKM 2018. LNCS (LNAI), vol. 10758, pp. 166–178. Springer, Cham (2018). https://doi.org/10.1007/978-3-319-75429-1_14
Papineni, K., Roukos, S., Ward, T., Zhu, W.-J.: BLEU: a method for automatic evaluation of machine translation. In: Proceedings of the 40th Annual Meeting on Association for Computational Linguistics, pp. 311–318. Association for Computational Linguistics (2002)
Parikh, D.M.: Handbook of Pharmaceutical Granulation Technology. CRC Press, Boca Raton (2016)
Pavlick, E., Rastogi, P., Ganitkevitch, J., Van Durme, B., Callison-Burch, C.: PPDB 2.0: Better paraphrase ranking, fine-grained entailment relations, word embeddings, and style classification (2015)
Prakash, A., et al.: Neural paraphrase generation with stacked residual LSTM networks. arXiv preprint arXiv:1610.03098 (2016)
Rus, V., Lintean, M.: A comparison of greedy and optimal assessment of natural language student input using word-to-word similarity metrics. In: Proceedings of the Seventh Workshop on Building Educational Applications Using NLP, pp. 157–162. Association for Computational Linguistics (2012)
See, A., Liu, P.J., Manning, C.D.: Get to the point: Summarization with pointer-generator networks. arXiv preprint arXiv:1704.04368 (2017)
Sennrich, R., et al.: Nematus: a toolkit for neural machine translation. arXiv preprint arXiv:1703.04357 (2017)
Serban, I.V., et al.: Multiresolution recurrent neural networks: An application to dialogue response generation. arXiv preprint arXiv:1606.00776 (2016)
Snover, M., Dorr, B., Schwartz, R., Micciulla, L., Makhoul, J.: A study of translation edit rate with targeted human annotation. In: Proceedings of Association for Machine Translation in the Americas, vol. 200 (2006)
Socher, R., Huang, E.H., Pennington, J., Ng, A.Y., Manning, C.D.: Dynamic pooling and unfolding recursive autoencoders for paraphrase detection. In: NIPS, vol. 24, pp. 801–809 (2011)
Sutskever, I., Vinyals, O., Le, Q.V.: Sequence to sequence learning with neural networks. In: Advances in Neural Information Processing Systems, pp. 3104–3112 (2014)
Vinyals, O., Kaiser, Ł., Koo, T., Petrov, S., Sutskever, I., Hinton, G.: Grammar as a foreign language. In: Advances in Neural Information Processing Systems, pp. 2773–2781 (2015)
Wieting, J., Bansal, M., Gimpel, K., Livescu, K., Roth, D.: From paraphrase database to compositional paraphrase model and back. arXiv preprint arXiv:1506.03487 (2015)
Wu, Y., et al.: Google’s neural machine translation system: Bridging the gap between human and machine translation. arXiv preprint arXiv:1609.08144 (2016)
Wubben, S., Van Den Bosch, A., Krahmer, E.: Paraphrase generation as monolingual translation: data and evaluation. In: Proceedings of the 6th International Natural Language Generation Conference, pp. 203–207. Association for Computational Linguistics (2010)
Xu, K., et al.: Show, attend and tell: neural image caption generation with visual attention. In: International Conference on Machine Learning, pp. 2048–2057 (2015)
Zhao, S., Lan, X., Liu, T., Li, S.: Application-driven statistical paraphrase generation. In: Proceedings of the Joint Conference of the 47th Annual Meeting of the ACL and the 4th International Joint Conference on Natural Language Processing of the AFNLP: Volume 2-Volume 2, pp. 834–842. Association for Computational Linguistics (2009)
Zhao, S., Niu, C., Zhou, M., Liu, T., Li, S.: Combining multiple resources to improve SMT-based paraphrasing model. In: ACL, pp. 1021–1029 (2008)
Zhao, S., Wang, H., Lan, X., Liu, T.: Leveraging multiple MT engines for paraphrase generation. In: Proceedings of the 23rd International Conference on Computational Linguistics, pp. 1326–1334. Association for Computational Linguistics (2010)
Acknowledgment
This paper is supported by The Vietnam National University, Hanoi (VNU), under Project No. QG.17.69 and The Vietnam National Foundation for Science and Technology Development (NAFOSTED) under grant number 102.01-2014.22.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2018 Springer Nature Switzerland AG
About this paper
Cite this paper
Nguyen-Ngoc, K., Le, AC., Nguyen, VH. (2018). A Hierarchical Conditional Attention-Based Neural Networks for Paraphrase Generation. In: Kaenampornpan, M., Malaka, R., Nguyen, D., Schwind, N. (eds) Multi-disciplinary Trends in Artificial Intelligence. MIWAI 2018. Lecture Notes in Computer Science(), vol 11248. Springer, Cham. https://doi.org/10.1007/978-3-030-03014-8_14
Download citation
DOI: https://doi.org/10.1007/978-3-030-03014-8_14
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-03013-1
Online ISBN: 978-3-030-03014-8
eBook Packages: Computer ScienceComputer Science (R0)