Skip to main content
SpringerLink
Log in

DuReSE: Rewriting Incomplete Utterances via Neural Sequence Editing

  • Published:
Neural Processing Letters Aims and scope Submit manuscript

Abstract

Utterance rewriting condenses a sparse, multi-turn context into a single, self-contained utterance. It has demonstrated superb effectiveness in response generation. Prior techniques mainly rely on machine translation technology to translate the last (incomplete) utterance in the context into a complete utterance. Such a rewriting paradigm contrasts with the main characteristic of the task, that is, the source and target utterances are mostly the same with only a small portion of local edits. Therefore, they rely heavily on large amounts of data to fit the translation model. This paper proposes DuReSE (dialogue utterance rewritten via sequence editing), a neural utterance editor designed for utterance rewriting. DuReSE shapes utterance rewriting as a sentence editing task and then predicts a small set of edit operations for each word in the dialogue context. The model edits the incomplete utterance in two phases where (1) an in-place editor performs word-level editing, and (2) a post-editor then refines the rewritten utterance. Experiments on three benchmark data sets have suggested that DuReSE outperforms baseline models in terms of EM, BLEU, and ROUGE. Specifically, DuReSE outperforms the baselines by up to 14.1% in EM, indicating that it can retrieve more informative and important information, compared with existing techniques.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3

Similar content being viewed by others

Notes

  1. https://github.com/google-research/bert.

  2. https://www.douban.com/group.

References

  1. Awasthi A, Sarawagi S, Goyal R et al (2019) Parallel iterative edit models for local sequence transduction. In: EMNLP, pp 4251–4261

  2. Bahdanau D, Cho K, Bengio Y (2015) Neural machine translation by jointly learning to align and translate. In: ICLR

  3. Cao Z, Li W, Li S et al (2018) Retrieve, rerank and rewrite: Soft template based neural summarization. In: ACL, pp 152–161

  4. Choi E, He H, Iyyer M et al (2018) Quac: Question answering in context. In: EMNLP, pp 2174–2184

  5. de Barcelos Silva A, Gomes MM, da Costa CA et al (2020) Intelligent personal assistants: a systematic literature review. Expert Syst Appl 147(113):193

    Google Scholar 

  6. Devlin J, Chang MW, Lee K et al (2019) Bert: pre-training of deep bidirectional transformers for language understanding. In: NAACL, pp 4171–4186

  7. Elgohary A, Peskov D, Boyd-Graber J (2019) Can you unpack that? Learning to rewrite questions-in-context. In: EMNLP, pp 5920–5926

  8. Eric M, Krishnan L, Charette F et al (2017) Key-value retrieval networks for task-oriented dialogue. In: SIGDIAL, pp 37–49

  9. Gu J, Lu Z, Li H et al (2016) Incorporating copying mechanism in sequence-to-sequence learning. In: ACL, pp 1631–1640

  10. Joshi M, Levy O, Zettlemoyer L et al (2019) Bert for coreference resolution: baselines and analysis. In: EMNLP, pp 5807–5812

  11. Joshi M, Chen D, Liu Y et al (2020) Spanbert: improving pre-training by representing and predicting spans. Trans Assoc Comput Linguist 8:64–77

    Article  Google Scholar 

  12. Kingma DP, Ba J (2015) Adam: a method for stochastic optimization. In: ICLR

  13. Kumar V, Joshi S (2016) Non-sentential question resolution using sequence to sequence learning. In: COLING, pp 2022–2031

  14. Lee K, He L, Lewis M et al (2017) End-to-end neural coreference resolution. In: EMNLP, pp 188–197

  15. Li J, Jia R, He H et al (2018) Delete, retrieve, generate: a simple approach to sentiment and style transfer. In: ACL, pp 1865–1874

  16. Lin CY (2004) ROUGE: A package for automatic evaluation of summaries. In: Text Summarization Branches Out. ACL, pp 74–81

  17. Liu Q, Chen B, Lou JG et al (2020) Incomplete utterance rewriting as semantic segmentation. In: EMNLP, pp 2846–2857

  18. Liu Y, Ott M, Goyal N et al (2019) Roberta: A robustly optimized bert pretraining approach. arXiv preprint arXiv:1907.11692

  19. Lu Y, Lin H, Tang J et al (2022) End-to-end neural event coreference resolution. Artif Intell 303(103):632

    MATH  Google Scholar 

  20. Malmi E, Krause S, Rothe S et al (2019) Encode, tag, realize: High-precision text editing. In: EMNLP, pp 5057–5068

  21. Ortega D, Vu NT (2017) Neural-based context representation learning for dialog act classification. arXiv preprint arXiv:1708.02561

  22. Pan Z, Bai K, Wang Y et al (2019) Improving open-domain dialogue systems via multi-turn incomplete utterance restoration. In: EMNLP, pp 1824–1833

  23. Panthaplackel S, Allamanis M, Brockschmidt M (2021) Copy that! editing sequences by copying spans. In: The AAAI conference on artificial intelligence, pp 13,622–13,630

  24. Papineni K, Roukos S, Ward T et al (2002) Bleu: a method for automatic evaluation of machine translation. In: ACL, pp 311-318

  25. Pragst L, Rach N, Minker W et al (2018) On the vector representation of utterances in dialogue context. In: LREC

  26. Rastogi P, Gupta A, Chen T et al (2019) Scaling multi-domain dialogue state tracking via query reformulation. In: ACL, pp 97–105

  27. Regan M, Rastogi P, Gupta A et al (2019) A dataset for resolving referring expressions in spoken dialogue via contextual query rewrites (CQR). arXiv preprint arXiv:1903.11783

  28. Ren G, Ni X, Malik M et al (2018) Conversational query understanding using sequence to sequence modeling. In: WWW, pp 1715–1724

  29. Russo C, Madani K, Rinaldi AM (2021) Knowledge acquisition and design using semantics and perception: a case study for autonomous robots. Neural Process Lett 53(5):3153–3168

    Article  Google Scholar 

  30. See A, Liu PJ, Manning CD (2017) Get to the point: summarization with pointer-generator networks. In: ACL, pp 1073–1083

  31. Song L, Xu K, Zhang Y, et al (2020) Zpr2: Joint zero pronoun recovery and resolution using multi-task learning and bert. In: ACL, pp 5429–5434

  32. Su H, Shen X, Zhang R et al (2019) Improving multi-turn dialogue modelling with utterance rewriter. In: ACL, pp 22–31

  33. Vaswani A, Shazeer N, Parmar N et al (2017) Attention is all you need. NIPS 30:5998–6008

    Google Scholar 

  34. Vinyals O, Fortunato M, Jaitly N (2015) Pointer networks. In: NIPS, pp 2692–2700

  35. Wiseman S, Shieber SM, Rush AM (2018) Learning neural templates for text generation. In: EMNLP, pp 3174–3187

  36. Wu W, Wang F, Yuan A et al (2020) CorefQA: coreference resolution as query-based span prediction. In: ACL, pp 6953–6963

  37. Zaidan AA, Zaidan BB (2020) A review on intelligent process for smart home applications based on IoT: coherent taxonomy, motivation, open challenges, and recommendations. Artif Intell Rev 53(1):141–165

    Article  Google Scholar 

  38. Zheng Q, Zhao P, Li Y et al (2021) Spectrum interference-based two-level data augmentation method in deep learning for automatic modulation classification. Neural Comput Appl 33(13):7723–7745

    Article  Google Scholar 

  39. Zheng Q, Zhao P, Zhang D et al (2021) MR-DCAE: Manifold regularization-based deep convolutional autoencoder for unauthorized broadcasting identification. Int J Intell Syst 36(12):7204–7238

    Article  Google Scholar 

Download references

Acknowledgements

This research is supported by National Natural Science Foundation of China (Grant No. 62102244, 62032004 and 62272296) and CCF-Tencent Open Research Fund (RAGR20220129).

Author information

Author notes

  1. Xiaodong Gu and Yuting Chen contributed to this work.

Authors and Affiliations

  1. School of Electronic Information and Electrical Engineering, Shanghai Jiao Tong University, Shanghai, 200240, China

    Wenhui Jiang, Xiaodong Gu, Yuting Chen & Beijun Shen

Authors
  1. Wenhui Jiang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Xiaodong Gu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Yuting Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Beijun Shen
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Beijun Shen.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Jiang, W., Gu, X., Chen, Y. et al. DuReSE: Rewriting Incomplete Utterances via Neural Sequence Editing. Neural Process Lett 55, 8713–8730 (2023). https://doi.org/10.1007/s11063-023-11174-8

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11063-023-11174-8

Keywords

Search

Navigation