Skip to main content

Search Challenges in Natural Language Generation with Complex Optimization Objectives

KI - Künstliche Intelligenz volume 30, pages 63–69 (2016)Cite this article

Abstract

Automatic natural language generation (NLG) is a difficult problem already when merely trying to come up with natural-sounding utterances. Ubiquituous applications, in particular companion technologies, pose the additional challenge of flexible adaptation to a user or a situation. This requires optimizing complex objectives such as information density, in combinatorial search spaces described using declarative input languages. We believe that AI search and planning is a natural match for these problems, and could substantially contribute to solving them effectively. We illustrate this using a concrete example NLG framework, give a summary of the relevant optimization objectives, and provide an initial list of research challenges.

This is a preview of subscription content, access via your institution.

Access options

Buy single article

Instant access to the full article PDF.

USD 39.95

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Fig. 1

Notes

  1. Surprisal values based on [8] obtained from http://tinyurl.com/pltagdemo.

References

  1. Bonet B, Haslum P, Hickmott SL, Thiébaux S (2008) Directed unfolding of petri nets. Trans Petri Nets Mod Concurr 1:172–198

    Article  Google Scholar 

  2. Cahill A, van Genabith J. Robust pcfg-based generation using automatically acquired LFG approximations. In: Calzolari et al. [3]

  3. Calzolari N, Cardie C, Isabelle P (eds.) (2006) Proceedings of the 21st International Conference on Computational Linguistics (ACL’06). ACL

  4. Carroll JA, Oepen S (2005) High efficiency realization for a wide-coverage unification grammar. In: Natural language processing–IJCNLP, pp 165–176

  5. Crocker MW, Demberg V, Teich E (2015) Information density and linguistic encoding (ideal). KI - Künstliche intelligenz. doi:10.1007/s13218-015-0391-y

  6. Crundall D, Bains M, Chapman P, Underwood G (2005) Regulating conversation during driving: a problem for mobile telephones? Transp Res Part F Traffic Psychol Behav 8(3):197–211

    Article  Google Scholar 

  7. Demberg V, Keller F (2008) Data from eye-tracking corpora as evidence for theories of syntactic processing complexity. Cognition 109(2):193–210

    Article  Google Scholar 

  8. Demberg V, Keller F, Koller A (2013) Incremental, predictive parsing with psycholinguistically motivated tree-adjoining grammar. Comput Linguist 39(4):1025–1066

    Article  Google Scholar 

  9. Demberg V, Sayeed A (2011) Linguistic cognitive load: implications for automotive uis. In: Adjunct proceedings of the 3rd international conference on automotive user interfaces and interactive vehicular applications (AutomotiveUI 2011)

  10. Dethlefs N, Hastie H, Rieser V, Lemon O (2012) Optimising incremental dialogue decisions using information density for interactive systems. In: EMNLP-CoNLL’12 Proceedings of the 2012 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning. Association for Computational Linguistics, Stroudsburg, pp 82–93

  11. Drews FA, Pasupathi M, Strayer DL (2008) Passenger and cell phone conversations in simulated driving. J Exp Psychol Appl 14(4):392–400

    Article  Google Scholar 

  12. Edelkamp S (2001) Planning with pattern databases. In: Cesta A, Borrajo D (eds) Proceedings of the 6th European conference on planning (ECP’01), pp 13–24, Springer, Verlag

  13. Frank SL, Otten LJ, Galli G, Vigliocco G (2015) The erp response to the amount of information conveyed by words in sentences. Brain Lang 140:1–11

    Article  Google Scholar 

  14. Gibson E (1998) Linguistic complexity: locality of syntactic dependencies. Cognition 68(1):1–76

    Article  Google Scholar 

  15. Gildea D, Temperley D (2010) Do grammars minimize dependency length? Cognit Sci 34:286–310

    Article  Google Scholar 

  16. Hale J (2001) A probabilistic earley parser as a psycholinguistic model. In: Proceedings of NAACL. NAACL, Carnegie Mellon University, Pittsburgh, pp 159–166

  17. Haslum P, Geffner H (2000) Admissible heuristics for optimal planning. In: Chien S, Kambhampati R, Knoblock C (eds)Proceedings of the 5th international conference on artificial intelligence planning systems (AIPS-00), AAAI Press, Menlo Park, Breckenridge CO, pp 140–149

  18. Helmert M, Haslum P, Hoffmann J, Nissim R (2014) Merge and shrink abstraction: a method for generating lower bounds in factored state spaces. J Assoc Comput Mach 61(3):16:1–16:63. doi:10.1145/2559951

    Article  MathSciNet  Google Scholar 

  19. Hoffmann J, Kissmann P, Torralba Á (2014) “Distance”? Who cares? Tailoring merge-and-shrink heuristics to detect unsolvability. In: Schaub T (ed) Proceedings of the 21st European conference on artificial intelligence (ECAI’14). IOS Press, Prague, Czech Republic

  20. Jaeger TF (2006) Redundancy and syntactic reduction in spontaneous speech. Unpublished dissertation, Stanford University

  21. Jaeger TF (2010) Redundancy and reduction: speakers manage syntactic information density. Cogn Psychol 61(1):23–62

    Article  MathSciNet  Google Scholar 

  22. Kay M (1996) Chart generation. In: Joshi AK, Palmer M (eds.) Proceedings of the 34th annual meeting of the association for computational linguistics, pp 200–204. Morgan Kaufmann/ACL

  23. Keenan J, Kintsch W (1973) Reading rate and of propositions retention as a function of the number in the base structure of sentences. Cogn Psychol 5:257–274

    Article  Google Scholar 

  24. Keyder E, Hoffmann J, Haslum P (2014) Improving delete relaxation heuristics through explicitly represented conjunctions. J Artif Intell Res 50:487–533

    MathSciNet  MATH  Google Scholar 

  25. Kuhn L, Price B, de Kleer J, Do M, Zhou R (2008) Heuristic search for target-value path problem. In: Proceedings of the 1st international symposium on search techniques in artificial intelligence and robotics

  26. Levy R (2008) Expectation-based syntactic comprehension. Cognition 106(3):1126–1177

    Article  Google Scholar 

  27. Levy R, Jaeger TF (2007) Speakers optimize information density through syntactic reduction. In: Schölkopf B, Platt JC, Hoffman T (eds) Advances in neural information processing systems 19, Proceedings of the twentieth annual conference on neural information processing systems. MIT Press, Cambridge, pp 849–856. http://papers.nips.cc/paper/3129-speakers-optimizeinformation-density-through-syntactic-reduction

  28. Linares LC, Stern R, Felner A (2014) Solving the target-value search problem. In: Edelkamp S, Bartak R (eds) Proceedings of the 7th annual symposium on combinatorial search (SOCS’14). AAAI Press

  29. McMillan KL (1993) Using unfoldings to avoid the state explosion problem in the verification of asynchronous circuits. In: von Bochmann G, Probst DK (eds) Proceedings of the 4th international workshop on computer aided verification (CAV’93), Lecture Notes in Computer Science, pp 164–177, Springer

  30. Nakatsu C, White M Learning to say it well: reranking realizations by predicted synthesis quality. In: Calzolari et al. [3]

  31. Rajkumar R, White M (2010) Designing agreement features for realization ranking. In: Proceedings of the 23rd international conference on computational linguistics: posters, pp 1032–1040

  32. Rajkumar R, White M (2011) Linguistically motivated complementizer choice in surface realization. In: Proceedings of the UCNLG+Eval: language generation and evaluation workshop. Association for Computational Linguistics, Edinburgh, pp 39–44. http://www.aclweb.org/anthology/W11-2706

  33. Rajkumar R, White M (2014) Better surface realization through psycholinguistics. Lang Linguist Compass 8(10):428–448

    Article  Google Scholar 

  34. Shannon CE (1948) A mathematical theory of communication. Bell Syst Tech J 27(3):379–423

    Article  MathSciNet  MATH  Google Scholar 

  35. Valmari A (1989) Stubborn sets for reduced state space generation. In: Proceedings of the 10th international conference on applications and theory of petri nets, pp 491–515

  36. Wehrle M, Helmert M (2014) Efficient stubborn sets: Generalized algorithms and selection strategies. In: Chien S, Do M, Fern A, Ruml W (eds) Proceedings of the 24th international conference on automated planning and scheduling (ICAPS’14). AAAI Press

  37. White M (2004) Reining in CCG chart realization. In: Belz A, Evans R, Piwek P (eds) Proceedings of the 3rd international conference atural language generation, lecture notes in computer science, vol 3123, pp 182–191, Springer

  38. White M (2006) Efficient realization of coordinate structures in combinatory categorial grammar. Res Lang Comput 4(1):39–75

    Article  MathSciNet  Google Scholar 

  39. White M, Rajkumar R (2009) Perceptron reranking for CCG realization. In: Proceedings of the 2009 conference on empirical methods in natural language processing vol 1, pp 410–419

  40. White M, Rajkumar R (2012) Minimal dependency length in realization ranking. In: Proceedings of the 2012 joint conference on empirical methods in natural language processing and computational natural language learning, pp 244–255

Download references

Acknowledgments

This work was partially supported by the DFG excellence cluster EXC 284 “Multimodal Computing and Interaction”, the DFG collaborative research center SFB 1102 “Information Density and Linguistic Encoding”, as well as the EU FP7 Programme under Grant Areement No. 295261 (MEALS). We thank Maximilian Schwenger for discussions. We are also grateful to Almaz for great Eritrean food.

Author information

Authors and Affiliations

  1. Saarland University, Saarbrücken, Germany

    Vera Demberg, Jörg Hoffmann, David M. Howcroft, Dietrich Klakow & Álvaro Torralba

Authors
  1. Vera Demberg
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jörg Hoffmann
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. David M. Howcroft
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Dietrich Klakow
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Álvaro Torralba
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Jörg Hoffmann.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Demberg, V., Hoffmann, J., Howcroft, D.M. et al. Search Challenges in Natural Language Generation with Complex Optimization Objectives. Künstl Intell 30, 63–69 (2016). https://doi.org/10.1007/s13218-015-0409-5

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s13218-015-0409-5

Keywords

  • Natural language processing
  • Search
  • Planning