Abstract
This paper contains the description and the analysis of the dataset collected during the Conversational Intelligence Challenge (ConvAI) which took place in 2017. During the evaluation round we collected over 4,000 dialogues from 10 chatbots and 1,000 volunteers. Here we provide the dataset statistics and outline some possible improvements for future data collection experiments.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
Notes
- 1.
The dataset is available at http://convai.io/2017/data/
- 2.
- 3.
References
Burtsev, M., Logacheva, V., Malykh, V., Serban, I., Lowe, R., Prabhumoye, S., Black, A. W., Rudnicky, A., and Bengio, Y. (2018). The First Conversational Intelligence Challenge. NIPS 2017 Competition track Springer Proceedings.
Lowe, R., Noseworthy, M., Serban, I. V., Angelard-Gontier, N., Bengio, Y., and Pineau, J. (2017). Towards an Automatic Turing Test: Learning to Evaluate Dialogue Responses. Acl, pages 1–19.
Rajpurkar, P., Zhang, J., Lopyrev, K., and Liang, P. (2016). SQuAD: 100,000+ Questions for Machine Comprehension of Text. Emnlp, (ii):2383–2392.
Serban, I. V., Lowe, R., Charlin, L., and Pineau, J. (2015). A Survey of Available Corpora for Building Data-Driven Dialogue Systems. CoRR, page 46.
Serban, I. V., Sordoni, A., Lowe, R., Charlin, L., Pineau, J., Courville, A., and Bengio, Y. (2016). A Hierarchical Latent Variable Encoder-Decoder Model for Generating Dialogues. Proceedings of the Advances in Neural Information Processing Systems 29 (NIPS 2016), pages 1–14.
Yu, Z., Xu, Z., Black, A. W., and Rudnicky, A. I. (2016). Chatbot Evaluation and Database Expansion via Crowdsourcing. WOCHAT workshop.
Acknowledgements
The work was supported by National Technology Initiative and PAO Sberbank project ID 0000000007417F630002.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2018 Springer International Publishing AG, part of Springer Nature
About this paper
Cite this paper
Logacheva, V., Burtsev, M., Malykh, V., Polulyakh, V., Seliverstov, A. (2018). ConvAI Dataset of Topic-Oriented Human-to-Chatbot Dialogues. In: Escalera, S., Weimer, M. (eds) The NIPS '17 Competition: Building Intelligent Systems. The Springer Series on Challenges in Machine Learning. Springer, Cham. https://doi.org/10.1007/978-3-319-94042-7_3
Download citation
DOI: https://doi.org/10.1007/978-3-319-94042-7_3
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-94041-0
Online ISBN: 978-3-319-94042-7
eBook Packages: Computer ScienceComputer Science (R0)