Abstract
Service providers — from public institutions to primary care facilities — need to constantly attend to clients’ inquiries to provide useful information and directive guidelines. Ensuring high quality service is challenging as it not only demands detailed domain-specific knowledge, but also the ability to quickly understand the clients’ issues through their diverse — and often casual — descriptions. This paper aims to provide a framework for the development of an automated information broker agent who performs the task of a helper. The main task of the agent is to interact with the client and direct them to obtain further services that cater their personalized need. To do so, the agent should accomplish a sequence of tasks that include natural language inquiry, knowledge gathering, reasoning, and giving feedback; in this way, it simulates a human helper to engage in interaction with the client. The framework combines a question-answering reasoning mechanism while utilizing domain-specific knowledge base. When the users cannot describe clearly their needs, the system tries to narrow down the possibilities by an iterative question-answering process, until it eventually identifies the target. In realizing our framework, we make a proof-of-concept project, Mandy, a primary care chatbot system created to assist healthcare staffs by automating the patient intake process. We describe in detail the system functionalities and design of the system, and evaluate our proof-of-concept on benchmark case studies.
Article PDF
Similar content being viewed by others
References
Ahmad, F., Hogg-Johnson, S., Stewart, D. E., Skinner, H. A., Glazier, R. H., and Levinson, W. (2009). Computer-assisted screening for intimate partner violence and controla randomized trial. Annals of internal medicine, 151(2): 93–102.
Araz, O. M. (2013). Integrating complex system dynamics of pandemic influenza with a multi-criteria decision making model for evaluating public health strategies. Journal of Systems Science and Systems Engineering, 22(3): 319–339.
Bakker, M. and Tsui, K.-L. (2017). Dynamic resource allocation for efficient patient scheduling: A data-driven approach. Journal of Systems Science and Systems Engineering, 26(4): 448–462.
Bendig, A. W. (1953). Twenty questions: an information analysis. Journal of Experimental Psychology, 46(5): 345.
Bengio, Y., Ducharme, R., Vincent, P., and Jauvin, C. (2003). A neural probabilistic language model. Journal of machine learning research, 3(Feb): 1137–1155.
Bernstein, S. L., Aronsky, D., Duseja, R., Epstein, S., Handel, D., Hwang, U., McCarthy, M., John McConnell, K., Pines, J. M., Rathlev, N., et al. (2009). The effect of emergency department crowding on clinically oriented outcomes. Academic Emergency Medicine, 16(1): 1–10.
Bickmore, T. and Giorgino, T. (2006). Health dialog systems for patients and consumers. Journal of biomedical informatics, 39(5): 556–571.
Bickmore, T. W., Pfeifer, L. M., Byron, D., Forsythe, S., Henault, L. E., Jack, B. W., Silliman, R., and Paasche-Orlow, M. K. (2010). Usability of conversational agents by patients with inadequate health literacy: evidence from two clinical trials. Journal of health communication, 15(S2): 197–210.
Black, L.-A., McTear, M., Black, N., Harper, R., and Lemon, M. (2005). Appraisal of a conversational artefact and its utility in remote patient monitoring. In Computer-Based Medical Systems, 2005. Proceedings. 18th IEEE Symposium on, pages 506–508. IEEE.
Bobrow, D. G. (1964). A question-answering system for high school algebra word problems. pages 591–614. ACM.
Bodenreider, O. (2004). The unified medical language system (umls): integrating biomedical terminology. Nucleic acids research, 32(suppl 1): D267–D270.
Caley, M. and Sidhu, K. (2011). Estimating the future healthcare costs of an aging population in the uk: expansion of morbidity and the need for preventative care. Journal of Public Health, 33(1): 117–122.
Chen, P., Cheung, Y., Lee, V. C., and Hart, A. (2016). Knowledge sharing via informal communities in a globally distributed organization. In International Symposium on Knowledge and Systems Sciences, pages 30–43. Springer.
Clancey, W. J. and Letsinger, R. (1982). NEOMYCIN: Reconfiguring a rule-based expert system for application to teaching. Department of Computer Science, Stanford University.
Dang, T.-T. and Ho, T.-B. (2017). Sequence-based measure for assessing drug-side effect causal relation from electronic medical records. In International Symposium on Knowledge and Systems Sciences, pages 53–65. Springer.
De Vine, L., Zuccon, G., Koopman, B., Sitbon, L., and Bruza, P. (2014). Medical semantic similarity with a neural language model. In Proceedings of the 23rd ACM international conference on conference on information and knowledge management, pages 1819–1822. ACM.
Delichatsios, H. K., Friedman, R. H., Glanz, K., Tennstedt, S., Smigelski, C., Pinto, B. M., Kelley, H., and Gillman, M. W. (2001). Randomized trial of a “talking computer” to improve adults’ eating habits. American Journal of Health Promotion, 15(4): 215–224.
Di Somma, S., Paladino, L., Vaughan, L., Lalle, I., Magrini, L., and Magnanti, M. (2015). Overcrowding in emergency department: an international issue. Internal and emergency medicine, 10(2): 171–175.
Dobbie, G. and Ross, K. (2017). Precision driven health: A new zealand research partnership. International Journal of Integrated Care, 17(3).
Edgren, L. (2006). Health consumer diversity and its implications. Journal of Systems Science and Systems Engineering, 15(1): 34–47.
Farzanfar, R., Frishkopf, S., Migneault, J., and Friedman, R. (2005). Telephone-linked care for physical activity: a qualitative evaluation of the use patterns of an information technology program for patients. Journal of Biomedical Informatics, 38(3): 220–228.
Ferrucci, D., Brown, E., Chu-Carroll, J., Fan, J., Gondek, D., Kalyanpur, A. A., Lally, A., Murdock, J. W., Nyberg, E., Prager, J., et al. (2010). Building watson: An overview of the deepqa project. AI magazine, 31(3): 59–79.
Green Jr, B. F., Wolf, A. K., Chomsky, C., and Laughery, K. (1961). Baseball: an automatic question-answerer. pages 219–224. ACM.
Herzog, O. and Rollinger, C.-R. (1991). Text understanding in LILOG: integrating computational linguistics and artificial intelligence: final report on the IBM Germany LILOG-Project. Springer.
High, R. (2012). The era of cognitive systems: An inside look at ibm watson and how it works. IBM Corporation, Redbooks.
Hubal, R. C. and Day, R. S. (2006). Informed consent procedures: An experimental test using a virtual character in a dialog systems training application. Journal of Biomedical Informatics, 39(5): 532–540.
Hunt, D. L., Haynes, R. B., Hanna, S. E., and Smith, K. (1998). Effects of computer-based clinical decision support systems on physician performance and patient outcomes: a systematic review. Jama, 280(15): 1339–1346.
Jiang, B., Tang, J., and Yan, C. (2017). Outpatient capacity allocation considering adding capacity to match high patient demand. Journal of Systems Science and Systems Engineering, 26(4): 487–516.
Jones, P., Chalmers, L., Wells, S., Ameratunga, S., Carswell, P., Ashton, T., Curtis, E., Reid, P., Stewart, J., Harper, A., et al. (2012). Implementing performance improvement in new zealand emergency departments: the six hour time target policy national research project protocol. BMC health services research, 12(1): 45.
Khoury, M. J. and Ioannidis, J. P. (2014). Big data meets public health. Science, 346(6213): 1054–1055.
Lipkin, M., Quill, T. E., and Napodano, R. J. (1984). The medical interview: a core curriculum for residencies in internal medicine. Annals of Internal Medicine, 100(2): 277–284.
Liu, Y., Ge, T., Mathews, K., Ji, H., and McGuinness, D. (2015). Exploiting task-oriented resources to learn word embeddings for clinical abbreviation expansion. Proceedings of BioNLP 15, pages 92–97.
Martínez-Pérez, B., de la Torre-Díez, I., López-Coronado, M., Sainz-De-Abajo, B., Robles, M., & García-Gómez, J. M. (2014). Mobile clinical decision support systems and applications: a literature and commercial review. Journal of medical systems, 38(1), 4.
McFillen, J. M., O’Neil, D. A., Balzer, W. K., and Varney, G. H. (2013). Organizational diagnosis: An evidence-based approach. Journal of Change Management, 13(2): 223–246.
Mikolov, T., Chen, K., Corrado, G., and Dean, J. (2013a). Efficient estimation of word representations in vector space. arXiv preprint arXiv: 1301.3781.
Mikolov, T., Sutskever, I., Chen, K., Corrado, G. S., and Dean, J. (2013b). Distributed representations of words and phrases and their compositionality. In Advances in neural information processing systems, pages 3111–3119.
Miller, R. A., Pople Jr, H. E., and Myers, J. D. (1982). Internist-i, an experimental computer-based diagnostic consultant for general internal medicine. New England Journal of Medicine, 307(8): 468–476.
Moskvina, A. and Liu, J. (2016a). How to build your network? a structural analysis. In Proceedings of the Twenty-Fifth International Joint Conference on Artificial Intelligence, pages 2597–2603. AAAI Press.
Moskvina, A. and Liu, J. (2016b). Integrating networks of equipotent nodes. In International Conference on Computational Social Networks, pages 39–50. Springer.
Ni, L., Lu, C., Liu, N., and Liu, J. (2017). Mandy: Towards a smart primary care chatbot application. In International Symposium on Knowledge and Systems Sciences, pages 38–52. Springer.
Nikfarjam, A., Sarker, A., O’Connor, K., Ginn, R., and Gonzalez, G. (2015). Pharmacovigilance from social media: mining adverse drug reaction mentions using sequence labeling with word embedding cluster features. Journal of the American Medical Informatics Association, 22(3): 671–681.
Pathak, J., Solbrig, H. R., Buntrock, J. D., Johnson, T. M., and Chute, C. G. (2009). Lexgrid: a framework for representing, storing, and querying biomedical terminologies from simple to sublime. Journal of the American Medical Informatics Association, 16(3): 305–315.
Pechsiri, C. and Sukharomana, R. (2017). Comparative study of using word co-occurrence to extract disease symptoms from web documents. In International Symposium on Knowledge and Systems Sciences, pages 91–100. Springer.
Ramelson, H. Z., Friedman, R. H., and Ockene, J. K. (1999). An automated telephone-based smoking cessation education and counseling system. Patient Education and Counseling, 36(2): 131–144.
Realdi, G., Previato, L., and Vitturi, N. (2008). Selection of diagnostic tests for clinical decision making and translation to a problem oriented medical record. Clinica Chimica Acta, 393(1): 37–43.
Richardson, D. B. (2006). Increase in patient mortality at 10 days associated with emergency department overcrowding. The Medical Journal of Australia, 184(5): 213–216.
Saygin, A. P., Cicekli, I., and Akman, V. (2000). Turing test: 50 years later. Minds and Machines, 10(4): 463–518.
Stern, S., Cifu, A., and Altkorn, D. (2014). Symptom to Diagnosis An Evidence Based Guide. McGraw Hill Professional.
Taber, J. M., Leyva, B., and Persoskie, A. (2015). Why do people avoid medical care? a qualitative study using national data. Journal of general internal medicine, 30(3): 290–297.
Tien, J. M. and Berg, D. (2003). A case for service systems engineering. Journal of systems science and systems engineering, 12(1): 13–38.
Tien, J. M. and Goldschmidt-Clermont, P. J. (2009). Healthcare: A complex service system. Journal of Systems Science and Systems Engineering, 18(3): 257–282.
Victor, L. Y., Buchanan, B. G., Shortliffe, E. H., Wraith, S. M., Davis, R., Scott, A. C., and Cohen, S. N. (1979). Evaluating the performance of a computer-based consultant. Computer programs in biomedicine, 9(1): 95–102.
Wang, C. and Tang, X. (2016). The online debate networks analysis: A case study of debates at tianya forum. In International Symposium on Knowledge and Systems Sciences, pages 140–150. Springer.
Warren, J. R. (1998). Better, more cost-effective intake interviews. IEEE Intelligent Systems and their Applications, 13(1): 40–48.
Weizenbaum, J. (1966). Eliza—a computer program for the study of natural language communication between man and machine. Communications of the ACM, 9(1): 36–45.
Wilensky, R., Chin, D. N., Luria, M., Martin, J., Mayfield, J., and Wu, D. (1988). The berkeley unix consultant project. Computational Linguistics, 14(4): 35–84.
Winograd, T. (1971). Procedures as a representation for data in a computer program for understanding natural language. Technical report.
Woods, W. A. and Kaplan, R. (1977). Lunar rocks in natural english: Explorations in natural language question answering. Linguistic structures processing, 5: 521–569.
Xu, J., Zhang, Y., Xu, H., et al. (2015). Clinical abbreviation disambiguation using neural word embeddings. Proceedings of BioNLP 15, pages 171–176.
Yan, B., Chen, Y., and Liu, J. (2017). Dynamic relationship building: exploitation versus exploration on a social network. In International Conference on Web Information Systems Engineering, pages 75–90. Springer.
Yan, B., Liu, Y., Liu, J., Cai, Y., Su, H., and Zheng, H. (2018). From the periphery to the center: Information brokerage in an evolving network. In Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence, IJCAI 2018, July 13–19, 2018, Stockholm, Sweden., pages 3912–3918.
Zhang, W., Jiang, Y., and Yoshida, T. (2016). Deep context identification of deceptive reviews using word vectors. In International Symposium on Knowledge and Systems Sciences, pages 213–224. Springer.
Acknowledgments
The first author is partially funded by a scholarship offered by Precision Driven Health in New Zealand, a public-private research partnership aimed at improving health outcomes through data science. Initial progress of the research was reported in the PDH &Orion healthBlog https://doi.org/orionhealth.com/global/knowledge-bub/blogs/meet-mandy-an-intelligent-and-interactive-medicare-system/ (Ni et al., 2017)
The authors would like to thank the anonymous referees for the constructive feedback to improve the quality of the paper.
Author information
Authors and Affiliations
Corresponding author
Additional information
Lin Ni is a Chatbot Developer at the National Institute for Health Innovation (NIHI), the University of Auckland. She received a B.E. degree from Northeastern University (China) in 2010, and a Postgraduate Diploma in Computer Science from the University of Auckland in 2017. Her working experience and research interests include Car Navigation systems, Recommendation systems, Multi-agent systems, and Chatbots.
Jiamou Liu is a lecturer in the Department of Computer Science at the University of Auckland. He was a senior lecturer at the Auckland University of Technology between 2011 and 2015 and a researcher at the Department of Computer Science of Leipzig University from 2009 to 2010. He received a PhD from The University of Auckland in 2010. His research interests include Social Network Analysis, Mutliagent Systems and Artificial Inteligence.
Rights and permissions
About this article
Cite this article
Ni, L., Liu, J. A Framework for Domain-Specific Natural Language Information Brokerage. J. Syst. Sci. Syst. Eng. 27, 559–585 (2018). https://doi.org/10.1007/s11518-018-5389-1
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11518-018-5389-1