Effects of Voice-Based Synthetic Assistant on Performance of Emergency Care Provider in Training

  • Praveen Damacharla
  • Parashar Dhakal
  • Sebastian Stumbo
  • Ahmad Y. JavaidEmail author
  • Subhashini Ganapathy
  • David A. Malek
  • Douglas C. Hodge
  • Vijay Devabhaktuni


As part of a perennial project, our team is actively engaged in developing new synthetic assistant (SA) technologies to assist in training combat medics and medical first responders. It is critical that medical first responders are well trained to deal with emergencies more effectively. This would require real-time monitoring and feedback for each trainee. Therefore, we introduced a voice-based SA to augment the training process of medical first responders and enhance their performance in the field. The potential benefits of SAs include a reduction in training costs and enhanced monitoring mechanisms. Despite the increased usage of voice-based personal assistants (PAs) in day-to-day life, the associated effects are commonly neglected for a study of human factors. Therefore, this paper focuses on performance analysis of the developed voice-based SA in emergency care provider training for a selected emergency treatment scenario. The research discussed in this paper follows design science in developing proposed technology; at length, we discussed architecture and development and presented working results of voice-based SA. The empirical testing was conducted on two groups as user studies using statistical analysis tools, one trained with conventional methods and the other with the help of SA. The statistical results demonstrated the amplification in training efficacy and performance of medical responders powered by SA. Furthermore, the paper also discusses the accuracy and time of task execution (t) and concludes with the guidelines for resolving the identified problems.


Emergency care provider Empirical study Human performance Performance analysis Personal assistant Synthetic assistant 



The University of Toledo and Round 1 Award from the Ohio Federal Research Jobs Commission (OFMJC) through Ohio Federal Research Network (OFRN) fund this research project; authors also appreciate support of the Paul A. Hotmer Family CSTAR (Cybersecurity and Teaming Research) Lab and EECS (Electrical Engineering and Computer Science) Department at the University of Toledo.


  1. Alison, K. C. (2013). Stop the bleeding: Understanding how our methods have evolved and how far we need to go. Journal of Emergency Medical Services.
  2. Amazon Web Services, I. (2017). Architecting for HIPAA security and compliance on Amazon Web Services. AWS Whitepapers, 18.Google Scholar
  3. Anderson, J. R. (2002). Spanning seven orders of magnitude: A challenge for cognitive modeling. Cognitive Science, 26(1), 85–112.MathSciNetCrossRefGoogle Scholar
  4. Anderson, J. R. (2007). How can the human mind occur in the physical universe? Oxford University Press.
  5. Ball, J., Myers, C., Heiberg, A., Cooke, N. J., Matessa, M., Freiman, M., & Rodgers, S. (2010). The synthetic teammate project. Computational and Mathematical Organization Theory, 16(3), 271–299.CrossRefGoogle Scholar
  6. Barnes, S. L., Bukoski, A., Kerby, J. D., Llerena, L., Armstrong, J. H., Strayhorn, C., & Consortium, U. o. M. C. C. T. (2016). Live tissue versus simulation training for emergency procedures: Is simulation ready to replace live tissue? Surgery, 160(4), 997–1007.CrossRefGoogle Scholar
  7. Bertrand, J., Babu, S., Polgreen, P., & Segre, A. (2010). Virtual agents based simulation for training healthcare workers in hand hygiene procedures. Paper presented at the Intelligent Virtual Agents.Google Scholar
  8. Bunch, L., Breedy, M., Bradshaw, J. M., Carvalho, M., Suri, N., Uszok, A., Marik, V. (2004). Software agents for process monitoring and notification. Paper presented at the Proceedings of the 2004 ACM symposium on Applied computing.Google Scholar
  9. Callaway, D. W., Smith, E. R., Cain, J., Shapiro, G., Burnett, W. T., McKay, S. D., & Mabry, R. (2011). Tactical emergency casualty care (TECC): Guidelines for the provision of prehospital trauma care in high threat environments. Journal of Special Operations Medicine, 11(3), 104–122.Google Scholar
  10. Demir, M., McNeese, N. J., Cooke, N. J., Ball, J. T., Myers, C., & Frieman, M. (2015). Synthetic teammate communication and coordination with humans. Proceedings of the Human Factors and Ergonomics Society Annual Meeting, 59(1), 951–955. Scholar
  11. Franklin, S., & Graesser, A. (1997). Is it an agent, or just a program?: A taxonomy for autonomous agents. In J. P. Müller, M. J. Wooldridge, & N. R. Jennings (Eds.), Intelligent agents III agent theories, architectures, and languages. Lecture notes in computer science (Lecture notes in artificial intelligence) (Vol. 1193). Berlin, Heidelberg: Springer.
  12. Gibbons, J. D., & Chakraborti, S. (2011). Nonparametric statistical inference. In M. Lovric (Ed.), International encyclopedia of statistical science. Berlin, Heidelberg: Springer. Scholar
  13. Graafland, M., Schraagen, J. M., & Schijven, M. P. (2012). Systematic review of serious games for medical education and surgical skills training. British Journal of Surgery, 99(10), 1322–1330.CrossRefGoogle Scholar
  14. Harden, S. (2011). 7 tips to turbocharge your surgical safety checklist. Outpatient Surgery Magazine, XII(08).
  15. Hauswald, J., Laurenzano, M. A., Zhang, Y., Li, C., Rovinski, A., Khurana, A., Tang, L. (2015). Sirius: An open end-to-end voice and vision personal assistant and its implications for future warehouse scale computers. Paper presented at the ACM SIGPLAN Notices.Google Scholar
  16. He, M., Wang, X., Todd, T. D., Zhao, D., & Kezys, V. (2007). Ad hoc assisted handoff for real-time voice in IEEE 802.11 infrastructure WLANs. International Journal of Wireless and Mobile Computing, 2(4), 324–336.CrossRefGoogle Scholar
  17. He, J., Chaparro, A., Nguyen, B., Burge, R. J., Crandall, J., Chaparro, B., & Cao, S. (2014). Texting while driving: Is speech-based text entry less risky than handheld text entry? Accident Analysis & Prevention, 72(Supplement C), 287–295. Scholar
  18. Henninger, A. E., Cutts, D., Loper, M., Lutz, R., Richbourg, R., Saunders, R., & Swenson, S. (2008). Live virtual constructive architecture roadmap (LVCAR) final report. Institute for Defense Analysis, Modeling & Simulation Coordination Office Project No. 06OC-TR-001.
  19. Hobgood, C., Hevia, A., Tamayo-Sarver, J. H., Weiner, B., & Riviello, R. (2005). The influence of the causes and contexts of medical errors on emergency medicine residents’ responses to their errors: An exploration. Academic Medicine, 80(8), 758–764.CrossRefGoogle Scholar
  20. Jeffries, P. R. (2009). Dreams for the future for clinical simulation: Research and development in this hot topic area will lead to more sophisticated manikins and robots possessing even more realistic features and functionality. What is known as the high-fidelity simulator today will look simplistic when compared to the next generation of simulators in the years to come. Nursing Education Perspectives, 30(2), 71–72.Google Scholar
  21. Jiang, J., Hassan Awadallah, A., Jones, R., Ozertem, U., Zitouni, I., Gurunath Kulkarni, R., & Khan, O. Z. (2015). Automatic online evaluation of intelligent assistants. Paper presented at the Proceedings of the 24th International Conference on World Wide Web.Google Scholar
  22. Johnson, M., Bradshaw, J. M., Feltovich, P. J., Jonker, C. M., Van Riemsdijk, M. B., & Sierhuis, M. (2014). Coactive design: Designing support for interdependence in joint activity. Journal of Human-Robot Interaction, 3(1), 43–69.CrossRefGoogle Scholar
  23. Kragh Jr., J. F., Walters, T. J., Baer, D. G., Fox, C. J., Wade, C. E., Salinas, J., & Holcomb, J. B. (2009). Survival with emergency tourniquet use to stop bleeding in major limb trauma. Annals of Surgery, 249(1), 1–7.CrossRefGoogle Scholar
  24. Lee, H. Y., Kim, Y. L., & Lee, S. M. (2015). Effects of virtual reality-based training and task-oriented training on balance performance in stroke patients. Journal of Physical Therapy Science, 27(6), 1883–1888.CrossRefGoogle Scholar
  25. Makary, M. A., & Daniel, M. (2016). Medical error—The third leading cause of death in the US. BMJ, 353.
  26. Mitchell, T. M., Caruana, R., Freitag, D., McDermott, J., & Zabowski, D. (1994). Experience with a learning personal assistant. Communications of the ACM, 37(7), 80–91.CrossRefGoogle Scholar
  27. Moorman, D. W. (2007). Communication, teams, and medical mistakes. Annals of Surgery, 245(2), 173–175.CrossRefGoogle Scholar
  28. Moorthy, A. E., & Vu, K.-P. L. (2014). Voice activated personal assistant: Acceptability of use in the public space. Paper presented at the International Conference on Human Interface and the Management of Information.Google Scholar
  29. Myers, K., Berry, P., Blythe, J., Conley, K., Gervasio, M., McGuinness, D. L., & Tambe, M. (2007). An intelligent personal assistant for task and time management. AI Magazine, 28(2), 47.Google Scholar
  30. Nagpal, K., Arora, S., Vats, A., Wong, H. W., Sevdalis, N., Vincent, C., & Moorthy, K. (2012). Failures in communication and information transfer across the surgical care pathway: interview study. BMJ Quality and Safety, 21(10), 843–849. Scholar
  31. Naismith, L. M., Cheung, J. J., Ringsted, C., & Cavalcanti, R. B. (2015). Limitations of subjective cognitive load measures in simulation-based procedural training. Medical Education, 49(8), 805–814.CrossRefGoogle Scholar
  32. Newell, A. (1994). Unified theories of cognition. Harvard University Press.
  33. Ong, M.-S., Biomed, E. M., & Coiera, E. (2011). A systematic review of failures in handoff communication during intrahospital transfers. The Joint Commission Journal on Quality and Patient Safety, 37(6), 274–278, AP1–AP8. Scholar
  34. Pratt, J. W. (1959). Remarks on zeros and ties in the Wilcoxon signed rank procedures. Journal of the American Statistical Association, 54(287), 655–667.MathSciNetCrossRefzbMATHGoogle Scholar
  35. Ruxton, G. D. (2006). The unequal variance t-test is an underused alternative to Student's t-test and the Mann–Whitney U test. Behavioral Ecology, 17(4), 688–690.CrossRefGoogle Scholar
  36. Sarikaya, R. (2017). The technology behind personal digital assistants: An overview of the system architecture and key components. IEEE Signal Processing Magazine, 34(1), 67–81.CrossRefGoogle Scholar
  37. Savage, L. E., Forestier, M. C., Withers, L. N., Tien, C. H., & Pannell, C. D. (2011). Tactical combat casualty care in the Canadian forces: Lessons learned from the Afghan war. Canadian Journal of Surgery, 54(6 Suppl), S118–S123.CrossRefGoogle Scholar
  38. Schweitzer, E. J. (2012). Reconciliation of the cloud computing model with US federal electronic health record regulations. Journal of the American Medical Informatics Association, 19(2), 161–165. Scholar
  39. Spector, J. M., Merrill, M. D., Elen, J., & Bishop, M. (2014). Handbook of research on educational communications and technology. 4th edn. New York: Springer-Verlag. Scholar
  40. Sun, M., Chen, Y.-N., Hua, Z., Tamres-Rudnicky, Y., Dash, A., & Rudnicky, A. I. (2016). App Dialogue: Multi-app dialogues for intelligent assistants. Paper presented at the Proceedings of the 10th International Conference on Language Resources and Evaluation (LREC).Google Scholar
  41. Terry, K. (2009). Patient privacy, the new threats. Physicians. Practice, 19(3).
  42. Watson, D., Clark, L. A., & Tellegen, A. (1988). Development and validation of brief measures of positive and negative affect: The PANAS scales. Journal of Personality and Social Psychology, 54(6), 1063–1070.CrossRefGoogle Scholar
  43. Yorke-Smith, N., Saadati, S., Myers, K. L., & Morley, D. N. (2009). Like an intuitive and courteous butler: a proactive personal agent for task management. Paper presented at the Proceedings of The 8th International Conference on Autonomous Agents and Multiagent Systems-Volume 1.Google Scholar
  44. Zachary, W., Santarelli, T., Lyons, D., Bergondy, M., & Johnston, J. (2001). Using a community of intelligent synthetic entities to support operational team training. Defence Technical Information Center.
  45. Zhang, H., Jain, A., Khandelwal, G., Kaushik, C., Ge, S., & Hu, W. (2016). Bing developer assistant: improving developer productivity by recommending sample code. Paper presented at the Proceedings of the 2016 24th ACM SIGSOFT International Symposium on Foundations of Software Engineering.Google Scholar

Copyright information

© International Artificial Intelligence in Education Society 2018

Authors and Affiliations

  • Praveen Damacharla
    • 1
  • Parashar Dhakal
    • 1
  • Sebastian Stumbo
    • 2
  • Ahmad Y. Javaid
    • 1
    Email author
  • Subhashini Ganapathy
    • 2
  • David A. Malek
    • 3
  • Douglas C. Hodge
    • 1
    • 4
  • Vijay Devabhaktuni
    • 1
  1. 1.Department of EECSThe University of ToledoToledoUSA
  2. 2.Department of Biomedical, Industrial & Human FactorsWright State UniversityDaytonUSA
  3. 3.Wright State Research InstituteWright State UniversityDaytonUSA
  4. 4.National Center for Medical ReadinessWright State UniversityDaytonUSA

Personalised recommendations