Automatically Testing of Multimodal Interactive Applications

  • Le Thanh Long
  • Nguyen Thanh Binh
  • Ioannis Parissis
Conference paper
Part of the Communications in Computer and Information Science book series (CCIS, volume 692)


Testing interactive multimodal applications is particularly important and requires a lot of effort. Automating this activity can result to significant development cost reduction and quality improvement. In this paper, we propose an approach for automating the test generation of such multimodal applications. This approach is based on the definition of a test modeling language, TTT. The objective of the TTT language is to provide a means for expressing abstract test scenarios for interactive multimodal applications, including non-deterministic choices and action occurrence probabilities that can be used to automate the test generation. Then, we built the TTTEST tool that supports to generate tests for multimodal events and to check the validity of CARE properties of this kind of applications. The approach is illustrated on a case study.


Interactive multimodal applications Test modeling language CARE properties 


  1. 1.
    Coutaz, J., Nigay, L., Salber, D., Blandford, A., May, J., Young, R.M.: Four easy pieces for assessing the usability of multimodal interaction: the care properties. In: INTERACT, pp. 115–120. Chapman & Hall (1995)Google Scholar
  2. 2.
    Duke, D.J., Harrison, M.D.: Abstract interaction objects. Comput. Graph. Forum 12(3), 25–36 (1993)CrossRefGoogle Scholar
  3. 3.
    Paternò, F., Faconti, G.: On the use of LOTOS to describe graphical interaction. In: HCI 1992: Proceedings of the Conference on People and Computers VII, pp. 155–173. Cambridge University Press, New York (1993)Google Scholar
  4. 4.
    Le, T.L., Nguyen, T.B., Parissis, I.: A new test modeling language for interactive applications based on task trees. In: Proceedings of the 4th International Symposium on Information and Communication Technology, pp. 285–293 (2013)Google Scholar
  5. 5.
    Le, T.L., Binh, N.T., Parissis, I.: Testing Multimodal Interactive Applications By Means of The TTT Language, Domain Specific Model-Based Approaches To Verification And Validation - Amaretto 2016. In: Conjunction with the 4th International Conference on Model-Driven Engineering and Software Development - MODELSWARD 2016, Rome, Italy, 19 February 2016Google Scholar
  6. 6.
    Madani, L., Parissis, I.: Automatically testing interactive applications using extendedtask trees. J. Log. Algebr. Program. 78(6), 454–471 (2009)MathSciNetCrossRefzbMATHGoogle Scholar
  7. 7.
    Madani, L., Oriat, C., Parissis, I., Bouchet, J., Nigay, L.: Synchronous testing of multimodal systems: an operational profile-based approach. In: 16th International Symposium on Software Reliability Engineering (ISSRE 2005), Chicago, IL, USA, pp. 325–334, 8–11 November 2005Google Scholar
  8. 8.
    Madani, L., Parissis, I.: Automatically testing interactive multimodal systems using task trees and fusion models. In: 6th International Workshop on Automation of Software Test (AST 2011), Hawai, USA (2011)Google Scholar
  9. 9.
    Musa, J.: Operational profiles in software-reliability engineering. IEEE Softw. 10, 14–32 (1993)CrossRefGoogle Scholar
  10. 10.
    Palanque, P., Bastide, R.: Verification of interactive software by analysis of its formal specification. In: INTERACT 1995, Norway (1995)Google Scholar
  11. 11.
    Shehady, R.K., Siewiorek, D.P.: A method to automate user interface testing using variable finite state machines. In: FTCS 1997: Proceedings of the 27th International Symposium on Fault-Tolerant Computing (FTCS 1997), p. 80, Washington, DC, USA. IEEE Computer Society (1997)Google Scholar
  12. 12.
    Aıt-Ameur, Y., Kamel, N.: A generic formal specification of fusion of modalities in a multimodal HCI. In: Jacquart, R., (ed.) IFIP Congress Topical Sessions, pp. 415–420. Kluwer (2004)Google Scholar
  13. 13.
    TerBeek, M.H., Faconti, G.P., Massink, M., Palanque, P.A., Winckler, M.: Resilience of interaction techniques to interrupts: a formal model-based approach. In: Gross, T., Gulliksen, J., Kotzé, P., Oestreicher, L., Palanque, P., Prates, ROliveira, Winckler, M. (eds.) INTERACT 2009, Part I. LNCS, vol. 5726, pp. 494–509. Springer, Heidelberg (2009). doi: 10.1007/978-3-642-03655-2_56 CrossRefGoogle Scholar
  14. 14.
    Paternò, F., Mancini, C., Meniconi, S.: ConcurTaskTrees: a diagrammatic notation for specifying task models. In: Howard, S., Hammond, J., Lindgaard, G. (eds.) Proceedings of the 6th IFIP TC 13 International Conference on Human-Computer Interaction (INTERACT 1997), Sydney, Australia, pp. 362–369. Chapman & Hall, Boca Raton (1997)Google Scholar
  15. 15.
    Palanque, P., Winckler, M., Ladry, J.-F., TerBeek, M.H., Faconti, G., Massink, M.: A formal approach supporting the comparative predictive assessment of the interruption-tolerance of interactive systems. In: Calvary, G., Graham, T.C.N., Gray, P. (eds.) Proceedings of the ACM SIGCHI Symposium on Engineering Interactive Computing Systems (EICS 2009), Pittsburgh, PA, USA, pp. 211–220. ACM Press (2009)Google Scholar
  16. 16.
    Kamel, N., AïtAmeur, Y., Selouani, S.-A., Hamam, H.: A formal model to handle the adaptability of multimodal user interfaces. In: Liang, B., Whitaker, R.M. (eds.) Proceedings of the 1st International ICST Conference on Ambient Media and Systems (AMBI-SYS 2008), Quebec, Canada (2008)Google Scholar
  17. 17.
    Mohand-Oussaïd, L., Aït-Sadoune, I., AïtAmeur, Y., Ahmed-Nacer, M.: A formal model for output multimodal HCI - an Event-B formalization. Computing 97, 713–740 (2015)MathSciNetCrossRefGoogle Scholar

Copyright information

© Springer International Publishing AG 2017

Authors and Affiliations

  • Le Thanh Long
    • 1
  • Nguyen Thanh Binh
    • 2
  • Ioannis Parissis
    • 3
  1. 1.Department of ComputingDuy Tan UniversityDa NangViet Nam
  2. 2.IT FacultyThe University of Danang - University of Science and TechnologyDa NangViet Nam
  3. 3.Univ. Grenoble Alpes, LCISValenceFrance

Personalised recommendations