Rationalizing the Need of Architecture-Driven Testing of Interactive Systems

  • Alexandre Canny
  • Elodie Bouzekri
  • Célia Martinie
  • Philippe Palanque
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11262)


Testing interactive systems is known to be a complex task that cannot be exhaustive. Indeed, the infinite number of combination of user input and the complexity of information presentation exceed the practical limits of exhaustive and analytical approach to testing [31]. Most interactive software testing techniques are produced by applying and tuning techniques from the field of software testing to try to address the specificities of interactive applications. When some elements cannot be taken into account by the software testing technique, they are usually ignored. In this paper we propose to follow an opposite approach, starting from a generic architecture for interactive systems (including both software and hardware elements) for identifying in a systematic way, testing problems and testing needs. This architecture-driven approach makes it possible to identify how software testing knowledge and techniques can support interactive systems testing but also where the interactive systems engineering community should invest in order to test their idiosyncrasies too.


Architecture-driven testing Interactive system testing 


  1. 1.
    Accot, J., Chatty, S., Palanque, P.: A formal description of low level interaction and its application to multimodal interactive systems. In: Bodart, F., Vanderdonckt, J. (eds.) DSV-IS 1996. Eurographics, pp. 92–104. Springer, Vienna (1996). Scholar
  2. 2.
    Accot, J., Chatty, S., Maury, S., Palanque, P.: Formal transducers: models of devices and building bricks for the design of highly interactive systems. In: Harrison, M.D., Torres, J.C. (eds.) DSV-IS 1997. Eurographics, pp. 143–159. Springer, Vienna (1997). Scholar
  3. 3.
    Abbaspour Asadollah, S., Inam, R., Hansson, H.: A survey on testing for cyber physical system. In: El-Fakih, K., Barlas, G., Yevtushenko, N. (eds.) ICTSS 2015. LNCS, vol. 9447, pp. 194–207. Springer, Cham (2015). Scholar
  4. 4.
    Avizienis, A., Laprie, J.C., Randell, B., Landwehr, C.: Basic concepts and taxonomy of dependable and secure computing. IEEE Trans. Dependable Secur. Comput. 1, 11–33 (2004)CrossRefGoogle Scholar
  5. 5.
    Bailly, G., Lecolinet, E., Nigay, L.: Visual menu techniques. ACM Comput. Surv. 49(4), 60:1–60:41 (2017)Google Scholar
  6. 6.
    Banerjee, I., Nguyen, B., Garousi, V., Memon, A.M.: Graphical user interface (GUI) testing: systematic mapping and repository. Inf. Softw. Technol. 55, 1679–1694 (2013)CrossRefGoogle Scholar
  7. 7.
    Bass, L., et al.: The arch model: Seeheim revisited. In: User Interface Developpers’ Workshop (1991)Google Scholar
  8. 8.
    Bellik, Y.: Multimodal interfaces: concepts, models and architecture, Ph.D. thesis, University Paris-South 11, Orsay (1995). (in French)Google Scholar
  9. 9.
    Bernhaupt, R., Cronel, M., Manciet, F., Martinie, C., Palanque, P.: Transparent automation for assessing and designing better interactions between operators and partly-autonomous interactive systems. In: ATACCS 2015, pp. 129–139 (2015)Google Scholar
  10. 10.
    Bouchet, J., Madani, L., Nigay, L., Oriat, C., Parissis, I.: Formal testing of multimodal interactive systems. In: Gulliksen, J., Harning, M.B., Palanque, P., van der Veer, G.C., Wesson, J. (eds.) EIS 2007. LNCS, vol. 4940, pp. 36–52. Springer, Heidelberg (2008). Scholar
  11. 11.
    Bourque, P., Fairley, R.E., IEEE Computer Society: Guide to the Software Engineering Body of Knowledge (SWEBOK(R)): Version 3.0. IEEE Computer Society Press, Los Alamitos (2014)Google Scholar
  12. 12.
    Campos, J.C., et al.: A more intelligent test case generation approach through task models manipulation. In: Proceedings of the ACM HCI. EICS, vol. 1, pp. 9:1–9:20 (2017)Google Scholar
  13. 13.
    Cockton, G., Woolrych, A.: Understanding inspection methods: lessons from an assessment of heuristic evaluation. In: Blandford, A., Vanderdonckt, J., Gray, P. (eds.) People and Computers XV—Interaction without Frontiers, pp. 171–191. Springer, London (2001). Scholar
  14. 14.
    Cronel, M., Dumas, B., Palanque, P., Canny, A.: MIODMIT: a generic architecture for dynamic multimodal interactive systems. In: Bogdan, C., et al. (eds.) Human-Centered and Error-Resilient Systems Development, HCSE 2018. LNCS, vol. 11262, pp. 109–129. Springer, Cham (2018)Google Scholar
  15. 15.
    Cuenca, F., Coninx, K., Vanacken, D., Luyten, K.: Graphical toolkits for rapid prototyping of multimodal systems: a survey. Interact. Comput. 27, 470–488 (2015)CrossRefGoogle Scholar
  16. 16.
    Dragicevic, P., Fekete, J.D.: Input device selection and interaction configuration with ICON. In: Blandford, A., Vanderdonckt, J., Gray, P. (eds.) People and Computers XV—Interaction without Frontiers, pp. 543–558. Springer, London (2001). Scholar
  17. 17.
    Dragicevic, P.: Un modèle d’interaction en entrée pour des systèmes interactifs multi-dispositifs hautement configurables. Ph.D. Université de Nantes (2004). (in French)Google Scholar
  18. 18.
    Göransson, B., Gulliksen, J., Boivie, I.: The usability design process - integrating user-centered systems design in the software development process. Softw. Process Improv. Pract. 8(2), 111–131 (2003)CrossRefGoogle Scholar
  19. 19.
    Greenberg, S.: Working through task-centered system design. In: Diaper, D., Stanton, N. (eds.) The Handbook of Task Analysis for Human-Computer Interaction. Lawrence Erlbaum Associates (2002)Google Scholar
  20. 20.
    Ha, T.T., Ghaffari, R.: Simulating Single and Multi-Touch Events for Testing a Touch Panel (2012).
  21. 21.
    ISO 9241-11. Ergonomics of human system interaction - Part 11. Usability: Definitions and concepts (2018)Google Scholar
  22. 22.
    Kelley, J.F.: An iterative design methodology for user-friendly natural language office information applications. ACM Trans. Inf. Syst. 2(1), 26–41 (1984)CrossRefGoogle Scholar
  23. 23.
    Lalanne, D., Nigay, L., Palanque, P., Robinson, P., Vanderdonckt, J., Ladry, J.F.: Fusion engines for multimodal input: a survey. In: ICMI, pp. 153–160. ACM (2009)Google Scholar
  24. 24.
    Lee, J.S., et al.: A 0.4 V driving multi-touch capacitive sensor with the driving signal frequency set to (n + 0.5) times the inverse of the LCD VCOM noise period. In: IEEE International Symposium on Circuits and Systems (ISCAS), pp. 682–685 (2014)Google Scholar
  25. 25.
    Lelli, V., Blouin, A., Baudry, B.: Classifying and qualifying GUI defects. Presented at the 8th IEEE International Conference on Software Testing, Verification and Validation, 13 April 2015Google Scholar
  26. 26.
    Lelli, V., Blouin, A., Baudry, B., Coulon, F.: On model-based testing advanced GUIs. In: 2015 IEEE Eighth International Conference on Software Testing, Verification and Validation Workshops (ICSTW), pp. 1–10 (2015)Google Scholar
  27. 27.
    Memon, A.M., Soffa, M.L., Pollack, M.E.: Coverage criteria for GUI testing. In: Proceedings of the 8th European Software Engineering Conference Held Jointly with 9th ACM SIGSOFT International Symposium on Foundations of Software Engineering, pp. 256–267. ACM, New York (2001)Google Scholar
  28. 28.
    Memon, A.M.: GUI testing: pitfalls and process. Computer 35(8), 87–88 (2002)CrossRefGoogle Scholar
  29. 29.
    Memon, A.M.: A comprehensive framework for testing graphical user interfaces. Ph.D. thesis, University of Pittsburgh, Pittsburgh (2001)Google Scholar
  30. 30.
    Memon, A.M., Nguyen, B.N.: Advances in automated model-based system testing of software applications with a GUI front-end. In: Zelkowitz, M.V. (ed.) Advances in Computers, pp. 121–162. Elsevier (2010)Google Scholar
  31. 31.
    Nguyen, B.N., Robbins, B., Banerjee, I., Memon, A.: GUITAR: an innovative tool for automated testing of GUI-driven software. Autom. Softw. Eng. 21, 65–105 (2014)CrossRefGoogle Scholar
  32. 32.
    Nielsen, J.: Usability Engineering. Morgan Kaufmann, San Francisco (1994)zbMATHGoogle Scholar
  33. 33.
    Palanque, P., Barboni, E., Martinie, C., Navarre, D., Winckler, M.: A model-based approach for supporting engineering usability evaluation of interaction techniques. In: Proceedings of EICS 2011, pp. 21–30. ACM (2011)Google Scholar
  34. 34.
    Pfaff, G.E. (ed.): Proceedings of IFIP/EG Workshop on User Interface Management Systems (November 1983, Seeheim, FRG). Springer, Berlin (1985)Google Scholar
  35. 35.
    Pirker, M., Bernhaupt, R.: Measuring user experience in the living room: results from an ethnographically oriented field study indicating major evaluation factors. In: Proceedings of the 9th European Conference on Interactive TV and Video (EuroITV 2011), pp. 79–82. ACM, New York (2011)Google Scholar
  36. 36.
    Rowley, D.E.: Usability testing in the field: bringing the laboratory to the user. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI 1994), pp. 252–257. ACM, New York (1994)Google Scholar
  37. 37.
    Sadeghi, A., Jabbarvand, R., Malek, S.: PATDroid: permission-aware GUI testing of android. In: Proceedings of the 2017 11th Joint Meeting on Foundations of Software Engineering, pp. 220–232. ACM, New York (2017)Google Scholar
  38. 38.
    Song, W., Qian, X., Huang, J.: EHBDroid: beyond GUI testing for android applications. In: Proceedings of the 32nd IEEE/ACM International Conference on Automated Software Engineering, pp. 27–37. IEEE Press, Piscataway (2017)Google Scholar
  39. 39.
    Thimbleby, H.: Reasons to question seven segment displays. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pp. 1431–1440. ACM, New York (2013)Google Scholar
  40. 40.
    Vu, T., et al.: Distinguishing users with capacitive touch communication. In: Mobicom 2012, pp. 197–208 (2012)Google Scholar

Copyright information

© IFIP International Federation for Information Processing 2019

Authors and Affiliations

  • Alexandre Canny
    • 1
  • Elodie Bouzekri
    • 1
  • Célia Martinie
    • 1
  • Philippe Palanque
    • 1
    • 2
  1. 1.ICS-IRIT, Université Paul Sabatier – Toulouse IIIToulouseFrance
  2. 2.Department of Industrial DesignTechnical University EindhovenEindhovenNetherlands

Personalised recommendations