Advertisement

Model Learning and Model-Based Testing

  • Bernhard K. Aichernig
  • Wojciech Mostowski
  • Mohammad Reza MousaviEmail author
  • Martin Tappler
  • Masoumeh Taromirad
Chapter
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11026)

Abstract

We present a survey of the recent research efforts in integrating model learning with model-based testing. We distinguished two strands of work in this domain, namely test-based learning (also called test-based modeling) and learning-based testing. We classify the results in terms of their underlying models, their test purpose and techniques, and their target domains.

Notes

Acknowledgments

The insightful comments of Karl Meinke and Neil Walkinshaw on an earlier draft led to improvements and are gratefully acknowledged.

The work of B. K. Aichernig and M. Tappler was supported by the TU Graz LEAD project “Dependable Internet of Things in Adverse Environments”. The work of M. R. Mousavi and M. Taromirad has been partially supported by the Swedish Research Council (Vetenskapsradet) award number: 621-2014-5057 (Effective Model-Based Testing of Concurrent Systems) and the Strategic Research Environment ELLIIT. The work of M. R. Mousavi has also been partially supported by the Swedish Knowledge Foundation (Stiftelsen for Kunskaps- och Kompetensutveckling) in the context of the AUTO-CAAS HöG project (number: 20140312).

References

  1. [Aar14]
    Aarts, F.: Tomte: bridging the gap between active learning and real-world systems. Ph.D. thesis, Department of Computer Science (2014)Google Scholar
  2. [AdRP13]
    Aarts, F., de Ruiter, J., Poll, E.: Formal models of bank cards for free. In: Proceedings of the 2013 IEEE Sixth International Conference on Software Testing, Verification and Validation Workshops, ICSTW 2013, pp. 461–468. IEEE Computer Society, Washington, DC (2013)Google Scholar
  3. [AFBKV15]
    Aarts, F., Fiterau-Brostean, P., Kuppens, H., Vaandrager, F.: Learning register automata with fresh value generation. In: Leucker, M., Rueda, C., Valencia, F.D. (eds.) ICTAC 2015. LNCS, vol. 9399, pp. 165–183. Springer, Cham (2015).  https://doi.org/10.1007/978-3-319-25150-9_11CrossRefGoogle Scholar
  4. [AHJW06]
    Arts, T., Hughes, J., Johansson, J., Wiger, U.T.: Testing telecoms software with QuviQ QuickCheck. In: Feeley, M., Trinder, P.W. (eds.) Proceedings of the 2006 ACM SIGPLAN Workshop on Erlang, Portland, Oregon, USA, 16 September 2006, pp. 2–10. ACM (2006)Google Scholar
  5. [AHK+12]
    Aarts, F., Heidarian, F., Kuppens, H., Olsen, P., Vaandrager, F.: Automata learning through counterexample guided abstraction refinement. In: Giannakopoulou, D., Méry, D. (eds.) FM 2012. LNCS, vol. 7436, pp. 10–27. Springer, Heidelberg (2012).  https://doi.org/10.1007/978-3-642-32759-9_4CrossRefGoogle Scholar
  6. [AKR15]
    Adamis, G., Kovács, G., Réthy, G.: Generating performance test model from conformance test logs. In: Fischer, J., Scheidgen, M., Schieferdecker, I., Reed, R. (eds.) SDL 2015. LNCS, vol. 9369, pp. 268–284. Springer, Cham (2015).  https://doi.org/10.1007/978-3-319-24912-4_19CrossRefGoogle Scholar
  7. [AKT+12]
    Aarts, F., Kuppens, H., Tretmans, J., Vaandrager, F.W., Verwer, S.: Learning and testing the bounded retransmission protocol. In: Heinz, J., de la Higuera, C., Oates, T. (eds.) Proceedings of the Eleventh International Conference on Grammatical Inference, ICGI 2012, University of Maryland, College Park, USA, 5–8 September 2012, JMLR Proceedings, vol. 21, pp. 4–18. JMLR.org (2012)Google Scholar
  8. [AKT+14]
    Aarts, F., Kuppens, H., Tretmans, J., Vaandrager, F.W., Verwer, S.: Improving active mealy machine learning for protocol conformance testing. Mach. Learn. 96(1–2), 189–224 (2014)MathSciNetCrossRefGoogle Scholar
  9. [AL13]
    Ansin, R., Lundberg, D.: Automated inference of excitable cell models as hybrid automata. Bachelor thesis. School of Computer Science and Communication, KTH Stockholm (2013)Google Scholar
  10. [Alp14]
    Alpaydin, E.: Introduction to Machine Learning, 3rd edn. MIT Press, Cambridge (2014)zbMATHGoogle Scholar
  11. [Ang87]
    Angluin, D.: Learning regular sets from queries and counterexamples. Inf. Comput. 75(2), 87–106 (1987)MathSciNetCrossRefGoogle Scholar
  12. [ARM16]
    Aerts, A., Reniers, M.A., Mousavi, M.R.: Model-based testing of cyber-physical systems. In: Song, H., Rawat, D.B., Jeschke, S., Brecher, C. (eds.) Cyber-Physical Systems Foundations, Principles and Applications, Chap. 19, pp. 287–304. Elsevier (2016)Google Scholar
  13. [ASJ+16]
    Argyros, G., Stais, I., Jana, S., Keromytis, A.D., Kiayias, A.: SFADiff: automated evasion attacks and fingerprinting using black-box differential automata learning. In: Weippl, E.R., Katzenbeisser, S., Kruegel, C., Myers, A.C., Halevi, S. (eds.) Proceedings of the 2016 ACM SIGSAC Conference on Computer and Communications Security, Vienna, Austria, 24–28 October 2016, pp. 1690–1701. ACM (2016)Google Scholar
  14. [ASV10]
    Aarts, F., Schmaltz, J., Vaandrager, F.W.: Inference and abstraction of the biometric passport. In: Margaria, T., Steffen, B. (eds.) ISoLA 2010. LNCS, vol. 6415, pp. 673–686. Springer, Heidelberg (2010).  https://doi.org/10.1007/978-3-642-16558-0_54CrossRefGoogle Scholar
  15. [AT10]
    Arts, T., Thompson, S.: From test cases to FSMs: augmented test-driven development and property inference. In: Proceedings of the 9th ACM SIGPLAN Workshop on Erlang, Erlang 2010 (2010)Google Scholar
  16. [AT17a]
    Aichernig, B.K., Tappler, M.: Learning from faults: mutation testing in active automata learning. In: Barrett, C., Davies, M., Kahsai, T. (eds.) NFM 2017. LNCS, vol. 10227, pp. 19–34. Springer, Cham (2017).  https://doi.org/10.1007/978-3-319-57288-8_2CrossRefGoogle Scholar
  17. [AT17b]
    Aichernig, B.K., Tappler, M.: Probabilistic black-box reachability checking. In: Lahiri, S.K., Reger, G. (eds.) RV 2017. LNCS, vol. 10548, pp. 50–67. Springer, Cham (2017).  https://doi.org/10.1007/978-3-319-67531-2_4CrossRefGoogle Scholar
  18. [BFFH14]
    Bonzanni, N., Feenstra, K.A., Fokkink, W., Heringa, J.: Petri nets are a biologist’s best friend. In: Fages, F., Piazza, C. (eds.) FMMB 2014. LNCS, vol. 8738, pp. 102–116. Springer, Cham (2014).  https://doi.org/10.1007/978-3-319-10398-3_8
  19. [BFFK09]
    Bonzanni, N., Feenstra, K.A., Fokkink, W., Krepska, E.: What can formal methods bring to systems biology? In: Cavalcanti, A., Dams, D.R. (eds.) FM 2009. LNCS, vol. 5850, pp. 16–22. Springer, Heidelberg (2009).  https://doi.org/10.1007/978-3-642-05089-3_2CrossRefGoogle Scholar
  20. [BG96]
    Bergadano, F., Gunetti, D.: Testing by means of inductive program learning. ACM Trans. Softw. Eng. Methodol. 5(2), 119–145 (1996)CrossRefGoogle Scholar
  21. [BGJ+05]
    Berg, T., Grinchtein, O., Jonsson, B., Leucker, M., Raffelt, H., Steffen, B.: On the correspondence between conformance testing and regular inference. In: Cerioli, M. (ed.) FASE 2005. LNCS, vol. 3442, pp. 175–189. Springer, Heidelberg (2005).  https://doi.org/10.1007/978-3-540-31984-9_14CrossRefGoogle Scholar
  22. [BIPT09]
    Bertolino, A., Inverardi, P., Pelliccione, P., Tivoli, M.: Automatic synthesis of behavior protocols for composable web-services. In: van Vliet, H., Issarny, V. (eds.) Proceedings of the 7th Joint Meeting of the European Software Engineering Conference and the ACM SIGSOFT International Symposium on Foundations of Software Engineering 2009, Amsterdam, The Netherlands, 24–28 August 2009, pp. 141–150. ACM (2009)Google Scholar
  23. [CBP+11]
    Cho, C.Y., Babić, D., Poosankam, P., Chen, K.Z., Wu, E.X., Song, D.: MACE: model-inference-assisted concolic exploration for protocol and vulnerability discovery. In: Proceedings of the 20th USENIX Conference on Security. USENIX Association (2011)Google Scholar
  24. [CdlHJ09]
    Combe, D., de la Higuera, C., Janodet, J.-C.: Zulu: an interactive learning competition. In: Yli-Jyrä, A., Kornai, A., Sakarovitch, J., Watson, B.W. (eds.) FSMNLP 2009. LNCS (LNAI), vol. 6062, pp. 139–146. Springer, Heidelberg (2010).  https://doi.org/10.1007/978-3-642-14684-8_15CrossRefGoogle Scholar
  25. [CHJS14]
    Cassel, S., Howar, F., Jonsson, B., Steffen, B.: Learning extended finite state machines. In: Giannakopoulou, D., Salaün, G. (eds.) SEFM 2014. LNCS, vol. 8702, pp. 250–264. Springer, Cham (2014).  https://doi.org/10.1007/978-3-319-10431-7_18CrossRefGoogle Scholar
  26. [CHJS16]
    Cassel, S., Howar, F., Jonsson, B., Steffen, B.: Active learning for extended finite state machines. Formal Aspects Comput. 28(2), 233–263 (2016)MathSciNetCrossRefGoogle Scholar
  27. [Cho78]
    Chow, T.S.: Testing software design modeled by finite-state machines. IEEE Trans. Softw. Eng. 4(3), 178–187 (1978)CrossRefGoogle Scholar
  28. [CNS13]
    Choi, W., Necula, G.C., Sen, K.: Guided GUI testing of android apps with minimal restart and approximate learning. In: Hosking, A.L., Eugster, P.T., Lopes, C.V. (eds.) Proceedings of the 2013 ACM SIGPLAN International Conference on Object Oriented Programming Systems Languages & Applications, OOPSLA 2013, Part of SPLASH 2013, Indianapolis, IN, USA, 26–31 October 2013, pp. 623–640. ACM (2013)Google Scholar
  29. [CO94]
    Carrasco, R.C., Oncina, J.: Learning stochastic regular grammars by means of a state merging method. In: Carrasco, R.C., Oncina, J. (eds.) ICGI 1994. LNCS, vol. 862, pp. 139–152. Springer, Heidelberg (1994).  https://doi.org/10.1007/3-540-58473-0_144CrossRefzbMATHGoogle Scholar
  30. [Col14]
    Collins, P.: Model-checking in systems biology - from micro to macro. In: Fages, F., Piazza, C. (eds.) FMMB 2014. LNCS, vol. 8738, pp. 1–22. Springer, Cham (2014).  https://doi.org/10.1007/978-3-319-10398-3_1CrossRefzbMATHGoogle Scholar
  31. [CSY99]
    Câmpeanu, C., Sântean, N., Yu, S.: Minimal cover-automata for finite languages. In: Champarnaud, J.-M., Ziadi, D., Maurel, D. (eds.) WIA 1998. LNCS, vol. 1660, pp. 43–56. Springer, Heidelberg (1999).  https://doi.org/10.1007/3-540-48057-9_4CrossRefzbMATHGoogle Scholar
  32. [DIMS12]
    Dinca, I., Ipate, F., Mierla, L., Stefanescu, A.: Learn and test for Event-B – a Rodin plugin. In: Derrick, J., et al. (eds.) ABZ 2012. LNCS, vol. 7316, pp. 361–364. Springer, Heidelberg (2012).  https://doi.org/10.1007/978-3-642-30885-7_32CrossRefGoogle Scholar
  33. [DIS12]
    Dinca, I., Ipate, F., Stefanescu, A.: Model learning and test generation for Event-B decomposition. In: Margaria, T., Steffen, B. (eds.) ISoLA 2012. LNCS, vol. 7609, pp. 539–553. Springer, Heidelberg (2012).  https://doi.org/10.1007/978-3-642-34026-0_40CrossRefGoogle Scholar
  34. [DLDvL08]
    Dupont, P., Lambeau, B., Damas, C., van Lamsweerde, A.: The QSM algorithm and its application to software behavior model induction. Appl. Artif. Intell. 22(1–2), 77–115 (2008)CrossRefGoogle Scholar
  35. [dRP15]
    de Ruiter, J., Poll, E.: Protocol state fuzzing of TLS implementations. In: Jung, J., Holz, T. (eds.) 24th USENIX Security Symposium, USENIX Security 15, Washington, D.C., USA, 12–14 August 2015, pp. 193–206. USENIX Association (2015)Google Scholar
  36. [EGPQ06]
    Elkind, E., Genest, B., Peled, D.A., Qu, H.: Grey-box checking. In: Najm, E., Pradat-Peyre, J.-F., Donzeau-Gouge, V.V. (eds.) FORTE 2006. LNCS, vol. 4229, pp. 420–435. Springer, Heidelberg (2006).  https://doi.org/10.1007/11888116_30CrossRefGoogle Scholar
  37. [FBJV14]
    Fiterău-Broştean, P., Janssen, R., Vaandrager, F.W.: Learning fragments of the TCP network protocol. In: Lang, F., Flammini, F. (eds.) FMICS 2014. LNCS, vol. 8718, pp. 78–93. Springer, Cham (2014).  https://doi.org/10.1007/978-3-319-10702-8_6CrossRefGoogle Scholar
  38. [FBJV16]
    Fiterău-Broştean, P., Janssen, R., Vaandrager, F.W.: Combining model learning and model checking to analyze TCP implementations. In: Chaudhuri, S., Farzan, A. (eds.) CAV 2016. LNCS, vol. 9780, pp. 454–471. Springer, Cham (2016).  https://doi.org/10.1007/978-3-319-41540-6_25CrossRefGoogle Scholar
  39. [FBLP+17]
    Fiterău-Broştean, P., Lenaerts, T., Poll, E., de Ruiter, J., Vaandrager, F.W., Verleg, P.: Model learning and model checking of SSH implementations. In: Erdogmus, H., Havelund, K. (eds.) Proceedings of the 24th ACM SIGSOFT International SPIN Symposium on Model Checking of Software, Santa Barbara, CA, USA, 10–14 July 2017, pp. 142–151. ACM (2017)Google Scholar
  40. [FvBK+91]
    Fujiwara, S., von Bochmann, G., Khendek, F., Amalou, M., Ghedamsi, A.: Test selection based on finite state models. IEEE Trans. Softw. Eng. 17(6), 591–603 (1991)CrossRefGoogle Scholar
  41. [GLPS08]
    Groz, R., Li, K., Petrenko, A., Shahbaz, M.: Modular system verification by inference, testing and reachability analysis. In: Suzuki, K., Higashino, T., Ulrich, A., Hasegawa, T. (eds.) FATES/TestCom -2008. LNCS, vol. 5047, pp. 216–233. Springer, Heidelberg (2008).  https://doi.org/10.1007/978-3-540-68524-1_16CrossRefGoogle Scholar
  42. [GPY02a]
    Groce, A., Peled, D.A., Yannakakis, M.: Adaptive model checking. In: Katoen, J.-P., Stevens, P. (eds.) TACAS 2002. LNCS, vol. 2280, pp. 357–370. Springer, Heidelberg (2002).  https://doi.org/10.1007/3-540-46002-0_25CrossRefGoogle Scholar
  43. [GPY02b]
    Groce, A., Peled, D.A., Yannakakis, M.: AMC: an adaptive model checker. In: Brinksma, E., Larsen, K.G. (eds.) CAV 2002. LNCS, vol. 2404, pp. 521–525. Springer, Heidelberg (2002).  https://doi.org/10.1007/3-540-45657-0_44CrossRefGoogle Scholar
  44. [GS16]
    Gebizli, C.Ş., Sözer, H.: Automated refinement of models for model-based testing using exploratory testing. Softw. Qual. J. 25(3), 1–27 (2016)Google Scholar
  45. [HGOR14]
    Hossen, K., Groz, R., Oriat, C., Richier, J.-L.: Automatic model inference of web applications for security testing. In: Seventh IEEE International Conference on Software Testing, Verification and Validation, ICST 2014 Workshops Proceedings, 31 March–4 April 2014, Cleveland, Ohio, USA, pp. 22–23. IEEE Computer Society (2014)Google Scholar
  46. [HHNS02]
    Hagerer, A., Hungar, H., Niese, O., Steffen, B.: Model generation by moderated regular extrapolation. In: Kutsche, R.-D., Weber, H. (eds.) FASE 2002. LNCS, vol. 2306, pp. 80–95. Springer, Heidelberg (2002).  https://doi.org/10.1007/3-540-45923-5_6CrossRefGoogle Scholar
  47. [HK08]
    Hung, P.N., Katayama, T.: Modular conformance testing and assume-guarantee verification for evolving component-based software. In: 15th Asia-Pacific Software Engineering Conference (APSEC 2008), 3–5 December 2008, Beijing, China, pp. 479–486. IEEE Computer Society (2008)Google Scholar
  48. [HNS03]
    Hungar, H., Niese, O., Steffen, B.: Domain-specific optimization in automata learning. In: Hunt, W.A., Somenzi, F. (eds.) CAV 2003. LNCS, vol. 2725, pp. 315–327. Springer, Heidelberg (2003).  https://doi.org/10.1007/978-3-540-45069-6_31CrossRefGoogle Scholar
  49. [HRD07]
    Henkel, J., Reichenbach, C., Diwan, A.: Discovering documentation for Java container classes. IEEE Trans. Softw. Eng. 33(8), 526–543 (2007)CrossRefGoogle Scholar
  50. [HSL08]
    Hsu, Y., Shu, G., Lee, D.: A model-based approach to security flaw detection of network protocol implementations. In: Proceedings of the 16th Annual IEEE International Conference on Network Protocols, ICNP 2008, Orlando, Florida, USA, 19–22 October 2008, pp. 114–123. IEEE Computer Society (2008)Google Scholar
  51. [HSM10]
    Howar, F., Steffen, B., Merten, M.: From ZULU to RERS. In: Margaria, T., Steffen, B. (eds.) ISoLA 2010. LNCS, vol. 6415, pp. 687–704. Springer, Heidelberg (2010).  https://doi.org/10.1007/978-3-642-16558-0_55CrossRefGoogle Scholar
  52. [HSM11]
    Howar, F., Steffen, B., Merten, M.: Automata learning with automated alphabet abstraction refinement. In: Jhala, R., Schmidt, D.A. (eds.) VMCAI 2011. LNCS, vol. 6538, pp. 263–277. Springer, Heidelberg (2011).  https://doi.org/10.1007/978-3-642-18275-4_19CrossRefGoogle Scholar
  53. [IHS15]
    Isberner, M., Howar, F., Steffen, B.: The open-source LearnLib. In: Kroening, D., Păsăreanu, C.S. (eds.) CAV 2015. LNCS, vol. 9206, pp. 487–495. Springer, Cham (2015).  https://doi.org/10.1007/978-3-319-21690-4_32CrossRefGoogle Scholar
  54. [ISD15]
    Ipate, F., Stefanescu, A., Dinca, I.: Model learning and test generation using cover automata. Comput. J. 58(5), 1140–1159 (2015)CrossRefGoogle Scholar
  55. [KMMV16]
    Kunze, S., Mostowski, W., Mousavi, M.R., Varshosaz, M.: Generation of failure models through automata learning. In: Workshop on Automotive Systems/Software Architectures (WASA 2016), pp. 22–25. IEEE Computer Society, April 2016Google Scholar
  56. [KMR]
    Khosrowjerdi, H., Meinke, K., Rasmusson, A.: Automated behavioral requirements testing for automotive ECU applications (2016, Submitted)Google Scholar
  57. [KV94]
    Kearns, M.J., Vazirani, U.V.: An Introduction to Computational Learning Theory. MIT Press, Cambridge (1994)Google Scholar
  58. [LCJ06]
    Lai, Z., Cheung, S.C., Jiang, Y.: Dynamic model learning using genetic algorithm under adaptive model checking framework. In: Sixth International Conference on Quality Software (QSIC 2006), 26–28 October 2006, Beijing, China, pp. 410–417. IEEE Computer Society (2006)Google Scholar
  59. [LGS06a]
    Li, K., Groz, R., Shahbaz, M.: Integration testing of components guided by incremental state machine learning. In: McMinn, P. (ed.) Testing: Academia and Industry Conference - Practice and Research Techniques (TAIC PART 2006), 29–31 August 2006, Windsor, United Kingdom, pp. 59–70. IEEE Computer Society (2006)Google Scholar
  60. [LGS06b]
    Li, K., Groz, R., Shahbaz, M.: Integration testing of distributed components based on learning parameterized I/O models. In: Najm, E., Pradat-Peyre, J.-F., Donzeau-Gouge, V.V. (eds.) FORTE 2006. LNCS, vol. 4229, pp. 436–450. Springer, Heidelberg (2006).  https://doi.org/10.1007/11888116_31CrossRefzbMATHGoogle Scholar
  61. [LS14]
    Lachmann, R., Schaefer, I.: Towards efficient and effective testing in automotive software development. In: Plödereder, E., Grunske, L., Schneider, E., Ull, D. (eds.) 44. Jahrestagung der Gesellschaft für Informatik, Informatik 2014, Big Data - Komplexität meistern, 22–26 September 2014, Stuttgart, Deutschland. LNI, vol. 232, pp. 2181–2192. GI (2014)Google Scholar
  62. [LY94]
    Lee, D., Yannakakis, M.: Testing finite-state machines: state identification and verification. IEEE Trans. Comput. 43(3), 306–320 (1994)MathSciNetCrossRefGoogle Scholar
  63. [MCWKK09]
    Comparetti, P.M., Wondracek, G., Krügel, C., Kirda, E.: Prospex: protocol specification extraction. In: 30th IEEE Symposium on Security and Privacy (S&P 2009), 17–20 May 2009, Oakland, California, USA, pp. 110–125. IEEE Computer Society (2009)Google Scholar
  64. [Mei04]
    Meinke, K.: Automated black-box testing of functional correctness using function approximation. SIGSOFT Softw. Eng. Notes 29(4), 143–153 (2004)CrossRefGoogle Scholar
  65. [MHR+06]
    Margaria, T., Hinchey, M.G., Raffelt, H., Rash, J.L., Rouff, C.A., Steffen, B.: Completing and adapting models of biological processes. In: Pan, Y., Rammig, F.J., Schmeck, H., Solar, M. (eds.) BICC 2006. IIFIP, vol. 216, pp. 43–54. Springer, Boston, MA (2006).  https://doi.org/10.1007/978-0-387-34733-2_5CrossRefGoogle Scholar
  66. [Mit97]
    Mitchel, T.M.: Machine Learning. McGraw Hill, New York (1997)Google Scholar
  67. [MN10]
    Meinke, K., Niu, F.: A learning-based approach to unit testing of numerical software. In: Petrenko, A., Simão, A., Maldonado, J.C. (eds.) ICTSS 2010. LNCS, vol. 6435, pp. 221–235. Springer, Heidelberg (2010).  https://doi.org/10.1007/978-3-642-16573-3_16CrossRefGoogle Scholar
  68. [MN15]
    Meinke, K., Nycander, P.: Learning-based testing of distributed microservice architectures: correctness and fault injection. In: Bianculli, D., Calinescu, R., Rumpe, B. (eds.) SEFM 2015. LNCS, vol. 9509, pp. 3–10. Springer, Heidelberg (2015).  https://doi.org/10.1007/978-3-662-49224-6_1CrossRefGoogle Scholar
  69. [MNRS04]
    Margaria, T., Niese, O., Raffelt, H., Steffen, B.: Efficient test-based model generation for legacy reactive systems. In: 2004 Ninth IEEE International High-Level Design Validation and Test Workshop, pp. 95–100. IEEE (2004)Google Scholar
  70. [MPS+09]
    Mostowski, W., Poll, E., Schmaltz, J., Tretmans, J., Wichers Schreur, R.: Model-based testing of electronic passports. In: Alpuente, M., Cook, B., Joubert, C. (eds.) FMICS 2009. LNCS, vol. 5825, pp. 207–209. Springer, Heidelberg (2009).  https://doi.org/10.1007/978-3-642-04570-7_19CrossRefGoogle Scholar
  71. [MS11]
    Meinke, K., Sindhu, M.A.: Incremental learning-based testing for reactive systems. In: Gogolla, M., Wolff, B. (eds.) TAP 2011. LNCS, vol. 6706, pp. 134–151. Springer, Heidelberg (2011).  https://doi.org/10.1007/978-3-642-21768-5_11CrossRefGoogle Scholar
  72. [MSB11]
    Myers, G.J., Sandler, C., Badgett, T.: The Art of Software Testing, 3rd edn. Wiley Publishing, Hoboken (2011)Google Scholar
  73. [Nie03]
    Niese, O.: An integrated approach to testing complex systems. Ph.D. thesis, Dortmund University of Technology (2003)Google Scholar
  74. [OG92]
    Oncina, J., Garcia, P.: Identifying regular languages in polynomial time. In: Advances in Structural and Syntactic Pattern Recognition. Series in Machine Perception and Artificial Intelligence, vol. 5, pp. 99–108. World Scientific (1992)Google Scholar
  75. [ORT+07]
    Oostdijk, M., Rusu, V., Tretmans, J., de Vries, R.G., Willemse, T.A.C.: Integrating verification, testing, and learning for cryptographic protocols. In: Davies, J., Gibbons, J. (eds.) IFM 2007. LNCS, vol. 4591, pp. 538–557. Springer, Heidelberg (2007).  https://doi.org/10.1007/978-3-540-73210-5_28CrossRefGoogle Scholar
  76. [PLG+14]
    Petrenko, A., Li, K., Groz, R., Hossen, K., Oriat, C.: Inferring approximated models for systems engineering. In: 15th International IEEE Symposium on High-Assurance Systems Engineering, HASE 2014, Miami Beach, FL, USA, 9–11 January 2014, pp. 249–253. IEEE Computer Society (2014)Google Scholar
  77. [PVY99]
    Peled, D., Vardi, M.Y., Yannakakis, M.: Black box checking. In: Wu, J., Chanson, S.T., Gao, Q. (eds.) PSTV 1999, FORTE 1999. IAICT, vol. 28, pp. 225–240. Springer, Boston, MA (1999).  https://doi.org/10.1007/978-0-387-35578-8_13CrossRefGoogle Scholar
  78. [PW15]
    Papadopoulos, P., Walkinshaw, N.: Black-box test generation from inferred models. In: Harrison, R., Bener, A.B., Turhan, B. (eds.) 4th IEEE/ACM International Workshop on Realizing Artificial Intelligence Synergies in Software Engineering, RAISE 2015, Florence, Italy, 17 May 2015, pp. 19–24. IEEE Computer Society (2015)Google Scholar
  79. [RMSM08]
    Raffelt, H., Margaria, T., Steffen, B., Merten, M.: Hybrid test of web applications with webtest. In: Bultan, T., Xie, T. (eds.) Proceedings of the 2008 Workshop on Testing, Analysis, and Verification of Web Services and Applications, Held in Conjunction with the ACM SIGSOFT International Symposium on Software Testing and Analysis (ISSTA 2008), TAV-WEB 2008, Seattle, Washington, USA, 21 July 2008, pp. 1–7. ACM (2008)Google Scholar
  80. [RMSM09]
    Raffelt, H., Merten, M., Steffen, B., Margaria, T.: Dynamic testing via automata learning. STTT 11(4), 307–324 (2009)CrossRefGoogle Scholar
  81. [RS93]
    Rivest, R.L., Schapire, R.E.: Inference of finite automata using homing sequences. Inf. Comput. 103(2), 299–347 (1993)MathSciNetCrossRefGoogle Scholar
  82. [SAP+17]
    Sivakorn, S., Argyros, G., Pei, K., Keromytis, A.D., Jana, S.: HVLearn: automated black-box analysis of hostname verification in SSL/TLS implementations. In: 2017 IEEE Symposium on Security and Privacy, SP 2017, San Jose, CA, USA, 22–26 May 2017, pp. 521–538. IEEE Computer Society (2017)Google Scholar
  83. [SG09]
    Shahbaz, M., Groz, R.: Inferring mealy machines. In: Cavalcanti, A., Dams, D.R. (eds.) FM 2009. LNCS, vol. 5850, pp. 207–222. Springer, Heidelberg (2009).  https://doi.org/10.1007/978-3-642-05089-3_14CrossRefGoogle Scholar
  84. [SG14]
    Shahbaz, M., Groz, R.: Analysis and testing of black-box component-based systems by inferring partial models. Softw. Test. Verification Reliab. 24(4), 253–288 (2014)CrossRefGoogle Scholar
  85. [SHL08]
    Shu, G., Hsu, Y., Lee, D.: Detecting communication protocol security flaws by formal fuzz testing and machine learning. In: Suzuki, K., Higashino, T., Yasumoto, K., El-Fakih, K. (eds.) FORTE 2008. LNCS, vol. 5048, pp. 299–304. Springer, Heidelberg (2008).  https://doi.org/10.1007/978-3-540-68855-6_19CrossRefGoogle Scholar
  86. [SHM11]
    Steffen, B., Howar, F., Merten, M.: Introduction to active automata learning from a practical perspective. In: Bernardo, M., Issarny, V. (eds.) SFM 2011. LNCS, vol. 6659, pp. 256–296. Springer, Heidelberg (2011).  https://doi.org/10.1007/978-3-642-21455-4_8CrossRefGoogle Scholar
  87. [SL07]
    Shu, G., Lee, D.: Testing security properties of protocol implementations - a machine learning based approach. In: 27th IEEE International Conference on Distributed Computing Systems (ICDCS 2007), 25–29 June 2007, Toronto, Ontario, Canada, p. 25. IEEE Computer Society (2007)Google Scholar
  88. [SLBW15]
    Schulze, C., Lindvall, M., Bjorgvinsson, S., Wiegand, R.: Model generation to support model-based testing applied on the NASA DAT web-application - an experience report. In: 26th IEEE International Symposium on Software Reliability Engineering, ISSRE 2015, Gaithersbury, MD, USA, 2–5 November 2015, pp. 77–87. IEEE Computer Society (2015)Google Scholar
  89. [SLG07a]
    Shahbaz, M., Li, K., Groz, R.: Learning and integration of parameterized components through testing. In: Petrenko, A., Veanes, M., Tretmans, J., Grieskamp, W. (eds.) FATES/TestCom - 2007. LNCS, vol. 4581, pp. 319–334. Springer, Heidelberg (2007).  https://doi.org/10.1007/978-3-540-73066-8_22CrossRefGoogle Scholar
  90. [SLG07b]
    Shahbaz, M., Li, K., Groz, R.: Learning parameterized state machine model for integration testing. In: 31st Annual International Computer Software and Applications Conference, COMPSAC 2007, Beijing, China, 24–27 July 2007, vol. 2, pp. 755–760. IEEE Computer Society (2007)Google Scholar
  91. [SMVJ15]
    Smeenk, W., Moerman, J., Vaandrager, F.W., Jansen, D.N.: Applying automata learning to embedded control software. In: Butler, M., Conchon, S., Zaïdi, F. (eds.) ICFEM 2015. LNCS, vol. 9407, pp. 67–83. Springer, Cham (2015).  https://doi.org/10.1007/978-3-319-25423-4_5CrossRefGoogle Scholar
  92. [SPK07]
    Shahbaz, M., Parreaux, B., Klay, F.: Model inference approach for detecting feature interactions in integrated systems. In: du Bousquet, L., Richier, J.-L. (eds.) Feature Interactions in Software and Communication Systems IX, International Conference on Feature Interactions in Software and Communication Systems, ICFI 2007, 3–5 September 2007, Grenoble, France, pp. 161–171. IOS Press (2007)Google Scholar
  93. [TAB17]
    Tappler, M., Aichernig, B.K., Bloem, R.: Model-based testing IoT communication via active automata learning. In: 2017 IEEE International Conference on Software Testing, Verification and Validation, ICST 2017, Tokyo, Japan, 13–17 March 2017, pp. 276–287 (2017)Google Scholar
  94. [TB03]
    Tretmans, J., Brinksma, E.: TorX: automated model-based testing. In: Hartman, A., Dussa-Ziegler, K. (eds.) First European Conference on Model-Driven Software Engineering, pp. 31–43, December 2003Google Scholar
  95. [Tre11]
    Tretmans, J.: Model-based testing and some steps towards test-based modelling. In: Bernardo, M., Issarny, V. (eds.) SFM 2011. LNCS, vol. 6659, pp. 297–326. Springer, Heidelberg (2011).  https://doi.org/10.1007/978-3-642-21455-4_9CrossRefGoogle Scholar
  96. [UL07]
    Utting, M., Legeard, B.: Practical Model-Based Testing - A Tools Approach. Morgan Kaufmann, Burlington (2007)Google Scholar
  97. [UPL12]
    Utting, M., Pretschner, A., Legeard, B.: A taxonomy of model-based testing approaches. Softw. Test. Verification Reliab. 22(5), 297–312 (2012)CrossRefGoogle Scholar
  98. [Vas73]
    Vasilevskii, M.P.: Failure diagnosis of automata. Cybernetics 9(4), 653–665 (1973)MathSciNetCrossRefGoogle Scholar
  99. [VT14]
    Volpato, M., Tretmans, J.: Active learning of nondeterministic systems from an ioco perspective. In: Margaria, T., Steffen, B. (eds.) ISoLA 2014. LNCS, vol. 8802, pp. 220–235. Springer, Heidelberg (2014).  https://doi.org/10.1007/978-3-662-45234-9_16CrossRefGoogle Scholar
  100. [VT15]
    Volpato, M., Tretmans, J.: Approximate active learning of nondeterministic input output transition systems. ECEASST 72 (2015)Google Scholar
  101. [WBDP10]
    Walkinshaw, N., Bogdanov, K., Derrick, J., Paris, J.: Increasing functional coverage by inductive testing: a case study. In: Petrenko, A., Simão, A., Maldonado, J.C. (eds.) ICTSS 2010. LNCS, vol. 6435, pp. 126–141. Springer, Heidelberg (2010).  https://doi.org/10.1007/978-3-642-16573-3_10CrossRefGoogle Scholar
  102. [WBHS07]
    Walkinshaw, N., Bogdanov, K., Holcombe, M., Salahuddin, S.: Reverse engineering state machines by interactive grammar inference. In: 14th Working Conference on Reverse Engineering (WCRE 2007), 28–31 October 2007, Vancouver, BC, Canada, pp. 209–218. IEEE Computer Society (2007)Google Scholar
  103. [WDG09]
    Walkinshaw, N., Derrick, J., Guo, Q.: Iterative refinement of reverse-engineered models by model-based testing. In: Cavalcanti, A., Dams, D.R. (eds.) FM 2009. LNCS, vol. 5850, pp. 305–320. Springer, Heidelberg (2009).  https://doi.org/10.1007/978-3-642-05089-3_20CrossRefGoogle Scholar
  104. [Wey83]
    Weyuker, E.J.: Assessing test data adequacy through program inference. ACM Trans. Program. Lang. Syst. 5(4), 641–655 (1983)CrossRefGoogle Scholar
  105. [WF17]
    Walkinshaw, N., Fraser, G.: Uncertainty-driven black-box test data generation. In: 2017 IEEE International Conference on Software Testing, Verification and Validation, ICST 2017, Tokyo, Japan, 13–17 March 2017, pp. 253–263 (2017)Google Scholar
  106. [YCM09]
    Yeh, T., Chang, T.-H., Miller, R.C.: Sikuli: using GUI screenshots for search and automation. In: Proceedings of the 22nd Annual ACM Symposium on User Interface Software and Technology, pp. 183–192. ACM (2009)Google Scholar

Copyright information

© Springer International Publishing AG, part of Springer Nature 2018

Authors and Affiliations

  • Bernhard K. Aichernig
    • 1
  • Wojciech Mostowski
    • 2
  • Mohammad Reza Mousavi
    • 2
    • 3
    Email author
  • Martin Tappler
    • 1
  • Masoumeh Taromirad
    • 2
  1. 1.Institute of Software TechnologyGraz University of TechnologyGrazAustria
  2. 2.Centre for Research on Embedded SystemsHalmstad UniversityHalmstadSweden
  3. 3.Department of InformaticsUniversity of LeicesterLeicesterUK

Personalised recommendations