Empirical Software Engineering

, Volume 21, Issue 3, pp 811–853 | Cite as

Inferring extended finite state machine models from software executions

Article

Abstract

The ability to reverse-engineer models of software behaviour is valuable for a wide range of software maintenance, validation and verification tasks. Current reverse-engineering techniques focus either on control-specific behaviour (e.g., in the form of Finite State Machines), or data-specific behaviour (e.g., as pre / post-conditions or invariants). However, typical software behaviour is usually a product of the two; models must combine both aspects to fully represent the software’s operation. Extended Finite State Machines (EFSMs) provide such a model. Although attempts have been made to infer EFSMs, these have been problematic. The models inferred by these techniques can be non-deterministic, the inference algorithms can be inflexible, and only applicable to traces with specific characteristics. This paper presents a novel EFSM inference technique that addresses the problems of inflexibility and non-determinism. It also adapts an experimental technique from the field of Machine Learning to evaluate EFSM inference techniques, and applies it to three diverse software systems.

Keywords

Reverse engineering EFSMs Dynamic analysis 

References

  1. Aarts F, Heidarian F, Kuppens H, Olsen P, Vaandrager F (2012) Automata learning through counterexample-guided abstraction refinement. In: In Proceedings FM 2012, 18th International Symposium on Formal MethodsGoogle Scholar
  2. Ammons G, Bodík R, Larus JR (2002) Mining specifications. In: POPL 2002, Portland, Oregon, pp 4–16Google Scholar
  3. Androutsopoulos K, Gold N, Harman M, Li Z, Tratt L (2009) A theoretical and empirical study of EFSM dependence. In: 2009 IEEE International Conference on Software Maintenance, ICSM 2009. IEEE, pp 287–296Google Scholar
  4. Angluin D (1987) Learning Regular Sets from Queries and Counterexamples. Inf Comput 75:87–106MathSciNetCrossRefMATHGoogle Scholar
  5. Arts T, Earle CB, Derrick J (2004) Development of a verified Erlang program for resource locking. Int J Softw Tools Technol Transfer 5(2–3):205–220CrossRefGoogle Scholar
  6. Biermann AW, Feldman JA (1972) On the synthesis of finite-state machines from samples of their behaviour. IEEE Trans Comput C 21:592–597MathSciNetCrossRefMATHGoogle Scholar
  7. Börger E, Stärk RF (2003) Abstract State Machines: A Method for High-level System Design and Analysis. SpringerGoogle Scholar
  8. Lindig CVD, Wasylkowski A, Zeller A (2006) Mining object behavior with ADABU. In: Proceedings of the 2006 international workshop on Dynamic systems analysis. ACM, pp 17–24Google Scholar
  9. Cesarini F, Thompson S (2011) Erlang by Example. O’Reilly MediaGoogle Scholar
  10. Cheng K, Krishnakumar A (1993) Automatic functional test generation using the extended finite state machine model. In: 30th Conference on Design Automation. ACM, pp 86–91Google Scholar
  11. Clarke E, Grumberg O, Jha S, Lu Y, Veith H (2000) Counterexample-guided abstraction refinement. In: Computer aided verification. Springer, pp 154–169Google Scholar
  12. Cook J, Wolf A (1998) Discovering models of software processes from event-based data. ACM Trans Softw Eng Methodol 7(3):215–249CrossRefGoogle Scholar
  13. Dallmeier V, Knopp N, Mallon C, Fraser G, Hack S, Zeller A (2012) Automatically generating test cases for specification mining. IEEE Trans Softw Eng 38(2):243–257CrossRefGoogle Scholar
  14. Damas C, Lambeau B, Dupont P, van Lamsweerde A (2005) Generating annotated behavior models from end-user scenarios. IEEE Trans Softw Eng 31(12)Google Scholar
  15. Damm W, Harel D (2001) Lscs: Breathing life into message sequence charts. Formal Methods in System Design 19(1):45–80CrossRefMATHGoogle Scholar
  16. De La Higuera C (2005) A bibliographical study of grammatical inference. Pattern Recog 38(9):1332–1348CrossRefGoogle Scholar
  17. Ernst MD, Cockrell J, Griswold WG, Notkin D (2001) Dynamically discovering likely program invariants to support program evolution. IEEE Trans Softw Eng 27(2):1–25CrossRefGoogle Scholar
  18. Fraser G, Walkinshaw N (2012) Behaviourally adequate software testing. In: Software Testing, Verification and Validation (ICST) 2012. IEEE, pp 300–309Google Scholar
  19. Freund Y, Schapire R (1995) A desicion-theoretic generalization of on-line learning and an application to boosting. In: Computational learning theory. Springer, pp 23–37Google Scholar
  20. Gold EM (1967) Language identification in the limit. Inf Control 10:447–474CrossRefMATHGoogle Scholar
  21. Gransden T, Walkinshaw N, Raman R (2014) Mining State-Based Models from Proof Corpora. In: Proceedings of Conferences on Intelligence Mathematics - Mathematical Knowledge Management Track - CICM’14, vol 8543Google Scholar
  22. Hall M, Frank E, Holmes G, Pfahringer B, Reutemann P, Witten IH (2009) The weka data mining software: an update. SIGKDD Explor Newsl 11:10–18CrossRefGoogle Scholar
  23. Hierons RM, Bogdanov K, Bowen JP, Cleaveland R, Derrick J, Dick J, Gheorghe M, Harman M, Kapoor K, Krause P et al (2009) Using formal specifications to support testing. ACM Comput Surv (CSUR) 41(2):9CrossRefGoogle Scholar
  24. Holcombe M (1988) X-machines as a basis for dynamic system specification. Softw Eng J 3(2):69– 76CrossRefGoogle Scholar
  25. Howar F, Steffen B, Jonsson B, Cassel S (2012) Inferring canonical register automata. In: Verification, Model Checking, and Abstract Interpretation. Springer, pp 251–266Google Scholar
  26. Howden WE (1982) Weak mutation testing and completeness of test sets. IEEE Trans Softw Eng 4:371–379CrossRefGoogle Scholar
  27. Just R, Schweiggert F, Kapfhammer GM (2011) MAJOR: An efficient and extensible tool for mutation analysis in a Java compiler. In: Automated Software Engineering (ASE). IEEE/ACM, pp 612–615Google Scholar
  28. Kohavi R (1995) A study of cross-validation and bootstrap for accuracy estimation and model selection. In: International joint Conference on artificial intelligence, vol 14. Morgan Kaufmann Publishers Inc., pp 1137–1145Google Scholar
  29. Kramer J, Magee J, Sloman M, Lister A (1983) Conic: an integrated approach to distributed computer control systems. IEE Proc 130(1):1–10CrossRefGoogle Scholar
  30. Krka I, Brun Y, Medvidovic N (2014) Automatic mining of specifications from invocation traces and method invariants. In: ACM SIGSOFT International Symposium on Foundations of Software Engineering (FSE), Hong Kong, ChinaGoogle Scholar
  31. Lang KJ, Pearlmutter BA, Price RA (1998) Results of the Abbadingo One DFA learning competition and a new evidence-driven state merging algorithm. In: Honavar V, Slutzki G (eds) Proceedings of the 4th International Colloquium on Grammatical Inference, vol 1433. Springer-Verlag, pp 1–12Google Scholar
  32. Lee C, Chen F, Roşu G (2011) Mining parametric specifications. In: Proceedings of the 33rd International Conference on Software Engineering. ACM, pp 591–600Google Scholar
  33. Li H, Thompson S (2011) A User-extensible Refactoring Tool for Erlang Programs. Tech. rep., University of Kent, http://www.cs.kent.ac.uk/pubs/2011/3171
  34. Lo D, Khoo SC (2006) QUARK: Empirical assessment of automaton-based specification miners. In: 2006 IEEE Computer Society on Reverse Engineering, (WCRE’06), pp 51–60Google Scholar
  35. Lo D, Maoz S (2012) Scenario-based and value-based specification mining: better together. Autom Softw Eng 19(4):423–458CrossRefGoogle Scholar
  36. Lo D, Cheng H, Han J, Khoo SC, Sun C (2009) Classification of software behaviors for failure detection: a discriminative pattern mining approach. In: Proceedings of the 15th ACM SIGKDD international conference on Knowledge discovery and data mining. ACM, pp 557–566Google Scholar
  37. Lo D, Mariani L, Santoro M (2012) Learning extended FSA from software: An empirical assessment. J Syst Softw 85(9):2063–2076. doi10.1016/j.jss.2012.04.001 CrossRefGoogle Scholar
  38. Lorenzoli D, Mariani L, Pezzè M (2008) Automatic generation of software behavioral models. In: 2008 ACM/IEEE 30th International Conference on Software Engineering, (ICSE’08). ACM, pp 501– 510Google Scholar
  39. Mitchell T (1997) Machine Learning. McGraw-HillGoogle Scholar
  40. Quinlan JR (1993) C4.5: Programs for Machine Learning. Morgan Kaufmann, San MateoGoogle Scholar
  41. Sokolova M, Lapalme G (2009) A systematic analysis of performance measures for classification tasks. Inf Process Manag 45(4):427–437CrossRefGoogle Scholar
  42. Taylor R, Hall M, Bogdanov K, Derrick J (2012) Using behaviour inference to optimise regression test sets. In: Testing Software and Systems (ICTSS’12). Springer, pp 184–199Google Scholar
  43. Valdes A, Skinner K (2000) Adaptive, model-based monitoring for cyber attack detection. In: Recent Advances in Intrusion Detection. Springer, pp 80–93Google Scholar
  44. Valiant L (1984) A theory of the learnable. Commun ACM 27(11):1134–1142CrossRefMATHGoogle Scholar
  45. Walkinshaw N, Bogdanov K (2013) Automated comparison of state-based software models in terms of their language and structure. ACM Trans Softw Eng Methodol 22 (2)Google Scholar
  46. Walkinshaw N, Bogdanov K, Holcombe M, Salahuddin S (2007) Reverse engineering state machines by interactive grammar inference. In: 2007 14th Working Conference on Reverse Engineering, WCRE 2007. IEEE, pp 209–218Google Scholar
  47. Walkinshaw N, Bogdanov K, Ali S, Holcombe M (2008) Automated discovery of state transitions and their functions in source code. Software Testing. Verification and Reliability (STVR) 18(2):99– 121CrossRefGoogle Scholar
  48. Walkinshaw N, Derrick J, Guo Q (2009) Iterative refinement of reverse-engineered models by model-based testing. In: International conference on Formal Methods (FM’09). Springer, pp 305–320Google Scholar
  49. Walkinshaw N, Bogdanov K, Derrick J, Paris J (2010) Increasing functional coverage by inductive testing: A case study. In: Testing Software and Systems (ICTSS’10), pp 126–141Google Scholar
  50. Walkinshaw N, Lambeau B, Damas C, Bogdanov K, Dupont P (2012) STAMINA: a competition to encourage the development and assessment of software model inference techniques. Empir Softw Eng:1–34Google Scholar
  51. Walkinshaw N, Taylor R, Derrick J (2013) Inferring extended finite state machine models from software executions. In: 2013 20th Working Conference on Reverse Engineering (WCRE). IEEE, pp 301–310Google Scholar
  52. Weiss SM, Kapouleas I (1989) An empirical comparison of pattern recognition, neural nets, and machine learning classification methods. In: Proceedings of the Eleventh International Joint Conference on Artificial Intelligence. Morgan Kaufmann, pp 781–787Google Scholar
  53. Wolpert DH (1996) The lack of a priori distinctions between learning algorithms. Neural comput 8(7):1341–1390CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media New York 2015

Authors and Affiliations

  1. 1.Department of Computer ScienceThe University of LeicesterLeicesterUK
  2. 2.Department of Computer ScienceThe University of SheffieldSheffieldUK

Personalised recommendations