Empirical Software Engineering

, Volume 21, Issue 6, pp 2459–2502 | Cite as

Improving the performance of OCL constraint solving with novel heuristics for logical operations: a search-based approach

  • Shaukat Ali
  • Muhammad Zohaib Iqbal
  • Maham Khalid
  • Andrea Arcuri
Article

Abstract

A common practice to specify constraints on the Unified Modeling Language (UML) models is using the Object Constraint Language (OCL). Such constraints serve various purposes, ranging from simply providing precise meaning to the models to supporting complex verification and validation activities. In many applications, these constraints have to be solved to obtain values satisfying the constraints, for example, in the case of model-based testing (MBT) to generate test data for the purpose of generating executable test cases. In our previous work, we proposed novel heuristics for various OCL constructs to efficiently solve them using search algorithms. These heuristics are enhanced in this paper to further improve the performance of OCL constraint solving. We performed an empirical evaluation comprising of three case studies using three search algorithms: Alternating Variable Method (AVM), (1 + 1) Evolutionary Algorithm (EA), and a Genetic Algorithm (GA) and in addition Random Search (RS) was used as a comparison baseline. In the first case study, we evaluated each heuristics using carefully designed artificial problems. In the second case study, we evaluated the heuristics on various constraints of Cisco’s Video Conferencing Systems defined to support MBT. Finally, the third case study is about EU-Rent Car Rental specification and is obtained from the literature. The results of the empirical evaluation showed that (1 + 1) EA and AVM with the improved heuristics significantly outperform the rest of the algorithms.

Keywords

UML OCL Search-based testing Test data Empirical evaluation 

References

  1. Aertryck Lv, Jensen T (2003) UML-Casting: test synthesis from UML models using constraint resolution. Paper presented at the Approches Formelles dans l’Assistance au Développement de Logiciels (AFADL’2003)Google Scholar
  2. Afzal W, Torkar R, Feldt R (2009) A systematic review of search-based testing for non-functional system properties. Inf Softw Technol 51:957–976CrossRefGoogle Scholar
  3. Ali S et al., Briand LC, Hemmati H, Panesar-Walawege RK (2009) A systematic review of the application and empirical investigation of search-based test case generation IEEE transactions on software engineering 99Google Scholar
  4. Ali S, Briand L, Arcuri A, Walawege S (2011a) an industrial application of robustness testing using aspect-oriented modeling, UML/MARTE, and search algorithms. Paper presented at the ACM/IEEE 14th International Conference on Model Driven Engineering Languages and Systems (Models 2011)Google Scholar
  5. Ali S, Iqbal MZ, Arcuri A, Briand L (2011b) A Search-based OCL constraint solver for model-based test data generation. Paper presented at the Proceedings of the 11th International Conference On Quality Software (QSIC 2011)Google Scholar
  6. Ali S, Briand L, Arcuri A, Walawege S (2011c) An industrial application of robustness testing using aspect-oriented modeling, UML/MARTE, and search algorithms. Paper presented at the ACM/IEEE 14th International Conference on Model Driven Engineering Languages and Systems (Models 2011)Google Scholar
  7. Ali S, Briand LC, Hemmati H (2012) Modeling robustness behavior using aspect-oriented modeling to support robustness testing of industrial systems. Softw Syst Model 11:633–670CrossRefGoogle Scholar
  8. Ali S, Iqbal MZ, Arcuri A, Briand L (2013) Generating test data from OCL constraints with search techniques. IEEE Trans Softw Eng 39:1376–1402. doi:10.1109/tse.2013.17 CrossRefGoogle Scholar
  9. Ali S, Iqbal MZ, Arcuri A (2014) Improved heuristics for solving OCL constraints using search algorithms. Paper presented at the Proceedings of the 2014 conference on Genetic and evolutionary computation, Vancouver, BC, CanadaGoogle Scholar
  10. Arcuri A (2010) It does matter how you normalise the branch distance in search based software testing. Paper presented at the Proceedings of the 2010 Third International Conference on Software Testing, Verification and ValidationGoogle Scholar
  11. Arcuri A (2011) It really does matter how you normalize the branch distance in search-based software testing software testing. Verification Reliab. doi:10.1002/stvr.457 Google Scholar
  12. Arcuri A, Briand. L (2011) A practical guide for using statistical tests to assess randomized algorithms in software engineering. Paper presented at the International Conference on Software Engineering (ICSE)Google Scholar
  13. Arcuri A, Fraser G (2011) On parameter tuning in search based software engineering. Paper presented at the International Symposium on Search Based Software Engineering (SSBSE)Google Scholar
  14. Arcuri A, Fraser G (2013) Parameter tuning or default values? An empirical investigation in search-based software engineering. Empir Softw Eng 18:594–623. doi:10.1007/s10664-013-9249-9 CrossRefGoogle Scholar
  15. Arcuri A, Iqbal MZ, Briand L (2012) Random testing: theoretical results and practical implications. IEEE Trans Softw Eng 38:258–277CrossRefGoogle Scholar
  16. Bao-Lin L, Zhi-shu L, Qing L, Hong CY (2007) Test case automate generation from uml sequence diagram and ocl expression. Paper presented at the International Conference on cimputational Intelligence and SecurityGoogle Scholar
  17. Bertolino A (2007) Software testing research: achievements, challenges, dreams. Paper presented at the 2007 Future of Software EngineeringGoogle Scholar
  18. Binder RV (1999) Testing object-oriented systems: models, patterns, and tools. Addison-Wesley Longman Publishing Co., Inc, AmsterdamGoogle Scholar
  19. Bordbar B, Anastasakis K (2005) UML2Alloy: a tool for lightweight modelling of Discrete Event Systems. Paper presented at the IADIS International Conference in Applied ComputingGoogle Scholar
  20. Brucker AD, Krieger MP, Longuet D, Wolff B (2010) A specification-based test case generation method for UML/OCL. Paper presented at the Worksshop on OCL and Textual Modelling, MoDELSGoogle Scholar
  21. Burke EK, Kendall G (2006) Search methodologies: introductory tutorials in optimization and decision support techniques. SpringerGoogle Scholar
  22. Cabot J, Gogolla M (2012) Object Constraint Language (OCL): A Definitive Guide. In: Bernardo M, Cortellessa V, Pierantonio A (eds) Formal methods for model-driven engineering, vol 7320. Lecture notes in computer science. Springer, Berlin, pp 58–90. doi:10.1007/978-3-642-30982-3_3 Google Scholar
  23. Cabot J, Claris R, Riera D (2008) Verification of UML/OCL class diagrams using constraint programming. Paper presented at the Proceedings of the 2008 I.E. International Conference on Software Testing Verification and Validation WorkshopGoogle Scholar
  24. Clarke J et al. (2003) Reformulating software engineering as a search problem IEE Software 150:161–175Google Scholar
  25. Clavel M, Dios MAGd (2009) Checking unsatisfiability for OCL constraints. Paper presented at the In the proceedings of the 9th OCL 2009 Workshop at the UML/MoDELS ConferencesGoogle Scholar
  26. Coley DA (1997) An introduction to genetic algorithms for scientists and engineers. World Scientific Publishing CompanyGoogle Scholar
  27. Costal D, Gómez C, Queralt A, Raventós R, Teniente E (2008) Improving the definition of general constraints in UML. Softw Syst Model 7:469–486. doi:10.1007/s10270-007-0078-4 CrossRefGoogle Scholar
  28. Distefano D, Katoen J-P, Rensink A (2000) Towards model checking OCL. Paper presented at the ECOOP-Workshop on Defining Precise Semantics for UMLGoogle Scholar
  29. Drechsler R, Drechsler N (2002) Evolutionary algorithms for embedded system design. Kluwer Academic Publishers, DordrechtMATHGoogle Scholar
  30. Fraser G, Arcuri A (2011) EvoSuite: Automatic Test Suite Generation for Object-Oriented Software. In: Zeller A (ed) ACM Symposium on the Foundations of Software Engineering (FSE), 2011. ACM, pp 416–419Google Scholar
  31. Frias L, Queralt A, Ramon AO (2003) U-Rent car rentals specificationGoogle Scholar
  32. Harman M, Jones BF (2001) Search-based software engineering. Inf Softw Technol 43:833–839CrossRefGoogle Scholar
  33. Harman M, A.Mansouri S, Zhang Y (2009) Search based software engineering: A comprehensive analysis and review of trends techniques and applications. King’s College,Technical Report TR-09-03Google Scholar
  34. Harman M, Mansouri A, Zhang Y (2012) Search based software engineering: Trends, techniques and applications to appear in ACM computing surveysGoogle Scholar
  35. Hong L, Tao Y, Ali S, Kunming N, Li Z Zen-CC: an automated and incremental conformance checking solution to support interactive product configuration. In: Software Reliability Engineering (ISSRE), 2014 I.E. 25th International Symposium on, 3–6 Nov. 2014 2014. pp 13–22. doi:10.1109/ISSRE.2014.13
  36. Iqbal MZ, Ali S, Yue T, Briand L (2012a) Experiences of Applying UML/MARTE on Three Industrial Projects. Paper presented at the ACM/IEEE International Conference on Model Driven Engineering Languages and Systems (MODELS)Google Scholar
  37. Iqbal MZ, Arcuri A, Briand L (2012b) Empirical investigation of search algorithms for environment model-based testing of real-time embedded software In: International Symposium on Software Testing and Analysis (ISSTA), ACMGoogle Scholar
  38. Iqbal M, Arcuri A, Briand L (2012c) Combining search-based and adaptive random testing strategies for environment model-based testing of real-time embedded systems. In: Fraser G, Teixeira de Souza J (eds) Search Based Software Engineering, vol 7515. Lecture Notes in Computer Science. Springer Berlin Heidelberg, pp 136–151. doi:10.1007/978-3-642-33119-0_11
  39. Iqbal MZ, Ali S, Yue T, Briand L (2012d) Experiences of Applying UML/MARTE on three industrial projects. Paper presented at the ACM/IEEE International Conference on Model Driven Engineering Languages and Systems (MODELS)Google Scholar
  40. Jackson D, Schechter I, Shlyahter H (2000) Alcoa: the alloy constraint analyzer. Paper presented at the Proceedings of the 22nd international conference on Software engineering, Limerick, IrelandGoogle Scholar
  41. Korel B (1990) Automated software test data generation. IEEE Trans Softw Eng 16:870–879. doi:10.1109/32.57624 CrossRefGoogle Scholar
  42. Krieger M, Knapp A (2008) Executing underspecified OCL operation contracts with a SAT Solver. Paper presented at the 8th International Workshop on OCL Concepts and ToolsGoogle Scholar
  43. Mantere T, Alander JT (2005) Evolutionary software engineering, a review. Appl Soft Comput 5:315–331CrossRefGoogle Scholar
  44. Mathur AP (2008) Foundations of software testing. Pearson EducationGoogle Scholar
  45. McMinn P (2004a) Search-based software test data generation: a survey. Softw Test Verification Reliab 14:52Google Scholar
  46. McMinn P (2004b) Search-based software test data generation: a survey: research articles. Softw Test Verif Reliab 14:105–156. doi:10.1002/stvr.v14:2 CrossRefGoogle Scholar
  47. McMinn P (2004c) Search-based software test data generation: a survey. Softw Test Verif Reliab 14:105–156. doi:10.1002/stvr.v14:2 CrossRefGoogle Scholar
  48. OCL (2011) Object constraint language specification, version 2.2. Object Management Group (OMG). http://www.omg.org/spec/OCL/2.2/
  49. Sheskin DJ (2007) Handbook of parametric and nonparametric statistical procedures. Chapman and Hall/CRCGoogle Scholar
  50. Souza JTd, Maia CL, Freitas FGd, Coutinho DP (2010) The human competitiveness of search based software engineering. Paper presented at the Proceedings of the 2nd International Symposium on Search Based Software EngineeringGoogle Scholar
  51. Tairas R, Cabot J (2012) Cloning in DSLs: Experiments with OCL. In: Sloane A, Aßmann U (eds) Software language engineering, vol 6940. Lecture notes in computer science. Springer, Berlin, pp 60–76. doi:10.1007/978-3-642-28830-2_4 Google Scholar
  52. Wang C, Pastore F, Goknil A, Briand L, Iqbal. MZ (2015) Automatic generation of system test cases from use case specifications. Paper presented at the International Symposium on Software Testing and Analysis (ISSTA)Google Scholar
  53. Wilson B (1994) EU-Rent car rentals case study. Model Systems & Brian Wilson AssociatesGoogle Scholar
  54. Winkelmann J, Taentzer G, Ehrig K, Küster JM (2008) Translation of Restricted OCL Constraints into Graph Constraints for Generating Meta Model Instances by Graph Grammars Electron. Notes Theor Comput Sci 211:159–170. doi:10.1016/j.entcs.2008.04.038 CrossRefMATHGoogle Scholar
  55. Wohlin C, Runeson P, Höst M (1999) Experimentation in software engineering: an introduction. Springer, HeidelbergMATHGoogle Scholar

Copyright information

© Springer Science+Business Media New York 2015

Authors and Affiliations

  • Shaukat Ali
    • 1
  • Muhammad Zohaib Iqbal
    • 2
    • 3
  • Maham Khalid
    • 2
  • Andrea Arcuri
    • 3
    • 4
  1. 1.Simula Research LaboratoryFornebuNorway
  2. 2.Quest LabNational University of Computer & Emerging SciencesIslamabadPakistan
  3. 3.SnT CentreUniversity of LuxembourgWalferdangeLuxembourg
  4. 4.ScientaOsloNorway

Personalised recommendations