Verifying Code Generation Tools for the B-Method Using Tests: A Case Study

  • Anamaria M. Moreira
  • Cleverton Hentz
  • David Déharbe
  • Ernesto C. B. de Matos
  • João B. Souza Neto
  • Valério de MedeirosJr.
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 9154)


In this paper, we present a case study where two code generators for the B-Method were validated using software testing techniques. Our testing strategy is a combination of Grammar-Based Testing (GBT) and Model-Based Testing (MBT) techniques. The strategy consists of two steps. In the first step, grammar-based coverage criteria are used to generate a wide and meaningful set of test input models to validate the parsing capabilities of the code generators. In the second step, a MBT tool is used to validate the correctness of the output produced by these tools. The MBT tool generates a set of tests based on the same input model used by the code generation tools. The generated code is considered correct (consistent with the input model) if it passes this set of tests. Using this testing strategy, we were able to find problems in both code generation tools with moderate effort.


Model-Based Testing Grammar-Based Testing B-Method Code generation 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Abrial, J.-R.: The B-book: assigning programs to meanings. Cambridge University Press (1996)Google Scholar
  2. 2.
    Ammann, P., Offutt, J.: Introduction to Software Testing. Cambridge University Press, New York (2008)CrossRefGoogle Scholar
  3. 3.
    ClearSy. Atelier B User Manual Version 4.0. Clearsy System Engineering (2009)Google Scholar
  4. 4.
    Déharbe, D., Medeiros, Jr., V.: Proposal: Translation of B Implementations to LLVM-IR. In: SBMF, Brasília - DF, SBMF (2013)Google Scholar
  5. 5.
    Goerigk, W., Dold, A., Gaul, T., Goos, G., Heberle, A., Von Henke, F.W., Hoffmann, U., Langmaack, H., Pfeifer, H., Ruess, H., et al.: Compiler correctness and implementation verification: The verifix approach (1996)Google Scholar
  6. 6.
    Gogolla, M., Vallecillo, A.: Tractable model transformation testing. In: France, R.B., Kuester, J.M., Bordbar, B., Paige, R.F. (eds.) ECMFA 2011. LNCS, vol. 6698, pp. 221–235. Springer, Heidelberg (2011) CrossRefGoogle Scholar
  7. 7.
    Härtel, J., Härtel, L., Lämmel, R.: Test-Data generation for Xtext. In: Combemale, B., Pearce, D.J., Barais, O., Vinju, J.J. (eds.) SLE 2014. LNCS, vol. 8706, pp. 342–351. Springer, Heidelberg (2014) Google Scholar
  8. 8.
    Hentz, C.: Automatic Generation of Tests from Language Descriptions (Text in Portuguese). Master’s thesis, UFRN, Natal, Brazil (2010)Google Scholar
  9. 9.
    Hoffman, D.M., Ly-Gagnon, D., Strooper, P., Wang, H.-Y.: Grammar-based test generation with yougen. Software: Practice and Experience 41(4), 427–447 (2011)Google Scholar
  10. 10.
    Kossatchev, A.S., Posypkin, M.A.: Survey of compiler testing methods. Program. Comput. Softw. 31(1), 10–19 (2005)CrossRefGoogle Scholar
  11. 11.
    Lämmel, R.: Grammar testing. In: Hussmann, H. (ed.) FASE 2001. LNCS, vol. 2029, pp. 201–216. Springer, Heidelberg (2001) CrossRefGoogle Scholar
  12. 12.
    Lämmel, R., Schulte, W.: Controllable combinatorial coverage in grammar-based testing. In: Uyar, M.U., Duale, A.Y., Fecko, M.A. (eds.) TestCom 2006. LNCS, vol. 3964, pp. 19–38. Springer, Heidelberg (2006) CrossRefGoogle Scholar
  13. 13.
    Lattner, C., Adve, V.S.: LLVM: A compilation framework for lifelong program analysis & transformation. In: 2nd IEEE/ACM International Symposium on Code Generation and Optimization, pp. 75–88 (2004)Google Scholar
  14. 14.
    Leroy, X.: Formal verification of a realistic compiler. Commun. ACM 52(7), 107–115 (2009)CrossRefGoogle Scholar
  15. 15.
    Li, N., Offutt, J.: An empirical analysis of test oracle strategies for model-based testing. In: IEEE 7th International Conference on Software Testing, Verification and Validation (April 2014)Google Scholar
  16. 16.
    Moreira, A.M., Hentz, C., Ramalho, V.: Application of a Syntax-based Testing Method and Tool to Software Product Lines. In: 7th Brazilian Workshop on Systematic and Automated Software Testing, Brasília - DF (2013)Google Scholar
  17. 17.
    Necula, G.C.: Translation validation for an optimizing compiler. ACM Sigplan Notices 35(5), 83–94 (2000)CrossRefGoogle Scholar
  18. 18.
    Purdom, P.: A sentence generator for testing parsers. BIT Numerical Mathematics 12(4), 366–375 (1972)MathSciNetCrossRefGoogle Scholar
  19. 19.
    Stuermer, I., Conrad, M., Doerr, H., Pepper, P.: Systematic testing of model-based code generators. IEEE Transactions on Software Engineering 33(9), 622–634 (2007)CrossRefGoogle Scholar
  20. 20.
    Wimmer, M., Burgueño, L.: Testing M2T/T2M transformations. In: Moreira, A., Schätz, B., Gray, J., Vallecillo, A., Clarke, P. (eds.) MODELS 2013. LNCS, vol. 8107, pp. 203–219. Springer, Heidelberg (2013) CrossRefGoogle Scholar
  21. 21.
    Zuck, L., Pnueli, A., Fang, Y., Goldberg, B.: VOC: A Methodology for the Translation Validation of Optimizing Compilers. Journal of Universal Computer Science 9(3), 223–247 (2003)Google Scholar

Copyright information

© Springer International Publishing Switzerland 2015

Authors and Affiliations

  • Anamaria M. Moreira
    • 2
  • Cleverton Hentz
    • 1
  • David Déharbe
    • 1
  • Ernesto C. B. de Matos
    • 1
  • João B. Souza Neto
    • 1
  • Valério de MedeirosJr.
    • 1
  1. 1.Federal University of Rio Grande do NorteNatalBrazil
  2. 2.Federal University of Rio de JaneiroRio de JaneiroBrazil

Personalised recommendations