Test Program Generation for a Microprocessor

A Case-Study
  • Achim D. Brucker
  • Abderrahmane Feliachi
  • Yakoub Nemouchi
  • Burkhart Wolff
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 7942)


Certifications of critical security or safety system properties are becoming increasingly important for a wide range of products. Certifying large systems like operating systems up to Common Criteria EAL 4 is common practice today, and higher certification levels are at the brink of becoming reality.

To reach EAL 7 one has to formally verify properties on the specification as well as test the implementation thoroughly. This includes tests of the used hardware platform underlying a proof architecture to be certified. In this paper, we address the latter problem: we present a case study that uses a formal model of a microprocessor and generate test programs from it. These test programs validate that a microprocessor implements the specified instruction set correctly.

We built our case study on an existing model that was, together with an operating system, developed in Isabelle/HOL. We use HOL-TestGen, a model-based testing environment which is an extension of Isabelle/HOL. We develop several conformance test scenarios, where processor models were used to synthesize test programs that were run against real hardware in the loop. Our test case generation approach directly benefits from the existing models and formal proofs in Isabelle/HOL.


test program generation symbolic test case generations black box testing white box testing theorem proving interactive testing 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Appenzeller, D.P., Kuehlmann, A.: Formal verification of a powerpc microprocessor. In: Proceedings of the 1995 IEEE International Conference on Computer Design: VLSI in Computers and Processors, ICCD 1995, pp. 79–84 (October 1995), doi:10.1109/ICCD.1995.528794Google Scholar
  2. 2.
    Beyer, S.: Putting it all together - Formal Verification of the VAMP. PhD thesis, Saarland University, Saarbrücken, Germany (2005)Google Scholar
  3. 3.
    Beyer, S., Jacobi, C., Kröning, D., Leinenbach, D., Paul, W.J.: Putting it all together – formal verification of the vamp. Int. J. Softw. Tools Technol. Transf. 8(4), 411–430 (2006) ISSN 1433-2779CrossRefGoogle Scholar
  4. 4.
    Biswas, P., Freeman, A., Yamada, K., Nakagawa, N., Uchiyama, K.: Functional verification of the superscalar sh-4 microprocessor. In: Proceeding of the IEEE Compcon 1997, pp. 115–120 (February 1997), doi:10.1109/CMPCON.1997.584682Google Scholar
  5. 5.
    Brucker, A.D., Wolff, B.: HOL TestGen: An interactive test-case generation framework. In: Chechik, M., Wirsing, M. (eds.) FASE 2009. LNCS, vol. 5503, pp. 417–420. Springer, Heidelberg (2009)CrossRefGoogle Scholar
  6. 6.
    Brucker, A.D., Wolff, B.: On theorem prover-based testing. Formal Aspects of Computing, FAC (2012), doi:10.1007/s00165-012-0222-y, ISSN 0934-5043Google Scholar
  7. 7.
    Brucker, A.D., Brügger, L., Krieger, M.P., Wolff, B.: HOL-TestGen 1.7.0 user guide. Technical Report 1551, Laboratoire en Recherche en Informatique (LRI), Université Paris-Sud 11, France (April 2012)Google Scholar
  8. 8.
    Common Criteria. Common criteria for information technology security evaluation (version 3.1), Part 3: Security assurance components (September 2006) Available as document CCMB-2006-09-003Google Scholar
  9. 9.
    de Moura, L., Bjørner, N.: Z3: An efficient smt solver. In: Ramakrishnan, C.R., Rehof, J. (eds.) TACAS 2008. LNCS, vol. 4963, pp. 337–340. Springer, Heidelberg (2008)CrossRefGoogle Scholar
  10. 10.
    Dorrenbacher, J.: Formal Specification and Verification of Microkernel. PhD thesis, Saarland University, Saarbrücken, Germany (2010)Google Scholar
  11. 11.
    Fallah, F., Takayama, K.: A new functional test program generation methodology. In: Proceedings of the 2001 International Conference on Computer Design, ICCD 2001, pp. 76–81 (2001), doi:10.1109/ICCD.2001.955006Google Scholar
  12. 12.
    Fox, A.: Formal specification and verification of arm6. In: Basin, D., Wolff, B. (eds.) TPHOLs 2003. LNCS, vol. 2758, pp. 25–40. Springer, Heidelberg (2003)CrossRefGoogle Scholar
  13. 13.
    Harrison, J.: Formal verification at intel. In: LICS, pp. 45–54. IEEE Computer Society (2003), doi:10.1109/LICS.2003.1210044, ISBN 0-7695-1884-2Google Scholar
  14. 14.
    Hayes, J.P.: Fault modeling for digital mos integrated circuits. IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems 3(3), 200–208 (1984), doi:10.1109/TCAD.1984.1270076, ISSN 0278-0070CrossRefGoogle Scholar
  15. 15.
    Hennessy, J.L., Patterson, D.A.: Computer Architecture: A Quantitative Approach, 4th edn. Morgan Kaufmann Publishers Inc., San Francisco (2006) ISBN 0123704901Google Scholar
  16. 16.
    Hilderman, V., Baghai, T.: Avionics Certification: A Complete Guide to DO-178 (Software), DO-254 (Hardware). Avionics Communications Inc. (2007) ISBN 978-1-885544-25-4Google Scholar
  17. 17.
    Kamkin, A., Kornykhin, E., Vorobyev, D.: Reconfigurable model-based test program generator for microprocessors. In: IEEE International Conference on Software Testing Verification and Validation Workshop, pp. 47–54 (2011), doi:10.1109/ICSTW.2011.35Google Scholar
  18. 18.
    Leroy, X.: Formal verification of a realistic compiler. Communications of the ACM 52(7), 107–115 (2009), doi:10.1145/1538788.1538814, ISSN 0001-0782 CrossRefGoogle Scholar
  19. 19.
    Mishra, P., Dutt, N.: Specification-driven directed test generation for validation of pipelined processors. ACM Trans. Design Autom. Electr. Syst. 13(3) (2008)Google Scholar
  20. 20.
    Nipkow, T., Paulson, L.C., Wenzel, M.T.: Isabelle/HOL. LNCS, vol. 2283. Springer, Heidelberg (2002)zbMATHGoogle Scholar
  21. 21.
    Russinoff, D.M.: A mechanically checked proof of correctness of the amd k5 floating point square root microcode. Formal Methods in System Design 14(1), 75–125 (1999)CrossRefGoogle Scholar
  22. 22.
    Shen, H., Ma, L., Zhang, H.: Crpg: a configurable random test-program generator for microprocessors. In: IEEE International Symposium on Circuits and Systems, ISCAS 2005, vol. 4, pp. 4171–4174 (May 2005), doi:10.1109/ISCAS.2005.1465550Google Scholar
  23. 23.
    Srinivasan, S.K., Velev, M.N.: Formal verification of an intel xscale processor model with scoreboarding, specialized execution pipelines, and impress data-memory exceptions. In: MEMOCODE, vol. 7, pp. 65–74. IEEE Computer Society (2003), doi:10.1109/MEMCOD.2003.1210090, ISBN 0-7695-1923-7Google Scholar
  24. 24.
    Wenzel, M., Wolff, B.: Building formal method tools in the Isabelle/Isar framework. In: Schneider, K., Brandt, J. (eds.) TPHOLs 2007. LNCS, vol. 4732, pp. 352–367. Springer, Heidelberg (2007)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2013

Authors and Affiliations

  • Achim D. Brucker
    • 1
  • Abderrahmane Feliachi
    • 2
    • 3
  • Yakoub Nemouchi
    • 2
    • 3
  • Burkhart Wolff
    • 2
    • 3
  1. 1.SAP AGKarlsruheGermany
  2. 2.Laboratoire LRI, UMR8623Univ. Paris-SudOrsayFrance
  3. 3.CNRSOrsayFrance

Personalised recommendations