Skip to main content
SpringerLink
Log in

Scalable and precise estimation and debugging of the worst-case execution time for analysis-friendly processors: a comeback of model checking

  • Regular Paper
  • Published:
International Journal on Software Tools for Technology Transfer Aims and scope Submit manuscript

Abstract

Estimating the worst-case execution time (WCET) of an application is an essential step in the context of developing real-time or safety-critical software, but it is also a complex and error-prone process. Conventional approaches require at least some manual inputs from the user, such as loop bounds and infeasible path information, which are hard to obtain and can lead to unsafe results if they are incorrect. This is aggravated by the lack of a comprehensive explanation of the estimate, i.e., a specific trace showing how the estimated WCET was reached. In this article, we revisit the use of Model Checking as an analysis technique for WCET estimation. Model Checking has been explored before, but did not prevail due to its poor scalability. We address this by shifting the analysis to the source code level, where code transformations can be applied that retain the timing behavior, but reduce the complexity. Furthermore, we show how Model Checking enables the reconstruction of a concrete trace of the WCET path, which can be examined in a debugger environment. A prerequisite for our approach is the use of analysis-friendly processors. This is in line with recent calls by the research community, since modern processors have reached a complexity that refutes timing analysis. Our experiments show that fast and precise estimates can be achieved with Model Checking, that its scalability can even exceed current approaches, and that new opportunities arise in the direction of “timing debugging”.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

Similar content being viewed by others

Notes

  1. We have empirically chosen \(N_{\mathrm {assert}}=10\); placing either more or less assertions usually take longer, because either the computational effort is growing, or more iterations are required.

  2. Note that this only holds true for cache-less processors.

  3. For PapaBench, we simulated the PPM and GPS inputs to enable the software transitioning into automatic flight mode. However, not all subprograms were triggered, i.a., approaching and error handling stayed inactive. A worst-case simulation is practically impossible here.

  4. Note that this code works correctly for 16-bit targets, as \(\forall n \in {\texttt {uint16}}, \exists i \le 255, \text { s.t. } \texttt {divides(i,n)}\).

References

  1. Abella, J., Hernández, C., Quiñones, E., Cazorla, F.J., Conmy, P.R., Azkarate-askasua, M., Pérez, J., Mezzetti, E., Vardanega, T.: WCET analysis methods: pitfalls and challenges on their trustworthiness. In: Proceedings of the International Symposium on Industrial Embedded Systems (SIES), pp. 39–48 (2015)

  2. Al-Bataineh, O., Reynolds, M., French, T.: Accelerating worst case execution time analysis of timed automata models with cyclic behaviour. Formal Aspects of Computing 27(5), 917–949 (2015)

    Article  MathSciNet  MATH  Google Scholar 

  3. Altenbernd, P., Gustafsson, J., Lisper, B., Stappert, F.: Early execution time-estimation through automatically generated timing models. Real-Time Syst. 52(6), 731–760 (2016)

    Article  Google Scholar 

  4. Axer, P., Ernst, R., Falk, H., Girault, A., Grund, D., Guan, N., Jonsson, B., Marwedel, P., Reineke, J., Rochange, C., Sebastian, M., von Hanxleden, R., Wilhelm, R., Yi, W.: Building timing predictable embedded systems. ACM Trans. Embed. Comput. Syst. 13(4), 82:1–82:37 (2014)

    Article  Google Scholar 

  5. Béchennec, J., Cassez, F.: Computation of WCET using program slicing and real-time model-checking. CoRR (2011). arXiv:1105.1633

  6. Becker, M., Neumair, M., Söhn, A., Chakraborty, S.: Approaches for software verification of an emergency recovery system for micro air vehicles. In: F. Koornneef, C. van Gulijk (eds.) Proceedings of the Computer Safety, Reliability, and Security—34th International Conference (SAFECOMP), Lecture Notes in Computer Science, vol. 9337, pp. 369–385. Springer, Berlin (2015)

  7. Bernat, G., Davis, R., Merriam, N., Tuffen, J., Gardner, A., Bennett, M., Armstrong, D.: Identifying opportunities for worst-case execution time reduction in an avionics system. Ada User J. 28(3), 189–195 (2007)

    Google Scholar 

  8. Beyer, D.: Status report on software verification—(competition summary SV-COMP 2014). In: E. Ábrahám, K. Havelund (eds.) Proceedings of the 20th International Conference on Tools and Algorithms for the Construction and Analysis of Systems (TACAS), Lecture Notes in Computer Science, vol. 8413, pp. 373–388. Springer, New York (2014)

  9. Blazy, S., Maroneze, A.O., Pichardie, D.: Formal verification of loop bound estimation for WCET analysis. In: E. Cohen, A. Rybalchenko (eds.) Proceedings of the 5th International Conference on Verified Software: Theories, Tools, Experiments (VSTTE), Lecture Notes in Computer Science, vol. 8164, pp. 281–303. Springer, New York (2014)

  10. Brandner, F., Hepp, S., Jordan, A.: Static profiling of the worst-case in real-time programs. In: L. Cucu-Grosjean, N. Navet, C. Rochange, J.H. Anderson (eds.) Proceedings of the 20th International Conference on Real-Time and Network Systems (RTNS), pp. 101–110. ACM (2012)

  11. Cerný, P., Henzinger, T.A., Kovács, L., Radhakrishna, A., Zwirchmayr, J.: Segment abstraction for worst-case execution time analysis. In: J. Vitek (ed.) Proceedings of the 24th European Symposium on Programming Languages and Systems (ESOP), Lecture Notes in Computer Science, vol. 9032, pp. 105–131. Springer, New York (2015)

  12. Chattopadhyay, S., Roychoudhury, A.: Scalable and precise refinement of cache timing analysis via path-sensitive verification. Real-Time Syst. 49(4), 517–562 (2013)

    Article  MATH  Google Scholar 

  13. Clarke, E.M., Kroening, D., Lerda, F.: A tool for checking ANSI-C programs. In: K. Jensen, A. Podelski (eds.) Proceedings of the 10th International Conference on Tools and Algorithms for the Construction and Analysis of Systems (TACAS), Lecture Notes in Computer Science, vol. 2988, pp. 168–176. Springer, New York (2004)

  14. Clarke Jr., E.M., Grumberg, O., Peled, D.A.: Model Checking. MIT Press, Cambridge (1999)

    MATH  Google Scholar 

  15. Dalsgaard, A.E., Olesen, M.C., Toft, M., Hansen, R.R., Larsen, K.G.: METAMOC: modular execution time analysis using model checking. In: Lisper [42], pp. 113–123

  16. Darke, P., Chimdyalwar, B., Venkatesh, R., Shrotri, U., Metta, R.: Over-approximating loops to prove properties using bounded model checking. In: W. Nebel, D. Atienza (eds.) Proceedings of the Design, Automation & Test in Europe Conference & Exhibition (DATE), pp. 1407–1412. ACM (2015)

  17. Demyanova, Y., Pani, T., Veith, H., Zuleger, F.: Empirical software metrics for benchmarking of verification tools. In: D. Kroening, C.S. Pasareanu (eds.) Proceedings of the 27th International Conference on Computer Aided Verification (CAV), Lecture Notes in Computer Science, vol. 9206, pp. 561–579. Springer, New York (2015)

  18. Ding, H., Liang, Y., Mitra, T.: WCET-centric partial instruction cache locking. In: P. Groeneveld, D. Sciuto, S. Hassoun (eds.) Proceedings of the 49th Annual Design Automation Conference (DAC), pp. 412–420. ACM (2012)

  19. Edwards, S.A., Kim, S., Lee, E.A., Liu, I., Patel, H.D., Schoeberl, M.: A disruptive computer design idea: architectures with repeatable timing. In: Proceedings of the 27th International Conference on Computer Design (ICCD), pp. 54–59. IEEE Computer Society (2009)

  20. Ermedahl, A., Fredriksson, J., Gustafsson, J., Altenbernd, P.: Deriving the worst-case execution time input values. In: Proceedings of the 21st Euromicro Conference on Real-Time Systems (ECRTS), pp. 45–54. IEEE Computer Society (2009)

  21. Ermedahl, A., Stappert, F., Engblom, J.: Clustered worst-case execution-time calculation. IEEE Trans. Comput. 54(9), 1104–1122 (2005)

    Article  Google Scholar 

  22. Ferdinand, C., Heckmann, R., Le Sergent, T., Lopes, D., Martin, B., Fornari, X., Martin, F.: Combining a high-level design tool for safety-critical systems with a tool for WCET analysis of executables. In: Proceedings of the 4th European Congress on Embedded Real Time Software (ERTS). SIA/AAAF/SEE (2008)

  23. Fuhrmann, I., Broman, D., von Hanxleden, R., Schulz-Rosengarten, A.: Time for reactive system modeling: interactive timing analysis with hotspot highlighting. In: A. Plantec, F. Singhoff, S. Faucou, L.M. Pinho (eds.) Proceedings of the 24th International Conference on Real-Time Networks and Systems (RTNS), pp. 289–298. ACM (2016)

  24. Goossens, K., Azevedo, A., Chandrasekar, K., Gomony, M.D., Goossens, S., Koedam, M., Li, Y., Mirzoyan, D., Molnos, A.M., Nejad, A.B., Nelson, A., Sinha, S.: Virtual execution platforms for mixed-time-criticality systems: the CompSOC architecture and design flow. SIGBED Rev. 10(3), 23–34 (2013)

    Article  Google Scholar 

  25. Gulwani, S., Jain, S., Koskinen, E.: Control-flow refinement and progress invariants for bound analysis. In: M. Hind, A. Diwan (eds.) Proceedigs of the Conference on Programming Language Design and Implementation (PLDI), pp. 375–385. ACM (2009)

  26. Gustafsson, J., Betts, A., Ermedahl, A., Lisper, B.: The Mälardalen WCET benchmarks: Past, present and future. In: Lisper [42], pp. 136–146

  27. Gustafsson, J., Ermedahl, A., Sandberg, C., Lisper, B.: Automatic derivation of loop bounds and infeasible paths for WCET analysis using abstract execution. In: Proceedings of the 27th International Real-Time Systems Symposium (RTSS), pp. 57–66 (2006)

  28. Harmon, T., Klefstad, R.: Interactive back-annotation of worst-case execution time analysis for java microprocessors. In: Proceedings of the 13th International Conference on Embedded and Real-Time Computing Systems and Applications (RTCSA), pp. 209–216. IEEE Computer Society (2007)

  29. Hatcliff, J., Dwyer, M.B., Zheng, H.: Slicing software for model construction. Higher-Order Symb. Comput. 13(4), 315–353 (2000)

    Article  MATH  Google Scholar 

  30. Healy, C.A., Sjödin, M., Rustagi, V., Whalley, D.B., van Engelen, R.: Supporting timing analysis by automatic bounding of loop iterations. Real-Time Syst. 18(2/3), 129–156 (2000)

    Article  Google Scholar 

  31. Henry, J., Asavoae, M., Monniaux, D., Maiza, C.: How to compute worst-case execution time by optimization modulo theory and a clever encoding of program semantics. In: Y. Zhang, P. Kulkarni (eds.) Proceedings of the 15th Conference on Languages, Compilers and Tools for Embedded Systems (LCTES), pp. 43–52. ACM (2014)

  32. Holsti, N.: Computing time as a program variable: a way around infeasible paths. In: R. Kirner (ed.) Proceedings of the 8th International Workshop on Worst-Case Execution Time (WCET) Analysis, OASICS, vol. 8. Internationales Begegnungs- und Forschungszentrum fuer Informatik (IBFI), Schloss Dagstuhl, Germany (2008)

  33. Holsti, N., Saarinen, S.: Status of the Bound-T WCET tool. Space Systems Finland Ltd, Espoo (2002)

    Google Scholar 

  34. Kim, S., Patel, H.D., Edwards, S.A.: Using a model checker to determine worst-case execution time. Technical report, Columbia University (2009). CUCS-038-09

  35. Kirner, R., Puschner, P.P.: Obstacles in worst-case execution time analysis. In: Proceedings of the 11th IEEE International Symposium on Object-Oriented Real-Time Distributed Computing (ISORC), pp. 333–339. IEEE Computer Society (2008)

  36. Knoop, J., Kovács, L., Zwirchmayr, J.: Symbolic loop bound computation for WCET analysis. In: E.M. Clarke, I. Virbitskaite, A. Voronkov (eds.) Proceedings of the 8th International Conference Perspectives of Systems Informatics (PSI), Revised Selected Papers, Lecture Notes in Computer Science, vol. 7162, pp. 227–242. Springer, New York (2012)

  37. Ko, L., Healy, C.A., Ratliff, E., Arnold, R.D., Whalley, D.B., Harmon, M.G.: Supporting the specification and analysis of timing constraints. In: Proceedings of the 2nd Real-Time Technology and Applications Symposium (RTAS), pp. 170–178. IEEE Computer Society (1996)

  38. Kuhnel, C.: AVR RISC Microcontroller Handbook, 1st edn. Newnes, Boston (1998)

    Google Scholar 

  39. Kuo, M.M.Y., Yoong, L.H., Andalam, S., Roop, P.S.: Determining the worst-case reaction time of IEC 61499 function blocks. In: Proceedings of the 8th IEEE International Conference on Industrial Informatics, pp. 1104–1109 (2010)

  40. Li, Y.T., Malik, S.: Performance analysis of embedded software using implicit path enumeration. IEEE Trans. Comput. Aided Design Integr. Circuits Syst. 16(12), 1477–1487 (1997)

    Article  Google Scholar 

  41. Lickly, B., Liu, I., Kim, S., Patel, H.D., Edwards, S.A., Lee, E.A.: Predictable programming on a precision timed architecture. In: E.R. Altman (ed.) Proceedings of the International Conference on Compilers, Architecture, and Synthesis for Embedded Systems, (CASES), pp. 137–146. ACM (2008)

  42. Lisper, B. (ed.): Proceedings of the 10th International Workshop on Worst-Case Execution Time Analysis (WCET), OASICS, vol. 15. Schloss Dagstuhl - Leibniz-Zentrum fuer Informatik, Germany (2010)

  43. Ltd., A.: ARM7TDMI Data Sheet, Doc ARM IHI 0042B edn. (2008). ABI release 2.06

  44. Lv, M., Gu, Z., Guan, N., Deng, Q., Yu, G.: Performance comparison of techniques on static path analysis of WCET. In: C. Xu, M. Guo (eds.) Proceedings of the International Conference on Embedded and Ubiquitous Computing (EUC), pp. 104–111. IEEE Computer Society (2008)

  45. Maiza, C., Raymond, P., Parent-Vigouroux, C., Bonenfant, A., Carrier, F., Cassé, H., Cuenot, P., Claraz, D., Halbwachs, N., Jahier, E., Li, H., Michiel, M.D., Mussot, V., Puaut, I., Rohou, E., Ruiz, J., Sotin, P., Sun, W.: The W-SEPT project: Towards semantic-aware WCET estimation. In: J. Reineke (ed.) International Workshop on Worst-Case Execution Time Analysis (WCET), OASICS, vol. 57, pp. 9:1–9:13. Schloss Dagstuhl - Leibniz-Zentrum fuer Informatik (2017)

  46. Marref, A.: Fully-automatic derivation of exact program-flow constraints for a tighter worst-case execution-time analysis. In: L. Carro, A.D. Pimentel (eds.) Proceedings of the International Conference on Embedded Computer Systems: Architectures, Modeling, and Simulation (SAMOS), pp. 200–208 (2011)

  47. Metta, R., Becker, M., Bokil, P., Chakraborty, S., Venkatesh, R.: TIC: a scalable model checking based approach to WCET estimation. In: T. Kuo, D.B. Whalley (eds.) Proceedings of the 17th Conference on Languages, Compilers, Tools, and Theory for Embedded Systems (LCTES), pp. 72–81. ACM (2016)

  48. Metzner, A.: Why model checking can improve WCET analysis. In: R. Alur, D.A. Peled (eds.) Proceedings of the 16th International Conference on Computer Aided Verification (CAV), Lecture Notes in Computer Science, vol. 3114, pp. 334–347. Springer, Berlin (2004)

  49. Mitra, T., Teich, J., Thiele, L.: Adaptive isolation for predictability and security (Dagstuhl seminar 16441). Dagstuhl Rep. 6(10), 120–153 (2017)

    Google Scholar 

  50. Mittal, S.: A survey of techniques for cache locking. ACM Trans. Design Autom. Electron. Syst. 21(3), 49:1–49:24 (2016)

    Article  Google Scholar 

  51. Nemer, F., Cassé, H., Sainrat, P., Bahsoun, J.P., Michiel, M.D.: PapaBench: a free real-time benchmark. In: Mueller, F. (ed.) International Workshop on Worst-Case Execution Time Analysis (WCET), OpenAccess Series in Informatics (OASIcs), vol. 4. Schloss Dagstuhl-Leibniz-Zentrum fuer Informatik, Dagstuhl (2006)

    Google Scholar 

  52. Park, C.Y., Shaw, A.C.: Experiments with a program timing tool based on source-level timing schema. IEEE Comput. 24(5), 48–57 (1991)

    Article  Google Scholar 

  53. Pingali, K., Bilardi, G.: APT: A data structure for optimal control dependence computation. In: D.W. Wall (ed.) Proceedings of the Conference on Programming Language Design and Implementation (PLDI), pp. 32–46. ACM (1995)

  54. Puschner, P.: Is WCET analysis a non-problem? Towards new software and hardware architectures. In: Bernat, G. (ed.) Proceedings of the 2nd International Workshop on Worst-Case Execution Time Analysis (WCET), pp. 89–92. Technical University of Vienna, Vienna (2002)

    Google Scholar 

  55. Puschner, P.P.: A tool for high-level language analysis of worst-case execution times. In: Proceedings of the 10th Euromicro Conference on Real-Time Systems (ECRTS), pp. 130–137. IEEE Computer Society (1998)

  56. Puschner, P.P., Koza, C.: Calculating the maximum execution time of real-time programs. Real-Time Syst. 1(2), 159–176 (1989)

    Article  Google Scholar 

  57. Puschner, P.P., Prokesch, D., Huber, B., Knoop, J., Hepp, S., Gebhard, G.: The T-CREST approach of compiler and WCET-analysis integration. In: Proceedings of the 16th International Symposium on Object/Component/Service-Oriented Real-Time Distributed Computing, (ISORC), pp. 1–8. IEEE Computer Society (2013)

  58. Raymond, P., Maiza, C., Parent-Vigouroux, C., Carrier, F.: Timing analysis enhancement for synchronous program. In: M. Auguin, R. de Simone, R.I. Davis, E. Grolleau (eds.) Proc. 21st International Conference on Real-Time Networks and Systems (RTNS), pp. 141–150. ACM (2013)

  59. Robertson, N., Seymour, P.: Graph minors XIII. The disjoint paths problem. J. Combin. Theory Ser. B 63(1), 65–110 (1995)

    Article  MathSciNet  MATH  Google Scholar 

  60. Schoeberl, M.: JOP: a Java optimized processor. In: Meersman, R., Tari, Z. (eds.) Proceedings of the International Workshop on the Move to Meaningful Internet Systems (OTM), pp. 346–359. Springer, Berlin (2003)

    Google Scholar 

  61. Souyris, J., Pavec, E.L., Himbert, G., Jégu, V., Borios, G., Heckmann, R.: Computing the worst case execution time of an avionics program by abstract interpretation. In: Proceedings of the 5th International Workshop on Worst-Case Execution Time (WCET) Analysis, pp. 21–24 (2005)

  62. Sun Microsystems Inc.: The SPARC Architecture Manual, Version 7. Sun Microsystems Inc., Mountain View (1987)

  63. Weiser, M.: Program slicing. In: S. Jeffrey, L.G. Stucki (eds.) Proceedings of the 5th International Conference on Software Engineering (ICSE), pp. 439–449. IEEE Computer Society (1981)

  64. Wilhelm, R.: Why AI + ILP is good for WCET, but MC is not, nor ILP alone. In: B. Steffen, G. Levi (eds.) Proceedings of the 5th International Conference on Verification, Model Checking, and Abstract Interpretation (VMCAI), Lecture Notes in Computer Science, vol. 2937, pp. 309–322. Springer, Berlin (2004)

  65. Wilhelm, R., Engblom, J., Ermedahl, A., Holsti, N., Thesing, S., Whalley, D., Bernat, G., Ferdinand, C., Heckmann, R., Mitra, T., Mueller, F., Puaut, I., Puschner, P., Staschulat, J., Stenström, P.: The worst-case execution time problem—overview of methods and survey of tools. ACM Trans. Embed. Comput. Syst. 7(3), 36:1–36:53 (2008)

    Article  Google Scholar 

  66. Wilhelm, R., Grund, D.: Computation takes time, but how much? Commun. ACM 57(2), 94–103 (2014)

    Article  Google Scholar 

  67. Zhao, W., Kulkarni, P.A., Whalley, D.B., Healy, C.A., Mueller, F., Uh, G.: Tuning the WCET of embedded applications. In: Proceedings of the 10th Real-Time and Embedded Technology and Applications Symposium (RTAS), pp. 472–481. IEEE Computer Society (2004)

Download references

Acknowledgements

The authors would like to thank the anonymous reviewers for their valuable comments and suggestions, and roadrunner for dedicating its eight brains to host an oracle.

Author information

Authors and Affiliations

  1. Chair of Real-Time Computer Systems, Technical University of Munich, Munich, Germany

    Martin Becker & Samarjit Chakraborty

  2. Tata Research Development and Design Centre, Pune, India

    Ravindra Metta & R. Venkatesh

Authors
  1. Martin Becker
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Ravindra Metta
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. R. Venkatesh
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Samarjit Chakraborty
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Martin Becker.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Becker, M., Metta, R., Venkatesh, R. et al. Scalable and precise estimation and debugging of the worst-case execution time for analysis-friendly processors: a comeback of model checking. Int J Softw Tools Technol Transfer 21, 515–543 (2019). https://doi.org/10.1007/s10009-018-0497-2

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10009-018-0497-2

Keywords

Search

Navigation