ML for ML: Learning Cost Semantics by Experiment

  • Ankush DasEmail author
  • Jan Hoffmann
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10205)


It is an open problem in static resource bound analysis to connect high-level resource bounds with the actual execution time and memory usage of compiled machine code. This paper proposes to use machine learning to derive a cost model for a high-level source language that approximates the execution cost of compiled programs on a specific hardware platform. The proposed technique starts by fixing a cost semantics for the source language in which certain constants are unknown. To learn the constants for a specific hardware, a machine learning algorithm measures the resource cost of a set of training programs and compares the cost with the prediction of the cost semantics. The quality of the learned cost model is evaluated by comparing the model with the measured cost on a set of independent control programs. The technique has been implemented for a subset of OCaml using Inria’s OCaml compiler on an Intel x86-64 and ARM 64-bit v8-A platform. The considered resources in the implementation are heap allocations and execution time. The training programs are deliberately simple, handwritten micro benchmarks and the control programs are retrieved from the standard library, an OCaml online tutorial, and local OCaml projects. Different machine learning techniques are applied, including (weighted) linear regression and (weighted) robust regression. To model the execution time of programs with garbage collection (GC), the system combines models for memory allocations and executions without GC, which are derived first. Experiments indicate that the derived cost semantics for the number of heap allocations on both hardware platforms is accurate. The error of the cost semantics on the control programs for the x86-64 architecture for execution time with and without GC is about 19.80% and 13.04%, respectively. The derived cost semantics are combined with RAML, a state-of-the-art system for automatically deriving resource bounds for OCaml programs. Using these semantics, RAML is for the first time able to make predictions about the actual worst-case execution time.


Execution Time Function Call Garbage Collector Robust Regression Memory Allocation 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.



This article is based on research that has been supported, in part, by AFRL under DARPA STAC award FA8750-15-C-0082, by NSF under grant 1319671 (VeriQ), and by a Google Research Award. Any opinions, findings, and conclusions contained in this document are those of the authors and do not necessarily reflect the views of the sponsoring organizations.


  1. 1.
    99 problems (solved) in OCaml. Accessed 16 Aug 2016
  2. 2.
  3. 3.
    Acar, U.A., Charguéraud, A., Rainey, M.: Oracle scheduling: controlling granularity in implicitly parallel languages. In: Proceedings of the 2011 ACM International Conference on Object Oriented Programming Systems Languages and Applications, OOPSLA 2011, pp. 499–518. ACM, New York (2011)Google Scholar
  4. 4.
    Albert, E., Arenas, P., Genaim, S., Gómez-Zamalloa, M., Puebla, G.: Automatic inference of resource consumption bounds. In: Bjørner, N., Voronkov, A. (eds.) LPAR 2012. LNCS, vol. 7180, pp. 1–11. Springer, Heidelberg (2012). doi: 10.1007/978-3-642-28717-6_1 CrossRefGoogle Scholar
  5. 5.
    Albert, E., Arenas, P., Genaim, S., Puebla, G., Zanardini, D.: Cost analysis of Java bytecode. In: Nicola, R. (ed.) ESOP 2007. LNCS, vol. 4421, pp. 157–172. Springer, Heidelberg (2007). doi: 10.1007/978-3-540-71316-6_12 CrossRefGoogle Scholar
  6. 6.
    Albert, E., Fernández, J.C., Román-Díez, G.: Non-cumulative resource analysis. In: Baier, C., Tinelli, C. (eds.) TACAS 2015. LNCS, vol. 9035, pp. 85–100. Springer, Heidelberg (2015). doi: 10.1007/978-3-662-46681-0_6 Google Scholar
  7. 7.
    Alias, C., Darte, A., Feautrier, P., Gonnord, L.: Multi-dimensional rankings, program termination, and complexity bounds of flowchart programs. In: Cousot, R., Martel, M. (eds.) SAS 2010. LNCS, vol. 6337, pp. 117–133. Springer, Heidelberg (2010). doi: 10.1007/978-3-642-15769-1_8 CrossRefGoogle Scholar
  8. 8.
    Austin, T., Larson, E., Ernst, D.: Simplescalar: an infrastructure for computer system modeling. Computer 35(2), 59–67 (2002)CrossRefGoogle Scholar
  9. 9.
    Avanzini, M., Lago, U.D., Moser, G.: Analysing the complexity of functional programs: higher-order meets first-order. In: 29th International Conference on Functional Programming (ICFP 2015) (2012)Google Scholar
  10. 10.
    Backes, M., Doychev, G., Köpf, B.: Preventing side-channel leaks in web traffic: a formal approach. In: Proceedings of 20th Network and Distributed Systems Security Symposium (NDSS) (2013)Google Scholar
  11. 11.
    Blanc, R., Henzinger, T.A., Hottelier, T., Kovács, L.: ABC: algebraic bound computation for loops. In: Clarke, E.M., Voronkov, A. (eds.) LPAR 2010. LNCS (LNAI), vol. 6355, pp. 103–118. Springer, Heidelberg (2010). doi: 10.1007/978-3-642-17511-4_7 CrossRefGoogle Scholar
  12. 12.
    Blelloch, G.E., Harper, R.: Cache and I/O efficent functional algorithms. In: Proceedings of the 40th Annual ACM SIGPLAN-SIGACT Symposium on Principles of Programming Languages, POPL 2013, pp. 39–50. ACM, New York (2013)Google Scholar
  13. 13.
    Brockschmidt, M., Emmes, F., Falke, S., Fuhs, C., Giesl, J.: Alternating runtime and size complexity analysis of integer programs. In: Ábrahám, E., Havelund, K. (eds.) TACAS 2014. LNCS, vol. 8413, pp. 140–155. Springer, Heidelberg (2014). doi: 10.1007/978-3-642-54862-8_10 CrossRefGoogle Scholar
  14. 14.
    Carbonneaux, Q., Hoffmann, J., Shao, Z.: Compositional certified resource bounds. In: 36th Conference on Programming Language Design and Implementation (PLDI 2015) (2015)Google Scholar
  15. 15.
    Černý, P., Henzinger, T.A., Kovács, L., Radhakrishna, A., Zwirchmayr, J.: Segment abstraction for worst-case execution time analysis. In: Vitek, J. (ed.) ESOP 2015. LNCS, vol. 9032, pp. 105–131. Springer, Heidelberg (2015). doi: 10.1007/978-3-662-46669-8_5 CrossRefGoogle Scholar
  16. 16.
    Chen, W.Y., Chang, P.P., Conte, T.M., Hwu, W.W.: The effect of code expanding optimizations on instruction cache design. IEEE Trans. Comput. 42(9), 1045–1057 (1993)CrossRefGoogle Scholar
  17. 17.
    Colin, A., Puaut, I.: Worst case execution time analysis for a processor with branch prediction. Real-Time Syst. 18(2), 249–274 (2000)CrossRefGoogle Scholar
  18. 18.
    Cousot, P., Cousot, R.: Abstract interpretation: a unified lattice model for static analysis of programs by construction or approximation of fixpoints. In: Proceedings of the 4th ACM SIGACT-SIGPLAN Symposium on Principles of Programming Languages, POPL 1977, pp. 238–252. ACM, New York (1977)Google Scholar
  19. 19.
    Danner, N., Licata, D.R., Ramyaa, R.: Denotational cost semantics for functional languages with inductive types. In: 29th International Conference on Functional Programming (ICFP 2015) (2012)Google Scholar
  20. 20.
    Doligez, D., Frisch, A., Garrigue, J., Rémy, D., Vouillon, J.: The OCaml system release 4.03.
  21. 21.
    Gulwani, S., Mehra, K.K., Chilimbi, T.M.: SPEED: precise and efficient static estimation of program computational complexity. In: 36th ACM Symposium on Principles of Programming Languages (POPL 2009), pp. 127–139 (2009)Google Scholar
  22. 22.
    Gulwani, S., Zuleger, F.: The reachability-bound problem. In: Conference on Programming Language Design and Implementation (PLDI 2010), pp. 292–304 (2010)Google Scholar
  23. 23.
    Hoffmann, J., Aehlig, K., Hofmann, M.: Multivariate amortized resource analysis. In: 38th Symposium on Principles of Programming Languages (POPL 2011) (2011)Google Scholar
  24. 24.
    Hoffmann, J., Das, A., Weng, S.: Towards automatic resource bound analysis for OCaml. In: Proceedings of the 44th ACM SIGPLAN Symposium on Principles of Programming Languages, POPL 2017, Paris, France, 18–20 January 2017, pp. 359–373 (2017)Google Scholar
  25. 25.
    Hofmann, M., Jost, S.: Static prediction of heap space usage for first-order functional programs. In: 30th ACM Symposium on Principles of Programming Languages (POPL 2003), pp. 185–197 (2003)Google Scholar
  26. 26.
    Huang, L., Jia, J., Yu, B., Chun, B.G., Maniatis, P., Naik, M.: Predicting execution time of computer programs using sparse polynomial regression. In: Lafferty, J.D., Williams, C.K.I., Shawe-Taylor, J., Zemel, R.S., Culotta, A. (eds.) Advances in Neural Information Processing Systems 23, pp. 883–891. Curran Associates, Inc., Red Hook (2010)Google Scholar
  27. 27.
    Lambert, J.M., Power, J.F.: Platform independent timing of Java virtual machine bytecode instructions. Electron. Notes Theor. Comput. Sci. 220, 97–113 (2008)CrossRefGoogle Scholar
  28. 28.
    Lawson, C., Hanson, R.: Solving Least Squares Problems. Classics in Applied Mathematics. Society for Industrial and Applied Mathematics, Philadelphia (1995)CrossRefzbMATHGoogle Scholar
  29. 29.
    Lim, S.S., Bae, Y.H., Jang, G.T., Rhee, B.D., Min, S.L., Park, C.Y., Shin, H., Park, K., Moon, S.M., Kim, C.S.: An accurate worst case timing analysis for RISC processors. IEEE Trans. Softw. Eng. 21(7), 593–604 (1995)CrossRefGoogle Scholar
  30. 30.
    Neter, J., Kutner, M.H., Nachtsheim, C.J., Wasserman, W.: Applied Linear Statistical Models, vol. 4. Irwin, Chicago (1996)Google Scholar
  31. 31.
    Ngo, V.C., Dehesa-Azuara, M., Fredrikson, M., Hoffmann, J.: Quantifying and Preventing Side Channels with Substructural Type Systems (2016). Working paperGoogle Scholar
  32. 32.
    Olivo, O., Dillig, I., Lin, C.: Static detection of asymptotic performance bugs in collection traversals. In: Conference on Programming Language Design and Implementation (PLDI 2015), pp. 369–378 (2015)Google Scholar
  33. 33.
    Rencher, A.C., Christensen, W.F.: Multivariate regression, pp. 339–383. Wiley (2012).
  34. 34.
    Rousseeuw, P.J., Leroy, A.M.: Robust Regression and Outlier Detection. Wiley, New York (1987)CrossRefzbMATHGoogle Scholar
  35. 35.
    Sinn, M., Zuleger, F., Veith, H.: A simple and scalable static analysis for bound analysis and amortized complexity analysis. In: Biere, A., Bloem, R. (eds.) CAV 2014. LNCS, vol. 8559, pp. 745–761. Springer, Heidelberg (2014). doi: 10.1007/978-3-319-08867-9_50 Google Scholar
  36. 36.
    Steele Jr., G.L.: Debunking the ‘expensive procedure call’ myth or, procedure call implementations considered harmful or, LAMBDA: the ultimate GOTO. In: Proceedings of the 1977 Annual Conference, ACM 1977, pp. 153–162. ACM, New York (1977)Google Scholar
  37. 37.
    Wilhelm, R., et al.: The worst-case execution-time problem – overview of methods and survey of tools. ACM Trans. Embed. Comput. Syst. 7(3), 36:1–36:53 (2008)CrossRefGoogle Scholar
  38. 38.
    Zuleger, F., Gulwani, S., Sinn, M., Veith, H.: Bound analysis of imperative programs with the size-change abstraction. In: Yahav, E. (ed.) SAS 2011. LNCS, vol. 6887, pp. 280–297. Springer, Heidelberg (2011). doi: 10.1007/978-3-642-23702-7_22 CrossRefGoogle Scholar

Copyright information

© Springer-Verlag GmbH Germany 2017

Authors and Affiliations

  1. 1.Carnegie Mellon UniversityPittsburghUSA

Personalised recommendations