Abstract
We provide a formal semantics for a large subset of the Lua programming language, in its version 5.2. The semantics is a major part of an ongoing effort to construct reliable tools to analyze Lua code. In this work, we present the details of several key aspects of the language, like the semantics of its only structured data-type (tables), its meta-programming mechanism (metatables), error handling, and how these mechanisms are used to define a complex dynamic semantics that must deal with several possible erroneous situations during run time, given the nature of the language. The semantics is mechanized in Redex, a DSL specially designed to specify and debug operational semantics. We validated the mechanization in two ways: first, by executing within Redex the test suite of the reference interpreter of Lua, and second, by specifying and performing random testing of its fundamental properties using the redex-check tool. Together, they evidence that our model soundly captures the semantics of the selected fragment of the language. Additionally, we address some of the performance problems that typically arise when testing a mechanization in Redex, by using a simple implementation of a reachability-based garbage collector that captures key aspects of Lua’s. By collecting syntactic garbage, we reduce the size of configurations during run time. Finally, we briefly discuss this avenue of development of our semantics, together with the implementation of a prototype tool to perform static analysis of Lua programs.
Similar content being viewed by others
Data availability
The claims made in the present work are supported by the provided mechanization. With it, it is possible to reproduce the results shown in the paper.
Code Availability
The mechanization is publicly available at https://github.com/Mallku2/lua-gc-redex-model.
Notes
Taken from http://lua-users.org/wiki/FuncTables .
Taken from http://lua-users.org/wiki/ObjectOrientationTutorial .
Our definition enforces left-to-right evaluation of expressions. Even if this is left unspecified in Lua’s reference manual, that is how expressions are evaluated in the two most popular implementations of Lua, the reference interpreter and LuaJIT (luajit.org).
This distinction helped simplify the model, as we do not need to define new notions of evaluation contexts to distinguish among either case—a task that proved to be cumbersome and difficult to maintain through the evolution of the model.
We thank the reviewers for inviting us to enrich our model with this and other features.
Note that LuaJIT 2.0, the other major implementation of (part of) Lua 5.2, does not perform caching of closures.
We do not include the value store since our subset implemented so far does not require it, but this may change in the future.
The language of patterns from Redex is expressive enough to be able to impose, for example, the expected representation invariant for a term that describes a finite functional mapping.
The first set of references from which reachability is computed.
While we do model tail calls, the actual testing of the mechanism in the official test suite consists in performing several recursive calls, beyond the stack limit, to actually test if the mechanism is being used. Given that performing such amount of recursive calls is not feasible within our mechanization (because of the time required), we do not include such tests.
On a system running Arch Linux updated to May, 2022, Racket v8.3, and 12 parallel processes on an Intel Core i7-10870H CPU @ 4GHz \(\times \) 8, with 16GB of RAM.
Remember that, at each iteration, a new local variable is created, to contain the corresponding value in the interval [1, 100].
Note about numeric representations: The official interpreter follows the IEEE 754 standard, while Racket has a richer set of numbers and behaviors, even including complex numbers. Therefore, it is required to convert to the required representation of numbers in order to correctly emulate the behavior of Lua.
To obtain these measures we added custom Racket code to our module Tests/RandomTesting/soundness/soundness_rand_test.rkt. We do not include it in the provided source code, tough the code reduces to counting well-formed configurations and recognizing the biggest configurations generated.
References
Adobe: Adobe Lightroom®. https://www.adobe.io/apis/creativecloud/lightroom.html (2019). Accessed 04 May 2020
Bodin, M., Chargueraud, A., Filaretti, D., Gardner, P., Maffeis, S., Naudziuniene, D., Schmitt, A., Smith, G.: A trusted mechanised JavaScript specification. In: POPL ’14 (2014)
Donnelly, K., Hallett, J.J., Kfoury, A.: Formal semantics of weak references. In: ISMM ’06 Proceedings of the 5th international symposium on Memory management, pp. 126–137 (2006)
Felleisen, M.: The calculi of lambda-v-cs conversion: a syntactic theory of control and state in imperative higher-order programming languages. PhD thesis, Indiana University (1987)
Felleisen, M., Finlder, R.B., Flatt, M.: Semantics Engineering with PLT Redex. The MIT Press, Cambridge (2009)
Gabay, Y., Kfoury, A.J.: A calculus for java’s reference objects. SIGPLAN Not 42(8), 9–17 (2007). https://doi.org/10.1145/1294297.1294299
Graham-Cumming, J.: CloudFlare’s new WAF: compiling to Lua. https://blog.cloudflare.com/cloudflares-new-waf-compiling-to-lua (2013). accessed 04 May 2020
Guha, A., Saftoiu, C., Krishnamurthi, S.: The essence of JavaScript. In: ECOOP ’10 (2010)
Haas, A., Rossberg, A., Schuff, D.L., Titzer, B.L., Holman, M., Gohman, D., Wagner, L., Zakai, A., Bastien, J.: Bringing the web up to speed with WebAssembly. In: Proceedings of the 38th ACM SIGPLAN Conference on Programming Language Design and Implementation, Association for Computing Machinery, New York, NY, USA, PLDI 2017, pp. 185–200. https://doi.org/10.1145/3062341.3062363 (2017)
Ierusalimschy, R.: Programming in Lua. Lua.org (2003)
Ierusalimschy, R., de Figueiredo, L.H., Celes, W.: Lua—an extensible extension language. Software 26(6), 635–652 (1996)
Ierusalimschy, R., de Figueiredo, L.H., Celes, W.: The evolution of an extension language: a history of lua. In: Brazilian Symposium on Programming Languages (2001)
Ierusalimschy, R., de Figueiredo, L.H., Celes, W.: Lua 5.2 Reference Manual. www.lua.org/manual/5.2/manual.html (2013)
Igarashi, A., Pierce, B.C., Wadler, P.: Featherweight Java: a minimal core calculus for Java and GJ. TOPLAS 23, 396–450 (2001)
Klein, C.: Randomized testing in PLT Redex. In: Proc. Scheme and Functional Programming, pp. 26–36 (2009)
Klein, C., McCarthy, J., Jaconette, S., Findler, R.B.: A semantics for context-sensitive reduction semantics. In: APLAS’11 (2011)
Klein, C., Clements, J., Dimoulas, C., Eastlund, C., Felleisen, M., Flatt, M., McCarthy, J.A., Rafkind, J., Tobin-Hochstadt, S., Findler, R.B.: Run your research: On the effectiveness of lightweight mechanization. In: Proceedings of the 39th Annual ACM SIGPLAN-SIGACT Symposium on Principles of Programming Languages, ACM, New York, NY, USA, POPL ’12, pp 285–296, https://doi.org/10.1145/2103656.2103691 (2012)
Krebbers, R., Wiedijk, F.: Separation logic for non-local control flow and block scope variables. In: FOSSACS’13, https://doi.org/10.1007/978-3-642-37075-5_17 (2013)
Leal, M.A., Ierusalimschy, R.: A formal semantics for finalizers. J UCS 11(7), 1198–1214 (2005). https://doi.org/10.3217/jucs-011-07-1198
Lin, H.: Operational Semantics for Featherweight Lua. Master’s thesis, San José State University (2015)
Lua Dev Team: Lua 5.2 test suite. https://www.lua.org/tests/ (2013). Accessed 04 May 2020
Lua Dev Team: Lua 5.2 reference manual. https://www.lua.org/manual/5.2/manual.html (2015). Accessed 04 May 2020
Lua Developers: Expression as statements. http://lua-users.org/wiki/ExpressionsAsStatements (2009). Accessed 04 May 2020
Lua Developers: Lua analyzers. http://lua-users.org/wiki/ProgramAnalysis (2014). Accessed 04 May 2020
Lua Developers: Lua implementations. http://lua-users.org/wiki/LuaImplementations (2018). Accessed 04 May 2020
Lua Developers: Lua directory. http://lua-users.org/wiki/LuaDirectory (2020). Accessed 04 May 2020
Lua Developers: Uses. https://www.lua.org/uses.html (2017). Accessed 04 May 2020
LuaTex: Luatex. http://www.luatex.org/languages.html (2018). Accessed 04 May 2020
Maffeis, S., Mitchell, J.C., Taly, A.: An operational semantics for JavaScript. In: APLAS ’08 (2008)
Manura, D.: Vararg the second class citizen. http://lua-users.org/wiki/VarargTheSecondClassCitizen (2007). Accessed 04 May 2020
Mascarenhas de Queiroz, F.: Optimized compilation of a dynamic language to a managed runtime environment. PhD thesis, Pontifícia Universidade Católica do Rio de Janeiro (2009)
Moura, A., Rodriguez, N., Ierusalimschy, R.: Coroutines in lua. J. Univers. Comput. Sci. 10(7), 910–25 (2004)
Pierce, B.C.: Types and Programming Languages, 1st edn. The MIT Press, Cambridge (2002)
Politz, J.G., Carroll, M.J., Lerner, B.S., Pombrio, J., Krishnamurthi, S.: A tested semantics for getters, setters, and eval in JavaScript. In: DLS ’12 (2012)
Politz, J.G., Martinez, A., Milano, M., Warren, S., Patterson, D., Li, J., Chitipothu, A., Krishnamurthi, S.: Python: the full monty: a tested semantics for the Python programming language. In: OOPSLA ’13 (2013)
Rossberg, A., (eds): Webassembly specification. https://webassembly.github.io/spec/core/ (2021). Accessed 20 Feb 2021
Soldevila, M., Ziliani, B., Silvestre, B., Fridlender, D., Mascarenhas, F.: Decoding Lua: Formal semantics for the developer and the semanticist. In: Proceedings of the 13th ACM SIGPLAN Dynamic Languages Symposium, DLS 2017 (2017)
Soldevila, M., Ziliani, B., Fridlender, D.: Understanding Lua’s garbage collection: Towards a formalized static analyzer. In: Proceedings of the 22nd International Symposium on Principles and Practice of Declarative Programming, PPDP 2020 (2020)
Acknowledgements
We thank Dr. Daniel Fridlender and Dr. Fabio Mascarenhas for their useful insights and support during the development of this research. We also thank wholeheartedly the anonymous reviewers, whose attention to detail helped us deliver a significantly improved paper.
Funding
This work was funded by the following projects: Consolidar II 33620180101063CB, UNC SECyT (Argentina) PICT D 2017-3315, ANPCyT (Argentina). The first author received a grant from CONICET, Argentina.
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of interest
The authors declare that they have no conflict of interest.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Springer Nature or its licensor holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Soldevila, M., Ziliani, B. & Silvestre, B. From Specification to Testing: Semantics Engineering for Lua 5.2. J Autom Reasoning 66, 905–952 (2022). https://doi.org/10.1007/s10817-022-09638-y
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10817-022-09638-y
Keywords
- Semantics
- Operational semantics
- Imperative languages
- Domain specific languages
- Lua
- Reduction semantics
- Randomized testing