Advertisement

Practical Mutation Testing for Smart Contracts

Conference paper
First Online:
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11737)

Abstract

Solidity smart contracts operate in a hostile environment, which introduces the need for the adequate application of testing techniques to ensure mitigation of the risk of a security incident. Mutation testing is one such technique. It allows for the evaluation of the efficiency of a test suite in detecting faults in a program, allowing developers to both assess and improve the quality of their test suites. In this paper, we propose a mutation testing framework and implement a prototype implementation called Vertigo that targets Solidity contracts for the Ethereum blockchain. We also show that mutation testing can be used to assess the test suites of real-world projects.

Keywords

Mutation testing Smart contract Solidity 
This is a preview of subscription content, log in to check access.

References

  1. 1.
  2. 2.
    Batch overlflow vulnerability - CVE-2018-10299. https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2018-10299
  3. 3.
  4. 4.
    eth-mutants: a mutation testing tool for smart contracts. https://github.com/federicobond/eth-mutants
  5. 5.
  6. 6.
  7. 7.
  8. 8.
    Parity Bug Security Alert. https://www.parity.io/security-alert-2/
  9. 9.
    PIT Mutation Testing. http://pitest.org/
  10. 10.
    Slither: Static Analyzer for Solidity. https://github.com/crytic/slither
  11. 11.
    Smart Contract Weakness Classification and Test Cases. https://smartcontractsecurity.github.io/SWC-registry/
  12. 12.
  13. 13.
  14. 14.
    The DAO Attacked: Code Issue Leads to \$60 Million Ether Theft - CoinDesk. https://www.coindesk.com/dao-attacked-code-issue-leads-60-million-ether-theft
  15. 15.
    Brent, L., et al.: Vandal: a scalable security analysis framework for smart contracts. CoRR (2018)Google Scholar
  16. 16.
    Budd, T.A., DeMillo, R.A., Lipton, R.J., Sayward, F.G.: The design of a prototype mutation system for program testing. In: Proceedings of the AFIPS National Computer Conference, vol. 74, pp. 623–627 (1978)Google Scholar
  17. 17.
    Budd, T.A., Gopal, A.S.: Program testing by specification mutation. Comput. Lang. 10(1), 63–73 (1985).  https://doi.org/10.1016/0096-0551(85)90011-6CrossRefzbMATHGoogle Scholar
  18. 18.
    Daran, M., Thévenod-Fosse, P.: Software error analysis. In: Proceedings of the 1996 International Symposium on Software Testing and Analysis - ISSTA 1996, vol. 21, pp. 158–171. ACM Press (1996).  https://doi.org/10.1145/229000.226313
  19. 19.
    Dijkstra, E.W.: Ewd 249 Notes on Structured Programming, 2nd edn. Department of Mathematics, Technische Hogeschool Eindhoven (1970)Google Scholar
  20. 20.
    Groce, A., Holmes, J., Marinov, D., Shi, A., Zhang, L.: An extensible, regular-expression-based tool for multi-language mutant generation. In: Proceedings of the 40th International Conference on Software Engineering Companion Proceeedings - ICSE 2018, pp. 25–28. ACM Press (2018).  https://doi.org/10.1145/3183440.3183485
  21. 21.
    Hildenbrandt, E., et al.: KEVM: a complete semantics of the Ethereum virtual machine. In: 2018 IEEE 31st Computer Security Foundations Symposium, pp. 204–217. IEEE (2018).  https://doi.org/10.1109/CSF.2018.00022
  22. 22.
    Hussain, S.: Mutation clustering. Master’s thesis, King’s College London, UK (2008)Google Scholar
  23. 23.
    Jia, Y., Harman, M.: An analysis and survey of the development of mutation testing. IEEE Trans. Softw. Eng. 37(5), 649–678 (2011).  https://doi.org/10.1109/TSE.2010.62CrossRefGoogle Scholar
  24. 24.
    Luu, L., Chu, D.H., Olickel, H., Saxena, P., Hobor, A.: Making smart contracts smarter. In: Proceedings of the 2016 ACM SIGSAC Conference on Computer and Communications Security - CCS 2016, pp. 254–269. ACM Press, New York (2016).  https://doi.org/10.1145/2976749.2978309
  25. 25.
    Nikolic, I., Kolluri, A., Sergey, I., Saxena, P., Hobor, A.: Finding the greedy, prodigal, and suicidal contracts at scale. In: Proceedings of the 34th Annual Computer Security Applications Conference. ACSAC 2018, pp. 653–663 (2018).  https://doi.org/10.1145/3274694.3274743
  26. 26.
    Offutt, A.J., Untch, R.H.: Mutation 2000: uniting the orthogonal. In: Wong, W.E. (ed.) Mutation Testing for the New Century, pp. 34–44. Springer, Boston (2001).  https://doi.org/10.1007/978-1-4757-5939-6_7CrossRefGoogle Scholar
  27. 27.
    Tsankov, P., Dan, A., Cohen, D.D., Gervais, A., Buenzli, F., Vechev, M.: Securify: practical security analysis of smart contracts. In: Proceedings of the 2018 ACM SIGSAC Conference on Computer and Communications Security, CCS 2018 (2018).  https://doi.org/10.1145/3243734.3243780
  28. 28.
    Wang, Y., et al.: Formal specification and verification of smart contracts for Azure blockchain. CoRR (2018)Google Scholar
  29. 29.
    Wong, W.E.: On mutation and data flow. Ph.D. thesis (1993)Google Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  1. 1.University of TwenteEnschedeThe Netherlands
  2. 2.TNOThe HagueThe Netherlands
  3. 3.ConsenSysNew YorkUSA

Personalised recommendations

Cite paper

Buy options