Skip to main content
SpringerLink
Log in

Practical Mutation Testing for Smart Contracts

  • Conference paper
  • First Online:
Data Privacy Management, Cryptocurrencies and Blockchain Technology (DPM 2019, CBT 2019)

Part of the book series: Lecture Notes in Computer Science ((LNSC,volume 11737))

Abstract

Solidity smart contracts operate in a hostile environment, which introduces the need for the adequate application of testing techniques to ensure mitigation of the risk of a security incident. Mutation testing is one such technique. It allows for the evaluation of the efficiency of a test suite in detecting faults in a program, allowing developers to both assess and improve the quality of their test suites. In this paper, we propose a mutation testing framework and implement a prototype implementation called Vertigo that targets Solidity contracts for the Ethereum blockchain. We also show that mutation testing can be used to assess the test suites of real-world projects.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 54.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 69.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    With freeze, we mean the act of blocking users from accessing the currency stored in the contract.

  2. 2.

    https://github.com/JoranHonig/vertigo.

References

  1. aragonOS. https://hack.aragon.org/docs/aragonos-intro.html

  2. Batch overlflow vulnerability - CVE-2018-10299. https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2018-10299

  3. CryptoKitties. https://www.cryptokitties.co/

  4. eth-mutants: a mutation testing tool for smart contracts. https://github.com/federicobond/eth-mutants

  5. Manticore. https://github.com/trailofbits/manticore

  6. Mythril. https://github.com/consensys/mythril

  7. openzeppelin-solidity. https://github.com/OpenZeppelin/openzeppelin-solidity

  8. Parity Bug Security Alert. https://www.parity.io/security-alert-2/

  9. PIT Mutation Testing. http://pitest.org/

  10. Slither: Static Analyzer for Solidity. https://github.com/crytic/slither

  11. Smart Contract Weakness Classification and Test Cases. https://smartcontractsecurity.github.io/SWC-registry/

  12. Solidity. https://github.com/ethereum/solidity

  13. SWC-129. https://smartcontractsecurity.github.io/SWC-registry/docs/SWC-129

  14. The DAO Attacked: Code Issue Leads to \$60 Million Ether Theft - CoinDesk. https://www.coindesk.com/dao-attacked-code-issue-leads-60-million-ether-theft

  15. Brent, L., et al.: Vandal: a scalable security analysis framework for smart contracts. CoRR (2018)

    Google Scholar 

  16. Budd, T.A., DeMillo, R.A., Lipton, R.J., Sayward, F.G.: The design of a prototype mutation system for program testing. In: Proceedings of the AFIPS National Computer Conference, vol. 74, pp. 623–627 (1978)

    Google Scholar 

  17. Budd, T.A., Gopal, A.S.: Program testing by specification mutation. Comput. Lang. 10(1), 63–73 (1985). https://doi.org/10.1016/0096-0551(85)90011-6

    Article  MATH  Google Scholar 

  18. Daran, M., Thévenod-Fosse, P.: Software error analysis. In: Proceedings of the 1996 International Symposium on Software Testing and Analysis - ISSTA 1996, vol. 21, pp. 158–171. ACM Press (1996). https://doi.org/10.1145/229000.226313

  19. Dijkstra, E.W.: Ewd 249 Notes on Structured Programming, 2nd edn. Department of Mathematics, Technische Hogeschool Eindhoven (1970)

    Google Scholar 

  20. Groce, A., Holmes, J., Marinov, D., Shi, A., Zhang, L.: An extensible, regular-expression-based tool for multi-language mutant generation. In: Proceedings of the 40th International Conference on Software Engineering Companion Proceeedings - ICSE 2018, pp. 25–28. ACM Press (2018). https://doi.org/10.1145/3183440.3183485

  21. Hildenbrandt, E., et al.: KEVM: a complete semantics of the Ethereum virtual machine. In: 2018 IEEE 31st Computer Security Foundations Symposium, pp. 204–217. IEEE (2018). https://doi.org/10.1109/CSF.2018.00022

  22. Hussain, S.: Mutation clustering. Master’s thesis, King’s College London, UK (2008)

    Google Scholar 

  23. Jia, Y., Harman, M.: An analysis and survey of the development of mutation testing. IEEE Trans. Softw. Eng. 37(5), 649–678 (2011). https://doi.org/10.1109/TSE.2010.62

    Article  Google Scholar 

  24. Luu, L., Chu, D.H., Olickel, H., Saxena, P., Hobor, A.: Making smart contracts smarter. In: Proceedings of the 2016 ACM SIGSAC Conference on Computer and Communications Security - CCS 2016, pp. 254–269. ACM Press, New York (2016). https://doi.org/10.1145/2976749.2978309

  25. Nikolic, I., Kolluri, A., Sergey, I., Saxena, P., Hobor, A.: Finding the greedy, prodigal, and suicidal contracts at scale. In: Proceedings of the 34th Annual Computer Security Applications Conference. ACSAC 2018, pp. 653–663 (2018). https://doi.org/10.1145/3274694.3274743

  26. Offutt, A.J., Untch, R.H.: Mutation 2000: uniting the orthogonal. In: Wong, W.E. (ed.) Mutation Testing for the New Century, pp. 34–44. Springer, Boston (2001). https://doi.org/10.1007/978-1-4757-5939-6_7

    Chapter  Google Scholar 

  27. Tsankov, P., Dan, A., Cohen, D.D., Gervais, A., Buenzli, F., Vechev, M.: Securify: practical security analysis of smart contracts. In: Proceedings of the 2018 ACM SIGSAC Conference on Computer and Communications Security, CCS 2018 (2018). https://doi.org/10.1145/3243734.3243780

  28. Wang, Y., et al.: Formal specification and verification of smart contracts for Azure blockchain. CoRR (2018)

    Google Scholar 

  29. Wong, W.E.: On mutation and data flow. Ph.D. thesis (1993)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. University of Twente, Enschede, The Netherlands

    Joran J. Honig, Maarten H. Everts & Marieke Huisman

  2. TNO, The Hague, The Netherlands

    Maarten H. Everts

  3. ConsenSys, New York, USA

    Joran J. Honig

Authors
  1. Joran J. Honig
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Maarten H. Everts
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Marieke Huisman
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Joran J. Honig .

Editor information

Editors and Affiliations

  1. Universitat Oberta de Catalunya, Barcelona, Spain

    Cristina Pérez-Solà

  2. Universitat Autonoma de Barcelona, Bellaterra, Spain

    Guillermo Navarro-Arribas

  3. University of Luxembourg, Esch-sur-Alzette, Luxembourg

    Alex Biryukov

  4. Institut Mines-Télécom, Evry, France

    Joaquin Garcia-Alfaro

Rights and permissions

Reprints and permissions

Copyright information

© 2019 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Honig, J.J., Everts, M.H., Huisman, M. (2019). Practical Mutation Testing for Smart Contracts. In: Pérez-Solà, C., Navarro-Arribas, G., Biryukov, A., Garcia-Alfaro, J. (eds) Data Privacy Management, Cryptocurrencies and Blockchain Technology. DPM CBT 2019 2019. Lecture Notes in Computer Science(), vol 11737. Springer, Cham. https://doi.org/10.1007/978-3-030-31500-9_19

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-31500-9_19

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-31499-6

  • Online ISBN: 978-3-030-31500-9

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation