Skip to main content
SpringerLink
Log in

A User-Centric Evaluation of Smart Contract Analysis Tools in Decentralised Finance (DeFi)

  • Conference paper
  • First Online:
Proceedings of the International Conference on Cybersecurity, Situational Awareness and Social Media

Part of the book series: Springer Proceedings in Complexity ((SPCOM))

  • 458 Accesses

Abstract

Blockchain and smart contract technology have led to the creation of an alternative financial system called Decentralised Finance (DeFi) which has grown exponentially in the last year alone to a current value of $76B. Without a central custodian or regulator, non-technical users may find it difficult to assess the security of their favourite projects. In this trustless environment, can the current state-of-the-art smart contract analysis tools be used by non-technical users to protect investors from incurring losses and improving the security in the space? In the paper, we review the literature focusing on well-known vulnerabilities of financial smart contracts and show the scale of successful DeFi attacks. By analysing the root cause of recent exploits of contracts, we assess the feasibility of detecting these vulnerabilities by automatic verification. We investigate 21 analysis tools for detecting vulnerabilities in smart contracts with an in-depth evaluation of six tools: Slither, Mythril, DerScanner, Manticore, Oyente and Securify v2. The tools were evaluated for their efficiency and accuracy against a custom dataset containing 28 vulnerable and 16 healthy smart contracts and are ultimately rated based on how useful they may be from a DeFi user perspective. The results indicate that, while Slither received the highest rating, none of the existing tools can successfully assist DeFi users at present due to lack of reliability or lack of simplicity for the targeted market.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 169.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 159.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 219.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    https://defipulse.com/.

  2. 2.

    https://blog.defiyield.app/yearn-finance-exploit-explained-a10b07c280c8.

  3. 3.

    https://www.coindesk.com/binance-smart-chain-defi-exchange-uranium-finance-exploit.

  4. 4.

    https://tron.network/static/doc/white_paper_v_2_0.pdf.

  5. 5.

    https://github.com/EOSIO/Documentation/blob/master/TechnicalWhitePaper.md.

  6. 6.

    https://www.terra.money/Terra_White_paper.pdf.

  7. 7.

    https://solana.com/solana-whitepaper.pdf.

  8. 8.

    https://blog.trailofbits.com/2019/01/18/empire-hacking-ethereum-edition-2.

  9. 9.

    https://docs.openzeppelin.com/contracts/2.x.

  10. 10.

    https://github.com/AlphaFinanceLab/homora-v2/tree/master/audits.

  11. 11.

    https://peckshield-94632.medium.com/the-spartan-incident-root-cause-analysis-b14135d3415f.

  12. 12.

    https://capturetheether.com/.

  13. 13.

    https://ethernaut.openzeppelin.com/.

  14. 14.

    https://derscanner.com/.

  15. 15.

    https://consensys.github.io/smart-contract-best-practices/known_attacks.

  16. 16.

    https://www.rekt.news.

  17. 17.

    https://docs.soliditylang.org/en/develop/security-considerations.html?highlight=check%20effects#use-the-checks-effects-interactions-pattern.

  18. 18.

    https://docs.soliditylang.org/en/v0.5.3/contracts.html#function-modifiers.

  19. 19.

    https://docs.soliditylang.org/en/v0.8.3/080-breaking-changes.html#silent-changes-of-the-semantics.

  20. 20.

    https://docs.soliditylang.org/en/v0.6.11/contracts.html.

  21. 21.

    https://consensys.net/blog/developers/solidity-best-practices-for-smart-contract-security/.

  22. 22.

    https://consensys.github.io/smart-contract-best-practices/recommendations/.

References

  1. Nakamoto, S.: Bitcoin: a peer-to-peer electronic cash system (2008)

    Google Scholar 

  2. Buterin, V.: Ethereum white paper—a next generation smart contract & decentralized application platform. ethereum.org (2015)

  3. Atzei, N., Bartoletti, M., Cimoli, T.: A survey of attacks on Ethereum smart contracts (SoK). In: Principles of Security and Trust (2017)

    Google Scholar 

  4. Understanding the DAO Hack. http://www.coindesk.com/understanding-dao-hack-journalists

  5. Leid, A., Van der Merwe B., Visser W.: Testing Ethereum smart contracts: a comparison of symbolic analysis and fuzz testing tools. In: South African Institute of Computer Scientists and Information Technologists (2020)

    Google Scholar 

  6. Mense, A., Flatscher, M.: Security vulnerabilities in Ethereum smart contracts. In: 20th International Conference on Information Integration and Web-Based Applications & Services (2018)

    Google Scholar 

  7. Dika, A., Nowostawski, M.: Security vulnerabilities in Ethereum smart contracts. In: IEEE International Conference on Internet of Things (iThings) and IEEE Green Computing and Communications (GreenCom) and IEEE Cyber, Physical and Social Computing (CPSCom) and IEEE Smart Data (SmartData) (2018)

    Google Scholar 

  8. Huang, Y., Brian, Y., Li, R., Zhao, L., Shi, P.: Smart contract security: a software lifecycle perspective. IEEE Access 150184–150202 (2019)

    Google Scholar 

  9. Sayeed, S., Marco-Gisbert H., Caira, T.: Smart contract: attacks and protections. IEEE Access 24416–24427 (2020)

    Google Scholar 

  10. Almakhour, M., Sliman, L., Samhat, A.E., Mellouk, A.: Verification of smart contracts: a survey. Pervasive Mobile Comput. (Elsevier) (2020)

    Google Scholar 

  11. Petrović, N., Tošić, M.: Semantic approach to smart contract verification. Facta Universitatis. Series: Autom. Control Robot. 19(1), 021–037 (2020)

    Google Scholar 

  12. Gudgeon L., Perez, D., Harz, D., Gervais A., Livshits, B.: The decentralized financial crisis: attacking DeFi (2020)

    Google Scholar 

  13. He, D., Deng, Z., Zhang, Y., Chan, S., Cheng, Y., Guizani, N.: Smart contract vulnerability analysis and security audit. IEEE Netw. 34(5), 276–282 (2020)

    Article  Google Scholar 

  14. SWC registry—smart contract weakness classification and test cases [Online]. https://swcregistry.io. Accessed 14 May 2021

  15. Yang, Q., Zeng, X., Zhang, Y., Hu, W.: New loan system based on smart contract. In: The 2019 ACM International Symposium on Blockchain and Secure Critical Infrastructure, pp. 121–126 (2019)

    Google Scholar 

  16. Oosthoek, K.: Flash crash for cash: cyber threats in decentralized finance. Preprint arXiv:2106.10740 (2021)

  17. Schär, F.: Decentralized finance: on blockchain- and smart contract-based financial markets. Federal Reserve Bank of St. Louis Review (2021)

    Google Scholar 

  18. Imeri, A., Agoulmine, N., Khadraoui, D.: Smart contract modeling and verification techniques: a survey. In: 8th International Workshop on ADVANCEs in ICT Infrastructures and Services (ADVANCE 2020) (2020)

    Google Scholar 

  19. Ghaleb, A., Pattabiraman, K.: How effective are smart contract analysis tools? Evaluating. In: 29th ACM SIGSOFT International Symposium on Software Testing and Analysis, pp. 415–427 (2020)

    Google Scholar 

  20. Durieux, T., Ferreira, J., Abreu, R., Cruz, P.: Empirical review of automated analysis tools on 47,587 Ethereum smart contracts. In: ACM/IEEE 42nd International Conference on Software Engineering (2020)

    Google Scholar 

  21. Liu, Y., Li, Y., Lin, S., Zhao, R.: Towards automated verification of smart contract fairness. In: Proceedings of the 28th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering (2020)

    Google Scholar 

  22. Brent, L., Grech, N., Lagouvardos, S., Scholz, B., Smaragdakis, Y.: Ethainter: a smart contract security analyzer for composite vulnerabilities. In: 41st ACM SIGPLAN Conference on Programming Language Design and Implementation (2020)

    Google Scholar 

  23. Chen, T., Li, X., Luo, X., Zhang, X.: Under-optimized smart contracts devour your money. In: IEEE 24th International Conference on Software Analysis, Evolution and Reengineering (SANER) (2017)

    Google Scholar 

  24. Tikhomirov, S., Voskresenskaya, E., Ivanitskiy, I., Takhaviev, R., Marchenko, E., Alexandrov, E.: SmartCheck: static analysis of Ethereum smart contracts. In: 1st International Workshop on Emerging Trends in Software Engineering for Blockchain (2018)

    Google Scholar 

  25. Luu, L, Chu, D.-H., Olickel, H., Saxena, P., Hobor, A.: Making smart contracts smarter. In: Proceedings of the 2016 ACM SIGSAC Conference on Computer and Communications Security, pp. 254–269 (2016)

    Google Scholar 

  26. Mueller, B.: Github—Mythril [Online]. https://github.com/b-mueller/mythril/. Accessed 16 July 2021

  27. Feist, J., Grieco, G., Groce, A., Slither: a static analysis framework for smart contracts. In: 2019 IEEE/ACM 2nd International Workshop on Emerging Trends in Software Engineering for Blockchain (WETSEB), pp. 8–15 (2019)

    Google Scholar 

  28. Tsankov, P., Dan, A., Drachsler-Cohen, D., Gervais, A., Buenzli, F., Vechev, M.: Securify: Practical security analysis of smart contracts. In: Proceedings of the 2018 ACM SIGSAC Conference on Computer and Communications Security, pp. 67–82 (2018)

    Google Scholar 

  29. Amani, S., Bégel, M., Borti, M., Staples, M.: Towards verifying ethereum smart contract bytecode in Isabelle/HOL. In: 7th ACM SIGPLAN International Conference on Certified Programs and Proofs (2018)

    Google Scholar 

  30. Bhargavan, K., Delignat-Lavaud, A., Fournet, C., Gollamudi, A., Gonthier, G., Kobeissi, N.: Formal verification of smart contracts: short paper. In: 2016 ACM Workshop on Programming Languages and Analysis for Security (PLAS ‘16) (2016)

    Google Scholar 

  31. Mossberg, M., Manzano, F., Hennenfent, E., Groce, A., Grieco, G., Feist, J., Brunson, T., Dinaburg, A.: Manticore: a user-friendly symbolic execution framework for binaries and smart contracts. In: 2019 34th IEEE/ACM International Conference on Automated Software Engineering (ASE), pp. 1186–1189 (2019)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. School of Computing, Dublin City University, Dublin, Ireland

    Gonzalo Faura & Cezary Siewiersky

  2. Lero, School of Computing, Dublin City University, Dublin, Ireland

    Irina Tal

Authors
  1. Gonzalo Faura
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Cezary Siewiersky
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Irina Tal
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Irina Tal .

Editor information

Editors and Affiliations

  1. Research Series Ltd, London, UK

    Cyril Onwubiko

  2. Dublin City University, Dublin, Ireland

    Pierangelo Rosati

  3. Temple University, Philadelphia, PA, USA

    Aunshul Rege

  4. University of Oxford, Oxford, UK

    Arnau Erola

  5. Lupovis, Glasgow, UK

    Xavier Bellekens

  6. Ain Shams University, Cairo, Egypt

    Hanan Hindy

  7. University of Stavanger, Stavanger, Norway

    Martin Gilje Jaatun

Appendix—Rating Criteria

Appendix—Rating Criteria

Installation

  1. 1.

    We were not successful installing the tool.

  2. 2.

    Installation was cumbersome and had multiple dependencies.

  3. 3.

    Only manual installation is available.

  4. 4.

    Docker image is available.

  5. 5.

    Tool is based online and no installation is required.

Additional Features

  1. 1.

    Features limited to main scan and no option to extension.

  2. 2.

    Few basic options only.

  3. 3.

    Few additional options but scan types can be selected.

  4. 4.

    Additional scanners can be added. Arguments allow flexible selection of checks and modification of solidity version.

  5. 5.

    Tool has an excellent selection of additional options and offers significant extendibility which are also easy to use. This may include easy access to additional documentation related to vulnerability, option to scan contract by providing its address or GitHub link, presentation of data in different formats (including graphs), showing functions dependency, etc.

Effectiveness

  1. 1.

    <=20% of vulnerabilities found.

  2. 2.

    >20% of vulnerabilities found.

  3. 3.

    >50% of vulnerabilities found.

  4. 4.

    >70% of vulnerabilities found.

  5. 5.

    Tool successfully detected >90% of prepared vulnerabilities and correctly indicated the line and type of vulnerability.

Liveliness

  1. 1.

    Latest commits older than 2 years.

  2. 2.

    Latest commits are older than 1 year, but newer than 2 years.

  3. 3.

    Latest commits over 6 months, but less than 1 year.

  4. 4.

    Latest commits between 3 and 6 months.

  5. 5.

    Latest commits to git library within the last 3 months.

Open Source

  1. 1.

    Proprietary tool.

  2. 2.

    Open source and community below 100.

  3. 3.

    Open source and community over 100.

  4. 4.

    Open source and community over 200.

  5. 5.

    Tool is open source and git community is over 300 members.

Friendly Output

  1. 1.

    General names of issue category only. No description or links.

  2. 2.

    Names of issues are descriptive, but lack clear explanation.

  3. 3.

    Simple categories of issues or short descriptions.

  4. 4.

    Categories are clear and descriptions of issues are also provided. No links with further information.

  5. 5.

    The tool uses clear ranks to indicate the severity of the problem, gives clear description and links where additional information could be found.

Accuracy

  1. 1.

    6 or more false positive issues indicated (high or medium category).

  2. 2.

    5 or less false positive issues indicated (high or medium category).

  3. 3.

    2 or less false positive issues indicated (high or medium category).

  4. 4.

    No false positive with critical or high-severity issues indicated. Some recommendations may be not correct.

  5. 5.

    The tool did not indicate any false positive vulnerabilities. All warnings or recommendations are correct.

Rights and permissions

Reprints and permissions

Copyright information

© 2023 The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Faura, G., Siewiersky, C., Tal, I. (2023). A User-Centric Evaluation of Smart Contract Analysis Tools in Decentralised Finance (DeFi). In: Onwubiko, C., et al. Proceedings of the International Conference on Cybersecurity, Situational Awareness and Social Media. Springer Proceedings in Complexity. Springer, Singapore. https://doi.org/10.1007/978-981-19-6414-5_25

Download citation

Publish with us

Policies and ethics

Search

Navigation