Skip to main content
SpringerLink
Log in

BanditFuzz: Fuzzing SMT Solvers with Multi-agent Reinforcement Learning

  • Conference paper
  • First Online:
Formal Methods (FM 2021)

Part of the book series: Lecture Notes in Computer Science ((LNPSE,volume 13047))

Included in the following conference series:

Abstract

We present BanditFuzz, a multi-agent reinforcement learning (RL) guided performance fuzzer for state-of-the-art Satisfiability Modulo Theories (SMT) solvers. BanditFuzz constructs inputs that expose performance issues in a set of target solvers relative to a set of reference solvers, and is the first performance fuzzer that supports the entirety of the theories in the SMT-LIB initiative. Another useful feature of BanditFuzz is that users can specify the size of inputs they want, thus enabling developers to construct very small inputs that zero-in on a performance problem in their SMT solver relative to other competitive solvers. We evaluate BanditFuzz across 52 logics from SMT-COMP ’20 targeting competition-winning solvers against runner-ups. We baseline BanditFuzz against random fuzzing and a single agent algorithm and observe a significant improvement, with up to a 82.6% improvement in the margin of PAR-2 scores across baselines on their respective benchmarks. Furthermore, we reached out to developers and contributors of the CVC4, Z3, and Bitwuzla solvers and provide case studies of how BanditFuzz was able to expose surprising performance deficiencies in each of these tools.

This work was supported in part by NSF grants CNS-1739816 and CCF-1837132, by the DARPA LOGiCS project under contract FA8750-20-C-0156, by the iCyPhy center, and by gifts from Intel, Amazon, and Microsoft.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 89.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 119.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    BitwuzlatimedoutonallbenchmarksproducedbyBanditFuzz.

References

  1. Armando, A., Mantovani, J., Platania, L.: Bounded model checking of software using SMT solvers instead of SAT solvers. Int. J. Softw. Tools Technol. Transf. 11(1), 69–83 (2009)

    Article  Google Scholar 

  2. Baldwin, S.: Compute Canada: advancing computational research. In: Journal of Physics: Conference Series, vol. 341, p. 012001. IOP Publishing (2012)

    Google Scholar 

  3. Barrett, C., et al.: CVC4. In: Gopalakrishnan, G., Qadeer, S. (eds.) CAV 2011. LNCS, vol. 6806, pp. 171–177. Springer, Heidelberg (2011). https://doi.org/10.1007/978-3-642-22110-1_14http://www.cs.stanford.edu/~barrett/pubs/BCD+11.pdf

    Chapter  Google Scholar 

  4. Barrett, C., Fontaine, P., Tinelli, C.: The satisfiability modulo theories library (SMT-LIB). www.SMT-LIB.org (2016)

  5. Berzish, M., Ganesh, V., Zheng, Y.: Z3str3: a string solver with theory-aware heuristics. In: 2017 Formal Methods in Computer Aided Design (FMCAD), pp. 55–59. IEEE (2017)

    Google Scholar 

  6. Berzish, M., Mora, F., Kulczynski, M., Nowotka, D., Ganesh, V.: Z3str4 string solver: system description. In: SMT-COMP 2020 (2020)

    Google Scholar 

  7. Blotsky, D., Mora, F., Berzish, M., Zheng, Y., Kabir, I., Ganesh, V.: StringFuzz: a fuzzer for string solvers. In: Chockler, H., Weissenbacher, G. (eds.) CAV 2018, Part II. LNCS, vol. 10982, pp. 45–51. Springer, Cham (2018). https://doi.org/10.1007/978-3-319-96142-2_6

    Chapter  Google Scholar 

  8. Böttinger, K., Godefroid, P., Singh, R.: Deep reinforcement fuzzing. arXiv preprint arXiv:1801.04589 (2018)

  9. Brain, M., Schanda, F., Sun, Y.: Building better bit-blasting for floating-point problems. In: Vojnar, T., Zhang, L. (eds.) TACAS 2019, Part I. LNCS, vol. 11427, pp. 79–98. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-17462-0_5

    Chapter  Google Scholar 

  10. Brummayer, R., Biere, A.: Fuzzing and delta-debugging SMT solvers. In: Proceedings of the 7th International Workshop on Satisfiability Modulo Theories, pp. 1–5. ACM (2009)

    Google Scholar 

  11. Bugariu, A., Müller, P.: Automatically testing string solvers. In: International Conference on Software Engineering (ICSE), 2020. ETH Zurich (2020)

    Google Scholar 

  12. Cadar, C., Ganesh, V., Pawlowski, P.M., Dill, D.L., Engler, D.R.: Exe: automatically generating inputs of death. ACM Trans. Inf. Syst. Secur. (TISSEC) 12(2), 10 (2008)

    Article  Google Scholar 

  13. Calzavara, S., Grishchenko, I., Maffei, M.: Horndroid: practical and sound static analysis of android applications by SMT solving. In: 2016 IEEE European Symposium on Security and Privacy (EuroS&P), pp. 47–62. IEEE (2016)

    Google Scholar 

  14. Cordeiro, L., Fischer, B., Marques-Silva, J.: SMT-based bounded model checking for embedded ANSI-C software. IEEE Trans. Softw. Eng. 38(4), 957–974 (2011)

    Article  Google Scholar 

  15. de Moura, L., Bjørner, N.: Z3: an efficient SMT solver. In: Ramakrishnan, C.R., Rehof, J. (eds.) TACAS 2008. LNCS, vol. 4963, pp. 337–340. Springer, Heidelberg (2008). https://doi.org/10.1007/978-3-540-78800-3_24

    Chapter  Google Scholar 

  16. Duchene, F.: Fuzz in the dark: genetic algorithm for black-box fuzzing. In: Black-Hat (2013)

    Google Scholar 

  17. Gerlich, R., Prause, C.R.: Optimizing the parameters of an evolutionary algorithm for fuzzing and test data generation. In: 2020 IEEE International Conference on Software Testing, Verification and Validation Workshops (ICSTW), pp. 338–345. IEEE (2020)

    Google Scholar 

  18. Godefroid, P., Peleg, H., Singh, R.: Learn&fuzz: machine learning for input fuzzing. In: Proceedings of the 32nd IEEE/ACM International Conference on Automated Software Engineering, pp. 50–59. IEEE Press (2017)

    Google Scholar 

  19. Gulwani, S., Srivastava, S., Venkatesan, R.: Program analysis as constraint solving. ACM SIGPLAN Not. 43(6), 281–292 (2008)

    Article  Google Scholar 

  20. Gupta, A.K., Nadarajah, S.: Handbook of Beta Distribution and its Applications. CRC Press, Boca Raton (2004)

    Book  Google Scholar 

  21. Harris, C.R., et al.: Array programming with NumPy. Nature 585(7825), 357–362 (2020)

    Article  Google Scholar 

  22. Healy, A., Monahan, R., Power, J.F.: Predicting SMT solver performance for software verification. In: Dubois, C., Masci, P., Méry, D. (eds.) Proceedings of the Third Workshop on Formal Integrated Development Environment, F-IDE@FM 2016, Limassol, Cyprus, November 8, 2016. EPTCS, vol. 240, pp. 20–37 (2016). https://doi.org/10.4204/EPTCS.240.2

  23. Jin, G., Song, L., Shi, X., Scherpelz, J., Lu, S.: Understanding and detecting real-world performance bugs. ACM SIGPLAN Not. 47(6), 77–88 (2012)

    Article  Google Scholar 

  24. Junker, M., Huuck, R., Fehnker, A., Knapp, A.: SMT-based false positive elimination in static program analysis. In: Aoki, T., Taguchi, K. (eds.) ICFEM 2012. LNCS, vol. 7635, pp. 316–331. Springer, Heidelberg (2012). https://doi.org/10.1007/978-3-642-34281-3_23

    Chapter  Google Scholar 

  25. Katz, G., Barrett, C., Dill, D.L., Julian, K., Kochenderfer, M.J.: Reluplex: an efficient SMT solver for verifying deep neural networks. In: Majumdar, R., Kunčak, V. (eds.) CAV 2017, Part I. LNCS, vol. 10426, pp. 97–117. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-63387-9_5

    Chapter  Google Scholar 

  26. Komuravelli, A., Gurfinkel, A., Chaki, S.: SMT-based model checking for recursive programs. Form. Methods Syst. Des. 48(3), 175–205 (2016)

    Article  Google Scholar 

  27. Lemieux, C., Padhye, R., Sen, K., Song, D.: PerfFuzz: automatically generating pathological inputs. In: Proceedings of the 27th ACM SIGSOFT International Symposium on Software Testing and Analysis, pp. 254–265 (2018)

    Google Scholar 

  28. Manes, V.J., et al.: Fuzzing: art, science, and engineering. arXiv preprint arXiv:1812.00140 (2018)

  29. Manès, V.J., Kim, S., Cha, S.K.: Ankou: guiding grey-box fuzzing towards combinatorial difference. In: Proceedings of the ACM/IEEE 42nd International Conference on Software Engineering, pp. 1024–1036 (2020)

    Google Scholar 

  30. Mansur, M.N., Christakis, M., Wüstholz, V., Zhang, F.: Detecting critical bugs in SMT solvers using blackbox mutational fuzzing. arXiv preprint arXiv:2004.05934 (2020)

  31. Heule, M., Matti Järvisalo, M.S.: Sat race 2019 (2019). http://sat-race-2019.ciirc.cvut.cz/

  32. Niemetz, A., Preiner, M.: Bitwuzla at the SMT-COMP 2020. CoRR abs/2006.01621 (2020). https://arxiv.org/abs/2006.01621

  33. Niemetz, A., Preiner, M.: Ternary propagation-based local search for more bit-precise reasoning. In: 2020 Formal Methods in Computer Aided Design, FMCAD 2020, Haifa, Israel, September 21–24, 2020, pp. 214–224. IEEE (2020). https://doi.org/10.34727/2020/isbn.978-3-85448-042-6_29

  34. Niemetz, A., Preiner, M., Biere, A.: Model-based API testing for SMT solvers. In: Proceedings of the 15th International Workshop on Satisfiability Modulo Theories, SMT, pp. 24–28 (2017)

    Google Scholar 

  35. Niemetz, A., Preiner, M., Biere, A.: Model-based API testing for SMT solvers. In: Brain, M., Hadarean, L. (eds.) Proceedings of the 15th International Workshop on Satisfiability Modulo Theories, SMT 2017), affiliated with the 29th International Conference on Computer Aided Verification, CAV 2017, Heidelberg, Germany, July 24–28, 2017, p. 10 (2017)

    Google Scholar 

  36. Patil, K., Kanade, A.: Greybox fuzzing as a contextual bandits problem. arXiv preprint arXiv:1806.03806 (2018)

  37. Pimpalkhare, N., Mora, F., Polgreen, E., Seshia, S.A.: MedleySolver: online SMT algorithm selection. In: Li, C.-M., Manyà, F. (eds.) SAT 2021. LNCS, vol. 12831, pp. 453–470. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-80223-3_31

    Chapter  Google Scholar 

  38. Rebert, A., et al.: Optimizing seed selection for fuzzing. In: USENIX Security Symposium, pp. 861–875 (2014)

    Google Scholar 

  39. Reynolds, A., Deters, M., Kuncak, V., Tinelli, C., Barrett, C.: Counterexample-guided quantifier instantiation for synthesis in SMT. In: Kroening, D., Păsăreanu, C.S. (eds.) CAV 2015, Part II. LNCS, vol. 9207, pp. 198–216. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-21668-3_12

    Chapter  Google Scholar 

  40. Russo, D.J., Van Roy, B., Kazerouni, A., Osband, I., Wen, Z., et al.: A tutorial on Thompson sampling. Found. Trends® Mach. Learn. 11(1), 1–96 (2018)

    Article  Google Scholar 

  41. Saavedra, G.J., Rodhouse, K.N., Dunlavy, D.M., Kegelmeyer, P.W.: A review of machine learning applications in fuzzing. arXiv preprint arXiv:1906.11133 (2019)

  42. Scott, J., Mora, F., Ganesh, V.: BanditFuzz: fuzzing SMT solvers with reinforcement learning. UWSpace. http://hdl.handle.net/10012/15753 (2020)

  43. Scott, J., Niemetz, A., Preiner, M., Nejati, S., Ganesh, V.: MachSMT: a machine learning-based algorithm selector for SMT solvers. In: TACAS 2021, Part II. LNCS, vol. 12652, pp. 303–325. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-72013-1_16

    Chapter  Google Scholar 

  44. Srivastava, S., Gulwani, S., Foster, J.S.: From program verification to program synthesis. In: Proceedings of the 37th Annual ACM SIGPLAN-SIGACT Symposium on Principles of Programming Languages, pp. 313–326 (2010)

    Google Scholar 

  45. Sutton, M., Greene, A., Amini, P.: Fuzzing: Brute Force Vulnerability Discovery. Pearson Education, London (2007)

    Google Scholar 

  46. Sutton, R.S., Barto, A.G.: Reinforcement Learning: An Introduction. MIT Press, Cambridge (2018)

    MATH  Google Scholar 

  47. Takanen, A., Demott, J.D., Miller, C.: Fuzzing for Software Security Testing and Quality Assurance. Artech House, USA (2008)

    MATH  Google Scholar 

  48. Winterer, D., Zhang, C., Su, Z.: On the unusual effectiveness of type-aware operator mutations for testing SMT solvers. Proc. ACM Program. Lang. 4(OOPSLA), 1–25 (2020)

    Article  Google Scholar 

  49. Winterer, D., Zhang, C., Su, Z.: Validating SMT solvers via semantic fusion. In: PLDI, pp. 718–730 (2020)

    Google Scholar 

  50. Woo, M., Cha, S.K., Gottlieb, S., Brumley, D.: Scheduling black-box mutational fuzzing. In: Proceedings of the 2013 ACM SIGSAC Conference on Computer & Communications Security, pp. 511–522. ACM (2013)

    Google Scholar 

  51. Zalewski, M.: afl-fuzz: making up grammar with a dictionary in hand (2015). https://lcamtuf.blogspot.com/2015/01/afl-fuzz-making-up-grammar-with.html

  52. Zalewski, M.: American Fuzzing Lop (2015)

    Google Scholar 

Download references

Acknowledgements

We would like to thank the following solver developers for their collaboration and feedback on BanditFuzz: Martin Brain, Aina Niemetz, and Mathias Preiner of the Bitwuzla and CVC4 teams, as well as Mitja Kulczynski and Murphy Berzish who developed Z3str4.

Author information

Authors and Affiliations

  1. University of Waterloo, Waterloo, ON, Canada

    Joseph Scott, Trishal Sudula, Hammad Rehman & Vijay Ganesh

  2. University of California, Berkeley, USA

    Federico Mora

Authors
  1. Joseph Scott
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Trishal Sudula
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Hammad Rehman
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Federico Mora
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Vijay Ganesh
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Joseph Scott .

Editor information

Editors and Affiliations

  1. University of Twente, Enschede, The Netherlands

    Marieke Huisman

  2. NASA Ames Research Center, Moffett Field, CA, USA

    Corina Păsăreanu

  3. Institute of Software, Chinese Academy of Sciences, Beijing, China

    Naijun Zhan

Rights and permissions

Reprints and permissions

Copyright information

© 2021 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Scott, J., Sudula, T., Rehman, H., Mora, F., Ganesh, V. (2021). BanditFuzz: Fuzzing SMT Solvers with Multi-agent Reinforcement Learning. In: Huisman, M., Păsăreanu, C., Zhan, N. (eds) Formal Methods. FM 2021. Lecture Notes in Computer Science(), vol 13047. Springer, Cham. https://doi.org/10.1007/978-3-030-90870-6_6

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-90870-6_6

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-90869-0

  • Online ISBN: 978-3-030-90870-6

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation