Skip to main content
SpringerLink
Log in

A systematic review of fuzzing

  • Application of soft computing
  • Published:
Soft Computing Aims and scope Submit manuscript

Abstract

Fuzzing is an important technique in software and security testing that involves continuously generating a large number of test cases against target programs to discover unexpected behaviors such as bugs, crashes, and vulnerabilities. Recently, fuzzing has advanced considerably owing to the emergence of new methods and corresponding tools. However, it still suffers from low coverage, ineffective detection of specific vulnerabilities, and difficulty in deploying complex applications. Therefore, to comprehensively survey the development of fuzzing techniques and analyze their advantages and existing challenges, this paper provides a comprehensive survey of the development of fuzzing techniques, summarizes the main research issues, and provides a categorized overview of the latest research advances and applications. The paper first introduces the background and related work on fuzzing. Research issues are subsequently addressed and summarized, along with the latest research developments. Furthermore, various customized fuzzing techniques in different applications are presented. Finally, the paper discusses future research directions.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Algorithm 1
Fig. 4
Fig. 5

Similar content being viewed by others

Data availability

Not applicable.

Code availability

Not applicable.

References

  • Abhishek A, Cris N (2012) Fuzzing for security. https://blog.chromium.org/2012/04/fuzzing-for-security.html. Accessed on 30 March 2021

  • Aschermann C, Schumilo S, Blazytko T, Gawlik R, Holz T (2019) REDQUEEN: fuzzing with input-to-state correspondence. In: Proceedings 2019 network and distributed system security symposium. https://doi.org/10.14722/ndss.2019.23371

  • Avci MG, Avci M (2019) An adaptive large neighborhood search approach for multiple traveling repairman problem with profits. Comput Oper Res 111:367–385. https://doi.org/10.1016/j.cor.2019.07.012

    Article  MathSciNet  Google Scholar 

  • Avgerinos T, Rebert A, Cha SK, Brumley D (2014) Enhancing symbolic execution with veritesting. In: Proceedings of the 36th international conference on software engineering, pp 1083–1094. https://doi.org/10.1145/2568225.2568293

  • Baldoni R, Coppa E, D’elia DC, Demetrescu C, Finocchi I (2018) A survey of symbolic execution techniques. ACM Comput Surv (CSUR) 51(3):1–39

    Article  Google Scholar 

  • Banks G, Cova M, Felmetsger V, Almeroth K, Kemmerer R, Vigna G (2006) SNOOZE: toward a stateful network protocol fuzzer. In: International conference on information security, pp 343–358. https://doi.org/10.1007/11836810_25

  • Beaman C, Redbourne M, Mummery JD, Hakak S (2022) Fuzzing vulnerability discovery techniques: survey, challenges and future directions. Comput Secur 120:1–13. https://doi.org/10.1016/j.cose.2022.102813

    Article  Google Scholar 

  • Bekrar S, Bekrar C, Groz R, Mounier L (2012) A taint based approach for smart fuzzing. In: 2012 IEEE fifth international conference on software testing, verification and validation, pp 818–825. https://doi.org/10.1109/icst.2012.182

  • Blazytko T, Aschermann C, Schlögel M, Abbasi A, Schumilo S, Wörner S, Holz T (2019) GRIMOIRE: synthesizing structure while fuzzing. In: 28th USENIX security symposium, pp 1985–2002

  • Blotsky D, Mora F, Berzish M, Zheng Y, Kabir I, Ganesh V (2018) Stringfuzz: a fuzzer for string solvers. In: International conference on computer aided verification, pp 45–51. https://doi.org/10.1007/978-3-319-96142-2_6

  • Böhme M, Pham V, Roychoudhury A (2019) Coverage-based greybox fuzzing as Markov chain. IEEE Trans Softw Eng 45(5):489–506. https://doi.org/10.1109/tse.2017.2785841

    Article  Google Scholar 

  • Böhme M, Falk B (2020) Fuzzing: on the exponential cost of vulnerability discovery. In: Proceedings of the 28th ACM joint meeting on European software engineering conference and symposium on the foundations of software engineering, pp 713–724. https://doi.org/10.1145/3368089.3409729

  • Böhme M, Pham VT, Nguyen MD, Roychoudhury A (2017) Directed greybox fuzzing. In: Proceedings of the 2017 ACM SIGSAC conference on computer and communications security, pp 2329–2344. https://doi.org/10.1145/3133956.3134020

  • Brad A (2009) Adobe reader and acrobat security initiative. https://blogs.adobe.com/security/2009/05/adobe_reader_and_acrobat_secur.html. Accessed on 30 March 2021

  • Brennan T, Saha S, Bultan T (2020) JVM fuzzing for JIT-induced side-channel detection. In: Proceedings of the ACM/IEEE 42nd international conference on software engineering, pp 1011–1023. https://doi.org/10.1145/3377811.3380432

  • Bugariu A, Müller P (2020) Automatically testing string solvers. In: Proceedings of the ACM/IEEE 42nd international conference on software engineering, pp 1459–1470. https://doi.org/10.1145/3377811.3380398

  • Chen Y, Ahmadi M, Farkhani RM, Wang B, Lu L (2020) MEUZZ: smart seed scheduling for hybrid fuzzing. In: International symposium on recent advances in intrusion detection, pp 77–92. https://doi.org/10.14722/ndss.2021.24486

  • Chen P, Chen H (2018) Angora: efficient fuzzing by principled search. In: 2018 IEEE symposium on security and privacy, pp 711–725. https://doi.org/10.1109/sp.2018.00046

  • Chen H, Guo S, Xue Y, Sui Y, Zhang C, Li Y, Wang H, Liu Y (2020) MUZZ: thread-aware grey-box fuzzing for effective bug hunting in multithreaded programs. In: 29th USENIX security symposium, pp 2325–2342

  • Chen Y, Jiang Y, Ma F, Liang J, Wang M, Zhou C, Jiao X, Su Z (2019) EnFuzz: ensemble fuzzing with seed synchronization among diverse fuzzers. In: 28th USENIX security symposium, pp 1967–1983

  • Chen P, Liu J, Chen H (2019) Matryoshka: fuzzing deeply nested branches. In: Proceedings of the 2019 ACM SIGSAC conference on computer and communications security, pp 499–513. https://doi.org/10.1145/3319535.3363225

  • Chen Y, Li P, Xu J, Guo S, Zhou R, Zhang Y, Wei T, Lu L (2020) Savior: towards bug-driven hybrid testing. In: 2020 IEEE symposium on security and privacy, pp 1580–1596. https://doi.org/10.1109/sp40000.2020.00002

  • Chen H, Xue Y, Li Y, Chen B, Xie X, Wu X, Liu Y (2018) Hawkeye: towards a desired directed grey-box fuzzer. In: Proceedings of the 2018 ACM SIGSAC conference on computer and communications security, pp 2095–2108. https://doi.org/10.1145/3243734.3243849

  • Chris E, Matt M, Tavis O (2011) Fuzzing at scale. https://security.googleblog.com/2011/08/fuzzing-at-scale.html. Accessed on 30 March 2021

  • Cisco secure development lifecycle (2018). https://www.cisco.com/c/en/us/about/trust-center/technology-built-in-security.html#~processes. Accessed on 6 Aug 2023

  • Clang (2007). https://clang.llvm.org/. Accessed on 1 March 2021

  • Corina J, Machiry A, Salls C, Shoshitaishvili Y, Hao S, Kruegel C, Vigna G (2017) Difuze: interface aware fuzzing for kernel drivers. In: Proceedings of the 2017 ACM SIGSAC conference on computer and communications security, pp 2123–2138. https://doi.org/10.1145/3133956.3134069

  • CVC4 (2021). https://cvc4.github.io/. Accessed on 30 March 2021

  • CVE-fuzzing-poc (2016). https://github.com/geeknik/cve-fuzzing-poc. Accessed on 30 March 2021

  • D’Angelo G, Farsimadan E, Ficco M, Palmieri F, Robustelli A (2023) Privacy-preserving malware detection in android-based IoT devices through federated Markov chains. Futur Gener Comput Syst 148:93–105. https://doi.org/10.1016/j.future.2023.05.021

    Article  Google Scholar 

  • D’Angelo G, Ficco M, Robustelli A (2023) An association rules-based approach for anomaly detection on can-bus. In: International conference on computational science and its applications. Springer, pp 174–190

  • Darpa cyber grand challenge. https://www.darpa.mil/program/cyber-grand-challenge. Accessed on 6 Aug 2023

  • Deng Y, Xia CS, Peng H, Yang C, Zhang L (2023) Large language models are zero-shot fuzzers: fuzzing deep-learning libraries via large language models. In: Proceedings of the 32nd ACM SIGSOFT international symposium on software testing and analysis, pp 423–435

  • Ding ZY, Goues CL (2021) An empirical study of oss-fuzz bugs. arXiv preprint arXiv:2103.11518

  • Donaldson AF, Clayton B, Harrison R, Mohsin H, Neto D, Teliman V, Watson H (2023) Industrial deployment of compiler fuzzing techniques for two GPU shading languages. In: 2023 IEEE conference on software testing, verification and validation, pp 374–385. https://doi.org/10.1109/ICST57152.2023.00042

  • Dynamorio. https://github.com/DynamoRIO/dynamorio. Accessed on 30 March 2021

  • Edwards SH (2001) A framework for practical, automated black-box testing of component-based software. Softw Test Veri Reliab 11(2):97–111. https://doi.org/10.1002/stvr.224

    Article  Google Scholar 

  • Eisele M, Maugeri M, Shriwas R, Huth C, Bella G (2022) Embedded fuzzing: a review of challenges, tools, and solutions. Cybersecurity 5(1–18):18. https://doi.org/10.1186/s42400-022-00123-y

    Article  Google Scholar 

  • Fioraldi A, Maier D, Eißfeldt H, Heuse M (2020) AFL++ : combining incremental steps of fuzzing research. In: 14th USENIX workshop on offensive technologies, pp 1–12

  • Frida. https://frida.re/. Accessed on 30 March 2021

  • Fuzzdata (2015). https://github.com/MozillaSecurity/fuzzdata.git. Accessed on 30 March 2021

  • Ganesh V, Leek T, Rinard M (2009) Taint-based directed whitebox fuzzing. In: 2009 IEEE 31st international conference on software engineering, pp 474–484. https://doi.org/10.1109/icse.2009.5070546

  • Gan S, Zhang C, Chen P, Zhao B, Qin X, Wu D, Chen Z (2020) GREYONE: data flow sensitive fuzzing. In: 29th USENIX security symposium, pp 2577–2594

  • Gan S, Zhang C, Qin X, Tu X, Li K, Pei Z, Chen Z (2018) Collafl: path sensitive fuzzing. In: 2018 IEEE symposium on security and privacy, pp 679–696. https://doi.org/10.1109/sp.2018.00040

  • Gascon H, Wressnegger C, Yamaguchi F, Arp D, Rieck K (2015) Pulsar: stateful black-box fuzzing of proprietary network protocols. In: Security and privacy in communication networks: 11th EAI international conference, SecureComm 2015, Dallas, TX, USA, 26–29 Oct 2015, Proceedings 11. Springer, pp 330–347. https://doi.org/10.1007/978-3-319-28865-9_18

  • GDB (1988). https://www.gnu.org/software/gdb/. Accessed on 30 March 2021

  • Github. https://github.com/. Accessed on 6 Aug 2023

  • Godefroid P (2020) Fuzzing: hack, art, and science. Commun ACM 63(2):70–76. https://doi.org/10.1145/3363824

    Article  Google Scholar 

  • Godefroid P, Levin MY, Molnar DA (2008) Automated whitebox fuzz testing. Netw Distrib Secur Symp 8:151–166

    Google Scholar 

  • Godefroid P, Kiezun A, Levin MY (2008) Grammar-based whitebox fuzzing. In: Proceedings of the 29th ACM SIGPLAN conference on programming language design and implementation, pp 206–215. https://doi.org/10.1145/1375581.1375607

  • Godefroid P, Kiezun A, Levin MY (2008) Grammar-based whitebox fuzzing. In: Proceedings of the 29th ACM SIGPLAN conference on programming language design and implementation, pp 206–215. https://doi.org/10.1145/1375581.1375607

  • Godefroid P, Peleg H, Singh R (2017) Learn &fuzz: machine learning for input fuzzing. In: 2017 32nd IEEE/ACM international conference on automated software engineering, pp 50–59. https://doi.org/10.1109/ase.2017.8115618

  • google: ClusterFuzz. https://github.com/google/clusterfuzz. Accessed on 30 March 2021

  • Gorbunov S, Rosenbloom A (2010) Autofuzz: automated network protocol fuzzing framework. Int J Comput Sci Netw Secur 10(8):239

    Google Scholar 

  • Güler E, Aschermann C, Abbasi A, Holz T (2019) AntiFuzz: impeding fuzzing audits of binary executables. In: 28th USENIX security symposium, pp 1931–1947

  • Güler E, Görz P, Geretto E, Jemmett A, Österlund S, Bos H, Giuffrida C, Holz T (2020) Cupid: automatic fuzzer selection for collaborative fuzzing. In: Annual computer security applications conference, pp 360–372. https://doi.org/10.1145/3427228.3427266

  • Han H, Cha SK (2017) IMF: inferred model-based fuzzer. In: Proceedings of the 2017 ACM SIGSAC conference on computer and communications security, pp 2345–2358 https://doi.org/10.1145/3133956.3134103

  • Han W, Joe B, Lee B, Song C, Shin I (2018) Enhancing memory error detection for large-scale applications and fuzz testing. In: Proceedings 2018 network and distributed system security symposium. https://doi.org/10.14722/ndss.2018.23312

  • He J, Balunović M, Ambroladze N, Tsankov P, Vechev M (2019) Learning to fuzz from symbolic execution with application to smart contracts. In: Proceedings of the 2019 ACM SIGSAC conference on computer and communications security, pp 531–548. https://doi.org/10.1145/3319535.3363230

  • Heelan S, Melham T, Kroening D (2019) Gollum: modular and greybox exploit generation for heap overflows in interpreters. In: Proceedings of the 2019 ACM SIGSAC conference on computer and communications security, pp 1–18. https://doi.org/10.1145/3319535.3354224

  • Henderson A, Yin H, Jin G, Han H, Deng H (2017) VDF: targeted evolutionary fuzz testing of virtual devices. In: International symposium on research in attacks, intrusions, and defenses, pp 3–25. https://doi.org/10.1007/978-3-319-66332-6_1

  • HonggFuzz (2015). https://honggfuzz.dev/. Accessed on 30 March 2021

  • Hou L, Su Y (2022) Swarm activity-based dynamic PSO for distribution decision. Int J Autom Control 16(3/4):503–517. https://doi.org/10.1504/ijaac.2022.10046277

    Article  Google Scholar 

  • Huang H, Yao P, Wu R, Shi Q, Zhang C (2020) PANGOLIN: incremental hybrid fuzzing with polyhedral path abstraction. In: 2020 IEEE symposium on security and privacy, pp 1613–1627. https://doi.org/10.1109/sp40000.2020.00063

  • IDA (2003). https://www.hex-rays.com/products/ida/. Accessed on 30 March 2021

  • Jack T, Li M (2016) When virtualization encounter AFL. In: Black Hat Europe

  • Jain V, Rawat S, Giuffrida C, Bos H (2018) TIFF: using input type inference to improve fuzzing. In: Proceedings of the 34th annual computer security applications conference, pp 505–517. https://doi.org/10.1145/3274694.3274746

  • Jeong DR, Kim K, Shivakumar B, Lee B, Shin I (2019) Razzer: finding kernel race bugs through fuzzing. In: 2019 IEEE symposium on security and privacy, pp 754–768. https://doi.org/10.1109/sp.2019.00017

  • Jesse H. TriforceAFL. https://github.com/nccgroup/TriforceAFL. Accessed on 30 March 2021

  • Jiang B, Liu Y, Chan W (2018) ContractFuzzer: fuzzing smart contracts for vulnerability detection. In: 2018 33rd IEEE/ACM international conference on automated software engineering, pp 259–269. https://doi.org/10.1145/3238147.3238177

  • Jones D. trinity. https://github.com/kernelslacker/trinity. Accessed on 30 March 2021

  • Ju Y, Dong J, Chen S (2021) Recovering surface normal and arbitrary images: a dual regression network for photometric stereo. IEEE Trans Image Process 30:3676–3690. https://doi.org/10.1109/TIP.2021.3064230

    Article  ADS  PubMed  Google Scholar 

  • Kim K, Jeong DR, Kim CH, Jang Y, Shin I, Lee B (2020) HFL: hybrid fuzzing on the Linux kernel. In: Proceedings of the 2020 annual network and distributed system security symposium, pp 1–17. https://doi.org/10.14722/ndss.2020.24018

  • Lemieux C, Padhye R, Sen K, Song D (2018) PerfFuzz: automatically generating pathological inputs. In: Proceedings of the 27th ACM SIGSOFT international symposium on software testing and analysis, pp 254–265. https://doi.org/10.1145/3213846.3213874

  • Lemieux C, Sen K (2018) FairFuzz: a targeted mutation strategy for increasing greybox fuzz testing coverage. In: Proceedings of the 33rd ACM/IEEE international conference on automated software engineering, pp 475–485. https://doi.org/10.1145/3238147.3238176

  • Li J, Zhao B, Zhang C (2018) Fuzzing: a survey. Cybersecurity 1(1):1–13. https://doi.org/10.1186/s42400-018-0002-y

    Article  Google Scholar 

  • Liang H, Pei X, Jia X, Shen W, Zhang J (2018) Fuzzing: state of the art. IEEE Trans Reliab 67(3):1199–1218. https://doi.org/10.1109/tr.2018.2834476

    Article  Google Scholar 

  • Liang H, Pei X, Jia X, Shen W, Zhang J (2018) Fuzzing: state of the art. IEEE Trans Reliab 67(3):1199–1218. https://doi.org/10.1145/3457913.3457934

    Article  Google Scholar 

  • Liang J, Jiang Y, Chen Y, Wang M, Zhou C, Sun J (2018) PAFL: extend fuzzing optimizations of single mode to industrial parallel mode. In: Proceedings of the 2018 26th ACM joint meeting on european software engineering conference and symposium on the foundations of software engineering, pp 809–814. https://doi.org/10.1145/3236024.3275525

  • Liang J, Wang M, Zhou C, Wu Z, Jiang Y, Liu J, Liu Z, Sun J (2022) PATA: fuzzing with path aware taint analysis. In: 2022 IEEE symposium on security and privacy, pp 1–17. https://doi.org/10.1109/sp46214.2022.9833594

  • Li Y, Chen B, Chandramohan M, Lin SW, Liu Y, Tiu A (2017) Steelix: program-state based binary fuzzing. In: Proceedings of the 2017 11th joint meeting on foundations of software engineering, pp 627–637. https://doi.org/10.1145/3106237.3106295

  • Li Y, Ji S, Chen Y, Liang S, Lee WH, Chen Y, Lyu C, Wu C, Beyah R, Cheng P, Lu K, Wang T (2021) UNIFUZZ: a holistic and pragmatic metrics-driven platform for evaluating fuzzers. In: 30th USENIX security symposium, pp 1–18

  • Lin P, Hong Z, Li Y, Wu L (2021) A priority based path searching method for improving hybrid fuzzing. Comput Secur 105:1–17. https://doi.org/10.1016/j.cose.2021.102242

    Article  Google Scholar 

  • Li X, Sun L, Qu H, Jang R, Yan Z (2021) OTA: an operation-oriented time allocation strategy for greybox fuzzing. In: 28th IEEE international conference on software analysis, evolution and reengineering, pp 108–118. https://doi.org/10.1109/saner50967.2021.00019

  • Liu C, Zou D, Luo P, Zhu BB, Jin H (2018) A heuristic framework to detect concurrency vulnerabilities. In: Proceedings of the 34th annual computer security applications conference, pp 529–541. https://doi.org/10.1145/3274694.3274718

  • Li Y, Xue Y, Chen H, Wu X, Zhang C, Xie X, Wang H, Liu Y (2019) Cerebro: context-aware adaptive fuzzing for effective vulnerability detection. In: Proceedings of the 2019 27th ACM joint meeting on European software engineering conference and symposium on the foundations of software engineering, pp 533–544. https://doi.org/10.1145/3338906.3338975

  • Lou B, Song J (2020) A study on using code coverage information extracted from binary to guide fuzzing. Int J Comput Sci Secur 14(5):200–210

    Google Scholar 

  • Luk CK, Cohn R, Muth R, Patil H, Klauser A, Lowney G, Wallace S, Reddi VJ, Hazelwood K (2005) Pin: building customized program analysis tools with dynamic instrumentation. ACM SIGPLAN Not 40(6):190–200. https://doi.org/10.1145/1065010.1065034

    Article  Google Scholar 

  • Luo W, Chai D, Run X, Wang J, Fang C, Chen Z (2021) Graph-based fuzz testing for deep learning inference engines. In: Proceedings of the 43rd international conference on software engineering, pp 288–299. https://doi.org/10.1109/ICSE43902.2021.00037

  • Luo Z, Zuo F, Shen Y, Jiao X, Chang W, Jiang Y (2020) ICS protocol fuzzing: coverage guided packet crack and generation. In: 2020 57th ACM/IEEE design automation conference, pp 1–6. https://doi.org/10.1109/DAC18072.2020.9218603

  • Lv W, Xiong J, Shi J, Huang Y, Qin S (2020) A deep convolution generative adversarial networks based fuzzing framework for industry control protocols. J Intell Manuf 32:441–457. https://doi.org/10.1007/s10845-020-01584-z

    Article  Google Scholar 

  • Lyu C, Ji S, Zhang C, Li Y, Lee WH, Song Y, Beyah R (2019) MOPT: optimized mutation scheduling for fuzzers. In: 28th USENIX security symposium, pp 1949–1966

  • Lyu C, Liang H, Ji S, Zhang X, Zhao B, Han M, Li Y, Wang Z, Wang W, Beyah R (2022) SLIME: program-sensitive energy allocation for fuzzing. In: Proceedings of the 31st ACM SIGSOFT international symposium on software testing and analysis, pp 365–377. https://doi.org/10.1145/3533767.3534385

  • Manès VJ, Han H, Han C, Cha SK, Egele M, Schwartz EJ, Woo M (2019) The art, science, and engineering of fuzzing: a survey. IEEE Trans Softw Eng 47(11):2312–2331. https://doi.org/10.1109/tse.2019.2946563

    Article  Google Scholar 

  • Mansur MN, Christakis M, Wüstholz V, Zhang F (2020) Detecting critical bugs in SMT solvers using blackbox mutational fuzzing. In: Proceedings of the 28th ACM joint meeting on European software engineering conference and symposium on the foundations of software engineering, pp 701–712. https://doi.org/10.1145/3368089.3409763

  • Max M, Kostya S (2016) Guided in-process fuzzing of Chrome components. https://security.googleblog.com/2016/08/guided-in-process-fuzzing-of-chrome.html. Accessed on 30 March 2021

  • Miller BP, Koski D, Lee CP, Maganty V, Murthy R, Natarajan A, Steidl J (1995) Fuzz Revisited: A re-examination of the reliability of UNIX utilities and services. Comput Sci Dept, University of Wisconsin. 1–23

  • Nagy S, Hicks M (2019) Full-speed fuzzing: reducing fuzzing overhead through coverage-guided tracing. In: 2019 IEEE symposium on security and privacy, pp 787–802. https://doi.org/10.1109/sp.2019.00069

  • Neystadt J (2008) Automated penetration testing with white-box fuzzing. Microsoft, February

  • Nguyen MD, Bardin S, Bonichon R, Groz R, Lemerre M (2020) Binary-level directed fuzzing for use-after-free vulnerabilities. In: 23rd International symposium on research in attacks, intrusions and defenses, pp 47–62

  • Nilizadeh S, Noller Y, Păsăreanu CS (2019) DifFuzz: Differential fuzzing for side-channel analysis. In: Proceedings of the 41st international conference on software engineering, pp 176–187. https://doi.org/10.1109/ICSE.2019.00034

  • Noller Y, Kersten R, Păsăreanu CS (2018) Badger: complexity analysis with fuzzing and symbolic execution. In: Proceedings of the 27th ACM SIGSOFT international symposium on software testing and analysis, pp 322–332. https://doi.org/10.1145/3213846.3213868

  • Odena A, Olsson C, Andersen D, Goodfellow I (2019) TensorFuzz: debugging neural networks with coverage-guided fuzzing. In: International conference on machine learning, pp 4901–4911

  • OllyDbg (2000). http://domoticx.com/windows-debugger-ollydbg-software/. Accessed on 30 March 2021

  • Onefuzz (2020). https://github.com/microsoft/onefuzz. Accessed on 23 March 2021

  • Pailoor S, Aday A, Jana S (2018) MoonShine: optimizing OS fuzzer seed selection with trace distillation. In: 27th USENIX security symposium, pp 729–743

  • PaiMei. https://github.com/OpenRCE/https://github.com/OpenRCE/paimei. Accessed on 30 March 2021

  • Peng H, Shoshitaishvili Y, Payer M (2018) T-Fuzz: fuzzing by program transformation. In: 2018 IEEE symposium on security and privacy, pp 697–710. https://doi.org/10.1109/SP.2018.00056

  • Petsios T, Zhao J, Keromytis AD, Jana S (2017) SlowFuzz: automated domain-independent detection of algorithmic complexity vulnerabilities. In: Proceedings of the 2017 ACM SIGSAC conference on computer and communications security, pp 2155–2168. https://doi.org/10.1145/3133956.3134073

  • Pham VT, Böhme M, Roychoudhury A (2016) Model-based whitebox fuzzing for program binaries. In: Proceedings of the 31st IEEE/ACM international conference on automated software engineering, pp 543–553. https://doi.org/10.1145/2970276.2970316

  • Pham VT, Böhme M, Roychoudhury A (2020) AFLNet: a greybox fuzzer for network protocols. In: 2020 IEEE 13th international conference on software testing, validation and verification, pp 460–465. https://doi.org/10.1109/icst46399.2020.00062

  • Pham VT, Böhme M, Santosa AE, Caciulescu AR, Roychoudhury A (2019) Smart greybox fuzzing. IEEE Trans Softw Eng. https://doi.org/10.1109/TSE.2019.2941681

    Article  Google Scholar 

  • Rawat S, Jain V, Kumar A, Cojocar L, Giuffrida C, Bos H (2017) VUzzer: application-aware evolutionary fuzzing. In: 24th Annual network and distributed system security symposium, pp 1–14. https://doi.org/10.14722/ndss.2017.23404

  • Saavedra GJ, Rodhouse KN, Dunlavy DM, Kegelmeyer PW (2019) A review of machine learning applications in fuzzing, pp 1–12. arXiv preprint arXiv:1906.11133

  • Schumilo S, Aschermann C, Abbasi A, Worner S, Holz T (2020) HYPER-CUBE: high-dimensional hypervisor fuzzing. In: 27th Annual network and distributed system security symposium, pp 23–26. https://doi.org/10.14722/ndss.2020.23096

  • Schumilo S, Aschermann C, Abbasi A, Wörner S, Holz T (2021) NYX: greybox hypervisor fuzzing using fast snapshots and affine types. In: 30th USENIX security symposium

  • Schumilo S, Aschermann C, Gawlik R, Schinzel S, Holz T (2017) kAFL: hardware-assisted feedback fuzzing for OS kernels. In: 26th USENIX security symposium, pp 167–182

  • Scott J, Mora F, Ganesh V (2020) Banditfuzz: a reinforcement-learning based performance fuzzer for SMT solvers. In: Software verification: 12th international conference, VSTTE 2020, and 13th international workshop, pp 68–86. https://doi.org/10.1007/978-3-030-63618-0_5

  • Serebryany K (2016) Continuous fuzzing with libFuzzer and AddressSanitizer. In: 2016 IEEE cybersecurity development, pp 157–157. https://doi.org/10.1109/secdev.2016.043

  • Serebryany K (2017) OSS-Fuzz—Google’s continuous fuzzing service for open source software. In: 26th USENIX security symposium, pp 1–28

  • She D, Shah A, Jana S (2022) Effective seed scheduling for fuzzing with graph centrality analysis. In: 2022 IEEE symposium on security and privacy, pp 2194–2211. https://doi.org/10.1109/sp46214.2022.9833761

  • Situ LY, Zuo ZQ, Guan L, Wang LZ, Li XD, Shi J, Liu P (2021) Vulnerable region-aware greybox fuzzing. J Comput Sci Technol 36:1212–1228. https://doi.org/10.1007/s11390-021-1196-0

    Article  Google Scholar 

  • Song D, Hetzelt F, Das D, Spensky C, Na Y, Volckaert S, Vigna G, Kruegel C, Seifert JP, Franz M (2019) PeriScope: an effective probing and fuzzing framework for the hardware-OS boundary. In: Proceedings 2019 network and distributed system security symposium, pp 1–15. https://doi.org/10.14722/ndss.2019.23176

  • Stephens N, Grosen J, Salls C, Dutcher A, Wang R, Corbetta J, Shoshitaishvili Y, Kruegel C, Vigna, G (2016) Driller: augmenting fuzzing through selective symbolic execution. In: 23rd Annual network and distributed system security symposium, pp 1–16. https://doi.org/10.14722/ndss.2016.23368

  • Sun L, Li X, Qu H, Zhang X (2020) AFLTurbo: speed up path discovery for greybox fuzzing. In: 2020 IEEE 31st international symposium on software reliability engineering, pp 81–91. https://doi.org/10.1109/issre5003.2020.00017

  • Sutton M, Greene A, Amini P (2007) Fuzzing: brute force vulnerability discovery. Pearson Education, London

    Google Scholar 

  • Takanen A, Demott JD, Miller C, Kettunen A (2018) Fuzzing for software security testing and quality assurance. Artech House, Norwood

    Google Scholar 

  • The home for Sanitizers (2019). https://github.com/google/sanitizers. Accessed on 30 March 2021

  • ThreadSanitizer (2019). https://clang.llvm.org/docs/ThreadSanitizer.html. Accessed on 30 March 2021

  • Trickel E, Pagani F, Zhu C, Dresel L, Vigna G, Kruegel C, Wang R, Bao T, Shoshitaishvili Y, Doupé A (2023) Toss a fault to your witcher: applying grey-box coverage-guided mutational fuzzing to detect SQL and command injection vulnerabilities. In: 2023 IEEE symposium on security and privacy (SP), pp 2658–2675. https://doi.org/10.1109/sp46215.2023.10179317

  • Tsankov P, Dashti MT, Basin D (2012) SECFUZZ: fuzz-testing security protocols. In: 2012 7th international workshop on automation of software test, pp 1–7. https://doi.org/10.1109/iwast.2012.6228985

  • UndefinedBehaviorSanitizer. https://clang.llvm.org/docs/UndefinedBehaviorSanitizer.html. Accessed on 30 March 2021

  • Viide J, Helin A, Laakso M, Pietikäinen P, Seppänen M, Halunen K, Puuperä R, Röning J (2008) Experiences with model inference assisted fuzzing. In: 2nd USENIX workshop on offensive technologies, vol 2, pp 1–2

  • Vinesh N, Rawat S, Bos H, Giuffrida C, Sethumadhavan M (2020) Confuzz—a concurrency fuzzer. In: 1st International conference on sustainable technologies for computational intelligence-proceedings of ICTSCI 2019, pp 667–691. https://doi.org/10.1007/978-981-15-0029-9_53

  • Vyukov D. Syzkaller. https://github.com/google/syzkaller. Accessed on 30 March 2021

  • Wang, J, Chen B, Wei L, Liu Y (2019) Superion: grammar-aware greybox fuzzing. In: 2019 IEEE/ACM 41st international conference on software engineering, pp 724–735. https://doi.org/10.1109/icse.2019.00081

  • Wang GG, Tan Y (2019) Improving metaheuristic algorithms with information feedback models. IEEE Trans Cybern 49(2):542–555. https://doi.org/10.1109/TCYB.2017.2780274

    Article  PubMed  Google Scholar 

  • Wang GG, Guo L, Gandomi AH, Hao GS, Wang H (2014) Chaotic krill herd algorithm. Inf Sci 274:17–34. https://doi.org/10.1016/j.ins.2014.02.123

    Article  MathSciNet  Google Scholar 

  • Wang Y, Wu Z, Wei Q, Wang Q (2019) NeuFuzz: efficient fuzzing with deep neural network. IEEE Access 7:36340–36352. https://doi.org/10.1109/access.2019.2903291

    Article  Google Scholar 

  • Wang Y, Jia P, Liu L, Huang C, Liu Z (2020) A systematic review of fuzzing based on machine learning techniques. PLoS ONE 15(8):1–20. https://doi.org/10.1371/journal.pone.0237749

    Article  CAS  Google Scholar 

  • Wang L, Pan Z, Wang J (2021) A review of reinforcement learning based intelligent optimization for manufacturing scheduling. Complex Syst Model Simul 1(4):257–270. https://doi.org/10.23919/CSMS.2021.0027

    Article  CAS  Google Scholar 

  • Wang X, Hu C, Ma R, Tian D, He J (2021) CMFuzz: context-aware adaptive mutation for fuzzers. Empir Softw Eng 26(1):1–34. https://doi.org/10.1007/s10664-020-09927-3

    Article  Google Scholar 

  • Wang F, Wang X, Sun S (2022) A reinforcement learning level-based particle swarm optimization algorithm for large-scale optimization. Inf Sci 602:298–312

    Article  Google Scholar 

  • Wang J, Chen B, Wei L, Liu Y (2017) Skyfire: data-driven seed generation for fuzzing. In: 2017 IEEE symposium on security and privacy, pp 579–594. https://doi.org/10.1109/SP.2017.23

  • Wang Y, Jia X, Liu Y, Zeng K, Bao T, Wu D, Su P (2020) Not all coverage measurements are equal: fuzzing by coverage accounting for input prioritization. In: 27th Annual network and distributed system security symposium, pp 1–17. https://doi.org/10.14722/ndss.2020.24422

  • Wang Z, Liblit B, Reps T (2020) TOFU: target-orienter fuzzer. arXiv preprint arXiv:2004.14375

  • Wang F, Shoshitaishvili Y (2017) Angr—the next generation of binary analysis. In: 2017 IEEE cybersecurity development, pp 8–9. https://doi.org/10.1109/SecDev.2017.14

  • Wang J, Song C, Yin H (2021) Reinforcement learning-based hierarchical seed scheduling for greybox fuzzing. In: Network and distributed system security symposium, pp 1–17. https://doi.org/10.14722/ndss.2021.24486

  • Wang H, Xie X, Li Y, Wen C, Li Y, Liu Y, Qin S, Chen H, Sui Y (2020) Typestate-guided fuzzer for discovering use-after-free vulnerabilities. In: 42nd International conference on software engineering, pp 999–1010. https://doi.org/10.1145/3377811.3380386

  • Wang Y, Zhang C, Xiang X, Zhao Z, Li W, Gong X, Liu B, Chen K, Zou W (2018) Revery: From proof-of-concept to exploitable. In: Proceedings of the 2018 ACM SIGSAC conference on computer and communications security, pp 1914—1927. https://doi.org/10.1145/3243734.3243847

  • Wang P, Zhou X, Lu K, Yue T, Liu Y (2020) Sok: the progress, challenges, and perspectives of directed greybox fuzzing. In: Challenges, and perspectives of directed greybox fuzzing

  • Wen C, Wang H, Li Y, Qin S, Liu Y, Xu Z, Chen H, Xie X, Pu G, Liu T (2020) MemLock: memory usage guided fuzzing. In: 42nd International conference on software engineering, pp 765–777 . https://doi.org/10.1145/3377811.3380396

  • Winterer D, Zhang C, Su Z (2020) On the unusual effectiveness of type-aware operator mutations for testing SMT solvers. Proc ACM Program Lang 4:1–25. https://doi.org/10.1145/3428261

    Article  Google Scholar 

  • Winterer D, Zhang C, Su Z (2020) Validating SMT solvers via semantic fusion. In: Proceedings of the 41st ACM SIGPLAN conference on programming language design and implementation, pp 718–730. https://doi.org/10.1145/3385412.3385985

  • Woo M, Cha SK, Gottlieb S, Brumley D (2013) Scheduling black-box mutational fuzzing. In: Proceedings of the 2013 ACM SIGSAC conference on computer and communications security, pp 511–522. https://doi.org/10.1145/2508859.2516736

  • Xie X, Ma L, Juefei-Xu F, Xue M, Chen H, Liu Y, Zhao J, Li B, Yin J, See S (2019) DeepHunter: a coverage-guided fuzz testing framework for deep neural networks. In: Proceedings of the 28th ACM SIGSOFT international symposium on software testing and analysis, pp 146–157. https://doi.org/10.1021/acs.jcim.8b00542.s002

  • Xu M, Kashyap S, Zhao H, Kim T (2020) Krace: data race fuzzing for kernel file systems. In: 2020 IEEE symposium on security and privacy, pp 1643–1660. https://doi.org/10.1109/sp40000.2020.00078

  • Ye G, Tang Z, Tan SH, Huang S, Fang D, Sun X, Bian L, Wang H, Wang Z (2021) Automated conformance testing for JavaScript engines via deep compiler fuzzing. In: 42nd ACM SIGPLAN conference on programming language design and implementation, pp 435–450

  • You W, Liu X, Ma S, Perry D, Zhang X, Liang B (2019) SLF: fuzzing without valid seed inputs. In: 2019 IEEE/ACM 41st international conference on software engineering, pp 712–723. https://doi.org/10.1109/icse.2019.00080

  • You W, Wang X, Ma S, Huang J, Zhang X, Wang X, Liang B (2019) ProFuzzer: on-the-fly input type probing for better zero-day vulnerability discovery. In: 2019 IEEE symposium on security and privacy, pp 769–786. https://doi.org/10.1109/sp.2019.00057

  • You W, Zong P, Chen K, Wang X, Liao X, Bian P, Liang B (2017) SemFuzz: semantics-based automatic generation of proof-of-concept exploits. In: Proceedings of the 2017 ACM SIGSAC conference on computer and communications security, pp 2139–2154. https://doi.org/10.1145/3133956.3134085

  • Yue T, Wang P, Tang Y, Wang E, Yu B, Lu K, Zhou X (2020) EcoFuzz: adaptive energy-saving greybox fuzzing as a variant of the adversarial multi-armed bandit. In: 29th USENIX security symposium, pp 2307–2324

  • Yun I, Lee S, Xu M, Jang Y, Kim T (2018) QSYM: a practical concolic execution engine tailored for hybrid fuzzing. In: 27th USENIX security symposium, pp 745–761

  • Z3 (2015). https://en.wikipedia.org/wiki/Z3_Theorem_Prover. Accessed on 30 March 2021

  • Zhang G, Wang PF, Yue T, Kong XD, Zhou X, Lu K (2022) ovAFLow: detecting memory corruption bugs with fuzzing-based taint inference. J Comput Sci Technol 37(2):405–422. https://doi.org/10.1007/s11390-021-1600-9

    Article  Google Scholar 

  • Zhang P, Ren B, Dong H, Dai Q (2022) CAGFuzz: coverage-guided adversarial generative fuzzing testing for image-based deep learning systems. IEEE Trans Softw Eng 48(11):4630–4646. https://doi.org/10.1109/TSE.2021.3124006

    Article  Google Scholar 

  • Zhang Q, Wang Y, Li J, Ma S (2020) Ethploit: from fuzzing to efficient exploit generation against smart contracts. In: 2020 IEEE 27th international conference on software analysis, evolution and reengineering, pp 116–126. https://doi.org/10.1109/SANER48275.2020.9054822

  • Zhang G, Wang P, Yue T, Kong X, Huang S, Zhou X, Lu K (2022) MobFuzz: adaptive multi-objective optimization in gray-box fuzzing. In: Network and distributed systems security symposium 2022, pp 1–18. https://doi.org/10.14722/ndss.2022.24314

  • Zhang Y, Wang Z, Yu W, Fang B (2021) Multi-level directed fuzzing for detecting use-after-free vulnerabilities. In: 2021 IEEE 20th international conference on trust, security and privacy in computing and communications, pp 569–576. https://doi.org/10.1109/trustcom53373.2021.00087

  • Zhao X, Qu H, Lv W, Li S, Xu J (2021) MooFuzz: many-objective optimization seed schedule for fuzzer. Mathematics 9:1–19. https://doi.org/10.3390/math9030205

    Article  Google Scholar 

  • Zhao X, Qu H, Xu J, Li S, Wang GG (2022) AMSFuzz: an adaptive mutation schedule for fuzzing. Expert Syst Appl 208:1–11. https://doi.org/10.1016/j.eswa.2022.118162

  • Zhao L, Duan Y, Yin H, Xuan J (2019) Send hardest problems my way: probabilistic path prioritization for hybrid fuzzing. In: Proceedings 2019 network and distributed system security symposium. https://doi.org/10.14722/ndss.2019.23504

  • Zhao H, Li Z, Wei H, Shi J, Huang Y (2019) SeqFuzzer: an industrial protocol fuzzing framework from a deep learning perspective. In: 2019 12th IEEE conference on software testing, validation and verification, pp 59–67. https://doi.org/10.1109/ICST.2019.00016

  • Zhou C, Wang M, Liang J, Liu Z, Jiang Y (2020) Zeror: speed up fuzzing with coverage-sensitive tracing and scheduling. In: 2020 35th IEEE/ACM international conference on automated software engineering, pp 858–870. https://doi.org/10.1145/3324884.3416572

  • Zlewski C. American Fuzzy Lop. http://lcamtuf.coredump.cx/afl. Accessed on 1 March 2021

Download references

Funding

This research was funded by the National Natural Science Foundation of China Grant Number 61827810.

Author information

Authors and Affiliations

  1. School of Information and Control Engineering, Qingdao University of Technology, Qingdao, China

    Xiaoqi Zhao

  2. College of Computer Science and Technology, Ocean University of China, Qingdao, China

    Haipeng Qu, Jianliang Xu, Xiaohui Li, Wenjie Lv & Gai-Ge Wang

Authors
  1. Xiaoqi Zhao
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Haipeng Qu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Jianliang Xu
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Xiaohui Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Wenjie Lv
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Gai-Ge Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Haipeng Qu.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zhao, X., Qu, H., Xu, J. et al. A systematic review of fuzzing. Soft Comput 28, 5493–5522 (2024). https://doi.org/10.1007/s00500-023-09306-2

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00500-023-09306-2

Keywords

Search

Navigation