Skip to main content
SpringerLink
Log in

Functional Verification for Agile Processor Development: A Case for Workflow Integration

  • Regular Paper
  • Published:
Journal of Computer Science and Technology Aims and scope Submit manuscript

Abstract

Agile hardware development methodology has been widely adopted over the past decade. Despite the research progress, the industry still doubts its applicability, especially for the functional verification of complicated processor chips. Functional verification commonly employs a simulation-based method of co-simulating the design under test with a reference model and checking the consistency of their outcomes given the same input stimuli. We observe limited collaboration and information exchange through the design and verification processes, dramatically leading to inefficiencies when applying the conventional functional verification workflow to agile development. In this paper, we propose workflow integration with collaborative task delegation and dynamic information exchange as the design principles to effectively address the challenges on functional verification under the agile development model. Based on workflow integration, we enhance the functional verification workflows with a series of novel methodologies and toolchains. The diff-rule based agile verification methodology (DRAV) reduces the overhead of building reference models with runtime execution information from designs under test. We present the RISC-V implementation for DRAV, DiffTest, which adopts information probes to extract internal design behaviors for co-simulation and debugging. It further integrates two plugins, namely XFUZZ for effective test generation guided by design coverage metrics and LightSSS for efficient fault analysis triggered by co-simulation mismatches. We present the integrated workflows for agile hardware development and demonstrate their effectiveness in designing and verifying RISC-V processors with 33 functional bugs found in NutShell. We also illustrate the efficiency of the proposed toolchains with a case study on a functional bug in the L2 cache of XiangShan.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

References

  1. Hennessy J L, Patterson D A. A new golden age for computer architecture. Communications of the ACM, 2019, 62(2): 48–60. https://doi.org/10.1145/3282307.

    Article  Google Scholar 

  2. Bao Y G, Carlson T E. Agile and open-source hardware. IEEE Micro, 2020, 40(4): 6–9. https://doi.org/10.1109/MM.2020.3002606.

    Article  Google Scholar 

  3. Lee Y, Waterman A, Cook H, Zimmer B, Keller B, Puggelli A, Kwak J, Jevtic R, Bailey S, Blagojevic M, Chiu P F, Avizienis R, Richards B, Bachrach J, Patterson D, Alon E, Nikolic B, Asanović K. An agile approach to building RISC-V microprocessors. IEEE Micro, 2016, 36(2): 8–20. https://doi.org/10.1109/MM.2016.11.

    Article  Google Scholar 

  4. Xu Y N, Yu Z H, Tang D, Chen G K, Chen L, Gou L R, Jin Y, Li Q R, Li X, Li Z J, Lin J W, Liu T, Liu Z G, Tan J Z, Wang H Q, Wang H Z, Wang K F, Zhang C Q, Zhang F W, Zhang L J, Zhang Z F, Zhao Y Y, Zhou Y Y, Zhou Y K, Zou J R, Cai Y, Huan D D, Li Z S, Zhao J Y, Chen Z H, He W, Quan Q Y, Liu X, Wang S, Shi K, Sun N H, Bao Y G. Towards developing high performance RISC-V processors using agile methodology. In Proc. the 55th IEEE/ACM International Symposium on Microarchitecture, Oct. 2022, pp.1178–1199. https://doi.org/10.1109/MICRO56248.2022.00080.

  5. Xu Y N, Yu Z H, Tang D, Cai Y, Huan D D, He W, Sun N H, Bao Y G. Toward developing high-performance RISC-V processors using agile methodology. IEEE Micro, 2023, 43(4): 98–106. https://doi.org/10.1109/MM.2023.3273562.

    Article  Google Scholar 

  6. Kabylkas N, Thorn T, Srinath S, Xekalakis P, Renau J. Effective processor verification with logic fuzzer enhanced co-simulation. In Proc. the 54th Annual IEEE/ACM International Symposium on Microarchitecture, Oct. 2021, pp.667–678. https://doi.org/10.1145/3466752.3480092.

  7. IEEE. IEEE standard for universal verification methodology language reference manual. IEEE Std 1800.2-2020, 2020, pp.1–458. https://doi.org/10.1109/IEEESTD.2020.9195920.

    Article  Google Scholar 

  8. Bachrach J, Vo H, Richards B, Lee Y, Waterman A, Avižienis R, Wawrzynek J, Asanović K. Chisel: Constructing hardware in a Scala embedded language. In Proc. the 49th Annual Design Automation Conference, Jun. 2012, pp.1216–1225. https://doi.org/10.1145/2228360.2228584.

  9. Beamer S, Donofrio D. Efficiently exploiting low activity factors to accelerate RTL simulation. In Proc. the 57th ACM/IEEE Design Automation Conference, July. 2020. https://doi.org/10.1109/DAC18072.2020.9218632.

  10. Yu S Z, Dong Y F, Liu J Y, Li Y, Wu Z L, Jansen D N, Zhang L J. CHA: Supporting SVA-like assertions in formal verification of chisel programs (tool paper). In Proc. the 20th International Conference on Software Engineering and Formal Methods, Sept. 2022, pp.324–331. https://doi.org/10.1007/978-3-031-17108-6_20.

  11. Karandikar S, Mao H, Kim D, Biancolin D, Amid A, Lee D, Pemberton N, Amaro E, Schmidt C, Chopra A, Huang Q J, Kovacs K, Nikolic B, Katz R, Bachrach J, Asanović K. FireSim: FPGA-accelerated cycle-exact scale-out system simulation in the public cloud. In Proc. the 45th ACM/IEEE Annual International Symposium on Computer Architecture, July. 2018, pp.29–42. https://doi.org/10.1109/ISCA.2018.00014.

  12. Kern C, Greenstreet M R. Formal verification in hardware design: A survey. ACM Trans. Design Automation of Electronic Systems, 1999, 4(2): 123–193. https://doi.org/10.1145/307988.307989.

    Article  Google Scholar 

  13. Asanović K, Avizienis R, Bachrach J, Beamer S, Biancolin D, Celio C, Cook H, Dabbelt D, Hauser J, Izraelevitz A, Karandikar S, Keller B, Kim D, Koenig J, Lee Y, Love E, Maas M, Magyar A, Mao H, Moreto M, Ou A, Patterson D A, Richards B, Schmidt C, Twigg S, Vo H, Waterman A. The rocket chip generator. Technical Report UCB/EECS-2016-17, EECS Department, University of California, Berkeley, 2016. https://www2.eecs.berkeley.edu/Pubs/TechRpts/2016/EECS-2016-17.html, July. 2023.

  14. Laeufer K, Koenig J, Kim D, Bachrach J, Sen K. RFUZZ: Coverage-directed fuzz testing of RTL on FPGAs. In Proc. the 2018 IEEE/ACM International Conference on Computer-Aided Design, Nov. 2018. https://doi.org/10.1145/3240765.3240842.

  15. Hur J, Song S, Kwon D, Baek E, Kim J, Lee B. DifuzzRTL: Differential fuzz testing to find CPU bugs. In Proc. the 42nd IEEE Symposium on Security and Privacy, May 2021, pp.1286–1303. https://doi.org/10.1109/SP40001.2021.00103.

  16. Trippel T, Shin K G, Chernyakhovsky A, Kelly G, Rizzo D, Hicks M. Fuzzing hardware like software. In Proc. the 31st USENIX Security Symposium, Aug. 2022, pp.3237–3254.

  17. Fioraldi A, Maier D C, Zhang D J, Balzarotti D. LibAFL: A framework to build modular and reusable fuzzers. In Proc. the 2022 ACM SIGSAC Conference on Computer and Communications Security, Nov. 2022, pp.1051–1065. https://doi.org/10.1145/3548606.3560602.

  18. Izraelevitz A, Koenig J, Li P, Lin R, Wang A, Magyar A, Kim D, Schmidt C, Markley C, Lawson J, Bachrach J. Reusability is FIRRTL ground: Hardware construction languages, compiler frameworks, and transformations. In Proc. the 2017 IEEE/ACM International Conference on Computer-Aided Design, Nov. 2017, pp.209–216. https://doi.org/10.1109/ICCAD.2017.8203780.

  19. Kim D, Celio C, Karandikar S, Biancolin D, Bachrach J, Asanović K. DESSERT: Debugging RTL effectively with state snapshotting for error replays across trillions of cycles. In Proc. the 28th International Conference on Field Programmable Logic and Applications, Aug. 2018, pp.76–764. https://doi.org/10.1109/FPL.2018.00021.

  20. Skinner H, Trapani Possignolo R, Wang S H, Renau J. LiveSim: A fast hot reload simulator for HDLs. In Proc. the 2020 IEEE International Symposium on Performance Analysis of Systems and Software, Aug. 2020, pp.126–135. https://doi.org/10.1109/ISPASS48437.2020.00028.

  21. Wang H, Zhang Z, Jin Y, Zhang L, Wang K. Nutshell: A Linux-compatible RISC-V processor designed by undergraduates. https://riscv.org/proceedings/2020/09/risc-vglobal-forum-proceedings/, July 2023.

  22. Wang K F, Xu Y N, Yu Z H, Tang D, Chen G K, Chen X, Gou L R, Hu X, Jin Y, Li Q R, Li X, Lin J W, Liu T, Liu Z G, Wang H Q, Wang H Z, Zhang C Q, Zhang F W, Zhang L J, Zhang Z F, Zhang Z Y, Zhao Y Y, Zhou Y Y, Zou J R, Cai Y, Huan D D, Li Z S, Zhao J Y, He W, Sun N H, Bao Y G. XiangShan open-source high performance RISC-V processor design and implementation. Journal of Computer Research and Development, 2023, 60(3): 476–493. (in Chinese) https://doi.org/10.7544/issn1000-1239.202221036.

    Article  Google Scholar 

  23. Lockhart D, Zibrat G, Batten C. PyMTL: A unified framework for vertically integrated computer architecture research. In Proc. the 47th Annual IEEE/ACM International Symposium on Microarchitecture, Dec. 2014, pp.280–292. https://doi.org/10.1109/MICRO.2014.50.

  24. Wang H Y, Beamer S. RepCut: Superlinear parallel RTL simulation with replication-aided partitioning. In Proc. the 28th ACM International Conference on Architectural Support for Programming Languages and Operating Systems, Mar. 2023, pp.572–585. https://doi.org/10.1145/3582016.3582034.

  25. Jiang S N, Ilbeyi B, Batten C. Mamba: Closing the performance gap in productive hardware development frameworks. In Proc. the 55th ACM/ESDA/IEEE Design Automation Conference, Jun. 2018. https://doi.org/10.1109/DAC.2018.8465576.

  26. Shi K, Xu S X, Diao Y H, Boland D, Bao Y G. ENCORE: Efficient architecture verification framework with FPGA acceleration. In Proc. the 2023 ACM/SIGDA International Symposium on Field Programmable Gate Arrays, Feb. 2023, pp.209–219. https://doi.org/10.1145/3543622.3573187.

  27. Xing Y, Lu H X, Gupta A, Malik S. Leveraging processor modeling and verification for general hardware modules. In Proc. the 2021 Design, Automation & Test in Europe Conference & Exhibition, Feb. 2021, pp.1130–1135. https://doi.org/10.23919/DATE51398.2021.9474194.

  28. Naylor M, Moore S. A generic synthesisable test bench. In Proc. the 2015 ACM/IEEE International Conference on Formal Methods and Models for Codesign, Sept. 2015, pp.128–137. https://doi.org/10.1109/MEMCOD.2015.7340479.

  29. Shen H H, Wei W L, Chen Y J, Chen B W, Guo Q. Coverage directed test generation: Godson experience. In Proc. the 17th Asian Test Symposium, Nov. 2008, pp.321–326. https://doi.org/10.1109/ATS.2008.42.

  30. Huang B Y, Zhang H C, Subramanyan P, Vizel Y, Gupta A, Malik S. Instruction-level abstraction (ILA): A uniform specification for system-on-chip (SoC) verification. ACM Trans. Design Automation of Electronic Systems, 2019, 24(1): Article No. 10. https://doi.org/10.1145/3282444.

  31. Canakci S, Delshadtehrani L, Eris F, Taylor M B, Egele M, Joshi A. DirectFuzz: Automated test generation for RTL designs using directed graybox fuzzing. In Proc. the 58th ACM/IEEE Design Automation Conference, Dec. 2021, pp.529–534. https://doi.org/10.1109/DAC18074.2021.9586289.

  32. Jiang S N, Ou Y H, Pan P T, Cheng K S, Zhang Y X, Batten C. PyH2: Using PyMTL3 to create productive and open-source hardware testing methodologies. IEEE Design & Test, 2021, 38(2): 53–61. https://doi.org/10.1109/MDAT.2020.3024144.

    Article  Google Scholar 

  33. Laeufer K, Iyer V, Biancolin D, Bachrach J, Nikolić B, Sen K. Simulator independent coverage for RTL hardware languages. In Proc. the 28th ACM International Conference on Architectural Support for Programming Languages and Operating Systems, Mar. 2023, pp.606–615. https://doi.org/10.1145/3582016.3582019.

Download references

Acknowledgments

The authors would like to thank Zhi-Wei Xu for his valuable insights into the idea. The proposed techniques in this paper have been extensively used by the XiangShan team. We give special thanks to our team members for their feedback and help on the work.

Conflict of Interest

The authors declare that they have no conflict of interest.

Author information

Authors and Affiliations

  1. State Key Lab of Processors, Institute of Computing Technology, Chinese Academy of Sciences, Beijing, 100190, China

    Yi-Nan Xu, Zi-Hao Yu, Kai-Fan Wang, Hua-Qiang Wang, Jia-Wei Lin, Yue Jin, Lin-Juan Zhang, Zi-Fei Zhang, Dan Tang, Sa Wang, Kan Shi, Ning-Hui Sun & Yun-Gang Bao

  2. University of Chinese Academy of Sciences, Beijing, 100049, China

    Yi-Nan Xu, Kai-Fan Wang, Hua-Qiang Wang, Jia-Wei Lin, Yue Jin, Lin-Juan Zhang, Zi-Fei Zhang, Ning-Hui Sun & Yun-Gang Bao

  3. Beijing Institute of Open Source Chip, Beijing, 100080, China

    Dan Tang

Authors
  1. Yi-Nan Xu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Zi-Hao Yu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Kai-Fan Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Hua-Qiang Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Jia-Wei Lin
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Yue Jin
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Lin-Juan Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Zi-Fei Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  9. Dan Tang
    View author publications

    You can also search for this author in PubMed Google Scholar

  10. Sa Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  11. Kan Shi
    View author publications

    You can also search for this author in PubMed Google Scholar

  12. Ning-Hui Sun
    View author publications

    You can also search for this author in PubMed Google Scholar

  13. Yun-Gang Bao
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Yun-Gang Bao.

Supplementary Information

ESM 1

(PDF 292 kb)

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Xu, YN., Yu, ZH., Wang, KF. et al. Functional Verification for Agile Processor Development: A Case for Workflow Integration. J. Comput. Sci. Technol. 38, 737–753 (2023). https://doi.org/10.1007/s11390-023-3285-8

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11390-023-3285-8

Keywords

Search

Navigation