Inter-FPGA interconnect topologies exploration for multi-FPGA systems


Prototyping using multi-FPGA systems offers significant advantages over simulation and emulation based pre-silicon verification techniques. Multi-FPGA prototyping follows a complex design flow where the quality of associated tools and the architecture of interconnect topology play a very important role in the performance of final prototyped design. A well designed interconnect topology may remain underutilized because of a poor routing tool and vice versa. This makes the selection of a good routing tool and the exploration of interconnect topologies extremely important for the quality of final design. In this work, we present a detailed comparison between six inter-FPGA interconnect topologies. We present a generic routing tool and for each topology, ten large, complex benchmarks are prototyped on four FPGA boards using this tool. Experimentation reveals that fully customized interconnect topology using a hybrid combination of direct two and multi point tracks gives the best frequency results for all the FPGA boards. On average, this topology gives 26.2, 28.5, 9.5, 32.1 and 12.4% better frequency results as compared to five other interconnect topologies. We also perform routing time comparison and the topology using generic hybrid combination of direct two and multi point tracks gives the best results. On average, this topology produces \(1.8\times \), \(2\times \), \(2\times \), \(9.2\times \), and \(4.4\times \) better results as compared to five other topologies under consideration. Frequency–time tradeoff analysis along with flexibility and setup time of different topologies is also performed. It reveals that a partially customized topology with hybrid combination of direct two and multi point tracks gives the best frequency–time tradeoff for smaller FPGA boards while a partially customized topology with switch-based and multi point connections gives the best results for larger FPGA boards with reasonable flexibility and moderate setup time.

This is a preview of subscription content, access via your institution.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18
Fig. 19
Fig. 20
Fig. 21
Fig. 22
Fig. 23
Fig. 24
Fig. 25


  1. 1.

    Santarini M (2005) Asic prototyping: Make versus buy, EDN, vol 11

  2. 2.

    Sigenics: Custom asic calculator, (2017)

  3. 3.

    AMD, (2007)

  4. 4.

    Pentium, (1994)

  5. 5.

    Huang C-Y, Yin Y-F, Hsu C-J, Huang TB, Chang T-M (2011) Soc hw/sw verification and validation. In: 16th Asia and South Pacific design automation conference (ASP-DAC 2011). IEEE, pp 297–300

  6. 6.

    Cadence, /tools/system-design-and-verification/simulation-and-testbench-verification/incisive-enterprise-simulator.html (2017)

  7. 7.

    Graphics M (2017)

  8. 8.

    Vcs: A functional veriifcation solution by synopsys, (2017)

  9. 9.

    Series CP ( 2017) pages / default.aspx

  10. 10.

    Zebu-server asic emulator by synopsys, /hardware-verification/emulation/Pages/default.aspx (2017)

  11. 11.

    Veloce MG (2017)

  12. 12.

    Kuon I, Rose J (2010) Quantifying and exploring the gap between FPGAs and ASICs, vol 1, Springer, Ed. Springer US

  13. 13.

    Babb J, Tessier R, Dahl M, Hanono S, Hoki D, Agarwal A (1997) Logic emulation with virtual wires. IEEE Trans Comput-Aided Des Integr Circuits Syst 16(6):609–626

    Article  Google Scholar 

  14. 14.

    Krupnova H (2004) Mapping multi-million gate SoCs on FPGAs: industrial methodology and experience. In: Design, automation and test in europe conference and exhibition, 2004. proceedings, vol 2. pp 1236–1241

  15. 15.

    Asaad S, Bellofatto R, Brezzo B, Haymes C, Kapur M, Parker B, Roewer T, Saha P, Takken T, Tierno J (2012) A cycle-accurate, cycle-reproducible multi-FPGA system for accelerating multi-core processor simulation. In: Proceedings of the ACM/SIGDA international symposium on field programmable gate arrays, ser. FPGA ’12. ACM, New York, pp 153–162. [Online].

  16. 16.

    Yang S (1991) Logic synthesis and optimization benchmarks user guide, version 3.0

  17. 17.

    Farooq U, Parvez H, Mehrez H, Marrakchi Z (2012) A new heterogeneous tree-based application specific FPGA and its comparison with mesh-based application specific FPGA. Microprocess Microsyst 36(8):588–605. [Online].

  18. 18.

    Stroobandt D, Verplaetse P, Van Campenhout J (2000) Generating synthetic benchmark circuits for evaluating cad tools. IEEE Trans Comput-Aided Des Integr Circuits Syst 19(9):1011–1022

    Article  Google Scholar 

  19. 19.

    Pouillon N, Greiner A (2010) Soc lib project. [Online].

  20. 20.

    Panades IM, Greiner A, Sheibanyrad A (2006) A low cost network-on-chip with guaranteed service well suited to the gals approach. In: Nano-networks and workshops, 2006. NanoNet ’06. 1st international conference on, 1–5

  21. 21.

    Certify, BasedPrototyping/Pages/Certify.aspx (2017)

  22. 22.

    Inagi M, Takashima Y, Nakamura Y (2009) Globally optimal time-multiplexing in inter-FPGA connections for accelerating multi-FPGA systems. In: Field programmable logic and applications, 2009. FPL 2009. International conference on, pp 212–217

  23. 23.

    Turki M, Marrakchi Z, Mehrez H, Abid M (2013) 9th International Symposium, ARC 2013, Los Angeles, CA, USA, March 25–27, 2013. Springer, ch. Iterative routing algorithm of inter-FPGA signals for multi-FPGA prototyping platform, 201–217

  24. 24.

    DiniGroup, (2017)

  25. 25.

    Tang Q, Mehrez H, Tuna M (Oct 2013) Routing algorithm for multi-FPGA based systems using multi-point physical tracks. In: Rapid system prototyping (RSP), 2013 international symposium on, pp 2–8

  26. 26.

    Farooq U, Chotin-Avot R, Azeem M, Ravoson M, Turki M, Mehrez H (2016) Inter-FPGA routing environment for performance exploration of multi-FPGA systems. In: Proceedings of the 27th international symposium on rapid system prototyping, ser. RSP ’16. New York: ACM, pp 107–113

  27. 27.

    Kim C, Shin H (1996) A performance-driven logic emulation system: FPGA network design and performance-driven partitioning. IEEE Trans Comput Aided Des Integr Circuits Syst 15(5):560–568

    Article  Google Scholar 

  28. 28.

    Hauck S, Borriello G, Ebeling C (1994) Mesh routing topologies for multi-FPGA systems. In: Proceedings 1994 IEEE international conference on computer design: VLSI in computers and processors, 170–177

  29. 29.

    Khalid MAS, Rose J (2000) A novel and efficient routing architecture for multi-FPGA systems. IEEE Trans Very Large Scale Integr VLSI Syst 8(1):30–39

    Article  Google Scholar 

  30. 30.

    Kuon I, Tessier R, Rose J (2008) Fpga architecture: survey and challenges. Found Trends Electron Des Autom 2:135–253

    Article  Google Scholar 

  31. 31.

    McMurchie L, Ebeling C (1995) Pathfinder: a negotiation-based performance-driven router for FPGAs. In: ACM international symposium on field-programmable gate arrays. ACM Press, New York, 111–117

  32. 32.

    Dijkstra EW (1959) A note on two problems in connexion with graphs. Numer Math 1(1):269–271

    MathSciNet  Article  MATH  Google Scholar 

  33. 33.

    Xilinx, (2017)

  34. 34.

    Altera, (2017)

  35. 35.

    Amos D, Lesea A, Richter R (2011) FPGA-based prototyping methodology manual: best practices in design-for-prototyping, Synopsys, Ed. Synopsys

Download references


This research work is done through the financial support of Systematic and Bpifrance. Authors would also like to thank Zied Marrakchi and Hayder Mrabet from Flexras Technologies for their suggestions and guidance to enhance the quality of this work.

Author information



Corresponding author

Correspondence to Umer Farooq.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Farooq, U., Mehrez, H. & Bhatti, M.K. Inter-FPGA interconnect topologies exploration for multi-FPGA systems. Des Autom Embed Syst 22, 117–140 (2018).

Download citation


  • Multi-FPGA prototyping
  • Interconnect topologies
  • Routing
  • Exploration