Abstract
With the Internet-of-things revolution, embedded devices are in charge of an ever increasing number of tasks ranging from sensing, up to Artificial Intelligence (AI) functions. In particular, AI is gaining importance since it can dramatically improve the QoS perceived by the final user and it allows to cope with problems whose algorithmic solution is hard to find. However, the associated computational requirements, mostly made of floating-point processing, impose a careful design and tuning of the computing platforms. In this scenario, there is a need for a set of benchmarks representative of the emerging AI applications and useful to compare the efficiency of different architectural solutions and computing platforms. In this paper we present a suite of benchmarks encompassing Computer Graphics, Computer Vision and Machine Learning applications, which are greatly used in many AI scenarios. Such benchmarks, differently from other suites, are kernels tailored to be effectively executed in bare-metal and specifically stress the floating-point support offered by the computing platform.
Keywords
- Benchmarks
- Machine learning
- Artificial intelligence
- Floating-point
- FPU.
This is a preview of subscription content, access via your institution.
Buying options
References
Lamp: Lightweight application-specific modular processor. http://www.lamp-platform.org
Freertos (2003). https://www.freertos.org
Spec cpu2006 (2006). https://www.spec.org/cpu2006/
Mbed (2009). https://www.mbed.com/
LSTMS for time series in pytorch (2019). https://www.jessicayung.com/lstms-for-time-series-in-pytorch/. Accessed 05 Aug 2019
Agosta, G., et al.: The RECIPE approach to challenges in deeply heterogeneous high performance systems. Microprocess. Microsyst. 77, 103185 (2020). https://doi.org/10.1016/j.micpro.2020.103185. ISSN 0141-9331
Agosta, G., et al.: Challenges in deeply heterogeneous high performance systems. In: 2019 22nd Euromicro Conference on Digital System Design (DSD), August 2019. https://doi.org/10.1109/DSD.2019.00068
Bienia, C.: Benchmarking Modern Multiprocessors. Ph.D. thesis, Princeton University, January 2011
Fan, W., Wang, K., Cayre, F., Xiong, Z.: 3D lighting-based image forgery detection using shape-from-shading. In: 2012 Proceedings of the 20th European Signal Processing Conference (EUSIPCO), pp. 1777–1781. IEEE (2012)
Fornaciari, al.: Reliable power and time-constraints-aware predictive management of heterogeneous exascale systems. In: Mudge, T.N., Pnevmatikatos, D.N. (eds.) Proceedings of the 18th International Conference on Embedded Computer Systems: Architectures, Modeling, and Simulation, Pythagorion, Greece, 15–19 July 2018, pp. 187–194. ACM (2018). https://doi.org/10.1145/3229631.3239368
Geiger, Andreas., Roser, Martin, Urtasun, Raquel: Efficient large-scale stereo matching. In: Kimmel, Ron, Klette, Reinhard, Sugimoto, Akihiro (eds.) ACCV 2010. LNCS, vol. 6492, pp. 25–38. Springer, Heidelberg (2011). https://doi.org/10.1007/978-3-642-19315-6_3
Geneva, P., Maley, J., Huang, G.: An efficient SCHMIDT-EKF for 3d visual-inertial slam. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 12105–12115 (2019)
Greff, K., Srivastava, R.K., Koutník, J., Steunebrink, B.R., Schmidhuber, J.: LSTM: a search space odyssey. IEEE Trans. Neural Networks Learn. Syst. 28(10), 2222–2232 (2016)
Gustafsson, J., Betts, A., Ermedahl, A., Lisper, B.: The mälardalen wcet benchmarks - past, present and future. In: Proceedings of the 10th International Workshop on Worst-Case Execution Time Analysis, July 2010
Guthaus, M.R., Ringenberg, J.S., Ernst, D., Austin, T.M., Mudge, T., Brown, R.B.: Mibench: a free, commercially representative embedded benchmark suite. In: 2001 IEEE International Workshop Proceedings of the Workload Characterization, 2001. WWC-4, pp. 3–14. WWC 2001 (2001)
Klein, G., Murray, D.: Parallel tracking and mapping for small ar workspaces. In: Proceedings of the 2007 6th IEEE and ACM International Symposium on Mixed and Augmented Reality, pp. 1–10. IEEE Computer Society (2007)
Leisch, F., Dimitriadou, E.: mlbench: Machine Learning Benchmark Problems, r package version 2.1-1 (2010)
Max, N.: Weights for computing vertex normals from facet normals. J. Graph. Tools 4(2), 1–6 (1999)
Möller, T.: A fast triangle-triangle intersection test. J. Graph. Tools 2(2), 25–30 (1997)
Montgomery, D.C., Peck, E.A., Vining, G.G.: Introduction to Linear Regression Analysis, vol. 821. Wiley, Hoboken (2012)
Paszke, A., et al.: Automatic Differentiation in Pytorch (2017)
Romanoni, A., Matteucci, M.: Mesh-based camera pairs selection and occlusion-aware masking for mesh refinement. Pattern Recogn. Lett. 125, 364–372 (2019)
Sansottera, A., Zoni, D., Cremonesi, P., Fornaciari, W.: Consolidation of multi-tier workloads with performance and reliability constraints. In: 2012 International Conference on High Performance Computing Simulation (HPCS), pp. 74–83 (2012). https://doi.org/10.1109/HPCSim.2012.6266893
Scotti, G., Zoni, D.: A fresh view on the microarchitectural design of FPGA-based risc cpus in the IoT era. J. Low Power Electron. Appl. 9, 19 (2019). https://doi.org/10.3390/jlpea9010009
Shiue, L.J., Jones, I., Peters, J.: A realtime GPU subdivision kernel. In: ACM Transactions on Graphics (TOG). vol. 24, pp. 1010–1015. ACM (2005)
Strecha, C., von Hansen, W., Van Gool, L., Fua, P., Thoennessen, U.: On benchmarking camera calibration and multi-view stereo for high resolution imagery. In: Computer Vision and Pattern Recognition, pp. 1–8. IEEE (2008)
Thomas, S., et al.: CortexSuite: a synthetic brain benchmark suite. In: International Symposium on Workload Characterization (IISWC), October 2014
Vu, H.H., Keriven, R., Labatut, P., Pons, J.P.: Towards high-resolution large-scale multi-view stereo. In: Computer Vision and Pattern Recognition, pp. 1430–1437. IEEE (2009)
Woo, S.C., Ohara, M., Torrie, E., Singh, J.P., Gupta, A.: The splash-2 programs: characterization and methodological considerations. In: Proceedings of the 22nd Annual International Symposium on Computer Architecture, pp. 24–36. ISCA 1995
Xu, J., Sun, Y., Zhang, L.: A mapping-based approach to eliminating self-intersection of offset paths on mesh surfaces for CNC machining. Computer-Aided Design 62, 131–142 (2015)
Zaharescu, Andrei., Boyer, Edmond, Horaud, Radu: TransforMesh : a topology-adaptive mesh-based approach to surface evolution. In: Yagi, Yasushi, Kang, Sing Bing, Kweon, In So, Zha, Hongbin (eds.) ACCV 2007. LNCS, vol. 4844, pp. 166–175. Springer, Heidelberg (2007). https://doi.org/10.1007/978-3-540-76390-1_17
Zoni, D., Cremona, L., Fornaciari, W.: All-digital control-theoretic scheme to optimize energy budget and allocation in multi-cores. IEEE Trans. Comput. 69(5), 706–721 (2020). https://doi.org/10.1109/TC.2019.2963859
Zoni, D., Cremona, L., Fornaciari, W.: All-digital energy-constrained controller for general-purpose accelerators and cpus. IEEE Embedded Syst. Lett. 12(1), 17–20 (2020). https://doi.org/10.1109/LES.2019.2914136
Zoni, D., Galimberti, A., Fornaciari, W.: Flexible and scalable FPGA-oriented design of multipliers for large binary polynomials. IEEE Access 8, 75809–75821 (2020). https://doi.org/10.1109/ACCESS.2020.2989423
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 Springer Nature Switzerland AG
About this paper
Cite this paper
Cremona, L., Fornaciari, W., Galimberti, A., Romanoni, A., Zoni, D. (2020). VGM-Bench: FPU Benchmark Suite for Computer Vision, Computer Graphics and Machine Learning Applications. In: Orailoglu, A., Jung, M., Reichenbach, M. (eds) Embedded Computer Systems: Architectures, Modeling, and Simulation. SAMOS 2020. Lecture Notes in Computer Science(), vol 12471. Springer, Cham. https://doi.org/10.1007/978-3-030-60939-9_23
Download citation
DOI: https://doi.org/10.1007/978-3-030-60939-9_23
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-60938-2
Online ISBN: 978-3-030-60939-9
eBook Packages: Computer ScienceComputer Science (R0)