Skip to main content
SpringerLink
Log in

Accuracy-Aware Compilers

  • Chapter
  • First Online:
Approximate Computing Techniques

  • 775 Accesses

Abstract

Many modern applications (such as multimedia processing, machine learning, and big data analytics) exhibit a natural tradeoff between the accuracy of the results they produce and the application’s execution time or energy consumption. These applications allow us to investigate new, more aggressive program optimization approaches. This chapter outlines the foundation of program optimization systems that expose and profitably exploit tradeoffs between the accuracy of the results that the program produces and the time and energy required to produce those results. These systems apply accuracy-aware program transformations that intentionally change the semantics of optimized programs. We discuss program analysis techniques that quantify the loss of accuracy introduced by program transformations.

We will present the fundamental techniques for the accuracy-aware compilation that help us discover and characterize the accuracy–performance tradeoffs in sequential programs. Through two case studies, we will describe two main directions in developing accuracy-aware compilers: (1) sensitivity profiling-based techniques speculatively execute candidate transformed programs and use dynamic program analysis to reason about their accuracy, while (2) analysis-based techniques use static analysis to derive formulas that characterize acceptable results and use mathematical optimization to find the best performing program versions, without executing them.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 79.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 99.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 129.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    Despite the name “accuracy metric” being traditionally used, these functions commonly compute the accuracy loss compared to some reference result. When these functions evaluate to a value close to 0, that means that the result of the approximate program is more accurate. They also do not always satisfy all the properties of mathematical metrics. For instance, distortion is not symmetric.

References

  1. Kildall, G. A. (1973). A unified approach to global program optimization. In Proceedings of the POPL.

    Google Scholar 

  2. Carbin, M., Kim, D., Misailovic, S., & Rinard, M. (2012). Proving acceptability properties of relaxed nondeterministic approximate programs. In Proceedings of the PLDI.

    Google Scholar 

  3. Rinard, M. (2006). Probabilistic accuracy bounds for fault-tolerant computations that discard tasks. In Proceedings of the ICS.

    Google Scholar 

  4. Chakradhar, S., Raghunathan, A., & Meng, J. (2009). Best-effort parallel execution framework for recognition and mining applications. In International Symposium on Parallel & Distributed Processing (IPDPS).

    Google Scholar 

  5. Meng, J., Raghunathan, A., Chakradhar, S., & Byna, S. (2010). Exploiting the forgiving nature of applications for scalable parallel execution. In International Symposium on Parallel & Distributed Processing (IPDPS).

    Google Scholar 

  6. Misailovic, S., Sidiroglou, S., Hoffmann, H., & Rinard, M. (2010). Quality of service profiling. In Proceedings of the ICSE.

    Google Scholar 

  7. Sidiroglou, S., Misailovic, S., Hoffmann, H., & Rinard, M. (2011). Managing performance vs. accuracy trade-offs with loop perforation. In Proceedings of the FSE.

    Google Scholar 

  8. Misailovic, S., Roy, D., & Rinard, M. (2011). Probabilistically accurate program transformations. In Proceedings of the SAS.

    Google Scholar 

  9. Zhu, Z., Misailovic, S., Kelner, J., & Rinard, M. (2012). Randomized accuracy-aware program transformations for efficient approximate computations. In Proceedings of the POPL.

    Google Scholar 

  10. Samadi, M., Jamshidi, D., Lee, J., & Mahlke, S. (2014). Paraprox: Pattern-based approximation for data parallel applications. In Proceedings of the ASPLOS.

    Google Scholar 

  11. Baek, W., & Chilimbi, T. M. (2010). Green: A framework for supporting energy-conscious programming using controlled approximation. In Proceedings of the PLDI.

    Google Scholar 

  12. Ansel, J., Wong, Y., Chan, C., Olszewski, M., Edelman, A., & Amarasinghe, S. (2011). Language and compiler support for auto-tuning variable-accuracy algorithms. In International Symposium on CGO.

    Google Scholar 

  13. Hoffmann, H., Sidiroglou, S., Carbin, M., Misailovic, S., Agarwal, A., & Rinard, M. (2011). Dynamic knobs for responsive power-aware computing. In Proceedings of the ASPLOS.

    Google Scholar 

  14. Rubio-González, C., Nguyen, C., Nguyen, H., Demmel, J., Kahan, W., Sen, K., Bailey, D., Iancu, C., & Hough, D. (2013). Precimonious: Tuning assistant for floating-point precision. In Proceedings of the SC.

    Google Scholar 

  15. Schkufza, E., Sharma, R., & Aiken, A. (2014). Stochastic optimization of floating-point programs with tunable precision. In Proceedings of the PLDI.

    Google Scholar 

  16. Han, S., Mao, H., & Dally, W. (2016). Deep compression: Compressing deep neural networks with pruning, trained quantization and Huffman coding. In International Conference on ICLR.

    Google Scholar 

  17. Cormode, G., & Garofalakis, M. (2007). Sketching probabilistic data streams. In Proceedings of the SIGMOD/PODS.

    Google Scholar 

  18. Rinard, M. (2007). Using early phase termination to eliminate load imbalances at barrier synchronization points. In Proceedings of the OOPSLA.

    Google Scholar 

  19. Misailovic, S., Sidiroglou, S., & Rinard, M. (2012). Dancing with uncertainty. In Proceedings of the RACES.

    Google Scholar 

  20. Misailovic, S., Kim, D., & Rinard, M. (2013). Parallelizing sequential programs with statistical accuracy tests. ACM Transactions Embedded Computing System Special Issue on Probabilistic Embedded Computing, 12(2s), 1–26.

    Article  Google Scholar 

  21. Campanoni, S., Holloway, G., Wei, G.-Y., & Brooks, D. (2015). Helix-up: Relaxing program semantics to unleash parallelization. In International Symposium on CGO.

    Google Scholar 

  22. Sampson, A., Dietl, W., Fortuna, E., Gnanapragasam, D., Ceze, L., & Grossman, D. (2011). EnerJ: Approximate data types for safe and general low-power computation. In Proceedings of the PLDI.

    Google Scholar 

  23. Esmaeilzadeh, H., Sampson, A., Ceze, L., & Burger, D. (2012). Neural acceleration for general-purpose approximate programs. In International Symposium on MICRO.

    Google Scholar 

  24. Liu, S., Pattabiraman, K., Moscibroda, T., & Zorn, B. (2011). Flikker: Saving DRAM refresh-power through critical data partitioning. In Proceedings of the ASPLOS.

    Google Scholar 

  25. Fernando, V., Franques, A., Abadal, S., Misailovic, S., & Torrellas, J. (2019). Replica: A wireless manycore for communication-intensive and approximate data. In Proceedings of the ASPLOS.

    Google Scholar 

  26. Chippa, V., Chakradhar, S., Roy, K., & Raghunathan, A. (2013). Analysis and characterization of inherent application resilience for approximate computing. In Proceedings of the DAC.

    Google Scholar 

  27. Misailovic, S., Carbin, M., Achour, S., Qi, Z., & Rinard, M. (2014). Chisel: Reliability- and accuracy-aware optimization of approximate computational kernels. In Proceedings of the OOPSLA.

    Google Scholar 

  28. Sharif, H., Srivastava, P., Huzaifa, M., Kotsifakou, M., Joshi, K., Sarita, Y., Zhao, N., Adve, V., Misailovic, S., & Adve, S. (2019). ApproxHPVM: A portable compiler IR for accuracy-aware optimizations. In Proceedings of ACM on Programming Languages, 3(OOPSLA).

    Google Scholar 

  29. Hoffmann, H., Misailovic, S., Sidiroglou, S., Agarwal, A., & Rinard, M. (2009, September). Using code perforation to improve performance, reduce energy consumption, and respond to failures. Tech. Rep. MIT-CSAIL-TR-2009-042, MIT.

    Google Scholar 

  30. Lattner, C., & Adve, V. (2004). LLVM: A compilation framework for lifelong program analysis & transformation. In International Symposium on CGO.

    Google Scholar 

  31. Ansel, J., Kamil, S., Veeramachaneni, K., Ragan-Kelley, J., Bosboom, J., O’Reilly, U. M., & Amarasinghe, S. (2014). OpenTuner: An extensible framework for program autotuning. In International Conference on PACT.

    Google Scholar 

  32. Sharif, H., Zhao, Y., Kotsifakou, M., Kothari, A., Schreiber, B., Wang, E., Sarita, Y., Zhao, N., Joshi, K., Adve, S., Misailovic, S., & Adve, S. (2021). ApproxTuner: A compiler and runtime system for adaptive approximations. In Proceedings of the PPoPP (pp. 262–277).

    Google Scholar 

  33. Geilen, M., & Basten, T. (2007). A calculator for pareto points. In Proceedings of the DATE.

    Google Scholar 

  34. P. B. Suite. http://parsec.cs.princeton.edu/

  35. X. V. T. Media. http://media.xiph.org/video/derf

  36. Park, J., Esmaeilzadeh, H., Zhang, X., Naik, M., & Harris, W. (2015). FlexJava: Language support for safe and modular approximate programming. In Proceedings of the FSE.

    Google Scholar 

  37. Carbin, M., Kim, D., Misailovic, S., & Rinard, M. (2013). Verified integrity properties for safe approximate program transformations. In Proceedings of the PEPM.

    Google Scholar 

  38. Lou, L., Nguyen, P., Lawrence, J., & Barnes, C. (2016). Image perforation: Automatically accelerating image pipelines by intelligently skipping samples. ACM Transactions on Graphics (TOG), 35(5), 1–14.

    Article  Google Scholar 

  39. Figurnov, M., Ibraimova, A., Vetrov, D., & Kohli, P. (2016). PerforatedCNNs: Acceleration through elimination of redundant convolutions.

    Google Scholar 

  40. Maier, D., Cosenza, B., & Juurlink, B. (2018). Local memory-aware kernel perforation. In International Symposium on CGO.

    Google Scholar 

  41. Li, S., Park, S., & Mahlke, S. (2018). Sculptor: Flexible approximation with selective dynamic loop perforation. In Proceedings of the ISC (pp. 341–351).

    Google Scholar 

  42. Xu, R., Koo, J., Kumar, R., Bai, P., Mitra, S., Misailovic, S., & Bagchi, S. (2018). VideoChef: Efficient approximation for streaming video processing pipelines. In Proceedings of the USENIX ATC.

    Google Scholar 

  43. Palem, K. (2005). Energy aware computing through probabilistic switching: A study of limits. IEEE Transactions on Computers, 54(9), 1123–1137.

    Article  Google Scholar 

  44. Leem, L., Cho, H., Bau, J., Jacobson, Q., & Mitra, S. (2010). ERSA: Error resilient system architecture for probabilistic applications. In Proceedings of the DATE.

    Google Scholar 

  45. Lee, K., Shrivastava, A., Issenin, I., Dutt, N., & Venkatasubramanian, N. (2006). Mitigating soft error failures for multimedia applications by selective data protection. In Proceedings of the CASES.

    Google Scholar 

  46. Carbin, M., Misailovic, S., & Rinard, M. (2013). Verifying quantitative reliability for programs that execute on unreliable hardware. In Proceedings of the OOPSLA.

    Google Scholar 

  47. Bjørner, N., Phan, A., & Fleckenstein, L. (2015). νZ-an optimizing SMT solver. In Internation Conference on TACAS.

    Google Scholar 

  48. Boston, B., Gong, Z., & Carbin, M. (2018). Leto: Verifying application-specific hardware fault tolerance with programmable execution models. Proceedings of the ACM on Programming Languages, 2(OOPSLA), 1–30.

    Google Scholar 

  49. Fernando, V., Joshi, K., & Misailovic, S. (2019). Verifying safety and accuracy of approximate parallel programs via canonical sequentialization. Proceedings of the ACM on Programming Languages, 3(OOPSLA), 1–29.

    Google Scholar 

  50. Sui, X., Lenharth, A., Fussell, D., & Pingali, K. (2016). Proactive control of approximate programs. In Proceedings of the ASPLOS.

    Google Scholar 

  51. Chiang, W., Baranowski, M., Briggs, I., Solovyev, A., Gopalakrishnan, G., & Rakamaric, Z. (2017). Rigorous floating-point mixed-precision tuning. In Proceedings of the POPL.

    Google Scholar 

  52. Izycheva, A., Darulova, E., & Seidl, H. (2019). Synthesizing efficient low-precision kernels. In International Symposium on ATVA.

    Google Scholar 

  53. Han, J., & Orshansky, M. (2013). Approximate computing: An emerging paradigm for energy-efficient design. In Proceedings of the ETS.

    Google Scholar 

  54. Xu, Q., Mytkowicz, T., & Kim, N.-S. (2016). Approximate computing: A survey. IEEE Design & Test, 33(1), 8–22.

    Article  Google Scholar 

  55. Stanley-Marbell, P., Alaghi, A., Carbin, M., Darulova, E., Dolecek, L., Gerstlauer, A., Gillani, G., Jevdjic, D., Moreau, T., Cacciotti, M., Daglis, A., Enright-Jerger, N., Falsafi, B., Misailovic, S., Sampson, A., & Zufferey, D. (2020). Exploiting errors for efficiency: A survey from circuits to applications. ACM Computing Surveys, 53(3), 51:1–51:39.

    Google Scholar 

  56. Carbin, M., & Rinard, M. (2010). Automatically identifying critical input regions and code in applications. In Proceedings of the ISSTA.

    Google Scholar 

  57. Roy, P., Ray, R., Wang, C., & Wong, W. (2014). ASAC: Automatic sensitivity analysis for approximate computing. In Proceedings of the LCTES.

    Google Scholar 

  58. Venkatagiri, R., Mahmoud, A., Hari, S., & Adve, S. (2016). Approxilyzer: Towards a systematic framework for instruction-level approximate computing and its application to hardware resiliency. In International Symposium on MICRO.

    Google Scholar 

  59. Nongpoh, B., Ray, R., Dutta, S., & Banerjee, A. (2017). Autosense: A framework for automated sensitivity analysis of program data. IEEE Transactions on Software Engineering, 43(12), 1110–1124.

    Article  Google Scholar 

  60. Gaffar, A., Mencer, O., Luk, W., Cheung, P., & Shirazi, N. (2002). Floating-point bitwidth analysis via automatic differentiation. In International Conference on FPT.

    Google Scholar 

  61. Chaudhuri, S., Gulwani, S., Lublinerman, R., & Navidpour, S. (2011). Proving programs robust. In Proceedings of the FSE.

    Google Scholar 

  62. Darulova, E., & Kuncak, V. (2014). Sound compilation of reals. In Proceedings of the POPL.

    Google Scholar 

  63. He, S., Lahiri, S., & Rakamaric, Z. (2018). Verifying relative safety, accuracy, and termination for program approximations. Journal of Automated Reasoning, 60(1), 23–42.

    Article  MathSciNet  Google Scholar 

  64. Barone, S., Traiola, M., Barbareschi, M., & Bosio, A. (2021). Multi-objective application-driven approximate design method. IEEE Access.

    Google Scholar 

  65. Dorn, J., Lacomis, J., Weimer, W., & Forrest, S. (2019). Automatically exploring tradeoffs between software output fidelity and energy costs. IEEE Transactions on Software Engineering, 45(3), 219–236.

    Article  Google Scholar 

  66. Ha, V., & Sentieys, O. (2021). Leveraging Bayesian optimization to speed up automatic precision tuning. In Proceedings of the DATE.

    Google Scholar 

  67. Vassiliadis, V., Riehme, J., Deussen, J., Parasyris, K., Antonopoulos, C., Bellas, N., Lalis, S., & Naumann, U. (2016). Towards automatic significance analysis for approximate computing. In International Symposium on CGO.

    Google Scholar 

  68. Sampson, A., Baixo, A., Ransford, B., Moreau, T., Yip, J., Ceze, L., & Oskin, M. (2015). Accept: A programmer-guided compiler framework for practical approximate computing. Tech. Rep. UW-CSE-15-01, University of Washington.

    Google Scholar 

  69. Samadi, M., Lee, J., Jamshidi, D., Hormati, A., & Mahlke, S. (2013). Sage: Self-tuning approximation for graphics engines. International Symposium on MICRO.

    Google Scholar 

  70. Goiri, I., Bianchini, R., Nagarakatte, S., & Nguyen, T. (2015). Approxhadoop: Bringing approximations to MapReduce frameworks. In Proceedings of the ASPLOS.

    Google Scholar 

  71. Ding, Y., Ansel, J., Veeramachaneni, K., Shen, X., O’Reilly, U. M., & Amarasinghe, S. (2015). Autotuning algorithmic choice for input sensitivity. In Proceedings of the PLDI.

    Google Scholar 

  72. Barati, S., Bartha, F., Biswas, S., Cartwright, R., Duracz, A., Fussell, D., Hoffmann, H., Imes, C., Miller, J., Mishra, N., Arvind, Nguyen, D., Palem, K., Pei, Y., Pingali, K., Sai, R., Wright, A., Yang, Y., & Zhang, S. (2019). Proteus: Language and runtime support for self-adaptive software development. IEEE Software, 36(2), 73–82.

    Google Scholar 

  73. Mahajan, D., Yazdanbakhsh, A., Park, J., Thwaites, B., & Esmaeilzadeh, H. (2016). Towards statistical guarantees in controlling quality tradeoffs for approximate acceleration. In International Symposium on Computer Architecture (ISCA).

    Google Scholar 

  74. Laurenzano, M., Hill, P., Samadi, M., Mahlke, S., Mars, J., & Tang, L. (2016). Input responsiveness: Using canary inputs to dynamically steer approximation. In Proceedings of the PLDI.

    Google Scholar 

  75. Misailovic, S. (2015). Accuracy-Aware Optimization of Approximate Programs. PhD thesis, Massachusetts Institute of Technology, Cambridge, MA.

    Google Scholar 

Download references

Acknowledgements

The presentation in this chapter is derived from the author’s Ph.D. dissertation [75]. The results from Case Study 1 appeared first in the FSE 2011 paper [7]. The results from Case Study 2 appeared first in the OOPSLA 2014 paper [27]. The author would like to thank all of his co-authors who contributed to bringing up the theory and practice of accuracy-aware program optimization; special gratitude goes to Martin Rinard, Hank Hoffmann, Stelios Sidiroglou, and Michael Carbin. The author would also like to thank Keyur Joshi, Olivier Sentieys, Hashim Sharif, and Yifan Zhao for providing feedback on the earlier versions of this chapter.

Author information

Authors and Affiliations

  1. University of Illinois Urbana-Champaign, Champaign, IL, USA

    Sasa Misailovic

Authors
  1. Sasa Misailovic
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Sasa Misailovic .

Editor information

Editors and Affiliations

  1. University of Lyon, ECL, INSA Lyon, CNRS, UCBL, CPE Lyon, INL,, Écully, France

    Alberto Bosio

  2. IETR, Institut National des Sciences Appliqué, Rennes, France

    Daniel Ménard

  3. University of Rennes, INRIA/IRISA, Lannion, France

    Olivier Sentieys

Rights and permissions

Reprints and permissions

Copyright information

© 2022 Springer Nature Switzerland AG

About this chapter

Check for updates. Verify currency and authenticity via CrossMark

Cite this chapter

Misailovic, S. (2022). Accuracy-Aware Compilers. In: Bosio, A., Ménard, D., Sentieys, O. (eds) Approximate Computing Techniques. Springer, Cham. https://doi.org/10.1007/978-3-030-94705-7_7

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-94705-7_7

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-94704-0

  • Online ISBN: 978-3-030-94705-7

  • eBook Packages: EngineeringEngineering (R0)

Publish with us

Policies and ethics

Search

Navigation