Skip to main content

Parallel Tempering MCMC Acceleration Using Reconfigurable Hardware

  • Conference paper

Part of the Lecture Notes in Computer Science book series (LNTCS,volume 7199)

Abstract

Markov Chain Monte Carlo (MCMC) is a family of algorithms which is used to draw samples from arbitrary probability distributions in order to estimate - otherwise intractable - integrals. When the distribution is complex, simple MCMC becomes inefficient and advanced variations are employed. This paper proposes a novel FPGA architecture to accelerate Parallel Tempering, a computationally expensive, popular MCMC method, which is designed to sample from multimodal distributions. The proposed architecture can be used to sample from any distribution. Moreover, the work demonstrates that MCMC is robust to reductions in the arithmetic precision used to evaluate the sampling distribution and this robustness is exploited to improve the FPGA’s performance. A 1072x speedup compared to software and a 3.84x speedup compared to a GPGPU implementation are achieved when performing Bayesian inference for a mixture model without any compromise on the quality of results, opening the way for the handling of previously intractable problems.

Keywords

  • Mixture Model
  • Markov Chain Monte Carlo
  • Markov Chain Monte Carlo Method
  • Target Distribution
  • Probability Evaluation

These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

This is a preview of subscription content, access via your institution.

Buying options

Chapter
USD   29.95
Price excludes VAT (Canada)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   39.99
Price excludes VAT (Canada)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   54.99
Price excludes VAT (Canada)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Andrieu, C., Roberts, G.O.: The pseudo-marginal approach for efficient Monte Carlo computations. The Annals of Statistics 37(2), 697–725 (2009)

    CrossRef  MathSciNet  MATH  Google Scholar 

  2. Asadi, N.B., Meng, T.H., Wong, W.H.: Reconfigurable computing for learning Bayesian networks. In: Proceedings of the 16th International ACM/SIGDA Symposium on Field Programmable Gate Arrays, FPGA 2008, pp. 203–211 (2008)

    Google Scholar 

  3. Byrd, J., Jarvis, S., Bhalerao, A.: Reducing the run-time of MCMC programs by multithreading on SMP architectures. In: IEEE International Symposium on Parallel and Distributed Processing, IPDPS 2008, pp. 1–8 (April 2008)

    Google Scholar 

  4. Chatzis, S.: A method for training finite mixture models under a fuzzy clustering principle. Fuzzy Sets and Systems 161(23), 3000–3013 (2010)

    CrossRef  MathSciNet  MATH  Google Scholar 

  5. de Dinechin, F., Pasca, B.: Designing Custom Arithmetic Data Paths with FloPoCo. IEEE Design and Test of Computers 28, 18–27 (2011)

    CrossRef  Google Scholar 

  6. Earl, D.J., Deem, M.W.: Parallel tempering: Theory, applications, and new perspectives. Phys. Chem. Chem. Phys. 7, 3910–3916 (2005)

    CrossRef  Google Scholar 

  7. Fielding, M., Nott, D.J., Liong, S.Y.: Efficient MCMC Schemes for Computationally Expensive Posterior Distributions. Technometrics 53(1), 16–28 (2011)

    CrossRef  MathSciNet  Google Scholar 

  8. Geyer, C.J.: Markov Chain Monte Carlo Maximum Likelihood. In: Proceedings of the 23rd Symposium on the Interface, Computing Science and Statistics, pp. 156–163 (1991)

    Google Scholar 

  9. Jasra, A., Stephens, D.A., Holmes, C.C.: On population-based simulation for static inference. Statistics and Computing, 263–279 (2007)

    Google Scholar 

  10. Kou, S.C., Zhou, Q., Wong, W.H.: Equi-energy sampler with applications in statistical inference and statistical mechanics. Ann. Statist. 34(4), 1581–1652 (2006)

    CrossRef  MathSciNet  MATH  Google Scholar 

  11. Lee, A., Yau, C., Giles, M.B., Doucet, A., Holmes, C.C.: On the Utility of Graphics Cards to Perform Massively Parallel Simulation of Advanced Monte Carlo Methods. Journal of Computational and Graphical Statistics 19(4), 769–789 (2010)

    CrossRef  Google Scholar 

  12. Li, Y., Mascagni, M., Gorin, A.: A decentralized parallel implementation for parallel tempering algorithm. Parallel Comput. 35, 269–283 (2009)

    CrossRef  MathSciNet  Google Scholar 

  13. Liu, J.S.: Monte Carlo strategies in scientific computing. Springer, Heidelberg (2001)

    MATH  Google Scholar 

  14. Mansinghka, V.K., Jonas, E.M., Tenenbaum, J.B.: Stochastic Digital Circuits for Probabilistic Inference. Technical Report MIT-CSAIL-TR-2008-069, Massachussets Institute of Technology (2008)

    Google Scholar 

  15. Saiprasert, C., Bouganis, C.-S., Constantinides, G.A.: Design of a Financial Application Driven Multivariate Gaussian Random Number Generator for an FPGA. In: Sirisuk, P., Morgan, F., El-Ghazawi, T., Amano, H. (eds.) ARC 2010. LNCS, vol. 5992, pp. 182–193. Springer, Heidelberg (2010)

    CrossRef  Google Scholar 

  16. Thomas, D.B., Luk, W., Leong, P.H., Villasenor, J.D.: Gaussian random number generators. ACM Comput. Surv. 39 (November 2007)

    Google Scholar 

  17. Tian, X., Bouganis, C.S.: A Run-Time Adaptive FPGA Architecture for Monte Carlo Simulations. In: 2011 International Conference on Field Programmable Logic and Applications (FPL), pp. 116–122 (September 2011)

    Google Scholar 

Download references

Author information

Authors and Affiliations

Authors

Editor information

Editors and Affiliations

Rights and permissions

Reprints and Permissions

Copyright information

© 2012 Springer-Verlag Berlin Heidelberg

About this paper

Cite this paper

Mingas, G., Bouganis, CS. (2012). Parallel Tempering MCMC Acceleration Using Reconfigurable Hardware. In: Choy, O.C.S., Cheung, R.C.C., Athanas, P., Sano, K. (eds) Reconfigurable Computing: Architectures, Tools and Applications. ARC 2012. Lecture Notes in Computer Science, vol 7199. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-28365-9_19

Download citation

  • DOI: https://doi.org/10.1007/978-3-642-28365-9_19

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-642-28364-2

  • Online ISBN: 978-3-642-28365-9

  • eBook Packages: Computer ScienceComputer Science (R0)