Parallel Tempering MCMC Acceleration Using Reconfigurable Hardware

  • Grigorios Mingas
  • Christos-Savvas Bouganis
Part of the Lecture Notes in Computer Science book series (LNCS, volume 7199)


Markov Chain Monte Carlo (MCMC) is a family of algorithms which is used to draw samples from arbitrary probability distributions in order to estimate - otherwise intractable - integrals. When the distribution is complex, simple MCMC becomes inefficient and advanced variations are employed. This paper proposes a novel FPGA architecture to accelerate Parallel Tempering, a computationally expensive, popular MCMC method, which is designed to sample from multimodal distributions. The proposed architecture can be used to sample from any distribution. Moreover, the work demonstrates that MCMC is robust to reductions in the arithmetic precision used to evaluate the sampling distribution and this robustness is exploited to improve the FPGA’s performance. A 1072x speedup compared to software and a 3.84x speedup compared to a GPGPU implementation are achieved when performing Bayesian inference for a mixture model without any compromise on the quality of results, opening the way for the handling of previously intractable problems.


Mixture Model Markov Chain Monte Carlo Markov Chain Monte Carlo Method Target Distribution Probability Evaluation 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Andrieu, C., Roberts, G.O.: The pseudo-marginal approach for efficient Monte Carlo computations. The Annals of Statistics 37(2), 697–725 (2009)MathSciNetCrossRefzbMATHGoogle Scholar
  2. 2.
    Asadi, N.B., Meng, T.H., Wong, W.H.: Reconfigurable computing for learning Bayesian networks. In: Proceedings of the 16th International ACM/SIGDA Symposium on Field Programmable Gate Arrays, FPGA 2008, pp. 203–211 (2008)Google Scholar
  3. 3.
    Byrd, J., Jarvis, S., Bhalerao, A.: Reducing the run-time of MCMC programs by multithreading on SMP architectures. In: IEEE International Symposium on Parallel and Distributed Processing, IPDPS 2008, pp. 1–8 (April 2008)Google Scholar
  4. 4.
    Chatzis, S.: A method for training finite mixture models under a fuzzy clustering principle. Fuzzy Sets and Systems 161(23), 3000–3013 (2010)MathSciNetCrossRefzbMATHGoogle Scholar
  5. 5.
    de Dinechin, F., Pasca, B.: Designing Custom Arithmetic Data Paths with FloPoCo. IEEE Design and Test of Computers 28, 18–27 (2011)CrossRefGoogle Scholar
  6. 6.
    Earl, D.J., Deem, M.W.: Parallel tempering: Theory, applications, and new perspectives. Phys. Chem. Chem. Phys. 7, 3910–3916 (2005)CrossRefGoogle Scholar
  7. 7.
    Fielding, M., Nott, D.J., Liong, S.Y.: Efficient MCMC Schemes for Computationally Expensive Posterior Distributions. Technometrics 53(1), 16–28 (2011)MathSciNetCrossRefGoogle Scholar
  8. 8.
    Geyer, C.J.: Markov Chain Monte Carlo Maximum Likelihood. In: Proceedings of the 23rd Symposium on the Interface, Computing Science and Statistics, pp. 156–163 (1991)Google Scholar
  9. 9.
    Jasra, A., Stephens, D.A., Holmes, C.C.: On population-based simulation for static inference. Statistics and Computing, 263–279 (2007)Google Scholar
  10. 10.
    Kou, S.C., Zhou, Q., Wong, W.H.: Equi-energy sampler with applications in statistical inference and statistical mechanics. Ann. Statist. 34(4), 1581–1652 (2006)MathSciNetCrossRefzbMATHGoogle Scholar
  11. 11.
    Lee, A., Yau, C., Giles, M.B., Doucet, A., Holmes, C.C.: On the Utility of Graphics Cards to Perform Massively Parallel Simulation of Advanced Monte Carlo Methods. Journal of Computational and Graphical Statistics 19(4), 769–789 (2010)CrossRefGoogle Scholar
  12. 12.
    Li, Y., Mascagni, M., Gorin, A.: A decentralized parallel implementation for parallel tempering algorithm. Parallel Comput. 35, 269–283 (2009)MathSciNetCrossRefGoogle Scholar
  13. 13.
    Liu, J.S.: Monte Carlo strategies in scientific computing. Springer, Heidelberg (2001)zbMATHGoogle Scholar
  14. 14.
    Mansinghka, V.K., Jonas, E.M., Tenenbaum, J.B.: Stochastic Digital Circuits for Probabilistic Inference. Technical Report MIT-CSAIL-TR-2008-069, Massachussets Institute of Technology (2008)Google Scholar
  15. 15.
    Saiprasert, C., Bouganis, C.-S., Constantinides, G.A.: Design of a Financial Application Driven Multivariate Gaussian Random Number Generator for an FPGA. In: Sirisuk, P., Morgan, F., El-Ghazawi, T., Amano, H. (eds.) ARC 2010. LNCS, vol. 5992, pp. 182–193. Springer, Heidelberg (2010)CrossRefGoogle Scholar
  16. 16.
    Thomas, D.B., Luk, W., Leong, P.H., Villasenor, J.D.: Gaussian random number generators. ACM Comput. Surv. 39 (November 2007)Google Scholar
  17. 17.
    Tian, X., Bouganis, C.S.: A Run-Time Adaptive FPGA Architecture for Monte Carlo Simulations. In: 2011 International Conference on Field Programmable Logic and Applications (FPL), pp. 116–122 (September 2011)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2012

Authors and Affiliations

  • Grigorios Mingas
    • 1
  • Christos-Savvas Bouganis
    • 1
  1. 1.Department of Electrical & Electronic EngineeringImperial College LondonLondonUnited Kingdom

Personalised recommendations