Chemical Boltzmann Machines

  • William Poole
  • Andrés Ortiz-Muñoz
  • Abhishek Behera
  • Nick S. Jones
  • Thomas E. Ouldridge
  • Erik Winfree
  • Manoj GopalkrishnanEmail author
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10467)


How smart can a micron-sized bag of chemicals be? How can an artificial or real cell make inferences about its environment? From which kinds of probability distributions can chemical reaction networks sample? We begin tackling these questions by showing three ways in which a stochastic chemical reaction network can implement a Boltzmann machine, a stochastic neural network model that can generate a wide range of probability distributions and compute conditional probabilities. The resulting models, and the associated theorems, provide a road map for constructing chemical reaction networks that exploit their native stochasticity as a computational resource. Finally, to show the potential of our models, we simulate a chemical Boltzmann machine to classify and generate MNIST digits in-silico.



This work was supported in part by U.S. National Science Foundation (NSF) graduate fellowships to WP and to AOM, by NSF grant CCF-1317694 to EW, and by the Gordon and Betty Moore Foundation through Grant GBMF2809 to the Caltech Programmable Molecular Technology Initiative (PMTI), by a Royal Society University Research Fellowship to TEO, and by a Bharti Centre for Communication in IIT Bombay award to AB.


  1. 1.
    Bray, D.: Protein molecules as computational elements in living cells. Nature 376(6538), 307 (1995)CrossRefGoogle Scholar
  2. 2.
    Bray, D.: Wetware: A Computer in Every Living Cell. Yale University Press, New Haven (2009)Google Scholar
  3. 3.
    McAdams, H.H., Arkin, A.: Stochastic mechanisms in gene expression. Proc. Natl. Acad. Sci. 94(3), 814–819 (1997)CrossRefGoogle Scholar
  4. 4.
    Elowitz, M.B., Levine, A.J., Siggia, E.D., Swain, P.S.: Stochastic gene expression in a single cell. Science 297(5584), 1183–1186 (2002)CrossRefGoogle Scholar
  5. 5.
    Perkins, T.J., Swain, P.S.: Strategies for cellular decision making. Mol. Syst. Biol. 5(1), 326 (2009)Google Scholar
  6. 6.
    Muroga, S.: Threshold Logic and Its Applications. Wiley Interscience, New York (1971)zbMATHGoogle Scholar
  7. 7.
    Hopfield, J.J.: Neural networks and physical systems with emergent collective computational abilities. Proc. Natl. Acad. Sci. 79(8), 2554–2558 (1982)MathSciNetCrossRefGoogle Scholar
  8. 8.
    Hinton, G.E., Sejnowski, T.J., Ackley, D.H.: Boltzmann Machines: Constraint Satisfaction Networks that Learn. Department of Computer Science, Carnegie-Mellon University, Pittsburgh (1984)Google Scholar
  9. 9.
    Bray, D.: Intracellular signalling as a parallel distributed process. J. Theor. Biol. 143(2), 215–231 (1990)CrossRefGoogle Scholar
  10. 10.
    Hellingwerf, K.J., Postma, P.W., Tommassen, J., Westerhoff, H.V.: Signal transduction in bacteria: phospho-neural network(s) in Escherichia coli. FEMS Microbiol. Rev. 16(4), 309–321 (1995)CrossRefGoogle Scholar
  11. 11.
    Mjolsness, E., Sharp, D.H., Reinitz, J.: A connectionist model of development. J. Theor. Biol. 152(4), 429–453 (1991)CrossRefGoogle Scholar
  12. 12.
    Mestl, T., Lemay, C., Glass, L.: Chaos in high-dimensional neural and gene networks. Physica D: Nonlin. Phenom. 98(1), 33–52 (1996)MathSciNetCrossRefzbMATHGoogle Scholar
  13. 13.
    Buchler, N.E., Gerland, U., Hwa, T.: On schemes of combinatorial transcription logic. Proc. Natl. Acad. Sci. 100(9), 5136–5141 (2003)CrossRefGoogle Scholar
  14. 14.
    Deutsch, J.M.: Collective regulation by non-coding RNA. arXiv preprint arXiv:1409.1899 (2014)
  15. 15.
    Deutsch, J.M.: Associative memory by collective regulation of non-coding RNA. arXiv preprint arXiv:1608.05494 (2016)
  16. 16.
    Hjelmfelt, A., Weinberger, E.D., Ross, J.: Chemical implementation of neural networks and turing machines. Proc. Natl. Acad. Sci. 88(24), 10983–10987 (1991)CrossRefzbMATHGoogle Scholar
  17. 17.
    Hjelmfelt, A., Ross, J.: Chemical implementation and thermodynamics of collective neural networks. Proc. Natl. Acad. Sci. 89(1), 388–391 (1992)CrossRefGoogle Scholar
  18. 18.
    Kim, J., Hopfield, J.J., Winfree, E.: Neural network computation by in vitro transcriptional circuits. In: Advances in Neural Information Processing Systems (NIPS), pp. 681–688 (2004)Google Scholar
  19. 19.
    Napp, N.E., Adams, R.P.: Message passing inference with chemical reaction networks. In: Advances in Neural Information Processing Systems (NIPS), pp. 2247–2255 (2013)Google Scholar
  20. 20.
    Gopalkrishnan, M.: A scheme for molecular computation of maximum likelihood estimators for log-linear models. In: Rondelez, Y., Woods, D. (eds.) DNA 2016. LNCS, vol. 9818, pp. 3–18. Springer, Cham (2016). doi: 10.1007/978-3-319-43994-5_1 CrossRefGoogle Scholar
  21. 21.
    Hjelmfelt, A., Schneider, F.W., Ross, J.: Pattern recognition in coupled chemical kinetic systems. Science 260, 335–335 (1993)CrossRefGoogle Scholar
  22. 22.
    Kim, J., White, K.S., Winfree, E.: Construction of an in vitro bistable circuit from synthetic transcriptional switches. Mol. Syst. Biol. 2, 68 (2006)CrossRefGoogle Scholar
  23. 23.
    Kim, J., Winfree, E.: Synthetic in vitro transcriptional oscillators. Mol. Syst. Biol. 7, 465 (2011)CrossRefGoogle Scholar
  24. 24.
    Qian, L., Winfree, E., Bruck, J.: Neural network computation with DNA strand displacement cascades. Nature 475(7356), 368–372 (2011)CrossRefGoogle Scholar
  25. 25.
    Lestas, I., Paulsson, J., Ross, N.E., Vinnicombe, G.: Noise in gene regulatory networks. IEEE Trans. Autom. Control 53, 189–200 (2008)MathSciNetCrossRefzbMATHGoogle Scholar
  26. 26.
    Lestas, I., Vinnicombe, G., Paulsson, J.: Fundamental limits on the suppression of molecular fluctuations. Nature 467(7312), 174–178 (2010)CrossRefGoogle Scholar
  27. 27.
    Veening, J.W., Smits, W.K., Kuipers, O.P.: Bistability, epigenetics, and bet-hedging in bacteria. Annu. Rev. Microbiol. 62, 193–210 (2008)CrossRefGoogle Scholar
  28. 28.
    Balázsi, G., van Oudenaarden, A., Collins, J.J.: Cellular decision making and biological noise: from microbes to mammals. Cell 144(6), 910–925 (2011)CrossRefGoogle Scholar
  29. 29.
    Tsimring, L.S.: Noise in biology. Rep. Prog. Phys. 77(2), 26601 (2014)CrossRefGoogle Scholar
  30. 30.
    Eldar, A., Elowitz, M.B.: Functional roles for noise in genetic circuits. Nature 467(7312), 167–173 (2010)CrossRefGoogle Scholar
  31. 31.
    Mansinghka, V.K.: Natively probabilistic computation. Ph.D. thesis, Massachusetts Institute of Technology (2009)Google Scholar
  32. 32.
    Wang, S., Zhang, X., Li, Y., Bashizade, R., Yang, S., Dwyer, C., Lebeck, A.R.: Accelerating Markov random field inference using molecular optical Gibbs sampling units. In: Proceedings of the 43rd International Symposium on Computer Architecture, pp. 558–569. IEEE Press (2016)Google Scholar
  33. 33.
    Fiser, J., Berkes, P., Orbán, G., Lengyel, M.: Statistically optimal perception and learning: from behavior to neural representations. Trends Cogn. Sci. 14(3), 119–130 (2010)CrossRefGoogle Scholar
  34. 34.
    Pouget, A., Beck, J.M., Ma, W.J., Latham, P.E.: Probabilistic brains: knowns and unknowns. Nat. Neurosci. 16(9), 1170–1178 (2013)CrossRefGoogle Scholar
  35. 35.
    Ackley, D.H., Hinton, G.E., Sejnowski, T.J.: A learning algorithm for Boltzmann machines. Cogn. Sci. 9(1), 147–169 (1985)CrossRefGoogle Scholar
  36. 36.
    Tanaka, T.: Mean-field theory of Boltzmann machine learning. Phys. Rev. E 58(2), 2302 (1998)CrossRefGoogle Scholar
  37. 37.
    Tang, Y., Sutskever, I.: Data normalization in the learning of restricted Boltzmann machines. Department of Computer Science, University of Toronto, Technical report UTML-TR-11-2 (2011)Google Scholar
  38. 38.
    Taylor, G.W., Hinton, G.E.: Factored conditional restricted Boltzmann machines for modeling motion style. In: Proceedings of the 26th Annual International Conference on Machine Learning (ICML), pp. 1025–1032. ACM (2009)Google Scholar
  39. 39.
    Casella, G., George, E.I.: Explaining the Gibbs sampler. Am. Stat. 46(3), 167–174 (1992)MathSciNetGoogle Scholar
  40. 40.
    Gillespie, D.T.: Stochastic simulation of chemical kinetics. Annu. Rev. Phys. Chem. 58, 35–55 (2007)CrossRefGoogle Scholar
  41. 41.
    Qian, H.: Phosphorylation energy hypothesis: open chemical systems and their biological functions. Annu. Rev. Phys. Chem. 58, 113–142 (2007)CrossRefGoogle Scholar
  42. 42.
    Beard, D.A., Qian, H.: Chemical Biophysics: Quantitative Analysis of Cellular Systems. Cambridge University Press, Cambridge (2008)CrossRefzbMATHGoogle Scholar
  43. 43.
    Ouldridge, T.E.: The importance of thermodynamics for molecular systems, the importance of molecular systems for thermodynamics. arXiv preprint arXiv:1702.00360 (2017)
  44. 44.
    Joshi, B.: A detailed balanced reaction network is sufficient but not necessary for its Markov chain to be detailed balanced. arXiv preprint arXiv:1312.4196 (2013)
  45. 45.
    Erez, A., Byrd, T.A., Vogel, R.M., Altan-Bonnet, G., Mugler, A.: Criticality of biochemical feedback. arXiv preprint arXiv:1703.04194 (2017)
  46. 46.
    Anderson, D.F., Craciun, G., Kurtz, T.G.: Product-form stationary distributions for deficiency zero chemical reaction networks. Bull. Math. Biol. 72(8), 1947–1970 (2010)MathSciNetCrossRefzbMATHGoogle Scholar
  47. 47.
    LeCun, Y., Cortes, C., Burges, C.J.C.: The MNIST database of handwritten digits (1998)Google Scholar

Copyright information

© Springer International Publishing AG 2017

Authors and Affiliations

  • William Poole
    • 1
  • Andrés Ortiz-Muñoz
    • 1
  • Abhishek Behera
    • 2
  • Nick S. Jones
    • 3
  • Thomas E. Ouldridge
    • 3
  • Erik Winfree
    • 1
  • Manoj Gopalkrishnan
    • 2
    Email author
  1. 1.California Institute of TechnologyPasadenaUSA
  2. 2.India Institute of Technology BombayMumbaiIndia
  3. 3.Imperial College LondonLondonUK

Personalised recommendations