Journal of Computer Science and Technology

, Volume 25, Issue 1, pp 154–168 | Cite as

Computational Cellular Dynamics Based on the Chemical Master Equation: A Challenge for Understanding Complexity



Modern molecular biology has always been a great source of inspiration for computational science. Half a century ago, the challenge from understanding macromolecular dynamics has led the way for computations to be part of the tool set to study molecular biology. Twenty-five years ago, the demand from genome science has inspired an entire generation of computer scientists with an interest in discrete mathematics to join the field that is now called bioinformatics. In this paper, we shall lay out a new mathematical theory for dynamics of biochemical reaction systems in a small volume (i.e., mesoscopic) in terms of a stochastic, discrete-state continuous-time formulation, called the chemical master equation (CME). Similar to the wavefunction in quantum mechanics, the dynamically changing probability landscape associated with the state space provides a fundamental characterization of the biochemical reaction system. The stochastic trajectories of the dynamics are best known through the simulations using the Gillespie algorithm. In contrast to the Metropolis algorithm, this Monte Carlo sampling technique does not follow a process with detailed balance. We shall show several examples how CMEs are used to model cellular biochemical systems. We shall also illustrate the computational challenges involved: multiscale phenomena, the interplay between stochasticity and nonlinearity, and how macroscopic determinism arises from mesoscopic dynamics. We point out recent advances in computing solutions to the CME, including exact solution of the steady state landscape and stochastic differential equations that offer alternatives to the Gilespie algorithm. We argue that the CME is an ideal system from which one can learn to understand “complex behavior” and complexity theory, and from which important biological insight can be gained.


biochemical networks cellular signaling epigenetics master equation nonlinear reactions stochastic modeling 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. [1]
    Kurtz T G. The relationship between stochastic and deterministic models for chemical reactions. J. Chem. Phys., 1972, 57(7): 2976–2978.CrossRefGoogle Scholar
  2. [2]
    Beard D A, Qian H. Chemical Biophysics: Quantitative Analysis of Cellular Systems. London: Cambridge Univ. Press, 2008.MATHGoogle Scholar
  3. [3]
    Wilkinson D J. Stochastic Modeling for Systems Biology. New York: Chapman & Hall/CRC, 2006.Google Scholar
  4. [4]
    Schlögl F. Chemical reaction models for non-equilibrium phase transition. Z. Physik., 1972, 253(2): 147–161.CrossRefGoogle Scholar
  5. [5]
    Murray J D. Mathematical Biology: An Introduction. 3rd Ed., New York: Springer, 2002.Google Scholar
  6. [6]
    Qian H, Saffarian S, Elson E L. Concentration fluctuations in a mesoscopic oscillating chemical reaction system. Proc. Natl. Acad. Sci. USA, 2002, 99(16): 10376–10381.MATHCrossRefMathSciNetGoogle Scholar
  7. [7]
    Taylor H M, Karlin S K. An Introduction to Stochastic Modeling. 3rd Ed., New York: Academic Press, 1998.MATHGoogle Scholar
  8. [8]
    Resat H, Wiley H S, Dixon D A. Probability-weighted dynamic Monte Carlo method for reaction kinetics simulations. J. Phys. Chem. B, 2001, 105(44): 11026–11034.CrossRefGoogle Scholar
  9. [9]
    Gardiner C W. Handbook of Stochastic Methods for Physics, Chemistry and the Natural Sciences. 3rd Ed., New York: Springer, 2004.MATHGoogle Scholar
  10. [10]
    van Kampen N G. Stochastic Processes in Physics and Chemistry. 3rd Ed., Amsterdam: Elsevier Science, 2007.Google Scholar
  11. [11]
    Vellela M, Qian H. Stochastic dynamics and nonequilibrium thermodynamics of a bistable chemical system: The Schlögl model revisited. J. R. Soc. Interf., 2009, 6(39): 925–940.Google Scholar
  12. [12]
    Qian H, Qian M, Tang X. Thermodynamics of the general diffusion process: Time-reversibility and entropy production. J. Stat. Phys., 2002, 107(5/6): 1129–1141.MATHCrossRefMathSciNetGoogle Scholar
  13. [13]
    Schrödinger E. What Is Life? The Physical Aspect of the Living Cell. New York: Cambridge Univ. Press, 1944.Google Scholar
  14. [14]
    Nicolis G, Prigogine I. Self-Organization in Nonequilibrium Systems. New York: Wiley-Interscience, 1977.MATHGoogle Scholar
  15. [15]
    Hänggi P, Grabert H, Talkner P, Thomas H. Bistable systems: Master equation versus Fokker-Planck modeling. Phys. Rev. A., 1984, 29(1): 371–378.CrossRefMathSciNetGoogle Scholar
  16. [16]
    Baras F, Mansour M M, Pearson J E. Microscopic simulation of chemical bistability in homogeneous systems. J. Chem. Phys. 1996, 105(18): 8257–8261.CrossRefGoogle Scholar
  17. [17]
    Vellela M, Qian H. A quasistationary analysis of a stochastic chemical reaction: Keizer’s paradox. Bull. Math. Biol., 2007, 69(5): 1727–1746.MATHCrossRefMathSciNetGoogle Scholar
  18. [18]
    Keizer J. Statistical Thermodynamics of Nonequilibrium Processes. New York: Springer-Verlag, 1987.Google Scholar
  19. [19]
    Bishop L, Qian H. Stochastic bistability and bifurcation in a mesoscopic signaling system with autocatalytic kinase. Biophys. J., 2010. (in the press)Google Scholar
  20. [20]
    Kussell E, Kishony R, Balaban N Q, Leibler S. Bacterial persistence: A model of survival in changing environments. Genetics, 2005, 169(4): 1804–1807.CrossRefGoogle Scholar
  21. [21]
    Turner B M. Histone acetylation and an epigenetic code. Bioessays, 2000, 22(9): 836–845.CrossRefGoogle Scholar
  22. [22]
    Jones P A, Takai D. The role of DNA methylation in mammalian epigenetics. Science, 2001, 293(5532): 1068–1070.CrossRefGoogle Scholar
  23. [23]
    Dodd I B, Micheelsen M A, Sneppen K, Thon G. Theoretical analysis of epigenetic cell memory by nucleosome modification. Cell, 2007, 129(4): 813–822.CrossRefGoogle Scholar
  24. [24]
    Zhu X M, Yin L, Hood L, Ao P. Robustness, stability and efficiency of phage λ genetic switch: Dynamical structure analysis. J. Bioinf. Compt. Biol., 2004, 2(4): 785–817.CrossRefGoogle Scholar
  25. [25]
    Ptashne M. On the use of the word “epigenetic”. Curr. Biol., 2007, 17(7): R233–R236.CrossRefGoogle Scholar
  26. [26]
    Mino H, Rubinstein J T, White J A. Comparison of algorithms for the simulation of action potentials with stochastic sodium channels. Ann. Biomed. Eng., 2002, 30(4): 578–587.CrossRefGoogle Scholar
  27. [27]
    Fox R F. Stochastic versions of the Hodgkin-Huxley equations. Biophys. J., 1997, 72(5): 2069–2074.CrossRefGoogle Scholar
  28. [28]
    Lamb H. Hydrodynamic. New York: Dover, 1945.Google Scholar
  29. [29]
    Morton-Firth C J, Bray D. Predicting temporal fluctuations in an intracellular signalling pathway J. Theoret. Biol., 1998, 192(1): 117–128.CrossRefGoogle Scholar
  30. [30]
    Elf J, Ehrenberg M. Fast evaluation of fluctuations in biochemical networks with the linear noise approximation. Genome Res., 2003, 13(11): 2475–2484.CrossRefGoogle Scholar
  31. [31]
    Vellela M, Qian H. On Poincaré-Hill cycle map of rotational random walk: Locating stochastic limit cycle in reversible Schnakenberg model. Proc. Roy. Soc. A: Math. Phys. Engr. Sci., 2009. (in the press)Google Scholar
  32. [32]
    Dill K A, Bromberg S, Yue K, Fiebig K M, Yee D P, Thomas P D, Chan H S. Principles of protein-folding — A perspective from simple exact models. Prot. Sci., 1995, 4(4): 561–602.CrossRefGoogle Scholar
  33. [33]
    Šali A, Shakhnovich E I, Karplus M. How does a protein fold? Nature, 1994, 369(6477): 248–251.CrossRefGoogle Scholar
  34. [34]
    Socci N D, Onuchic J N. Folding kinetics of protein like heteropolymer. J. Chem. Phys., 1994, 101: 1519–1528.CrossRefGoogle Scholar
  35. [35]
    Shrivastava I, Vishveshwara S, Cieplak M, Maritan A, Banavar J R. Lattice model for rapidly folding protein-like heteropolymers. Proc. Natl. Acad. Sci. U.S.A, 1995, 92(20): 9206–9209.CrossRefGoogle Scholar
  36. [36]
    Klimov D K, Thirumalai D. Criterion that determines the foldability of proteins. Phys. Rev. Lett., 1996, 76(21): 4070–4073.CrossRefGoogle Scholar
  37. [37]
    Cieplak M, Henkel M, Karbowski J, Banavar J R. Master equation approach to protein folding and kinetic traps. Phys. Rev. Lett., 1998, 80(16): 3654–3657.CrossRefGoogle Scholar
  38. [38]
    Mélin R, Li H, Wingreen N, Tang C. Designability, thermodynamic stability, and dynamics in protein folding: A lattice model study. J. Chem. Phys., 1999, 110(2): 1252–1262.CrossRefGoogle Scholar
  39. [39]
    Ozkan S B, Bahar I, Dill K A. Transition states and the meaning of ϕ-values in protein folding kinetics. Nature Struct. Biol., 2001, 8(9): 765–769.CrossRefGoogle Scholar
  40. [40]
    Kachalo S, Lu H, Liang J. Protein folding dynamics via quantification of kinematic energy landscape. Phys. Rev. Lett., 2006, 96(5): 058106.CrossRefGoogle Scholar
  41. [41]
    Chan H S, Dill K A. Compact polymers. Macromolecules, 1989, 22(12): 4559–4573.CrossRefGoogle Scholar
  42. [42]
    Chan H S, Dill K A. The effects of internal constraints on the configurations of chain molecules. J. Chem. Phys., 1990, 92(5): 3118–3135.CrossRefGoogle Scholar
  43. [43]
    Liang J, Zhang J, Chen R. Statistical geometry of packing defects of lattice chain polymer from enumeration and sequential Monte Carlo method. J. Chem. Phys., 2002, 117(7): 3511–3521.CrossRefGoogle Scholar
  44. [44]
    Zhang J, Chen Y, Chen R, Liang J. Importance of chirality and reduced flexibility of protein side chains: A study with square and tetrahedral lattice models. J. Chem. Phys., 2004, 121(1): 592–603.CrossRefGoogle Scholar
  45. [45]
    Williams P D, Pollock D D, Goldstein R A. Evolution of functionality in lattice proteins. J. Mole. Graph. Modelling, 2001, 19(1): 150–156.CrossRefGoogle Scholar
  46. [46]
    Bloom J D, Wilke C O, Arnold F H, Adami C. Stability and the evolvability of function in a model protein. Biophys. J., 2004, 86(5): 2758–2764.CrossRefGoogle Scholar
  47. [47]
    Lu H M, Liang J. A model study of protein nascent chain and cotranslational folding using hydrophobic-polar residues. Prot. Struct. Funct. Bioinf., 2008, 70(2): 442–449.CrossRefGoogle Scholar
  48. [48]
    Cao Y, Liang J. Optimal enumeration of state space of finitely buffered stochastic molecular networks and exact computation of steady state landscape probability. BMC Syst. Biol., 2008, 2: 30.CrossRefGoogle Scholar
  49. [49]
    Barrett R, Berry M, Chan T F, Demmel J, Donato J, Dongarra J, Eijkhout V, Pozo R, Romine C, van der Vorst H. Templates for the Solution of Linear Systems: Building Blocks for Iterative Methods. 2nd Ed., Philadelphia, PA: SIAM, 1994.Google Scholar
  50. [50]
    Lehoucq R, Sorensen D, Yang C. Arpack Users’ Guide: Solution of Large Scale Eigenvalue Problems with Implicitly Restarted Arnoldi Methods. Philadelphia, PA: SIAM, 1998.Google Scholar
  51. [51]
    Cao Y, Lu H M, Liang J. Stochastic probability landscape model for switching efficiency, robustness, and differential threshold for induction of genetic circuit in phage λ. In Proc. the 30th Annual Int. Conf. IEEE Eng. Med. Biol. Soc., Vancouver, Canada, Aug. 20–24, 2008, pp.611–614.Google Scholar
  52. [52]
    Gardner T S, Canter C R, Collins J J. Construction of a genetic toggle switch in Escherichia coli. Nature, 2000, 403(6767): 339–342.CrossRefGoogle Scholar
  53. [53]
    Kepler T B, Elston T C. Stochasticity in transcriptional regulation: Origins, consequences, and mathematical representations. Biophys. J., 2001, 81(6): 3116–3136.CrossRefGoogle Scholar
  54. [54]
    Schultz D, Onuchic J N, Wolynes P G. Understanding stochastic simulations of the smallest genetic networks. J. Chem. Phys., 2007, 126(24): 245102.CrossRefGoogle Scholar
  55. [55]
    Kim K Y, Wang, J. Potential energy landscape and robustness of a gene regulatory network: Toggle Switch. PLoS Comput. Biol., 2007, 3(3): e60.CrossRefMathSciNetGoogle Scholar
  56. [56]
    Wang J, Xu L, Wang E. Potential landscape and flux framework of nonequilibrium networks: Robustness, dissipation, and coherence of biochemical oscillations. Proc. Natl. Acad. Sci. U.S.A., 2008, 105(34): 12271–12276.CrossRefGoogle Scholar
  57. [57]
    Ptashne M. Genetic Switch: Phage Lambda Revisited. New York: Cold Spring Harbor Laboratory Press, 2004.Google Scholar
  58. [58]
    Arkin A, Ross J, McAdams H H. Stochastic kinetic analysis of developmental pathway bifurcation in phage λ-infected Escherichia coli cells. Genetics, 1998, 149(44): 1633–1648.Google Scholar
  59. [59]
    Aurell E, Brown S, Johanson J, Sneppen K. Stability puzzles in phage λ. Phys. Rev. E., 2002, 65(5): 051914.CrossRefGoogle Scholar
  60. [60]
    Munsky B, Khammash M. The finite state projection algorithm for the solution of the chemical master equation. J. Chem. Phys., 2006, 124(4): 044104.CrossRefGoogle Scholar
  61. [61]
    Munsky B, Khammash M. A multiple time interval finite state projection algorithm for the solution to the chemical master equation. J. Comput. Phys., 2007, 226(1): 818–835.MATHCrossRefMathSciNetGoogle Scholar
  62. [62]
    Macnamara S, Bersani A M, Burrage K, Sidje R B. Stochastic chemical kinetics and the total quasi-steady-state assumption: Application to the stochastic simulation algorithm and chemical master equation. J. Chem. Phys., 2008, 129(9): 095105.CrossRefGoogle Scholar
  63. [63]
    Datta B N. Numerical Linear Algebra and Applications. Brooks/Cole Pub. Co., 1995.Google Scholar
  64. [64]
    Golub G H, van Loan C F. Matrix Computations. Johns Hopkins Univ. Press, 1996.Google Scholar
  65. [65]
    Sidje R B. Expokit: A software package for computing matrix exponentials. ACM Trans. Math. Softw., 1998, 24(1): 130–156.MATHCrossRefGoogle Scholar
  66. [66]
    Lu H M, Liang J. Perturbation-based Markovian transmission model for probing allosteric dynamics of large macromolecular assembling: A study of GroEL-GroES. PLoS Comput. Biol., 2009, 5(10): e1000526.CrossRefGoogle Scholar
  67. [67]
    Cao Y, Gillespie D T, Petzold L R. The slow-scale stochastic simulation algorithm. J. Chem. Phys., 2005, 122(1): 14116.CrossRefGoogle Scholar
  68. [68]
    Cao Y, Liang J. Nonlinear coupling for improved stochastic network model: A study of Schnakenberg model. In Proc. the 3rd Symp. Optimiz. Syst. Biol., Zhangjiajie, China, Sept. 20–22, 2009, pp.379–386.Google Scholar
  69. [69]
    Schnakenberg J. Simple chemical reaction systems with limit cycle behaviour. J. Theoret. Biol., 1979, 81(3): 389–400.CrossRefMathSciNetGoogle Scholar
  70. [70]
    Qian H. Open-system nonequilibrium steady state: Statistical thermodynamics, fluctuations, and chemical oscillations. J. Phys. Chem. B, 2006, 110(31): 15063–15074.CrossRefGoogle Scholar
  71. [71]
    Goutsias J. Classical versus stochastic kinetics modeling of biochemical reaction systems. Biophys. J., 2007, 92(7): 2350–2365.CrossRefGoogle Scholar
  72. [72]
    Uribe C A, Verghese G C. Mass fluctuation kinetics: Capturing stochastic effects in systems of chemical reactions through coupled mean-variance computations. J. Chem. Phys., 2007, 126(2): 024109.CrossRefGoogle Scholar
  73. [73]
    Keizer J. On the macroscopi equivalence of descriptions of fluctuations for chemical reactions. J. Math. Phys., 1977, 18: 1316–1321.CrossRefGoogle Scholar
  74. [74]
    Mitchell M. Complexity: A Guided Tour. London: Oxford Univ. Press, 2009.Google Scholar
  75. [75]
    Laughlin R B, Pines D, Schmalian J, Stojković B P, Wolynes P G. The middle way. Proc. Natl. Acad. Sci. USA, 2000, 97(1): 32–37.CrossRefGoogle Scholar
  76. [76]
    Qian H, Shi P Z, Xing J. Stochastic bifurcation, slow fluctuations, and bistability as an origin of biochemical complexity. Physical Chemistry Chemical Physics, 2009, 11(24): 4861–4870.CrossRefGoogle Scholar

Copyright information

© Springer 2010

Authors and Affiliations

  1. 1.Department of BioengineeringUniversity of Illinois at ChicagoChicagoU.S.A.
  2. 2.Shanghai Center for Systems BiomedicineShanghai Jiao Tong UniversityShanghaiChina
  3. 3.Department of Applied MathematicsUniversity of WashingtonSeattleU.S.A.
  4. 4.Kavli Institute for Theoretical Physics ChinaChinese Academy of SciencesBeijingChina

Personalised recommendations