Improved Parameter Estimation in Kinetic Models: Selection and Tuning of Regularization Methods

  • Attila Gábor
  • Julio R. Banga
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 8859)


Kinetic models are being increasingly used as a systematic framework to understand function in biological systems. Calibration of these nonlinear dynamic models remains challenging due to the nonconvexity and ill-conditioning of the associated inverse problems. Nonconvexity can be dealt with suitable global optimization. Here, we focus on simultaneously dealing with ill-conditioning by making use of proper regularization methods. Regularized calibrations ensure the best trade-offs between bias and variance, thus reducing over-fitting. We present a critical comparison of several methods, and guidelines for properly tuning them. The performance of this procedure and its advantages are illustrated with a well known benchmark problem considering several scenarios of data availability and measurement noise.


Dynamic models parameter estimation Tikhonov regularization regularization tuning 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Almquist, J., Cvijovic, M., Hatzimanikatis, V., Nielsen, J., Jirstrand, M.: Kinetic models in industrial biotechnology - Improving cell factory performance. Metabolic Engineering 1–22 (April 2014)Google Scholar
  2. 2.
    Ashyraliyev, M., Fomekong-Nanfack, Y., Kaandorp, J.A., Blom, J.G.: Systems biology: parameter estimation for biochemical models. FEBS Journal 276(4), 886–902 (2009)CrossRefGoogle Scholar
  3. 3.
    Banga, J.R., Balsa-Canto, E.: Parameter estimation and optimal experimental design. Essays in Biochemistry 45, 195–210 (2008)CrossRefGoogle Scholar
  4. 4.
    Bansal, L., Chu, Y., Laird, C., Hahn, J.: Regularization of Inverse Problems to Determine Transcription Factor Profiles from Fluorescent Reporter Systems. AIChE Journal 58(12), 3751–3762 (2012)CrossRefGoogle Scholar
  5. 5.
    Bauer, F.: Some considerations concerning regularization and parameter choice algorithms. Inverse Problems 23(2), 837–858 (2007)MathSciNetCrossRefzbMATHGoogle Scholar
  6. 6.
    Bauer, F., Lukas, M.A.: Comparingparameter choice methods for regularization of ill-posed problems. Mathematics and Computers in Simulation 81(9), 1795–1841 (2011)MathSciNetCrossRefzbMATHGoogle Scholar
  7. 7.
    Dennis, J.E., Gay, D.M., Welsch, R.E.: An Adaptive Nonlinear Least-Squares Algorithm. ACM Transaction on Mathematical Software 7(3), 348–368 (1981)CrossRefzbMATHGoogle Scholar
  8. 8.
    Engl, H.W., Flamm, C., Kügler, P., Lu, J., Müller, S., Schuster, P., Philipp, K.: Inverse problems in systems biology. Inverse Problems 25(12), 123014 (2009)CrossRefGoogle Scholar
  9. 9.
    Engl, H.W., Hanke, M., Neubauer, A.: Regularization of Inverse Problems. Kluwer Academic Publishers (1996)Google Scholar
  10. 10.
    Hansen, P.C., O’Leary, D.P.: The use of the L-Curve in the regularization of discrete ill-posed problems. SIAM Journal on Scientific Computing 14(6), 1487–1503 (1993)MathSciNetCrossRefzbMATHGoogle Scholar
  11. 11.
    Hindmarsh, A.C., Brown, P.N., Grant, K.E., Lee, S.L., Serban, R., Shumaker, D.E., Woodward, C.S.: {SUNDIALS: Suite of Nonlinear and Differential / Algebraic Equation Solvers}. ACM Transaction on Mathematical Software 31(3), 363–396 (2005)MathSciNetCrossRefzbMATHGoogle Scholar
  12. 12.
    Kaltenbacher, B., Neubauer, A., Scherzer, O.: Iterative Regularization Methods for Nonlinear Ill-Posed Problems. Radon Series on Computational and Applied Mathematics. Walter de Gruyter, Berlin, New York (2008)CrossRefzbMATHGoogle Scholar
  13. 13.
    Kitano, H. (ed.): Foundations of Systems Biology. The MIT Press (2001)Google Scholar
  14. 14.
    Kravaris, C., Hahn, J., Chu, Y.: Advances and selected recent developments in state and parameter estimation. Computers & Chemical Engineering 51, 111–123 (2013), CrossRefGoogle Scholar
  15. 15.
    Lepskii, O.: On a Problem of Adaptive Estimation in Gaussian White Noise. Theory of Probability & Its Applications 35(3), 454–466 (1991)MathSciNetCrossRefGoogle Scholar
  16. 16.
    Link, H., Christodoulou, D., Sauer, U.: Advancing metabolic models with kinetic information. Current Opinion in Biotechnology 29, 8–14 (2014)CrossRefGoogle Scholar
  17. 17.
    Ljung, L., Chen, T.: What can regularization offer for estimation of dynamical systems? In: 11th IFAC International Workshop on Adaptation and Learning in Control and Signal Processing. IFAC, vol. 5 (2013)Google Scholar
  18. 18.
    Miró, A., Pozo, C., Guillén-Gosálbez, G., Egea, J.A., Jiménez, L.: Deterministic global optimization algorithm based on outer approximation for the parameter estimation of nonlinear dynamic biological systems. BMC Bioinformatics 13(1), 90 (2012)CrossRefGoogle Scholar
  19. 19.
    Moles, C.G., Mendes, P., Banga, J.R.: Parameter Estimation in Biochemical Pathways: A Comparison of Global Optimization Methods. Genome Research 13, 2467–2474 (2003)CrossRefGoogle Scholar
  20. 20.
    Morozov, V.A.: Methods for Solving Incorrectly Posed Problems. Springer (1984)Google Scholar
  21. 21.
    Palm, R.: Numerical Comparison of Regularization Algorithms for Solving Ill-Posed Problems. Ph.D. thesis, University of Tartu, EstoniaGoogle Scholar
  22. 22.
    Papamichail, I., Adjiman, C.S.: Global optimization of dynamic systems. Comput. Chem. Eng. 28, 403–415 (2004)CrossRefGoogle Scholar
  23. 23.
    Regiska, T.: A Regularization Parameter in Discrete Ill-Posed Problems. SIAM Journal on Scientific Computing 17(3), 740–749 (1996)MathSciNetCrossRefGoogle Scholar
  24. 24.
    Reichel, L., Rodriguez, G.: Old and new parameter choice rules for discrete ill-posed problems. Numerical Algorithms 63(1), 65–87 (2012)MathSciNetCrossRefGoogle Scholar
  25. 25.
    Rodriguez-Fernandez, M., Egea, J.A., Banga, J.R.: Novel metaheuristic for parameter estimation in nonlinear dynamic biological systems. BMC Bioinformatics 7, 483 (2006)CrossRefGoogle Scholar
  26. 26.
    Rodriguez-Fernandez, M., Mendes, P., Banga, J.R.: A hybrid approach for efficient and robust parameter estimation in biochemical pathways. Bio Systems 83(2-3), 248–265 (2006)CrossRefGoogle Scholar
  27. 27.
    Schmidt, M., Fung, G., Rosaless, R.: Optimization Methods for l1-Regularization. Tech. rep. (2009),
  28. 28.
    Stanford, N.J., Lubitz, T., Smallbone, K., Klipp, E., Mendes, P., Liebermeister, W.: Systematic construction of kinetic models from genome-scale metabolic networks. PloS one 8(11), e79195 (2012)Google Scholar
  29. 29.
    Tibshirani, R.: Regression Shrinkage and Selection via the Lasso. Journal of the Royal Statistical Society: Series B (Statistical Methodology) 58(1), 267–288 (1996)MathSciNetzbMATHGoogle Scholar
  30. 30.
    Walter, E., Prorizato, L.: Identification of Parametric Models from experimental data. Springer (1997)Google Scholar
  31. 31.
    Wang, H., Wang, X.C.: Parameter estimation for metabolic networks with two stage Bregman regularization homotopy inversion algorithm. Journal of Theoretical Biology 343, 199–207 (2014)CrossRefGoogle Scholar
  32. 32.
    Zou, H., Hastie, T.: Regularization and variable selection via the elastic net. Journal of the Royal Statistical Society: Series B (Statistical Methodology) 67(2), 301–320 (2005), MathSciNetCrossRefzbMATHGoogle Scholar

Copyright information

© Springer International Publishing Switzerland 2014

Authors and Affiliations

  • Attila Gábor
    • 1
  • Julio R. Banga
    • 1
  1. 1.BioProcess Engineering GroupIIM-CSICVigoSpain

Personalised recommendations