Hyperparameter optimization for image analysis: application to prostate tissue images and live cell data of virus-infected cells

  • Christian RitterEmail author
  • Thomas Wollmann
  • Patrick Bernhard
  • Manuel Gunkel
  • Delia M. Braun
  • Ji-Young Lee
  • Jan Meiners
  • Ronald Simon
  • Guido Sauter
  • Holger Erfle
  • Karsten Rippe
  • Ralf Bartenschlager
  • Karl Rohr
Original Article



Automated analysis of microscopy image data typically requires complex pipelines that involve multiple methods for different image analysis tasks. To achieve best results of the analysis pipelines, method-dependent hyperparameters need to be optimized. However, complex pipelines often suffer from the fact that calculation of the gradient of the loss function is analytically or computationally infeasible. Therefore, first- or higher-order optimization methods cannot be applied.


We developed a new framework for zero-order black-box hyperparameter optimization called HyperHyper, which has a modular architecture that separates hyperparameter sampling and optimization. We also developed a visualization of the loss function based on infimum projection to obtain further insights into the optimization problem.


We applied HyperHyper in three different experiments with different imaging modalities, and evaluated in total more than 400.000 hyperparameter combinations. HyperHyper was used for optimizing two pipelines for cell nuclei segmentation in prostate tissue microscopy images and two pipelines for detection of hepatitis C virus proteins in live cell microscopy data. We evaluated the impact of separating the sampling and optimization strategy using different optimizers and employed an infimum projection for visualizing the hyperparameter space.


The separation of sampling and optimization strategy of the proposed HyperHyper optimization framework improves the result of the investigated image analysis pipelines. Visualization of the loss function based on infimum projection enables gaining further insights on the optimization process.


Microscopy image analysis Hyperparameter optimization Optimization framework Visualization 



This work is funded by the Deutsche Forschungsgemeinschaft (DFG, German Research Foundation)—Projektnumber 240245660—SFB 1129 (projects P11, Z4) and the BMBF within the projects CancerTelSys (e:Med, #01ZX1602) and de.NBI (HD-HuB, #031A537C).

Compliance with ethical standards

Conflict of interest

The authors declare that they have no conflict of interest.

Ethical approval

All procedures performed in studies involving human participants were in accordance with the ethical standards of the institutional and/or national research committee and with the 1964 Helsinki Declaration and its later amendments or comparable ethical standards.

Informed consent

Informed consent was obtained from all individual participants included in the study.


  1. 1.
    Afgan E, Baker D, Batut B, van den Beek M, Bouvier D, Čech M, Chilton J, Clements D, Coraor N, Grüning BA, Guerler A, Hillman-Jackson J, Hiltemann S, Jalili V, Rasche H, Soranzo N, Goecks J, Taylor J, Nekrutenko A, Blankenberg D (2018) The Galaxy platform for accessible, reproducible and collaborative biomedical analyses: 2018 update. Nucleic Acids Res 46(1):537–544CrossRefGoogle Scholar
  2. 2.
    Chen T, Guestrin C (2016) Xgboost: a scalable tree boosting system. In: Proceedings of SIGKDD. ACM, pp 785–794Google Scholar
  3. 3.
    Chenouard N, Smal I, De Chaumont F, Maška M, Sbalzarini IF, Gong Y, Cardinale J, Carthel C, Coraluppi S, Winter M, Cohen AR, Godinez WJ, Rohr K, Kalaidzidis Y, Liang L, Duncan J, Shen H, Xu Y, Magnusson KE, Jaldén J, Blau HM, Paul-Gilloteaux P, Roudot P, Kervrann C, Waharte F, Tinevez JY, Shorte SL, Willemse J, Celler K, van Wezel GP, Dan HW, Tsai YS, Ortiz de Solórzano C, Olivo-Marin JC, Meijering E (2014) Objective comparison of particle tracking methods. Nat Methods 11(3):281–290CrossRefGoogle Scholar
  4. 4.
    Cleary K, Peters TM (2010) Image-guided interventions: technology review and clinical applications. Annu Rev Biomed Eng 12:119–142CrossRefGoogle Scholar
  5. 5.
    Cypko MA, Stoehr M, Kozniewski M, Druzdzel MJ, Dietz A, Berliner L, Lemke HU (2017) Validation workflow for a clinical Bayesian network model in multidisciplinary decision making in head and neck oncology treatment. Int J Comput Assist Radiol Surg 12(11):1959–1970CrossRefGoogle Scholar
  6. 6.
    Di Tommaso P, Chatzou M, Floden EW, Barja PP, Palumbo E, Notredame C (2017) Nextflow enables reproducible computational workflows. Nat Biotechnol 35(4):316–319CrossRefGoogle Scholar
  7. 7.
    Godinez WJ, Lampe M, Koch P, Eils R, Muller B, Rohr K (2012) Identifying virus-cell fusion in two-channel fluorescence microscopy image sequences based on a layered probabilistic approach. IEEE Trans Med Imaging 31(9):1786–1808CrossRefGoogle Scholar
  8. 8.
    Godinez WJ, Rohr K (2015) Tracking multiple particles in fluorescence time-lapse microscopy images via probabilistic data association. IEEE Trans Med Imaging 34(2):415–432CrossRefGoogle Scholar
  9. 9.
    Goldberg DE (1989) Genetic algorithms in search, optimization, and machine learning. Addison-Wesley, BostonGoogle Scholar
  10. 10.
    Golovin D, Solnik B, Moitra S, Kochanski G, Karro J, Sculley D (2017) Google vizier: a service for black-box optimization. In: Proceedings of SIGKDD. ACM, pp 1487–1495Google Scholar
  11. 11.
    Hertel L, Collado J, Sadowski P, Baldi P (2018) Sherpa: hyperparameter optimization for machine learning models. In: Proceedings of NIPS (submitted)Google Scholar
  12. 12.
    Hutter F, Hoos HH, Leyton-Brown K (2011) Sequential model-based optimization for general algorithm configuration. In: Proceedings of LION. Springer, pp 507–523Google Scholar
  13. 13.
    Kingma DP, Ba J (2014) Adam: a method for stochastic optimization. arXiv:1412.6980
  14. 14.
    Komer B, Bergstra J, Eliasmith C (2014) Hyperopt-sklearn: automatic hyperparameter configuration for scikit-learn. In: Proceedings of ICML workshop on AutoML, pp 2825–2830Google Scholar
  15. 15.
    Kuhn HW (2005) The Hungarian method for the assignment problem. NRL 2:7–21CrossRefGoogle Scholar
  16. 16.
    Milletari F, Navab N, Ahmadi SA (2016) V-net: fully convolutional neural networks for volumetric medical image segmentation. In: Proceedings of 3DV. IEEE, pp 565–571Google Scholar
  17. 17.
    Mitchell HB (2012) Data fusion: concepts and ideas. Springer, BerlinCrossRefGoogle Scholar
  18. 18.
    Parzen E (1962) On estimation of a probability density function and mode. Ann Math Stat 33(3):1065–1076CrossRefGoogle Scholar
  19. 19.
    Rahman SA, Koch P, Weichsel J, Godinez WJ, Schwarz U, Rohr K, Lamb DC, Kräusslich HG, Müller B (2014) Investigating the role of f-actin in human immunodeficiency virus assembly by live-cell microscopy. J Virol 88(14):7904–7914CrossRefGoogle Scholar
  20. 20.
    Ritter C, Imle A, Lee JY, Müller B, Fackler OT, Bartenschlager R, Rohr K (2018) Two-filter probabilistic data association for tracking of virus particles in fluorescence microscopy images. In: Proceedings of ISBI. IEEE, pp 957–960Google Scholar
  21. 21.
    Ronneberger O, Fischer P, Brox T (2015) U-Net: convolutional networks for biomedical image segmentation. In: Proceedings of MICCAI. Springer, pp 234–241Google Scholar
  22. 22.
    Sage D, Neumann FR, Hediger F, Gasser SM, Unser M (2005) Automatic tracking of individual fluorescence particles: application to the study of chromosome dynamics. IEEE Trans Image Process 14(9):1372–1383CrossRefGoogle Scholar
  23. 23.
    Snoek J, Larochelle H, Adams RP (2012) Practical Bayesian optimization of machine learning algorithms. In: Proceedings of the advances in neural information processing systems, pp 2951–2959Google Scholar
  24. 24.
    Svensson CM, Medyukhina A, Belyaev I, Al-Zaben N, Figge MT (2018) Untangling cell tracks: quantifying cell migration by time lapse image data analysis. Cytom Part A 93(3):357–370CrossRefGoogle Scholar
  25. 25.
    Tektonidis M, Rohr K (2017) Diffeomorphic multi-frame non-rigid registration of cell nuclei in 2D and 3D live cell images. IEEE Trans Image Process 26(3):1405–1417CrossRefGoogle Scholar
  26. 26.
    Thornton C, Hutter F, Hoos HH, Leyton-Brown K (2013) Auto-WEKA: combined selection and hyperparameter optimization of classification algorithms. In: Proceedigns of SIGKDD. ACM, pp 847–855Google Scholar
  27. 27.
    Ulman V, Maška M, Magnusson KE, Ronneberger O, Haubold C, Harder N, Matula P, Matula P, Svoboda D, Radojevic M, Smal I, Rohr K, Jaldén J, Blau HM, Dzyubachyk O, Lelieveldt B, Xiao P, Li Y, Cho SY, Dufour AC, Olivo-Marin JC, Reyes-Aldasoro CC, Solis-Lemus JA, Bensch R, Brox T, Stegmaier J, Mikut R, Wolf S, Hamprecht FA, Esteves T, Quelhas P, Demirel Ö, Malmström L, Jug F, Tomancak P, Meijering E, Muñoz-Barrutia A, Kozubek M, Ortiz-de Solorzano C (2017) An objective comparison of cell-tracking algorithms. Nat Methods 14(12):1141–1552CrossRefGoogle Scholar
  28. 28.
    Wang Y, Du S, Balakrishnan S, Singh A (2017) Stochastic zeroth-order optimization in high dimensions. arXiv:1710.10551
  29. 29.
    Wollmann T, Bernhard P, Gunkel M, Braun DM, Meiners J, Simon R, Sauter G, Erfle H, Rippe K, Rohr K (2019) Black-box hyperparameter optimization for nuclei segmentation in prostate tissue images. In: Proceedings of Bildverarbeitung für die Medizin. Springer, pp 345–350Google Scholar
  30. 30.
    Wollmann T, Erfle H, Eils R, Rohr K, Gunkel M (2017) Workflows for microscopy image analysis and cellular phenotyping. J Biotechnol 261:70–75CrossRefGoogle Scholar

Copyright information

© CARS 2019

Authors and Affiliations

  • Christian Ritter
    • 1
    Email author
  • Thomas Wollmann
    • 1
  • Patrick Bernhard
    • 1
  • Manuel Gunkel
    • 2
  • Delia M. Braun
    • 3
  • Ji-Young Lee
    • 5
    • 6
  • Jan Meiners
    • 4
  • Ronald Simon
    • 4
  • Guido Sauter
    • 4
  • Holger Erfle
    • 2
  • Karsten Rippe
    • 3
  • Ralf Bartenschlager
    • 5
    • 6
  • Karl Rohr
    • 1
  1. 1.Biomedical Computer Vision GroupBioQuant, IPMB, University of Heidelberg and DKFZHeidelbergGermany
  2. 2.High-Content Analysis of the Cell (HiCell) and Advanced Biological Screening FacilityBioQuant, University of HeidelbergHeidelbergGermany
  3. 3.Division of Chromatin NetworksDKFZ and BioQuantHeidelbergGermany
  4. 4.Department of PathologyUniversity Medical Center Hamburg-EppendorfHamburgGermany
  5. 5.Department of Infectious Diseases, Molecular VirologyUniversity Hospital HeidelbergHeidelbergGermany
  6. 6.German Center of Infection Research, Heidelberg Partner SiteHeidelbergGermany

Personalised recommendations