Skip to main content
SpringerLink
Log in

Inter-laboratory study of human in vitro toxicogenomics-based tests as alternative methods for evaluating chemical carcinogenicity: a bioinformatics perspective

Archives of Toxicology volume 90, pages 2215–2229 (2016)Cite this article

Abstract

The assessment of the carcinogenic potential of chemicals with alternative, human-based in vitro systems has become a major goal of toxicogenomics. The central read-out of these assays is the transcriptome, and while many studies exist that explored the gene expression responses of such systems, reports on robustness and reproducibility, when testing them independently in different laboratories, are still uncommon. Furthermore, there is limited knowledge about variability induced by the data analysis protocols. We have conducted an inter-laboratory study for testing chemical carcinogenicity evaluating two human in vitro assays: hepatoma-derived cells and hTERT-immortalized renal proximal tubule epithelial cells, representing liver and kidney as major target organs. Cellular systems were initially challenged with thirty compounds, genome-wide gene expression was measured with microarrays, and hazard classifiers were built from this training set. Subsequently, each system was independently established in three different laboratories, and gene expression measurements were conducted using anonymized compounds. Data analysis was performed independently by two separate groups applying different protocols for the assessment of inter-laboratory reproducibility and for the prediction of carcinogenic hazard. As a result, both workflows came to very similar conclusions with respect to (1) identification of experimental outliers, (2) overall assessment of robustness and inter-laboratory reproducibility and (3) re-classification of the unknown compounds to the respective toxicity classes. In summary, the developed bioinformatics workflows deliver accurate measures for inter-laboratory comparison studies, and the study can be used as guidance for validation of future carcinogenicity assays in order to implement testing of human in vitro alternatives to animal testing.

This is a preview of subscription content, access via your institution.

Access options

Buy single article

Instant access to the full article PDF.

USD 39.95

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

Similar content being viewed by others

References

  • Annys E, Billington R, Clayton R, Bremm KD, Graziano M, McKelvie J, Ragan I, Schwarz M, van der Laan JW, Wood C, Öberg M, Wester P, Woodward KN (2014) Advancing the 3Rs in regulatory toxicology—carcinogenicity testing: scope for harmonization and advancing the 3Rs in regulated sectors of the European Union. Regul Toxicol Pharmacol 69:234–242

    Article  Google Scholar 

  • Aschauer L, Gruber LN, Pfaller W, Limonciel A, Athersuch TJ, Cavill R, Khan A, Gstraunthaler G, Grillari J, Grillari R, Hewitt P, Leonard MO, Wilmes A, Jennings P (2013) Delineation of the key aspects in the regulation of epithelial monolayer formation. Mol Cell Biol 33:2535–2550

    Article  CAS  Google Scholar 

  • Ashburner M, Ball CA, Blake JA, Botstein D, Butler H, Cherry JM, Davis AP, Dolinski K, Dwight SS, Eppig JT, Harris MA, Hill DP, Issel-Tarver L, Kasarskis A, Lewis S, Matese JC, Richardson JE, Ringwald M, Rubin GM, Sherlock G (2000) Gene ontology: tool for the unification of biology. The Gene Ontology Consortium. Nat Genet 25:25–29

    Article  CAS  Google Scholar 

  • Caiment F, Tsamou M, Jennen D, Kleinjans J (2014) Assessing compound carcinogenicity in vitro connectivity mapping. Carcinogenesis 35:201–207

    Article  CAS  Google Scholar 

  • Cristianini N, Shawe-Taylor J (2000) An introduction to support vector machines and other kernel-based learning methods. Cambridge University Press, Cambridge

    Book  Google Scholar 

  • Dai M, Wang P, Boyd AD, Kostov G, Athey B, Jones EG, Bunney WE, Myers RM, Speed TP, Akil H, Watson SJ, Meng F (2005) Evolving gene/transcript definitions significantly alter the interpretation of GeneChip data. Nucleic Acids Res 33:e175

    Article  Google Scholar 

  • Doktorova TY, Yildirimman R, Vinken M, Vilardell M, Vanhaecke T, Gmuender H, Bort R, Brolen G, Holmgren G, Li R, Chesne C, van Delft J, Kleinjans J, Castell J, Bjorquist P, Herwig R, Rogiers V (2013) Transcriptomic responses generated by hepatocarcinogens in a battery of liver-based in vitro models. Carcinogenesis 34:1393–1402

    Article  CAS  Google Scholar 

  • Doktorova T, Yildirimman R, Ceelen L, Vilardell M, Vanhaecke T, Vinken M, Ates G, Heymans A, Gmuender H, Bort R, Corvi R, Phrakonkham P, Li R, Mouchet N, Chesne C, van Delft J, Kleinjans J, Castell J, Herwig R, Rogiers V (2014) Testing chemical carcinogenicity by using a transcriptomics HepaRG-based model. Exp Clin Sci J 13:623–637

    CAS  Google Scholar 

  • Ernst J, Bar-Joseph Z (2006) STEM: a tool for the analysis of short time series gene expression data. BMC Bioinform 7:191

    Article  Google Scholar 

  • Fan X, Shao L, Fang H, Tong W, Cheng Y (2011) Cross-platform comparison of microarray-based multiple-class prediction. PLoS ONE 6:e16067

    Article  CAS  Google Scholar 

  • Fowlkes EB, Mallows CL (1983) A method for comparing two hierarchical clusterings. J Am Stat Assoc 78:553–569

    Article  Google Scholar 

  • Ganter B et al (2005) Development of a large-scale chemogenomics database to improve drug candidate selection and to understand mechanisms of chemical toxicity and action. J Biotechnol 119:219–244

    Article  CAS  Google Scholar 

  • Gómez-Lechón MJ, Castell JV, Donato MT (2010) The use of hepatocytes to investigate drug toxicity. Methods Mol Biol 640:389–415

    Article  Google Scholar 

  • Gusenleitner D, Auerbach SS, Melia T, Gómez HF, Sherr DH, Monti S (2014) Genomic models of short-term exposure accurately predict long-term chemical carcinogenicity and identify putative mechanisms of action. PLoS ONE 9:e102579

    Article  Google Scholar 

  • Guyon I, Weston J, Barnhill S, Vapnik V (2002) Gene selection for cancer classification using support vector machines. Mach Learn 46:389–422

    Article  Google Scholar 

  • Hartung T, Bremer S, Casati S, Coecke S, Corvi R, Fortaner S, Gribaldo L, Halder M, Hoffmann S, Roi AJ, Prieto P, Sabbioni E, Scott L, Worth A, Zuang V (2004) A modular approach to the ECVAM principles on test validity. Altern Lab Anim 32:467–472

    CAS  PubMed  Google Scholar 

  • Ioannidis JP, Khoury MJ (2011) Improving validation practices in “omics” research. Science 334:1230–1232

    Article  CAS  Google Scholar 

  • Irizarry RA, Warren D, Spencer F, Kim IF, Biswal S, Frank BC, Gabrielson E, Garcia JG, Geoghegan J, Germino G, Griffin C, Hilmer SC, Hoffman E, Jedlicka AE, Kawasaki E, Martínez-Murillo F, Morsberger L, Lee H, Petersen D, Quackenbush J, Scott A, Wilson M, Yang Y, Ye SQ, Yu W (2005) Multiple-laboratory comparison of microarray platforms. Nat Methods 2:345–350

    Article  CAS  Google Scholar 

  • Jennen DG, Magkoufopoulou C, Ketelslegers HB, van Herwijnen MH, Kleinjans JC, van Delft JH (2010) Comparison of HepG2 and HepaRG by whole-genome gene expression analysis for the purpose of chemical hazard identification. Toxicol Sci 115:66–79

    Article  CAS  Google Scholar 

  • Jennings P, Aydin S, Bennett J, McBride R, Weiland C, Tuite N, Gruber LN, Perco P, Gaora PO, Ellinger-Ziegelbauer H, Ahr HJ, Kooten CV, Daha MR, Prieto P, Ryan MP, Pfaller W, McMorrow T (2009) Inter-laboratory comparison of human renal proximal tubule (HK-2) transcriptome alterations due to Cyclosporine A exposure and medium exhaustion. Toxicol In Vitro 23:486–499

    Article  CAS  Google Scholar 

  • Jennings P, Weiland C, Limonciel A, Bloch KM, Radford R, Aschauer L, McMorrow T, Wilmes A, Pfaller W, Ahr HJ, Slattery C, Lock EA, Ryan MP, Ellinger-Ziegelbauer H (2012) Transcriptomic alterations induced by Ochratoxin A in rat and human renal proximal tubular in vitro models and comparison to a rat in vivo model. Arch Toxicol 86:571–589

    Article  CAS  Google Scholar 

  • Kiyosawa N, Ando Y, Watanabe K, Niino N, Manabe S, Yamoto T (2009) Scoring multiple toxicological endpoints using a toxicogenomics database. Toxicol Lett 188:91–97

    Article  CAS  Google Scholar 

  • Lamb J, Crawford ED, Peck D, Modell JW, Blat IC, Wrobel MJ, Lerner J, Brunet JP, Subramanian A, Ross KN, Reich M, Hieronymus H, Wei G, Armstrong SA, Haggarty SJ, Clemons PA, Wei R, Carr SA, Lander ES, Golub TR (2006) The connectivity map: using gene-expression signatures to connect small molecules, genes and diseases. Science 313:1929–1935

    Article  CAS  Google Scholar 

  • Limonciel A, Aschauer L, Wilmes A, Prajczer S, Leonard MO, Pfaller W, Jennings P (2011) Lactate is an ideal non-invasive marker for evaluating temporal alterations in cell stress and toxicity in repeat dose testing regimes. Toxicol In Vitro 25:1855–1862

    Article  CAS  Google Scholar 

  • Limonciel A, Wilmes A, Aschauer L, Radford R, Bloch KM, McMorrow T, Pfaller W, van Delft JH, Slattery C, Ryan MP, Lock EA, Jennings P (2012) Oxidative stress induced by potassium bromate exposure results in altered tight junction protein expression in renal proximal tubule cells. Arch Toxicol 86:1741–1751

    Article  CAS  Google Scholar 

  • MAQC Consortium (2006) The microarray quality control (MAQC) project shows inter- and intraplatform reproducibility of gene expression measurements. Nat Biotechnol 24:1151–1161

    Article  Google Scholar 

  • MAQC Consortium (2010) The microarray quality control (MAQC)-II study of common practices for the development and validation of microarray-based predictive models. Nat Biotechnol 28:827–838

    Article  Google Scholar 

  • Paules RS, Aubrecht J, Corvi R, Garthoff B, Kleinjans JC (2011) Moving forward in human cancer risk assessment. Environ Health Perspect 119:739–743

    Article  Google Scholar 

  • Radford R, Slattery C, Jennings P, Blacque O, Pfaller W, Gmuender H, Van Delft J, Ryan MP, McMorrow T (2012) Carcinogens induce loss of the primary cilium in human renal proximal tubular epithelial cells independently of effects on the cell cycle. Am J Physiol Renal Physiol 302:F905–F916

    Article  CAS  Google Scholar 

  • Radford R, Frain H, Ryan MP, Slattery C, McMorrow T (2013) Mechanisms of chemical carcinogenesis in the kidneys. Int J Mol Sci 14:19416–19433

    Article  CAS  Google Scholar 

  • Ritchie ME, Phipson B, Wu D, Hu Y, Law CW, Shi W, Smyth GK (2015) Limma powers differential expression analyses for RNA-sequencing and microarray studies. Nucleic Acids Res 43:e47

    Article  Google Scholar 

  • Silva Lima B, van der Laan JW (2000) Mechanisms of nongenotoxic carcinogenesis and assessment of the human hazard. Regul Toxicol Pharmacol 32:135–143

    Article  CAS  Google Scholar 

  • Vinken M, Doktorova T, Ellinger-Ziegelbauer H, Ahr HJ, Lock E, Carmichael P, Roggen E, van Delft J, Kleinjans J, Castell J, Bort R, Donato T, Ryan M, Corvi R, Keun H, Ebbels T, Athersuch T, Sansone SA, Rocca-Serra P, Stierum R, Jennings P, Pfaller W, Gmuender H, Vanhaecke T, Rogiers V (2008) The carcinoGENOMICS project: critical selection of model compounds for the development of omics-based in vitro carcinogenicity screening assays. Mutat Res 659:202–210

    Article  CAS  Google Scholar 

  • Wang C, Gong B, Bushel PR, Thierry-Mieg J, Thierry-Mieg D, Xu J, Fang H, Hong H, Shen J, Su Z, Meehan J, Li X, Yang L, Li H, Łabaj PP, Kreil DP, Megherbi D, Gaj S, Caiment F, van Delft J, Kleinjans J, Scherer A, Devanarayan V, Wang J, Yang Y, Qian HR, Lancashire LJ, Bessarabova M, Nikolsky Y, Furlanello C, Chierici M, Albanese D, Jurman G, Riccadonna S, Filosi M, Visintainer R, Zhang KK, Li J, Hsieh JH, Svoboda DL, Fuscoe JC, Deng Y, Shi L, Paules RS, Auerbach SS, Tong W (2014) The concordance between RNA-seq and micorarray data depends on chemical treatment and transcript abundance. Nat Biotechnol 32:926–932

    Article  CAS  Google Scholar 

  • Wieser M, Stadler G, Jennings P, Streubel B, Pfaller W, Ambros P, Riedl C, Katinger H, Grillari J, Grillari-Voglauer R (2008) hTERT alone immortalizes epithelial cells of renal proximal tubules without changing their functional characteristics. Am J Physiol Renal Physiol 295:F1365–F1375

    Article  CAS  Google Scholar 

  • Yauk CL, Berndt ML, Williams A, Douglas GR (2004) Comprehensive comparison of six microarray technologies. Nucleic Acids Res 32:e124

    Article  Google Scholar 

  • van Delft J, Gaj S, Lienhard M, Albrecht MW, Kirpiy A, Brauers K, Claessen S, Lizarraga D, Lehrach H, Herwig R, Kleinjans J (2012) RNA-seq provides new insights in the transcriptome responses induced by the carcinogen benzo[a]pyrene. Toxicol Sci 130:427–439

    Article  Google Scholar 

  • Yildirimman R, Brolén G, Vilardell M, Eriksson G, Synnergren J, Gmuender H, Kamburov A, Ingelman-Sundberg M, Castell J, Lahoz A, Kleinjans J, van Delft J, Björquist P, Herwig R (2011) Human embryonic stem cell derived hepatocyte-like cells as a tool for in vitro hazard assessment of chemical carcinogenicity. Toxicol Sci 124:278–290

    Article  CAS  Google Scholar 

Download references

Acknowledgments

We thank Anja Heymans and Michael David for technical assistance. This work was supported by the European Commission under its 6th Framework Programme with the Grant carcinoGENOMICS (LSHB-CT-2006-037712) and under its 7th Framework Programme with the Grant diXa (283775).

Author information

Author notes

    Authors and Affiliations

    1. Department Computational Molecular Biology, Max-Planck-Institute for Molecular Genetics, Ihnestr. 73, 14195, Berlin, Germany

      R. Herwig, M. Vilardell & R. Yildirimman

    2. Genedata AG, Margarethenstrasse 38, 4053, Basel, Switzerland

      H. Gmuender, A. Brandenburg & T. Wittenberger

    3. European Union Reference Laboratory for Alternatives to Animal Testing (EURL ECVAM), Institute for Health and Consumer Protection (IHCP), European Commission Joint Research Centre, TP 126, Via E. Fermi 2749, 21027, Ispra, Italy

      R. Corvi & P. Phrakonkham

    4. School of Pharmacy and Biomolecular Sciences, Liverpool John Moores University, Byrom Street, Liverpool, L3 3AF, UK

      K. M. Bloch & E. A. Lock

    5. Department of Biochemistry and Molecular Biology, Faculty of Medicine, University of Valencia, Av. Blasco Ibanez 15, 46010, Valencia, Spain

      J. Castell

    6. Department of In Vitro Toxicology and Dermato-Cosmetology, Vrije Universiteit Brussel, Laarbeeklaan 103, 1090, Brussels, Belgium

      L. Ceelen, T. Y. Doktorova & V. Rogiers

    7. Biopredic International, Parc d’affaires de la Bretèche, Bldg. A4, 35760, St Gregoire, France

      C. Chesne

    8. Department of Toxicogenomics, Maastricht University, Maastricht, The Netherlands

      D. Jennen & J. Kleinjans

    9. Division of Physiology, Department of Physiology and Medical Physics, Medical University of Innsbruck, Innsbruck, Austria

      P. Jennings & A. Limonciel

    10. Conway Institute, School of Biomolecular and Biomedical Science, University College Dublin, Belfield, Dublin 4, Ireland

      T. McMorrow, R. Radford, C. Slattery & M. Ryan

    11. Department of Risk Analysis for Products in Development, Netherlands Organisation for Applied Scientific Research (TNO), Utrechtseweg 48, 3704 HE, Zeist, The Netherlands

      R. Stierum

    Authors
    1. R. Herwig
      View author publications

      You can also search for this author in PubMed Google Scholar

    2. H. Gmuender
      View author publications

      You can also search for this author in PubMed Google Scholar

    3. R. Corvi
      View author publications

      You can also search for this author in PubMed Google Scholar

    4. K. M. Bloch
      View author publications

      You can also search for this author in PubMed Google Scholar

    5. A. Brandenburg
      View author publications

      You can also search for this author in PubMed Google Scholar

    6. J. Castell
      View author publications

      You can also search for this author in PubMed Google Scholar

    7. L. Ceelen
      View author publications

      You can also search for this author in PubMed Google Scholar

    8. C. Chesne
      View author publications

      You can also search for this author in PubMed Google Scholar

    9. T. Y. Doktorova
      View author publications

      You can also search for this author in PubMed Google Scholar

    10. D. Jennen
      View author publications

      You can also search for this author in PubMed Google Scholar

    11. P. Jennings
      View author publications

      You can also search for this author in PubMed Google Scholar

    12. A. Limonciel
      View author publications

      You can also search for this author in PubMed Google Scholar

    13. E. A. Lock
      View author publications

      You can also search for this author in PubMed Google Scholar

    14. T. McMorrow
      View author publications

      You can also search for this author in PubMed Google Scholar

    15. P. Phrakonkham
      View author publications

      You can also search for this author in PubMed Google Scholar

    16. R. Radford
      View author publications

      You can also search for this author in PubMed Google Scholar

    17. C. Slattery
      View author publications

      You can also search for this author in PubMed Google Scholar

    18. R. Stierum
      View author publications

      You can also search for this author in PubMed Google Scholar

    19. M. Vilardell
      View author publications

      You can also search for this author in PubMed Google Scholar

    20. T. Wittenberger
      View author publications

      You can also search for this author in PubMed Google Scholar

    21. R. Yildirimman
      View author publications

      You can also search for this author in PubMed Google Scholar

    22. M. Ryan
      View author publications

      You can also search for this author in PubMed Google Scholar

    23. V. Rogiers
      View author publications

      You can also search for this author in PubMed Google Scholar

    24. J. Kleinjans
      View author publications

      You can also search for this author in PubMed Google Scholar

    Corresponding author

    Correspondence to R. Herwig.

    Ethics declarations

    Conflict of interest

    Hans Gmuender, Timo Wittenberger and Arndt Brandenburg are employees of GeneData AG, a company that provides bioinformatics service. Christophe Chesne is employee of Biopredic International, a company that provides human in vitro assays.

    Additional information

    R. Herwig, H. Gmuender, R. Corvi, M. Ryan, V. Rogiers and J. Kleinjans have contributed equally to this work. Except equally contributed authors, other authors are in alphabetical order.

    Electronic supplementary material

    Below is the link to the electronic supplementary material.

    Suppl. Fig. 1

    Human in vitro systems were challenged with three different coded compounds (IC10). Response data after 72-h exposure were compared with DMSO control experiments (triplicates). All experiments were independently performed in three different laboratories. After performance of microarray experiments (in a centralized laboratory), expression data were analyzed independently with two different workflows (TIFF 653 kb)

    Suppl. Fig. 2

    Overlap of ranked fold-changes (treatment vs. control) from pairwise experiments with the HepaRG assay (PDF 270 kb)

    Suppl. Fig. 3

    Overlap of ranked fold-changes (treatment vs. control) from pairwise experiments with the RPTEC/TERT1 assay (PDF 277 kb)

    Suppl. material. 4

    (PDF 172 kb)

    Rights and permissions

    Reprints and Permissions

    About this article

    Check for updates. Verify currency and authenticity via CrossMark

    Cite this article

    Herwig, R., Gmuender, H., Corvi, R. et al. Inter-laboratory study of human in vitro toxicogenomics-based tests as alternative methods for evaluating chemical carcinogenicity: a bioinformatics perspective. Arch Toxicol 90, 2215–2229 (2016). https://doi.org/10.1007/s00204-015-1617-3

    Download citation

    • Received:

    • Accepted:

    • Published:

    • Issue Date:

    • DOI: https://doi.org/10.1007/s00204-015-1617-3

    Keywords

    search

    Navigation