The assessment of the carcinogenic potential of chemicals with alternative, human-based in vitro systems has become a major goal of toxicogenomics. The central read-out of these assays is the transcriptome, and while many studies exist that explored the gene expression responses of such systems, reports on robustness and reproducibility, when testing them independently in different laboratories, are still uncommon. Furthermore, there is limited knowledge about variability induced by the data analysis protocols. We have conducted an inter-laboratory study for testing chemical carcinogenicity evaluating two human in vitro assays: hepatoma-derived cells and hTERT-immortalized renal proximal tubule epithelial cells, representing liver and kidney as major target organs. Cellular systems were initially challenged with thirty compounds, genome-wide gene expression was measured with microarrays, and hazard classifiers were built from this training set. Subsequently, each system was independently established in three different laboratories, and gene expression measurements were conducted using anonymized compounds. Data analysis was performed independently by two separate groups applying different protocols for the assessment of inter-laboratory reproducibility and for the prediction of carcinogenic hazard. As a result, both workflows came to very similar conclusions with respect to (1) identification of experimental outliers, (2) overall assessment of robustness and inter-laboratory reproducibility and (3) re-classification of the unknown compounds to the respective toxicity classes. In summary, the developed bioinformatics workflows deliver accurate measures for inter-laboratory comparison studies, and the study can be used as guidance for validation of future carcinogenicity assays in order to implement testing of human in vitro alternatives to animal testing.
This is a preview of subscription content, access via your institution.
Buy single article
Instant access to the full article PDF.
Price excludes VAT (USA)
Tax calculation will be finalised during checkout.
Similar content being viewed by others
Annys E, Billington R, Clayton R, Bremm KD, Graziano M, McKelvie J, Ragan I, Schwarz M, van der Laan JW, Wood C, Öberg M, Wester P, Woodward KN (2014) Advancing the 3Rs in regulatory toxicology—carcinogenicity testing: scope for harmonization and advancing the 3Rs in regulated sectors of the European Union. Regul Toxicol Pharmacol 69:234–242
Aschauer L, Gruber LN, Pfaller W, Limonciel A, Athersuch TJ, Cavill R, Khan A, Gstraunthaler G, Grillari J, Grillari R, Hewitt P, Leonard MO, Wilmes A, Jennings P (2013) Delineation of the key aspects in the regulation of epithelial monolayer formation. Mol Cell Biol 33:2535–2550
Ashburner M, Ball CA, Blake JA, Botstein D, Butler H, Cherry JM, Davis AP, Dolinski K, Dwight SS, Eppig JT, Harris MA, Hill DP, Issel-Tarver L, Kasarskis A, Lewis S, Matese JC, Richardson JE, Ringwald M, Rubin GM, Sherlock G (2000) Gene ontology: tool for the unification of biology. The Gene Ontology Consortium. Nat Genet 25:25–29
Caiment F, Tsamou M, Jennen D, Kleinjans J (2014) Assessing compound carcinogenicity in vitro connectivity mapping. Carcinogenesis 35:201–207
Cristianini N, Shawe-Taylor J (2000) An introduction to support vector machines and other kernel-based learning methods. Cambridge University Press, Cambridge
Dai M, Wang P, Boyd AD, Kostov G, Athey B, Jones EG, Bunney WE, Myers RM, Speed TP, Akil H, Watson SJ, Meng F (2005) Evolving gene/transcript definitions significantly alter the interpretation of GeneChip data. Nucleic Acids Res 33:e175
Doktorova TY, Yildirimman R, Vinken M, Vilardell M, Vanhaecke T, Gmuender H, Bort R, Brolen G, Holmgren G, Li R, Chesne C, van Delft J, Kleinjans J, Castell J, Bjorquist P, Herwig R, Rogiers V (2013) Transcriptomic responses generated by hepatocarcinogens in a battery of liver-based in vitro models. Carcinogenesis 34:1393–1402
Doktorova T, Yildirimman R, Ceelen L, Vilardell M, Vanhaecke T, Vinken M, Ates G, Heymans A, Gmuender H, Bort R, Corvi R, Phrakonkham P, Li R, Mouchet N, Chesne C, van Delft J, Kleinjans J, Castell J, Herwig R, Rogiers V (2014) Testing chemical carcinogenicity by using a transcriptomics HepaRG-based model. Exp Clin Sci J 13:623–637
Ernst J, Bar-Joseph Z (2006) STEM: a tool for the analysis of short time series gene expression data. BMC Bioinform 7:191
Fan X, Shao L, Fang H, Tong W, Cheng Y (2011) Cross-platform comparison of microarray-based multiple-class prediction. PLoS ONE 6:e16067
Fowlkes EB, Mallows CL (1983) A method for comparing two hierarchical clusterings. J Am Stat Assoc 78:553–569
Ganter B et al (2005) Development of a large-scale chemogenomics database to improve drug candidate selection and to understand mechanisms of chemical toxicity and action. J Biotechnol 119:219–244
Gómez-Lechón MJ, Castell JV, Donato MT (2010) The use of hepatocytes to investigate drug toxicity. Methods Mol Biol 640:389–415
Gusenleitner D, Auerbach SS, Melia T, Gómez HF, Sherr DH, Monti S (2014) Genomic models of short-term exposure accurately predict long-term chemical carcinogenicity and identify putative mechanisms of action. PLoS ONE 9:e102579
Guyon I, Weston J, Barnhill S, Vapnik V (2002) Gene selection for cancer classification using support vector machines. Mach Learn 46:389–422
Hartung T, Bremer S, Casati S, Coecke S, Corvi R, Fortaner S, Gribaldo L, Halder M, Hoffmann S, Roi AJ, Prieto P, Sabbioni E, Scott L, Worth A, Zuang V (2004) A modular approach to the ECVAM principles on test validity. Altern Lab Anim 32:467–472
Ioannidis JP, Khoury MJ (2011) Improving validation practices in “omics” research. Science 334:1230–1232
Irizarry RA, Warren D, Spencer F, Kim IF, Biswal S, Frank BC, Gabrielson E, Garcia JG, Geoghegan J, Germino G, Griffin C, Hilmer SC, Hoffman E, Jedlicka AE, Kawasaki E, Martínez-Murillo F, Morsberger L, Lee H, Petersen D, Quackenbush J, Scott A, Wilson M, Yang Y, Ye SQ, Yu W (2005) Multiple-laboratory comparison of microarray platforms. Nat Methods 2:345–350
Jennen DG, Magkoufopoulou C, Ketelslegers HB, van Herwijnen MH, Kleinjans JC, van Delft JH (2010) Comparison of HepG2 and HepaRG by whole-genome gene expression analysis for the purpose of chemical hazard identification. Toxicol Sci 115:66–79
Jennings P, Aydin S, Bennett J, McBride R, Weiland C, Tuite N, Gruber LN, Perco P, Gaora PO, Ellinger-Ziegelbauer H, Ahr HJ, Kooten CV, Daha MR, Prieto P, Ryan MP, Pfaller W, McMorrow T (2009) Inter-laboratory comparison of human renal proximal tubule (HK-2) transcriptome alterations due to Cyclosporine A exposure and medium exhaustion. Toxicol In Vitro 23:486–499
Jennings P, Weiland C, Limonciel A, Bloch KM, Radford R, Aschauer L, McMorrow T, Wilmes A, Pfaller W, Ahr HJ, Slattery C, Lock EA, Ryan MP, Ellinger-Ziegelbauer H (2012) Transcriptomic alterations induced by Ochratoxin A in rat and human renal proximal tubular in vitro models and comparison to a rat in vivo model. Arch Toxicol 86:571–589
Kiyosawa N, Ando Y, Watanabe K, Niino N, Manabe S, Yamoto T (2009) Scoring multiple toxicological endpoints using a toxicogenomics database. Toxicol Lett 188:91–97
Lamb J, Crawford ED, Peck D, Modell JW, Blat IC, Wrobel MJ, Lerner J, Brunet JP, Subramanian A, Ross KN, Reich M, Hieronymus H, Wei G, Armstrong SA, Haggarty SJ, Clemons PA, Wei R, Carr SA, Lander ES, Golub TR (2006) The connectivity map: using gene-expression signatures to connect small molecules, genes and diseases. Science 313:1929–1935
Limonciel A, Aschauer L, Wilmes A, Prajczer S, Leonard MO, Pfaller W, Jennings P (2011) Lactate is an ideal non-invasive marker for evaluating temporal alterations in cell stress and toxicity in repeat dose testing regimes. Toxicol In Vitro 25:1855–1862
Limonciel A, Wilmes A, Aschauer L, Radford R, Bloch KM, McMorrow T, Pfaller W, van Delft JH, Slattery C, Ryan MP, Lock EA, Jennings P (2012) Oxidative stress induced by potassium bromate exposure results in altered tight junction protein expression in renal proximal tubule cells. Arch Toxicol 86:1741–1751
MAQC Consortium (2006) The microarray quality control (MAQC) project shows inter- and intraplatform reproducibility of gene expression measurements. Nat Biotechnol 24:1151–1161
MAQC Consortium (2010) The microarray quality control (MAQC)-II study of common practices for the development and validation of microarray-based predictive models. Nat Biotechnol 28:827–838
Paules RS, Aubrecht J, Corvi R, Garthoff B, Kleinjans JC (2011) Moving forward in human cancer risk assessment. Environ Health Perspect 119:739–743
Radford R, Slattery C, Jennings P, Blacque O, Pfaller W, Gmuender H, Van Delft J, Ryan MP, McMorrow T (2012) Carcinogens induce loss of the primary cilium in human renal proximal tubular epithelial cells independently of effects on the cell cycle. Am J Physiol Renal Physiol 302:F905–F916
Radford R, Frain H, Ryan MP, Slattery C, McMorrow T (2013) Mechanisms of chemical carcinogenesis in the kidneys. Int J Mol Sci 14:19416–19433
Ritchie ME, Phipson B, Wu D, Hu Y, Law CW, Shi W, Smyth GK (2015) Limma powers differential expression analyses for RNA-sequencing and microarray studies. Nucleic Acids Res 43:e47
Silva Lima B, van der Laan JW (2000) Mechanisms of nongenotoxic carcinogenesis and assessment of the human hazard. Regul Toxicol Pharmacol 32:135–143
Vinken M, Doktorova T, Ellinger-Ziegelbauer H, Ahr HJ, Lock E, Carmichael P, Roggen E, van Delft J, Kleinjans J, Castell J, Bort R, Donato T, Ryan M, Corvi R, Keun H, Ebbels T, Athersuch T, Sansone SA, Rocca-Serra P, Stierum R, Jennings P, Pfaller W, Gmuender H, Vanhaecke T, Rogiers V (2008) The carcinoGENOMICS project: critical selection of model compounds for the development of omics-based in vitro carcinogenicity screening assays. Mutat Res 659:202–210
Wang C, Gong B, Bushel PR, Thierry-Mieg J, Thierry-Mieg D, Xu J, Fang H, Hong H, Shen J, Su Z, Meehan J, Li X, Yang L, Li H, Łabaj PP, Kreil DP, Megherbi D, Gaj S, Caiment F, van Delft J, Kleinjans J, Scherer A, Devanarayan V, Wang J, Yang Y, Qian HR, Lancashire LJ, Bessarabova M, Nikolsky Y, Furlanello C, Chierici M, Albanese D, Jurman G, Riccadonna S, Filosi M, Visintainer R, Zhang KK, Li J, Hsieh JH, Svoboda DL, Fuscoe JC, Deng Y, Shi L, Paules RS, Auerbach SS, Tong W (2014) The concordance between RNA-seq and micorarray data depends on chemical treatment and transcript abundance. Nat Biotechnol 32:926–932
Wieser M, Stadler G, Jennings P, Streubel B, Pfaller W, Ambros P, Riedl C, Katinger H, Grillari J, Grillari-Voglauer R (2008) hTERT alone immortalizes epithelial cells of renal proximal tubules without changing their functional characteristics. Am J Physiol Renal Physiol 295:F1365–F1375
Yauk CL, Berndt ML, Williams A, Douglas GR (2004) Comprehensive comparison of six microarray technologies. Nucleic Acids Res 32:e124
van Delft J, Gaj S, Lienhard M, Albrecht MW, Kirpiy A, Brauers K, Claessen S, Lizarraga D, Lehrach H, Herwig R, Kleinjans J (2012) RNA-seq provides new insights in the transcriptome responses induced by the carcinogen benzo[a]pyrene. Toxicol Sci 130:427–439
Yildirimman R, Brolén G, Vilardell M, Eriksson G, Synnergren J, Gmuender H, Kamburov A, Ingelman-Sundberg M, Castell J, Lahoz A, Kleinjans J, van Delft J, Björquist P, Herwig R (2011) Human embryonic stem cell derived hepatocyte-like cells as a tool for in vitro hazard assessment of chemical carcinogenicity. Toxicol Sci 124:278–290
We thank Anja Heymans and Michael David for technical assistance. This work was supported by the European Commission under its 6th Framework Programme with the Grant carcinoGENOMICS (LSHB-CT-2006-037712) and under its 7th Framework Programme with the Grant diXa (283775).
Conflict of interest
Hans Gmuender, Timo Wittenberger and Arndt Brandenburg are employees of GeneData AG, a company that provides bioinformatics service. Christophe Chesne is employee of Biopredic International, a company that provides human in vitro assays.
R. Herwig, H. Gmuender, R. Corvi, M. Ryan, V. Rogiers and J. Kleinjans have contributed equally to this work. Except equally contributed authors, other authors are in alphabetical order.
Electronic supplementary material
Below is the link to the electronic supplementary material.
Human in vitro systems were challenged with three different coded compounds (IC10). Response data after 72-h exposure were compared with DMSO control experiments (triplicates). All experiments were independently performed in three different laboratories. After performance of microarray experiments (in a centralized laboratory), expression data were analyzed independently with two different workflows (TIFF 653 kb)
Overlap of ranked fold-changes (treatment vs. control) from pairwise experiments with the HepaRG assay (PDF 270 kb)
Overlap of ranked fold-changes (treatment vs. control) from pairwise experiments with the RPTEC/TERT1 assay (PDF 277 kb)
(PDF 172 kb)
About this article
Cite this article
Herwig, R., Gmuender, H., Corvi, R. et al. Inter-laboratory study of human in vitro toxicogenomics-based tests as alternative methods for evaluating chemical carcinogenicity: a bioinformatics perspective. Arch Toxicol 90, 2215–2229 (2016). https://doi.org/10.1007/s00204-015-1617-3