A case study on the detailed reproducibility of a Human Cell Atlas project
- 19 Downloads
Reproducibility is a defining feature of a scientific discovery. Reproducibility can be at different levels for different types of study. The purpose of the Human Cell Atlas (HCA) project is to build maps of molecular signatures of all human cell types and states to serve as references for future discoveries. Constructing such a complex reference atlas must involve the assembly and aggregation of data from multiple labs, probably generated with different technologies. It has much higher requirements on reproducibility than individual research projects. To add another layer of complexity, the bioinformatics procedures involved for single-cell data have high flexibility and diversity. There are many factors in the processing and analysis of single-cell RNA-seq data that can shape the final results in different ways.
To study what levels of reproducibility can be reached in current practices, we conducted a detailed reproduction study for a well-documented recent publication on the atlas of human blood dendritic cells as an example to break down the bioinformatics steps and factors that are crucial for the reproducibility at different levels.
We found that the major scientific discovery can be well reproduced after some efforts, but there are also some differences in some details that may cause uncertainty in the future reference. This study provides a detailed case observation on the on-going discussions of the type of standards the HCA community should take when releasing data and publications to guarantee the reproducibility and reliability of the future atlas.
Current practices of releasing data and publications may not be adequate to guarantee the reproducibility of HCA. We propose building more stringent guidelines and standards on the information that needs to be provided along with publications for projects that evolved in the HCA program.
KeywordsHuman Cell Atlas reproducibility single cell bioinformatics
We thank Nir Hacohen, Alexandra-Chloé Villani and Orit Rozenblatt-Rosen (authors of the original paper) for their helpful discussion. This work is supported by CZI Human Cell Atlas Pilot Project and the National Natural Science Foundation of China (Nos. 61673231 and 61721003).
- 6.The 1000 Genomes Project Consortium. (2010) A map of human genome variation from population-scale sequencing. Nature, 467, 1061–1073Google Scholar
- 7.The 1000 Genomes Project Consortium. (2012) An integrated map of genetic variation from 1092 human genomes. Nature, 491, 56–65Google Scholar
- 9.The Human Cell Atlas Meeting Participants. (2017) The Human Cell Atlas. eLife, 6, e27041Google Scholar
- 24.Data Coordination–Human Cell Atlas (https://doi.org/www.humancellatlas.org/data-sharing)
- 26.van der Maaten, L. and Hinton, G. (2008) Visualizing data using t- SNE. J. Mach. Learn. Res., 9, 2579–2605.Google Scholar
- 28.Yuansheng Zhou, T. O. S. (2018) Using global t-SNE to preserve inter-cluster data structure. bioRxiv, Doi: https://doi.org/10.1101/331611