Processing and Analyzing Human Microbiome Data

  • Xuan Zhu
  • Jian Wang
  • Cielito Reyes-Gibby
  • Sanjay Shete
Part of the Methods in Molecular Biology book series (MIMB, volume 1666)


The human microbiome is associated with complex disorders such as diabetes, cancer, obesity and cardiovascular disorders. Recent technological developments have allowed researchers to fully quantify the composition of the microbiome using culture-independent approaches, resulting in a large amount of microbiome data, which provide invaluable opportunities to assess the important contributions of the microbiome to human health and disease. In this chapter, we discuss and evaluate multiple statistical approaches for processing, summarizing, and analyzing microbiome data. Specifically, we provide programming scripts for processing microbiome data using QIIME and calculating alpha and beta diversities, assessing the association between diversities and outcomes of interest using R programs, as well as interpretation of results. We illustrate the methods in the context of analyzing the foregut microbiome in esophageal adenocarcinoma.

Key words

Human microbiome Microbiome composition Alpha diversity Beta diversity 16S rRNA sequencing Association study Phylogenetic tree QIIME Esophageal adenocarcinoma 


  1. 1.
    Ursell LK, Metcalf JL, Parfrey LW, Knight R (2012) Defining the human microbiome. Nutr Rev 70(Suppl 1):S38–S44CrossRefPubMedPubMedCentralGoogle Scholar
  2. 2.
    Li H (2015) Microbiome, metagenomics, and high-dimensional compositional data analysis. Annu Rev Stat Appl 2:73–94CrossRefGoogle Scholar
  3. 3.
    Backhed F, Ley RE, Sonnenburg JL, Peterson DA, Gordon JI (2005) Host-bacterial mutualism in the human intestine. Science 307:1915–1920CrossRefPubMedGoogle Scholar
  4. 4.
    Human Microbiome Project (2016) About HMP metagenomic sequencing & analysis. analyses.php
  5. 5.
    Claus SP, Guillou H, Ellero-Simatos S (2016) The gut microbiota: a major player in the toxicity of environmental pollutants? NPJ Biofilms Microbiomes 2:16003CrossRefPubMedPubMedCentralGoogle Scholar
  6. 6.
    National Institutes of Health (2016) NIH Human Microbiome Project defines normal bacterial makeup of the body.
  7. 7.
    Hartstra AV, Bouter KEC, Backhed F, Nieuwdorp M (2015) Insights into the role of the microbiome in obesity and type 2 diabetes. Diabetes Care 38:159–165CrossRefPubMedGoogle Scholar
  8. 8.
    Tang WHW, Hazen SL (2014) The contributory role of gut microbiota in cardiovascular disease. J Clin Invest 124:4204–4211CrossRefPubMedPubMedCentralGoogle Scholar
  9. 9.
    Dulal S, Keku TO (2014) Gut microbiome and colorectal adenomas. Cancer J 20:225–231CrossRefPubMedPubMedCentralGoogle Scholar
  10. 10.
    Illumina (2016) Introduction to human microbiome analysis, Survey the genomes of entire communities.
  11. 11.
    Woo PCY, Lau SKP, Teng JLL, Tse H, Yuen KY (2008) Then and now: use of 16S rDNA gene sequencing for bacterial identification and discovery of novel bacteria in clinical microbiology laboratories. Clin Microbiol Infect 14:908–934CrossRefPubMedGoogle Scholar
  12. 12.
    Clarridge JE (2004) Impact of 16S rRNA gene sequence analysis for identification of bacteria on clinical microbiology and infectious diseases. Clin Microbiol Rev 17:840–862CrossRefPubMedPubMedCentralGoogle Scholar
  13. 13.
    Hamady M, Knight R (2009) Microbial community profiling for human microbiome projects: tools, techniques, and challenges. Genome Res 19:1141–1152CrossRefPubMedPubMedCentralGoogle Scholar
  14. 14.
    Fiona Stewart EY (2014) Addressing challenges in microbiome DNA analysis, NEB UK ExpressionsGoogle Scholar
  15. 15.
    Brooks JP (2016) Challenges for case-control studies with microbiome data. Ann Epidemiol 26:336–341Google Scholar
  16. 16.
    Yang L, Chaudhary N, Baghdadi J, Pei Z (2014) Microbiome in reflux disorders and esophageal adenocarcinoma. Cancer J 20:207–210CrossRefPubMedPubMedCentralGoogle Scholar
  17. 17.
    Gilles A, Meglecz E, Pech N, Ferreira S, Malausa T, Martin JF (2011) Accuracy and quality assessment of 454 GS-FLX Titanium pyrosequencing. BMC Genomics 12:245CrossRefPubMedPubMedCentralGoogle Scholar
  18. 18.
    Caporaso JG, Lauber CL, Walters WA, Berg-Lyons D, Huntley J, Fierer N, Owens SM, Betley J, Fraser L, Bauer M, Gormley N, Gilbert JA, Smith G, Knight R (2012) Ultra-high-throughput microbial community analysis on the Illumina HiSeq and MiSeq platforms. ISME J 6:1621–1624CrossRefPubMedPubMedCentralGoogle Scholar
  19. 19.
    Schloss PD, Westcott SL, Ryabin T, Hall JR, Hartmann M, Hollister EB, Lesniewski RA, Oakley BB, Parks DH, Robinson CJ, Sahl JW, Stres B, Thallinger GG, Van Horn DJ, Weber CF (2009) Introducing mothur: open-source, platform-independent, community-supported software for describing and comparing microbial communities. Appl Environ Microbiol 75:7537–7541CrossRefPubMedPubMedCentralGoogle Scholar
  20. 20.
    Caporaso JG, Kuczynski J, Stombaugh J, Bittinger K, Bushman FD, Costello EK, Fierer N, Pena AG, Goodrich JK, Gordon JI, Huttley GA, Kelley ST, Knights D, Koenig JE, Ley RE, Lozupone CA, McDonald D, Muegge BD, Pirrung M, Reeder J, Sevinsky JR, Turnbaugh PJ, Walters WA, Widmann J, Yatsunenko T, Zaneveld J, Knight R (2010) QIIME allows analysis of high-throughput community sequencing data. Nat Methods 7:335–336CrossRefPubMedPubMedCentralGoogle Scholar
  21. 21.
    Erica Plummer JT, Bulach DM, Garland SM, Tabrizi SN (2015) A comparison of three bioinformatics pipelines for the analysis of preterm gut microbiota using 16S rRNA gene sequencing data. J Proteomics Bioinform 8:283–291Google Scholar
  22. 22.
    Navas-Molina JA, Peralta-Sanchez JM, Gonzalez A, McMurdie PJ, Vazquez-Baeza Y, Xu ZJ, Ursell LK, Lauber C, Zhou HW, Song SJ, Huntley J, Ackermann GL, Berg-Lyons D, Holmes S, Caporaso JG, Knight R (2013) Advancing our understanding of the human microbiome using QIIME. Methods Enzymol 531:371–444CrossRefPubMedPubMedCentralGoogle Scholar
  23. 23.
    Mir K, Neuhaus K, Bossert M, Schober S (2013) Short barcodes for next generation sequencing. PLoS One 8:e82933CrossRefPubMedPubMedCentralGoogle Scholar
  24. 24.
    Edgar RC (2013) UPARSE: highly accurate OTU sequences from microbial amplicon reads. Nat Methods 10:996–998CrossRefPubMedGoogle Scholar
  25. 25.
    Scitable by nature education (2016) Primer.
  26. 26.
    De Beuf K, De Schrijver J, Thas O, Van Criekinge W, Irizarry RA, Clement L (2012) Improved base-calling and quality scores for 454 sequencing based on a Hurdle Poisson model. BMC Bioinformatics 13:303CrossRefPubMedPubMedCentralGoogle Scholar
  27. 27.
    Si XF, Baselga A, Leprieur F, Song X, Ding P (2016) Selective extinction drives taxonomic and functional alpha and beta diversities in island bird assemblages. J Anim Ecol 85:409–418CrossRefPubMedGoogle Scholar
  28. 28.
    McMurdie PJ, Holmes S (2013) Phyloseq: an R package for reproducible interactive analysis and graphics of microbiome census data. PLoS One 8:e61217CrossRefPubMedPubMedCentralGoogle Scholar
  29. 29.
    Hill MO (1973) Diversity and evenness: a unifying notation and its consequences. Ecology 54:427–432CrossRefGoogle Scholar
  30. 30.
    Li K, Bihan M, Yooseph S, Methe BA (2012) Analyses of the microbial diversity across the human microbiome. PLoS One 7:e32118CrossRefPubMedPubMedCentralGoogle Scholar
  31. 31.
    Lande R (1996) Statistics and partitioning of species diversity, and similarity among multiple communities. Oikos 76:5–13CrossRefGoogle Scholar
  32. 32.
    Basualdo CV (2011) Choosing the best non-parametric richness estimator for benthic macroinvertebrates databases. Rev Soc Entomol Argent 70(1–2):27–38Google Scholar
  33. 33.
    Sandra D, Williamson KB (2013) Species richness and diversity of a terrestrial insular environment: serpentine of the Barberton Greenstone Belt, South Africa. Int J Biodivers Conserv 5(5):296–310Google Scholar
  34. 34.
    Morris EK, Caruso T, Buscot F, Fischer M, Hancock C, Maier TS, Meiners T, Muller C, Obermaier E, Prati D, Socher SA, Sonnemann I, Waschke N, Wubet T, Wurst S, Rillig MC (2014) Choosing and using diversity indices: insights for ecological applications from the German Biodiversity Exploratories. Ecol Evol 4:3514–3524CrossRefPubMedPubMedCentralGoogle Scholar
  35. 35.
    Nagendra H (2002) Opposite trends in response for the Shannon and Simpson indices of landscape diversity. Appl Geogr 22:175–186CrossRefGoogle Scholar
  36. 36.
    Saucedo-Garcia A, Anaya AL, Espinosa-Garcia FJ, Gonzalez MC (2014) Diversity and communities of foliar endophytic fungi from different agroecosystems of Coffea arabica L. in two regions of Veracruz, Mexico. PLoS One 9:e98454CrossRefPubMedPubMedCentralGoogle Scholar
  37. 37.
    Williams VL, Witkowski ETF, Balkwill K (2005) Application of diversity indices to appraise plant availability in the traditional medicinal markets of Johannesburg, South Africa. Biodivers Conserv 14:2971–3001CrossRefGoogle Scholar
  38. 38.
    Colwell RK (2009) Biodiversity: concepts, patterns, and measurement. In: Levin SA (ed) The Princeton guide to ecology. Princeton University Press, Princeton, NJ, pp 257–263Google Scholar
  39. 39.
    Fisher RA, Corbet AS, Williams CB (1943) The relation between the number of species and the number of individuals in a random sample of an animal population. J Anim Ecol 12:42–58CrossRefGoogle Scholar
  40. 40.
    Magurran AE (2004) Measuring biological diversity. Blackwell Publishing, Oxford, UKGoogle Scholar
  41. 41.
    Hughes JB, Hellmann JJ, Ricketts TH, Bohannan BJ (2001) Counting the uncountable: statistical approaches to estimating microbial diversity. Appl Environ Microbiol 67:4399–4406CrossRefPubMedPubMedCentralGoogle Scholar
  42. 42.
    Chao A, Ma MC, Yang MCK (1993) Stopping rules and estimation for recapture debugging with unequal failure rates. Biometrika 80:193–201CrossRefGoogle Scholar
  43. 43.
    Gotelli NJ, Colwell RK (2010) Estimating species richness. In: Magurran AE, McGill BJ (eds) Frontiers in measuring biodiversity. Oxford University, New York, pp 39–54Google Scholar
  44. 44.
    Chao A, Chazdon RL, Colwell RK, Shen TJ (2005) A new statistical approach for assessing similarity of species composition with incidence and abundance data. Ecol Lett 8:148–159CrossRefGoogle Scholar
  45. 45.
    Soininen J (2010) Species turnover along abiotic and biotic gradients: patterns in space equal patterns in time? Bioscience 60:433–439CrossRefGoogle Scholar
  46. 46.
    Koleff P, Gaston KJ, Lennon JJ (2003) Measuring beta diversity for presence-absence data. J Anim Ecol 72:367–382CrossRefGoogle Scholar
  47. 47.
  48. 48.
    Shirkhorshidi AS, Aghabozorgi S, Wah TY (2015) A comparison study on similarity and dissimilarity measures in clustering continuous data. PLoS One 10:e0144059CrossRefPubMedPubMedCentralGoogle Scholar
  49. 49.
    Emran SM, Ye N (2002) Robustness of Chi-square and Canberra distance metrics for computer intrusion detection. Qual Reliab Eng Int 18:19–28CrossRefGoogle Scholar
  50. 50.
    Giuseppe Jurman SR, Visintainer R, Furlanello C (2009) Canberra distance on ranked lists. Advances in ranking–NIPS 09 workshop, pp 22–27Google Scholar
  51. 51.
    Hennig C, Hausdorf B (2006) Design of dissimilarity measures: a new dissimilarity between species distribution areas. In: Batagelj V, Bock H-H, Ferligoj A, Žiberna A (eds) Stud class data anal. Springer, Berlin, Heidelberg, pp 29–37Google Scholar
  52. 52.
    Anderson MJ, Millar RB (2004) Spatial variation and effects of habitat on temperate reef fish assemblages in northeastern New Zealand. J Exp Mar Biol Ecol 305:191–221CrossRefGoogle Scholar
  53. 53.
    Horn HS (1966) Measurement of overlap in comparative ecological studies. Am Nat 100:419CrossRefGoogle Scholar
  54. 54.
    Anderson MJ, Ellingsen KE, McArdle BH (2006) Multivariate dispersion as a measure of beta diversity. Ecol Lett 9:683–693CrossRefPubMedGoogle Scholar
  55. 55.
    Cao Y, Williams WP, Bark AW (1997) Similarity measure bias in river benthic Aufwuchs community analysis. Water Environ Res 69:95–106CrossRefGoogle Scholar
  56. 56.
    Lozupone C, Knight R (2005) UniFrac: a new phylogenetic method for comparing microbial communities. Appl Environ Microbiol 71:8228–8235CrossRefPubMedPubMedCentralGoogle Scholar
  57. 57.
    Clarke KR, Somerfield PJ, Chapman MG (2006) On resemblance measures for ecological studies, including taxonomic dissimilarities and a zero-adjusted Bray-Curtis coefficient for denuded assemblages. J Exp Mar Biol Ecol 330:55–80CrossRefGoogle Scholar
  58. 58.
    Fukuyama J, McMurdie PJ, Dethlefsen L, Relman DA, Holmes S (2012) Comparisons of distance methods for combining covariates and abundances in microbiome studies. Pac Symp Biocomput:213–224Google Scholar
  59. 59.
    Lozupone CA, Knight R (2007) Global patterns in bacterial diversity. Proc Natl Acad Sci U S A 104:11436–11440CrossRefPubMedPubMedCentralGoogle Scholar
  60. 60.
    Schloss PD (2008) Evaluating different approaches that test whether microbial communities have the same structure. ISME J 2:265–275CrossRefPubMedGoogle Scholar
  61. 61.
    Ives AR, Helmus MR (2010) Phylogenetic metrics of community similarity. Am Nat 176:E128–E142CrossRefPubMedGoogle Scholar
  62. 62.
    McArdle BH, Anderson MJ (2001) Fitting multivariate models to community data: a comment on distance-based redundancy analysis. Ecology 82:290–297CrossRefGoogle Scholar
  63. 63.
    Zhao N, Chen J, Carroll IM, Ringel-Kulka T, Epstein MP, Zhou H, Zhou JJ, Ringel Y, Li HZ, Wu MC (2015) Testing in microbiome-profiling studies with MiRKAT, the Microbiome regression-based kernel association test. Am J Hum Genet 96:797–807CrossRefPubMedPubMedCentralGoogle Scholar
  64. 64.
    Zhu X, Wang J, Peng B, Shete S (2016) Empirical estimation of sequencing error rates using smoothing splines. BMC Bioinformatics 17:177CrossRefPubMedPubMedCentralGoogle Scholar
  65. 65.
    Scealy JL, Welsh AH (2011) Regression for compositional data by using distributions defined on the hypersphere. J R Stat Soc B 73:351–375CrossRefGoogle Scholar
  66. 66.
    Kent JT (1982) The Fisher-Bingham distribution on the sphere. J R Stat Soc B 44:71–80Google Scholar
  67. 67.
    Aitchison J (1982) The statistical-analysis of compositional data. J R Stat Soc B 44:139–177Google Scholar
  68. 68.
    Shi PX, Zhang AR, Li HZ (2016) Regression analysis for microbiome compositional data. Ann Appl Stat 10:1019–1040CrossRefGoogle Scholar
  69. 69.
    Fisher CK, Mehta P (2014) Identifying keystone species in the human gut microbiome from metagenomic timeseries using sparse linear regression. PLoS One 9:e102451CrossRefPubMedPubMedCentralGoogle Scholar
  70. 70.
    Chen EZ, Li HZ (2016) A two-part mixed-effects model for analyzing longitudinal microbiome compositional data. Bioinformatics 32:2611–2617CrossRefPubMedGoogle Scholar
  71. 71.
    Gevers D, Knight R, Petrosino JF, Huang K, McGuire AL, Birren BW, Nelson KE, White O, Methe BA, Huttenhower C (2012) The Human Microbiome Project: a community resource for the healthy human microbiome. PLoS Biol 10:e1001377CrossRefPubMedPubMedCentralGoogle Scholar
  72. 72.
    Edgar RC (2016) UNCROSS: filtering of high-frequency cross-talk in 16S amplicon reads. doi: 10.1101/088666
  73. 73.
    McDonald D, Price MN, Goodrich J, Nawrocki EP, DeSantis TZ, Probst A, Andersen GL, Knight R, Hugenholtz P (2012) An improved Greengenes taxonomy with explicit ranks for ecological and evolutionary analyses of bacteria and archaea. ISME J 6:610–618CrossRefPubMedGoogle Scholar
  74. 74.
    DeSantis TZ, Hugenholtz P, Larsen N, Rojas M, Brodie EL, Keller K, Huber T, Dalevi D, Hu P, Andersen GL (2006) Greengenes, a chimera-checked 16S rRNA gene database and workbench compatible with ARB. Appl Environ Microbiol 72:5069–5072CrossRefPubMedPubMedCentralGoogle Scholar
  75. 75.
    R Core Team (2016) R: a language and environment for statistical computing. R foundation for statistical computing.
  76. 76.
    van den Boogaart KG, Tolosana R, Bren M (2014) compositions: Compositional Data Analysis. R Package Version 1:40–1.
  77. 77.
    Oksanen J, Guillaume Blanchet F, Friendly M, Kindt R, Legendre P, McGlinn D, Minchin PR, O’Hara RB, Simpson GL, Solymos P, Henry M, Stevens H (2016) vegan: Community Ecology Package. R package version 2.3-5.
  78. 78.
    Paradis E, Claude J, Strimmer K (2004) APE: analyses of phylogenetics and evolution in R language. Bioinformatics 20:289–290CrossRefPubMedGoogle Scholar
  79. 79.
    Bokulich NA, Subramanian S, Faith JJ, Gevers D, Gordon JI, Knight R, Mills DA, Caporaso JG (2013) Quality-filtering vastly improves diversity estimates from Illumina amplicon sequencing. Nat Methods 10:57–59Google Scholar
  80. 80.
    Walters WA, Caporaso JG, Lauber CL, Berg-Lyons D, Fierer N, Knight R (2011) PrimerProspector: de novo design and taxonomic analysis of barcoded polymerase chain reaction primers. Bioinformatics 27:1159–1161CrossRefPubMedPubMedCentralGoogle Scholar

Copyright information

© Springer Science+Business Media LLC 2017

Authors and Affiliations

  • Xuan Zhu
    • 1
  • Jian Wang
    • 1
  • Cielito Reyes-Gibby
    • 2
  • Sanjay Shete
    • 1
    • 3
  1. 1.Department of BiostatisticsThe University of Texas MD Anderson Cancer CenterHoustonUSA
  2. 2.Department of Emergency MedicineThe University of Texas MD Anderson Cancer CenterHoustonUSA
  3. 3.Department of EpidemiologyThe University of Texas MD Anderson Cancer CenterHoustonUSA

Personalised recommendations