Abstract
One of the most common pitfalls often found in high dimensional biological data sets are correlations between the features. This may lead to statistical and machine learning methodologies overvaluing or undervaluing these correlated predictors, while the truly relevant ones are ignored. In this paper, we will define a new method called pairwise permutation algorithm (PPA) with the aim of mitigating the correlation bias in feature importance values. Firstly, we provide a theoretical foundation, which builds upon previous work on permutation importance. PPA is then applied to a toy data set, where we demonstrate its ability to correct the correlation effect. We further test PPA on a microbiome shotgun dataset, to show that the PPA is already able to obtain biological relevant biomarkers.
T. Maasland and J. Pereira—Equal contribution to this work.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Kiers, H., Smilde, A.: A comparison of various methods for multivariate regression with highly collinear variables. Stat. Meth. Appl. 16, 193 (2007)
Breiman, L.: Random forests. Mach. Learn. 45(1), 5–32 (2001)
Fisher, A., Rudin, C., Dominici, F.: Model class reliance: variable importance measures for any machine learning model class, from the “Rashomon” perspective (2018)
Strobl, C., Boulesteix, A.L., Kneib, T., Augustin, T., Zeileis, A.: Conditional variable importance for random forests. BMC Bioinform. 9, 307 (2008). https://doi.org/10.1186/1471-2105-9-307
Lundberg, S.M., Lee, S.-I.: A unified approach to interpreting model predictions. In: Advances in Neural Information Processing Systems (2017)
Ribeiro, M., Singh, S., Guestrin, C.: Why should i trust you?: explaining the predictions of any classifier. eprint arXiv:1602.04938 (2016)
Pereira, J., Groen, A.K., Stroes, E.S.G., Levin, E.: Graph space embedding. In: Proceedings of the 28th International Joint Conference on Artificial Intelligence Main Track, pp. 3253–3259 (2019). https://doi.org/10.24963/ijcai.2019/451
Shrikumar, A., Greenside, P., Kundaje, A.: Learning important features through propagating activation differences. arXiv preprint arXiv:1704.02685 (2017)
Kootte, R.S., et al.: Improvement of insulin sensitivity after lean donor feces in metabolic syndrome is driven by baseline intestinal microbiota composition. Cell Metab. 26, 611–619 (2017)
Ojala, M., Garriga, G.C.: Permutation tests for studying classifier performance. J. Mach. Learn. Res. 11, 1833–1863 (2010)
Hooker, G., Mentch, L.: Please stop permuting features an explanation and alternatives. arXiv preprint arXiv:1905.03151v1 (2019)
Grömping, U.: Variable importance assessment in regression: linear regression versus Random Forest. Am. Stat. 63(4), 308–319 (2009). https://doi.org/10.1198/tast.2009.08199
Tolosi, L., Lengauer, T.: Classification with correlated features: unreliability of feature ranking and solutions. Bioinformatics 27(14), 1986–1994 (2011)
Gregorutti, B., Michel, B., Saint-Pierre, P.: Correlation and variable importance in random forests. Stat. Comput. 27(3), 659–678 (2016). https://doi.org/10.1007/s11222-016-9646-1
Imangaliyev, S., Keijser, B., Crielaard, W., Tsivtsivadze, E.: Personalized microbial network inference via co-regularized spectral clustering. Methods 83, 28–35 (2015)
Ishwaran, H., et al.: Variable importance in binary regression trees and forests. Electron. J. Stat. 1, 519–537 (2007)
Caruana, R., Niculescu-Mizil, A., Crew, G., et al.: Ensemble selection from libraries of models. In: 21st International Conference on Machine Learning, ICML 2004, vol. 18. ACM (2004)
Chen, T., Guestrin, C.: XGBoost: a scalable tree boosting system. In: Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 785–794. ACM (2016)
Meinshausen, N., Bühlmann, P.: Stability selection. J. R. Stat. Soc. Ser. B (Stat. Methodol.) 72, 417–473 (2010)
Vangay, P., Hillmann, B.M., Knights, D.: Microbiome Learning Repo (ML Repo): a public repository of microbiome regression and classification tasks. GigaScience 8, 1–12 (2019)
Qin, J., Li, Y., Cai, Z., et al.: A metagenome-wide association study of gut microbiota in type 2 diabetes. Nature 490, 55–60 (2012)
Cohen, M.R.: The New Chinese Medicine Handbook: An Innovative Guide to Integrating Eastern Wisdom with Western Practice for Modern Healing, Fair Winds Press (2015)
Chang, S.Y., Kim, D.-H., Han, M.J.: Physicochemical and sensory characteristics of soy yogurt fermented with Bifidobacterium breve K-110, Streptococcus thermophilus 3781, or Lactobacillus acidophilus Q509011. Food Sci. Biotechnol. 19, 107–113 (2010). https://doi.org/10.1007/s10068-010-0015-0
Bedani, R., Rossi, E.A., Isay Saad, S.M.: Impact of inulin and okara on Lactobacillus acidophilus La-5 and Bifidobacterium animalis Bb-12 viability in a fermented soy product and probiotic survival under in vitro simulated gastrointestinal conditions. Food Microbiol. 34(2), 382–389 (2013)
Kanda, H., Wang, H.L., Hesseltine, C.W., et al.: Yoghurt production by Lactobacillus fermentation of soybean milk. Process Biochem. 11(4), 23 (1976)
Kwon, D.Y., Daily, J.W., III., Kim, H.J.: Antidiabetic effects of fermented soybean products on type 2 diabetes. Nutr. Res. 30(1), 1–13 (2010)
Mueller, N.T., Odegaard, A.O., Gross, M.D., et al.: Soy intake and risk of type 2 diabetes mellitus in Chinese Singaporeans. Eur. J. Nutr. 51, 1033–1040 (2012)
Münger, L.H., Trimigno, A., Picone, G., et al.: Identification of urinary food intake biomarkers for milk, cheese, and soy-based drink by untargeted GC-MS and NMR in healthy humans. J. Proetome Res. 16(9), 3321–3335 (2017)
Cook, G.M., Wells, J.E., Russell, J.B.: Ability of Acidaminococcus Fermentans to oxidize trans-aconitate and decrease the accumulation of tricarballylate, a toxic end product of ruminal fermentation. Appl. Environ. Microbiol. 60(7), 2533–2537 (1994)
Moens, F., Verce, M., De Vuyst, L.: Lactate- and acetate-based cross-feeding interactions betweeen selected strains of Lactobacilli. Bifidobacteria and colon bacteria in the presence of inulin-type fructans. Int. J. Food Microbiol. 241, 225–236 (2017)
Saulnier, D.M.A., Spinler, J.K., Gibson, G.R., et al.: Mechanisms of Probiosis and Prebiosis: considerations for enhanced functional foods. Curr. Opin. Biotechnol. 20(2), 135–141 (2009)
de Goffau, M.C., Luopajärvi, K., Knip, M., et al.: Fecal microbiota composition differs between children with beta-cell autoimmunity and those without. Diabetes 62(4), 1238–1244 (2013)
Hur, K.Y., Lee, M.-S.: Gut microbiota and metabolic disorders. Diabetes Metab. J. 39(3), 198–203 (2015)
Hartstra, A.V., Bouter, K.E.C., Bäckhed, F., et al.: Insights into the role of the microbiome in obesity and type 2 diabetes. Diabetes Care 38(1), 159–165 (2015)
Murri, M., Leiva, I., Gomez-Zumaquero, J.M., et al.: Gut microbiota in children with type 1 diabetes differs from that in healthy children: a case-control study. BMC Med. 11, 46 (2013)
Noureldein, M.H., Bitar, S., Youssef, N.: Butyrate modulates Diabetes-linked gut dysbiosis: epigenetic and mechanistic modifications. J. Mol. Endocrinol. 64(1), 29–42 (2020)
Endesfelder, D., Engel, M., Davis-Richardson, A.G., et al.: Towards a functional hypothesis relating anti-islet cell autoimmunity to the dietary impact on microbial communities and butyrate production. Microbiome 4, 17 (2016)
Jia, L., Li, D., Feng, N., et al.: Anti-diabetic effects of clostridium butyricum CGMCC0313.1 through promoting the growth of gut butyrate-producing bacteria in Type 2 Diabetic Mice. Sci. Rep. 7(1), 7046 (2017)
Khan, S., Jena, G.: Sodium butyrate reduces insulin-resistance, fat accumulation and dyslipidemia in Type-2 Diabetic rat: a comparative study with metformin. Chem. Biol. Interact. 254, 124–134 (2016)
Donohoe, D.R., Garge, N., Zhang, X., et al.: The microbiome and butyrate regulate energy metabolism and autophagy in the mammalian colon. Cell Metab. 13(5), 517–526 (2012)
Pereira, J., Groen, A.K., Stroes, E.S.G., Levin, E.: Graph space embedding. In: Proceedings of the 28th International Joint Conference on Artificial Intelligence, IJCAI 2019 (2019)
Sharma, S., Taliyan, R.: Histone deacetylase inhibitors: future therapeutics for insulin resistance and type 2 diabetes. Pharmacol. Res. 113(Pt A), 320–326 (2016)
Dirice, E., Ng, R.W.S., Martinez, R., et al.: Isoform-selective inhibitor of histone deacetylase 3 (HDAC3) limits pancreatic islet infiltration and protects female nonobese diabetic mice from diabetes. J. Biol. Chem. 292(43), 17598–17608 (2017)
Khan, S., Jena, G.: The role of butyrate, a histone deacetylase inhibitor in diabetes mellitus: experimental evidence for therapeutic intervention. Epigenomics 7(4), 669–680 (2015)
Canfora, E.E., Jocken, J.W., Blaak, E.E.: Short-chain fatty acids in control of body weight and insulin sensitivity. Nat. Rev. Endocrinol. 11(10), 577–591 (2015)
Acknowledgments
We would like to thank Manon Balvers for helping with computational experiments.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 Springer Nature Switzerland AG
About this paper
Cite this paper
Maasland, T. et al. (2021). Interpretable Models via Pairwise Permutations Algorithm. In: Kamp, M., et al. Machine Learning and Principles and Practice of Knowledge Discovery in Databases. ECML PKDD 2021. Communications in Computer and Information Science, vol 1524. Springer, Cham. https://doi.org/10.1007/978-3-030-93736-2_2
Download citation
DOI: https://doi.org/10.1007/978-3-030-93736-2_2
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-93735-5
Online ISBN: 978-3-030-93736-2
eBook Packages: Computer ScienceComputer Science (R0)