Abstract
Over the last two decades, electronic nose research has produced thousands of research works. Many of them were describing the ability of the e-nose technology to solve diverse applications in domains ranging from food technology to safety, security, or health. It is, in fact, in the biomedical field where e-nose technology is finding a research niche in the last years. Although few success stories exist, most described applications never found the road to industrial or clinical exploitation. Most described methodologies were not reliable and were plagued by numerous problems that prevented practical application beyond the lab. This work emphasizes the need of external validation in machine olfaction. I describe some statistical and methodological pitfalls of the e-nose practice and I give some best practice recommendations for researchers in the field.
Similar content being viewed by others
References
Dragonieri S, Schot R, Mertens BJ, Le Cessie S, Gauw SA, Spanevello A, Resta O, Willard NO, Vink TJ, Rabe KF, Bel EH, Sterk PJ (2007) An electronic nose in the discrimination of patients with asthma and controls. J Allergy Clin Immunol 120:856–862
Montuschi P, Mores N, Trové A, Mondino C, Barnes PJ (2013) The electronic nose in respiratory medicine. Respi Int Rev Thorac Dis 85:72–84
Greulich T, Hattesohl A, Grabisch A, Koepke J, Schmid S, Noeske S, Nell C, Wencker M, Jörres RA, Vogelmeier CF, Köhler U, Koczulla AR (2013) Detection of obstructive sleep apnea by an electronic nose. Eur Respir J 42:145–155
Lazar Z, Fens N, van der Maten J, van der Schee MP, Wagener AH, de Nijs SB, Dijkers E, Sterk PJ (2010) Electronic nose breathprints are independent of acute changes in airway caliber in asthma. Sensors 10:9127–9138
Biller H, Holz O, Windt H, Koch W, Müller M, Jörres RA, Krug N, Hohlfeld JM (2011) Breath profiles by electronic nose correlate with systemic markers but not ozone response. Respir Med 105:1352–1363
Chapman E, Thomas PS, Stone E, Lewis C, Yates DH (2012) A breath test for malignant mesothelioma using an electronic nose. Eur Respir J 40:448–454
Hattesohl A, Jörres R, Dressel H, Schmid S, Vogelmeier C, Greulich T, Noeske S, Bals R, Koczulla AR (2011) Discrimination between COPD patients with and without alpha 1-antitrypsin deficiency using an electronic nose. Respirology 16:1258–1264
Valera J, Togores B, Cosio B (2012) Use of the electronic nose for diagnosing respiratory diseases. Arch Bronconeumología 48:187–188
Fens N, Zwinderman A, van der Schee M, de Nijs SB, Dijkers E, Roldaan AC, Cheung D, Bel EH, Sterk PJ (2009) Exhaled breath profiling enables discrimination of chronic obstructive pulmonary disease and asthma. Am J Respir Crit Care Med 180:1076–1082
Di Natale C, Macagnano A, Martinelli E, Paolesse R, D'Arcangelo G, Roscioni C, Finazzi-Agrò A, D'Amico A (2003) Lung cancer identification by the analysis of breath by means of an array of non-selective gas sensors. Biosens Bioelectron 18:1209–1218
Machado R, Laskowski D, Deffenderfer O, Burch T, Zheng S, Mazzone PJ, Mekhail T, Jennings C, Stoller JK, Pyle J, Duncan J, Dweik RA, Erzurum SC (2005) Detection of lung cancer by sensor array analyses of exhaled breath. Am J Respir Crit Care Med 171:1286–1291
Fens N, de Nijs S, Peters S, Dekker T, Knobel HH, Vink TJ, Willard NP, Zwinderman AH, Krouwels FH, Janssen HG, Lutter R, Sterk PJ (2011) Exhaled air molecular profiling in relation to inflammatory subtype and activity in COPD. Eur Respir J 38:1301–1309
Pennazza G, Marchetti E, Santonico M, Mantini G, Mummolo S, Marzo G, Paolesse R, D'Amico A, Di Natale C (2008) Application of a quartz microbalance based gas sensor array for the study of halitosis. J Breath Res 2:017009
Persaud K (2005) Medical applications of odor-sensing devices. The Int J Lower Extrem Wounds 4:50–6
Kodogiannis V, Lygouras J, Tarzynski A, Chowdrey H (2008) Artificial odor discrimination system using electronic nose and neural networks for the identification of urinary tract infection. IEEE Trans Inf Technol Biomed 12:707–703
Covington J, Wedlake L, Andreyev J, Ouaret N, Thomas MG, Nwokolo CU, Bardhan KD, Arasaradnam RP (2012) The detection of patients at risk of gastrointestinal toxicity during pelvic radiotherapy by electronic nose and FAIMS: a pilot study. Sensors 12:13002–13018
Marco S, Gutierrez-Galvez A (2012) Signal and Data Processing for Machine Olfaction and Chemical Sensing: A Review. IEEE Sensors J 12:3174–3184
Gutierrez-Osuna R (2002) Pattern analysis for machine olfaction: a review. IEEE Sensors J 162(1-2):46–55
Gutiérrez, A, Marco, S, Eds. (2009). Biologically Inspired Signal Processing for Chemical Sensing (Studies in Computational Intelligence. Vol. 188. Springer
Persaud KC, Marco S, Gutierrez-Galvez A, Eds (20139, Neuromorphic Olfaction, in Frontiers in Neuroengineering, CRC Press
Ioannidis J (2005) Why most published research findings are false. PLoS Med 2:e124
Cornfield J (1966) Sequential trials, sequential analysis and the likelihood principle. Am Statistian 20:18–23
Defernez M, Kemsley EK (1997) The use and misuse of chemometrics for treating classification problems. TrAC Trends Anal Chem 16:216–221
Marco S, Ortega A, Pardo A, Samitier J (1998) Gas identification with tin oxide sensor array and self-organizing maps: adaptive correction of sensor drifts. Instrum Meas Trans IEEE 47:316–321
Padilla M, Perera A, Montoliu I, Chaudry A, Persaud K, Marco S, (2010) Drift compensation of gas sensor array data by orthogonal signal correction. Vol. 100, pp. 28–35
Ziyatdinov A, Marco S, Chaudry A, Persaud K, Caminal P, Perera A (2010) Drift compensation of gas sensor array data by common principal component analysis. Sensors Actuators B: Chem 146:460–465
Martinelli E, Magna G, Vito SD, Di Fuccio R, Di Francia F, Vergara A, Di Natale C (2013) An adaptive classification model based on the Artificial Inmune System for chemical sensor drift mitigation. Sensors Actuators B: Chem 177:1017–1026
Di Carlo S, Falasconi M, Sanchez E, Scionti A, Squillero G, Tonda A (2011) Increasing pattern recognition accuracy for chemical sensing by evolutionary based drift compensation. Pattern Recog Lett 32:1594–1603
Gutierrez-Osuna R (2000) Drift reduction for metal-oxide sensor arrays using canonical correlation regression and partial least squares. Proceedings of the 7th International Symposium on Olfaction and Electronic Nose. IOP Press, London, pp 147–152
Artursson T, Eklov T, Lundstrom I, Martensson P, Sjostrom M, Holmberg M (2000) Drift correction for gas sensors using multivariate methods. J Chemom 14:711–723
Knobloch H, Turner C, Spooner A, Chambers M (2009) Methodological variation in headspace analysis of liquid samples using electronic nose. Sensors Actuators B: Chem 139:353–360
Kuske M, Rubio R, Romain AC, Nicolas J, Marco S (2005) Fuzzy k-NN applied to molds detection. Sensors Actuators B: Chem 106(1):52–60
Kuske M, Padilla M, Romain AC, Nicolas J, Rubio R, Marco S (2006) Detection of diverse mold species growing on building materials by gas sensor arrays and pattern recognition. Sensors Actuators B: Chem 119(1):33–40
Adam G, Lemaigre S, Romain A-C, Nicolas J, Delfosse P (2013) Evaluation of an electronic nose for the early detection of organic overload of anaerobic digesters. Bioprocess Biosyst Eng 36:23–33
Quality Assurance of Pharmaceuticals: A Compendium of Guidelines and Related Materials (2007) World Health Organization
Gujral P, Amrhein M, Wise BM, Bonvin D (2010) Framework for explicit drift correction in multivariate calibration models. J Chemom 24(7/8):534–543
Nimsuk N, Nakamoto T (2008) Study on the odor classification in dynamical concentration robust against humidity and temperature changes. Sensors Actuators B: Chem 134(1):252–257
Kashwan, KR, Bhuyan M (2005) Robust electronic-nose system with temperature and humidity drift compensation for tea and spice flavor discrimination. IEEE Asian Conference on Sensors and the International Conference on new Techniques in Pharmaceutical and Biomedical Research, 5-7 September 2005, Kuala Lumpur, Malasya, pp. 154–158
Romain AC, Nicolas J, Andre P (1997) In situ measurement of olfactive pollution with inorganic semiconductors: limitations due to humidity and temperature influence. Semin Food Anal 2:283–296
Tomic O, Eklöv T, Kvaal K, Haugen JE (2004) Recalibration of a gas-sensor array system related to sensor replacement. Anal Chim Acta 512:199–206
Marco S, Pardo A, Davide FA, Di Natale C, D'Amico A, Hierlemann A, Mitrovics J, Schwweizer M, Weimar U, Göpel W (1996) Different strategies for the identification of gas sensing systems. Sensors Actuators B: Chem 34(1):213–223
Marco S, Samitier J, Morante JR (1995) A novel time-domain method to analyze multicomponent exponential transients. Meas Sci Technol 6(2):135
Samitier J, Lopez-Villegas JM, Marco S, Camara L, Pardo A, Ruiz O, Morante JR (1994) A new method to analyze signal transients in chemical sensors. Sensors Actuators B: Chem 18(1):308–312
Gutierrez-Osuna R, Nagle HT, Schiffman SS (1999) Transient response analysis of an electronic nose using multi-exponential models. Sensors Actuators B: Chem 61(1):170–182
Vilanova X, Llobet E, Alcubilla R, Sueiras JE, Correig X (1996) Analysis of the conductance transient in thick-film tin oxide gas sensors. Sensors Actuators B: Chem 31(3):175–180
Kearns M, Ron D (1999) Algorithmic stability and sanity-check bounds for leave-one-out cross-validation. Neural Comput 11:1427–1453
Steyerberg E, Harrell F, Borsboom G, Eijkemans MJ, Vergouwe Y, Habbema JD (2001) Internal validation of predictive models: efficiency of some procedures for logistic regression analysis. J Clin Epidemiol 54:774–781
Lindgren F, Hansen B, Karcher W, Sjöström M, Eriksson L (1996) Model validation by permutation tests: applications to variable selection. J Chemom 10:521–532
Ojala M, Garriga GC (2010) Permutation tests for studying classifier performance. J Mach Learn Res 11:1833–1863
Westerhuis JA, Hoefsloot HC, Smit S, Vis DJ, Smilde AK, van Velzen EJ, van Duijnhoven JPM, van Dorsten FA (2008) Assessment of PLSDA cross validation. Metabolomics 4(1):81–89
Rubingh CM, Bijlsma S, Derks EP, Bobeldijk I, Verheij ER, Kochhar S, Smilde AK (2006) Assessing the performance of statistical validation tools for megavariate metabolomics data. Metabolomics 2:53–61
Fawcett T (2006) An introduction to ROC analysis. Pattern Recog Lett 27(8):861–874
Lasko TA, Bhagwat JG, Zou KH, Ohno-Machado L (2005) The use of receiver operating characteristic curves in biomedical informatics. J Biomed Inf 38:4–415
Gardner M, Altman D (1986) Confidence intervals rather than p-values: estimation rather than hypothesis testing. Br Med J 292:746–750
Goodman S (1999) Toward evidence-based medical statistics. 1. The p-value fallacy. Ann. Intern Med 130:995–1004
Goodman S (1999) Toward evidence-based medical statistics. 2. The Bayes factor. Ann Int Med 130:1005–1013
Kass R, Raftery A (1995) Bayes factors. J Am Stat Assoc 90:773–795
Broadhurst DI, Kell DB (2006) Statistical strategies for avoiding false discoveries in metabolomics and related experiments. Metabolomics 2:171–196
Kenny LC, Dunn WB, Ellis DI, Myers J, Baker PN, Kell DB (2005) Novel biomarkers for pre-eclampsia detected using metabolomics and machine learning. Metabolomics 1:227–234
Ransohoff DF (2005) Lessons from controversy: ovarian cancer screening and serum proteomics. J Natl Cancer Inst 97:315–319
Petricoin EF, Ardekani AM, Hitt BA, Levine PJ, Fusaro VA, Steinberg SM, Mills GB, Simone C, Fishman DA, Kohn EC, Liotta LA (2002) Use of proteomic patterns in serum to identify ovarian cancer. Lancet 359(9306):572–577
West-Nørager M, Bro R, Marini F, Høgdall EV, Høgdall CK, Nedergaard L, Heegaard NH (2009) Feasibility of serodiagnosis of ovarian cancer by mass spectrometry. Anal Chem 81:1907–1913
Vander Heyden Y, Nijhuis A, Smeyers-Verbeke J, Vandeginste BG, Massart DL (2001) Guidance for robustness/ruggedness tests in method validation. J Pharm Biomed Anal 24:723–753
Zeaiter M, Roger J-M, Bellon-Maurel V, Rutledge DN (2004) Robustness of models developed by multivariate calibration. Part I. TrAC Trends Anal Chem 23:157–170
Zeaiter M, Roger J-M, Bellon-Maurel V (2005) Robustness of models developed by multivariate calibration. Part II. The influence of pre-processing methods. TrAC Trends Anal Chem 24:437–445
Lavine BK (2009) Validation of classifiers in comprehensive chemometrics 3:587–598
Esbensen KH, Geladi P (2010) Principles of proper validation: use and abuse of resampling for validation. J Chemom 34:168–187
Filzmoser P, Liebmann B, Varmuza K (2009) Repeated double cross-validation. J Chemom 23:160–171
Rousseeuw J, Debruyne M, Engelen S, Hubert M (2006) Robustness and outlier detection in chemometrics. Crit Rev Anal Chem 36:221–242
Acknowledgments
This work was funded by the Spanish Ministerio de Economía y Competitividad under the project TEC2011-26143. Santiago Marco is member of the consolidated research group SGR2009-0753 by the Generalitat de Catalunya.
Author information
Authors and Affiliations
Corresponding author
Additional information
Published in the topical collection Chemosensors and Chemoreception with guest editors Jong-Heun Lee and Hyung-Gi Byun.
Rights and permissions
About this article
Cite this article
Marco, S. The need for external validation in machine olfaction: emphasis on health-related applications. Anal Bioanal Chem 406, 3941–3956 (2014). https://doi.org/10.1007/s00216-014-7807-7
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00216-014-7807-7