Improving deep learning performance with missing values via deletion and compensation
- 45 Downloads
Missing values in a dataset is one of the most common difficulties in real applications. Many different techniques based on machine learning have been proposed in the literature to face this problem. In this work, the great representation capability of the stacked denoising auto-encoders is used to obtain a new method of imputating missing values based on two ideas: deletion and compensation. This method improves imputation performance by artificially deleting values in the input features and using them as targets in the training process. Nevertheless, although the deletion of samples is demonstrated to be really efficient, it may cause an imbalance between the distributions of the training and the test sets. In order to solve this issue, a compensation mechanism is proposed based on a slight modification of the error function to be optimized. Experiments over several datasets show that the deletion and compensation not only involve improvements in imputation but also in classification in comparison with other classical techniques.
KeywordsMissing values Imputation Classification Deep learning
The work of A. R. Figueiras-Vidal has been partly supported by Grant Macro-ADOBE (TEC 2015-67719-P, MINECO/FEDER&FSE). The work of J.L. Sancho-Gómez has been partly supported by Grant AES 2017 (PI17/00771, MINECO/FEDER).
- 4.Quinlan JR (1993) C4.5: programs for machine learning. Morgan-Kaufmann, BurlingtonGoogle Scholar
- 7.Delalleau O, Courville A, Bengio Y (2008) Gaussian mixtures with missing data: an efficient EM training algorithm. In: Proceeding of the computing research association conference, Snowbird, p 155Google Scholar
- 8.Ghahramani Z, Jordan MI (1994) Supervised learning from incomplete data via an EM approach. In: Cowan JD, Tesauro G, Alspector J (eds) Advances in neural information processing systems, vol 6. Morgan-Kaufmann, Burlington, pp 120–127Google Scholar
- 15.Latif BA, Mercier G (2010) Self-organizing maps. https://doi.org/10.5772/9178
- 18.Smola AJ, Vishwanathan SVN, Hofmann T (2005) Kernel methods for missing variables. In: Proceedings of the 10th international workshop on artificial intelligence and statistics, pp 325–332Google Scholar
- 20.Bengio Y, Lecun Y (2007) Scaling learning algorithms towards AI. MIT Press, CambridgeGoogle Scholar
- 25.Gondara L, Wang K (2017) Multiple imputation using deep denoising autoencoders. arXiv:1705.02737v2
- 26.Sánchez-Morales A, Sancho-Gómez JL, Figueiras-Vidal AR (2017) Values deletion to improve deep imputation processes. In: International work-conference on the interplay between natural and artificial computation, IWINAC 2017, Coruna, pp 240–246. https://doi.org/10.1007/978-3-319-59773-7-25
- 27.Vincent P, Larochelle H, Bengio Y, Manzagol PA (2008) Extracting and composing robust features with denoising autoencoders. In: Proceedings of the 25th international conference on machine learning, ICML’08. ACM, New York, pp 1096–1103. https://doi.org/10.1145/1390156.1390294
- 32.Lichman M (2013) UCI machine learning repository. http://archive.ics.uci.edu/ml
- 33.Delve: data for evaluating learning in valid experiments. https://www.cs.toronto.edu/~delve/data/datasets.html