Skip to main content
SpringerLink
Log in

Concluding Remarks

  • Chapter
  • First Online:
Deep Learning and Missing Data in Engineering Systems

Part of the book series: Studies in Big Data ((SBD,volume 48))

  • 1248 Accesses

Abstract

This chapter gives a summary of the topics that were discussed in this book. These include deep learning and missing data mechanisms and patterns, as well as machine learning approaches to solve the problem. It also describes advanced and novel approaches such as, deep learning with ant colony optimization algorithm and deep learning with cuckoo search algorithm, to name a few. Also presented in this book are experiments that show the impact of using lower dimensions and different numbers of hidden layers in the deep autoencoder networks.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 129.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Hardcover Book
USD 169.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  • Abdella, M., & Marwala, T. (2005). The use of genetic algorithms and neural networks to approximate missing data in database. In 3rd International Conference on Computational Cybernetics, ICCC (pp. 207–212). IEEE.

    Google Scholar 

  • Alex, K., Sutskever, I., & Hinton, G. E. (2012). Imagenet classification with deep convolutional neural networks. In F. Pereira, C. J. C. Burges, L. Bottou, & K. Q. Weinberger (Eds.), Advances in Neural Information Processing Systems 25 (pp. 1097–1105). Curran Associates, Inc., (last accessed: May 2016). [Online]. Available: http://papers.nips.cc/paper/4824-imagenet-classification-with-deep-convolutional-neural-networks.pdf.

  • Ali, F. A., & Mohamed, A. T. (2016). A hybrid cuckoo search algorithm with Nelder-Mead method for solving global optimization problems. SpringerPlus, 5(1), 473. Springer International Publishing.

    Article  Google Scholar 

  • Allison, P. D. (2000). Multiple imputation for missing data. Sociological Methods & Research, 28(3), 301–309.

    Article  Google Scholar 

  • Allison, P. D. (2002). Missing data. Thousand Oaks, CA: Sage.

    Google Scholar 

  • Dempster, A. P., Laird, N. M., & Rubin, D. B. (1997). Maximum likelihood from incomplete data via the EM algorithm. Journal of the Royal Statistics Society, 39(1), 1–38.

    MathSciNet  MATH  Google Scholar 

  • Deng, L., Li, J., Huang, J.-T., Yao, K., Yu, D., Seide, F., Seltzer, M., Zweig, G., He, X., & Williams, J. (2013). Recent advances in deep learning for speech research at Microsoft. IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) (pp. 8604–8608).

    Google Scholar 

  • Deng, L., & Yu, D. (2014). Deep learning: methods and applications. Foundations and Trends in Signal Processing, 7(3–4), 197–387.

    Article  MathSciNet  Google Scholar 

  • Dorigo, M., Maniezzo, V., & Colorni, A. (1991). Positive feedback as a search strategy. Tech. Rep.

    Google Scholar 

  • Feng, X., Zhang, Y., & Glass, J. (2014). Speech feature denoising and dereverberation via deep autoencoders for noisy reverberant speech recognition. In IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) (pp. 1759–1763).

    Google Scholar 

  • Grossberg, S. (2013). Recurrent neural networks. Scholarpedia, 8(2), 1888.

    Article  Google Scholar 

  • Gupta, E., & Saxena, A. (2016). Performance evaluation of antlion optimizer based regulator in automatic generation control of interconnected power system. Journal of Engineering, 2016, 1–14.

    Article  Google Scholar 

  • Hinton, G. (2010). A practical guide to training restricted Boltzmann machines. Momentum, 9(1), 926.

    Google Scholar 

  • Hinton, G. E., Osindero, S., & Teh, Y. W. (2006). A fast learning algorithm for deep belief nets. Neural Computation, 18(7), 1527–1554.

    Article  MathSciNet  Google Scholar 

  • Hung, H. L., Chao, C. C., Cheng, C. H., & Huang, Y. F. (2010). Invasive weed optimization method based blind multiuser detection for mc-cdma interference suppression over multipath fading channel. International Conference on Systems, Man and Cybernatics (SMC) (pp. 2145–2150).

    Google Scholar 

  • Kalousis, A., & Hilario, M. (2000). Supervised knowledge discovery from incomplete data. In Proceedings of the 2nd International Conference on Data Mining. Retrieved October 2016, from http://cui.unige.ch/AI-group/research/metal/Papers/missingvalues.ps.

  • Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2012). Imagenet classification with deep convolutional neural networks. In Advances in neural information processing systems (pp. 1097–1105).

    Google Scholar 

  • Le Roux, N., & Bengio, Y. (2008). Representational power of restricted Boltzmann machines and deep belief networks. Neural Computation, 20(6), 1631–1649.

    Article  MathSciNet  Google Scholar 

  • LeCun, Y., Bengio, Y., & Hinton, G. E. (2015). Deep learning. Nature, 521, 436–444.

    Article  Google Scholar 

  • Leke, C., & Marwala, T. (2016). Missing data estimation in high-dimensional datasets: A swarm intelligence-deep neural network approach. In International Conference in Swarm Intelligence (pp. 259–270). Springer International Publishing.

    Google Scholar 

  • Liu, X. J., Yi, H., & Ni, Z.-H. (2013). Application of ant colony optimization algorithm in process planning optimization. Journal of Intelligent Manufacturing, 24(1), 1–13.

    Article  Google Scholar 

  • Marwala, T. (2009). Computational intelligence for missing data imputation: Estimation and management knowledge optimization techniques. New York: Information Science Reference.

    Book  Google Scholar 

  • Marwala, T. (2001). Probabilistic fault identification using a committee of neural networks and vibration data. Journal of Aircraft, 38(1), 138–146.

    Article  Google Scholar 

  • Marwala, T., & Chakraverty, S. (2006). Fault classification in structures with incomplete measured data using autoassociative neural networks and genetic algorithm, 90(4).

    Google Scholar 

  • Marwala, T. (2013). Economic modelling using artificial intelligence methods. UK: Springer.

    Book  Google Scholar 

  • Marwala, T. (2015). Causality, correlation, and artificial intelligence for rational decision making. Singapore: World Scientific.

    Book  Google Scholar 

  • Mehrabian, A. R., & Lucas, C. (2006). A novel numerical optimization algorithm inspired from weed colonization. Ecological Informatics, 1, 355–366.

    Article  Google Scholar 

  • Mikolov, T., Karafiát, M., Burget, L., ÄŒernocký, J., & Khudanpur, S. (2010). Recurrent neural network based language model. In Eleventh Annual Conference of the International Speech Communication Association.

    Google Scholar 

  • Mirjalili, S. (2015). The ant lion optimizer. Advances in Engineering Software, 8, 80–98.

    Article  Google Scholar 

  • Monteiro, M. S. R., Fontes, D. B. M. M., & Fontes, F. A. C. C. (2012). Ant colony optimization: a literature survey. Universidade do Porto, Faculdade de Economia do Porto, FEP Working Papers. Retrieved January 2016, from http://EconPapers.repec.org/RePEc:por:fepwps:474.

  • Pérez, A., Dennis, R. J., Gil, J. F. A., Róndon, M. A., & López, A. (2002). Use of the mean, hot deck and multiple imputation techniques to predict outcome in intensive care unit patients in Colombia. Journal of Statistics in Medicine, 21(24), 3885–3896.

    Article  Google Scholar 

  • Polikar, R., De Pasquale, J., Mohammed, H. S., Brown, G., & Kuncheva, L. I. (2010). Learn++mf: A random subspace approach for the missing feature problem. Pattern Recognition, 43(11), 3817–3832.

    Article  Google Scholar 

  • Ramoni, M., & Sebastiani, P. (2001). Robust learning with missing data. Journal of Machine Learning, 45(2), 147–170.

    Article  Google Scholar 

  • Rubin, D. (1978). Multiple imputations in sample surveys-a phenomenological Bayesian approach to nonresponse. Proceedings of the Survey Research Methods Section of the American Statistical Association, 1, 20–34.

    Google Scholar 

  • Satheeshkumar, R., & Shivakumar, R. (2016). Ant lion optimization approach for load frequency control of multi-area interconnected power systems. Circuits and Systems, 7, 2357–2383.

    Article  Google Scholar 

  • Simard, P. Y., Steinkraus, D., & Platt, J. C. (2003). Best practices for convolutional neural networks applied to visual document analysis (pp. 958). IEEE.

    Google Scholar 

  • Teodoro C. B., Leandro d-. S. C., & Luiz L. (2012). Bat-Inspired Optimization Approach for the Brushless DC Wheel Motor Problem. Transactions on Magnetics, IEEE, 48(2), 947–950.

    Google Scholar 

  • Tieleman, T. (2008). Training restricted boltzmann machines using approximations to the likelihood gradient. In Proceedings of the 25th International Conference on Machine Learning, ser. New York, NY, USA: ACM, pp. 1064–1071. [Online]. Available: http://doi.acm.org/10.1145/1390156.1390290. Accessed May 2016.

  • Tremblay, M. C., Dutta, K., & Vandermeer, D. (2010). Using data mining techniques to discover bias patterns in missing data. Journal of Data and Information Quality, 2(1).

    Article  Google Scholar 

  • Twala, B., & Cartwright, M. (2010). Ensemble missing data techniques for software effort prediction. Intelligent Data Analysis, 14(3), 299–331.

    Article  Google Scholar 

  • Twala, B. (2009). An empirical comparison of techniques for handling incomplete data using decision trees. Applied Artificial Intelligence, 23(5), 373–405.

    Article  Google Scholar 

  • Twala, B. E. T. H., Jones, M. C., & Hand, D. J. (2008). Good methods for coping with missing data in decision trees. Pattern Recognition Letters, 29(7), 950–956.

    Article  Google Scholar 

  • Twala, B., & Phorah, M. (2010). Predicting incomplete gene microarray data with the use of supervised learning algorithms. Pattern Recognition Letters, 31, 2061–2069.

    Article  Google Scholar 

  • Vasanthakumar, S., Kumarappan, N., Arulraj, R. & Vigneysh, T. (2015). Cuckoo Search Algorithm based Environmental Economic Dispatch of Microgrid System with Distributed Generation. In International Conference on Smart Technologies and Management for Computing, Communication, Controls, Energy and Materials (ICSTM) (pp. 575–580). IEEE.

    Google Scholar 

  • Veenhuis, C. (2010). Binary invasive weed optimization. In Second World Congress on Nature and Biologically Inspired Computing (pp. 449–454).

    Google Scholar 

  • Wang, J., Zhou, B., & Zhou, S. (2016). An improved cuckoo search optimization algorithm for the problem of chaotic systems parameter estimation. Computational Intelligence and Neuroscience, 8.

    Google Scholar 

  • Yamany, W., Tharwat, A., Fawzy, M., Gaber, T., & Hassanien, A. E. (2015). A new multilayer perceptrons trainer based on ant lion optimization algorithm. In Fourth International Conference on Information Science and Industrial Applications (ISI) (pp. 40–45).

    Google Scholar 

  • Yang, X. S., & Debb, S. (2009). Cuckoo Search via Levy Flights. World Congress on Nature and Biologically Inspired Computing (NaBIC), 48(2), 210–214.

    Article  Google Scholar 

  • Yang, X.-S. (2010a). Firefly algorithm, Levy flights and global optimization. In M. Bramer, R. Ellis, M. Petridis (Eds.), Research and Development in Intelligent Systems XXVI (pp. 209–218). London: Springer.

    Google Scholar 

  • Yang, X. S. (2010b). Firefly algorithm, stochastic test functions and design optimisation. International Journal of Bio-Inspired Computation, 2(2), 78–84.

    Article  Google Scholar 

  • Yang, X. S. (2010c). A New Metaheuristic Bat-Inspired Algorithm. In Nature Inspired Cooperative Strategies for Optimization (NISCO), Studies in Computational Intelligence (pp. 65–74). Springer, Heidelberg.

    Chapter  Google Scholar 

  • Yang, X. S. (2011). Bat Algorithm for Multi-objective Optimization. International Journal of Bio-Inspired Computation, 3(5), 267–274.

    Article  Google Scholar 

  • Yang, X. S., Hosseini, S. S. S., & Gandomi, A. H. (2012). Firefly algorithm for solving non-convex economic dispatch problems with valve loading effect. Applied Soft Computing, 12(3), 1180–1186.

    Article  Google Scholar 

  • Yang, X. S., & Debb, S. (2014). Cuckoo search: recent advances and applications. Neural Computing and Applications, 24(1), 169–174.

    Article  Google Scholar 

  • Zecchina, A. C., Simpsona, A. R., Maiera, H. R., Leonarda, M., Roberts, A. J., & Berrisforda, M. J. (2006). Application of two ant colony optimisation algorithms to water distribution system optimisation. Mathematical and Computer Modelling, 44(5–6), 451–468.

    Article  Google Scholar 

  • Zhang, X. L., & Wu, J. (2013). Deep belief networks based voice activity detection. IEEE Transactions on Audio, Speech and Language Processing, 21(4), 697–710.

    Article  Google Scholar 

  • Zhou, Y., Xie, J., Li, L. & Ma, M. (2014). Cloud Model Bat Algorithm. The Scientific World Journal, 2014(237102), 1–11. Hindawi Publishing Corporation.

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Faculty of Engineering and Built Environment, University of Johannesburg, Auckland Park, South Africa

    Collins Achepsah Leke & Tshilidzi Marwala

Authors
  1. Collins Achepsah Leke
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Tshilidzi Marwala
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Collins Achepsah Leke .

Rights and permissions

Reprints and permissions

Copyright information

© 2019 Springer Nature Switzerland AG

About this chapter

Check for updates. Verify currency and authenticity via CrossMark

Cite this chapter

Leke, C.A., Marwala, T. (2019). Concluding Remarks. In: Deep Learning and Missing Data in Engineering Systems. Studies in Big Data, vol 48. Springer, Cham. https://doi.org/10.1007/978-3-030-01180-2_11

Download citation

Publish with us

Policies and ethics

Search

Navigation