Regression Task on Big Data with Convolutional Neural Network

  • Chang Liu
  • Ziheng Wang
  • Su Wu
  • Shaozhi WuEmail author
  • Kai Xiao
Conference paper
Part of the Advances in Intelligent Systems and Computing book series (AISC, volume 921)


As one of most widely utilized methods in deep learning, convolutional neural network (CNN) has been proven effective in many machine learning applications, especially in the areas of image understanding and computer vision. However, CNN is mainly used for applications with the approach of classification, while its usage for regression is not well-studied. In this work, we propose a strategy based on CNN with Visual Geometry Group Network (VGG) for image regression task. We have applied this method on images of MNIST processed with labels of continuous number. In our study, the original discrete classes of handwriting numbers are converted into float numbers with respect to normal distribution, thereby the traditional classification task in MNIST becomes a regression one. In our study, different loss functions such as Mean Absolute Error (MAE) and Log-cosh have been applied and validated. Final results generated by model trained with CNN with VGG with 10-fold cross-validation can be obtained, where MAE is less than 0.25, compared to the much higher error of around 3 with the use of other loss functions and convolutional layers. The significantly reduced error suggests the applicability of our proposed method.


Regression CNN VGG Loss function Normal distribution 


  1. 1.
    Hubel, D.H., Wiesel, T.N.: Early exploration of the visual cortex. Neuron 20(3), 401–412 (1998)CrossRefGoogle Scholar
  2. 2.
    Fukushima, K., Miyake, S.: Neocognitron: a self-organizing neural network model for a mechanism of visual pattern recognition. In: Amari, S., Arbib, M.A. (eds.) Competition and Cooperation in Neural Nets, pp. 267–285. Springer, Berlin, Heidelberg (1982)CrossRefGoogle Scholar
  3. 3.
    LeCun, Y., et al.: Gradient-based learning applied to document recognition. In: Proceedings of the IEEE 86.11, pp. 2278–2324 (1998)Google Scholar
  4. 4.
    Hecht-Nielsen, R.: Theory of the backpropagation neural network. In: Neural networks for perception, pp. 65–93 (1992)Google Scholar
  5. 5.
    Krizhevsky, A., Sutskever, I., Hinton, G.E.: Imagenet classification with deep convolutional neural networks. In: Advances in Neural Information Processing Systems (2012)Google Scholar
  6. 6.
    Deng, J., et al.: Imagenet large scale visual recognition competition 2012 (ILSVRC2012). See (2012)Google Scholar
  7. 7.
    Paoletti, M.E., et al.: A new deep convolutional neural network for fast hyperspectral image classification. ISPRS J. Photogramm. Remote Sens. 145, 120–147 (2018)CrossRefGoogle Scholar
  8. 8.
    LeCun, Y.: The MNIST database of handwritten digits (1998).
  9. 9.
    Simard, P.Y., Steinkraus, D., Platt, J.C.: Best practices for convolutional neural networks applied to visual document analysis. IEEE (2003)Google Scholar
  10. 10.
    Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556 (2014)
  11. 11.
    Samuel, A.L.: Some studies in machine learning using the game of checkers. IBM J. Res. Dev. 3(3), 210–229 (1959)MathSciNetCrossRefGoogle Scholar
  12. 12.
    Ioffe, S., Szegedy, C.: Batch normalization: Accelerating deep network training by reducing internal covariate shift. arXiv preprint arXiv:1502.03167 (2015)
  13. 13.
    Clevert, D.-A., Unterthiner, T., Hochreiter, S.: Fast and accurate deep network learning by exponential linear units (elus). arXiv preprint arXiv:1511.07289 (2015)
  14. 14.
    Kingma, D.P., Ba, J.: Adam: a method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014)
  15. 15.
    Kohavi, R.: A study of cross-validation and bootstrap for accuracy estimation and model selection. In: Ijcai, vol. 14., no. 2 (1995)Google Scholar
  16. 16.
    Hinton, G.E., et al.: Improving neural networks by preventing co-adaptation of feature detectors. arXiv preprint arXiv:1207.0580 (2012)
  17. 17.
    Srivastava, N., et al.: Dropout: a simple way to prevent neural networks from overfitting. J. Mach. Learn. Res. 15(1), 1929–1958 (2014)MathSciNetzbMATHGoogle Scholar
  18. 18.
    Nair, V., Hinton, G.E.: Rectified linear units improve restricted boltzmann machines. In: Proceedings of the 27th International Conference on Machine Learning (ICML-10) (2010)Google Scholar
  19. 19.
    Willmott, C.J., Matsuura, K.: Advantages of the mean absolute error (MAE) over the root mean square error (RMSE) in assessing average model performance. Clim. Res. 30(1), 79–82 (2005)CrossRefGoogle Scholar

Copyright information

© Springer Nature Switzerland AG 2020

Authors and Affiliations

  • Chang Liu
    • 1
  • Ziheng Wang
    • 2
  • Su Wu
    • 1
  • Shaozhi Wu
    • 1
    Email author
  • Kai Xiao
    • 3
  1. 1.University of Electronic Science and Technology of ChinaChengduChina
  2. 2.Tongji UniversityShanghaiChina
  3. 3.Shanghai Jiao Tong UniversityShanghaiChina

Personalised recommendations