Advertisement

An improved residual network model for image recognition using a combination of snapshot ensembles and the cutout technique

  • Chakkrit TermritthikunEmail author
  • Yeshi Jamtsho
  • Paisarn Muneesawang
Article
  • 41 Downloads

Abstract

NUF-Net (Naresuan University and Fiber One Public Company Limited Network) is a new and improved Convolutional Neural Network (CNN) model based on the previously developed NU-LiteNet model. Improvements in accuracy were achieved by adding the identity mapping technique of the ResNet model and incorporating Snapshot Ensembles and the Cutout technique into the NU-LiteNet model. We modified the structure of the convolution layers by changing any filters of a size larger than 3 ×3, into a 3 ×3 filter, thereby significantly reducing processing time and reducing the error rate. To test the effectiveness of our modifications, we developed 10 variations of the NUF-Net-Residual model, one of which, termed NUF-Net-Residual-102, achieved significantly lower error rates than both ResNet and Wide-ResNet when using CIFAR-10, CIFAR-100 and Tiny-ImageNet datasets. The relative error rates were 2.94% for CIFAR-10, 17.57% for CIFAR-100 and 29.57% for Tiny-ImageNet. As well, NUF-Net-Residual-102 achieved a model parameter size of 31.65 million which is a lower value than for Wide-ResNet-32 (46.16 million), although higher than ResNet-1202 (19.42 million).

Keywords

Deep learning Image recognition Convolutional neural networks ResNet Snapshot ensembles Cutout technique 

Notes

Acknowledgements

The authors would like to acknowledge the financial support from the Thailand Research Fund through the Royal Golden Jubilee Ph.D. Program (Grant No. PHD/0101/2559). We would also like to extend our appreciation to Mr. Roy I. Morien of the Naresuan University Graduate School for his assistance in editing the English grammar and expression in the paper.

References

  1. 1.
    Cheng G, Yang C, Yao X, Guo L, Han J (2018) When deep learning meets metric learning: Remote sensing image scene classification via learning discriminative CNNs. IEEE transactions on geoscience and remote sensing 56(5):2811–2821CrossRefGoogle Scholar
  2. 2.
    Chollet F (2017) Xception: Deep learning with depthwise separable convolutions. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 1251–1258Google Scholar
  3. 3.
    DeVries T, Taylor GW (2017) Improved regularization of convolutional neural networks with cutout. arXiv preprint arXiv:1708.04552
  4. 4.
    El-Rahiem BA, Ahmed MAO, Reyad O, El-Rahaman HA, Amin M, El-Samie FA (2019) An efficient deep convolutional neural network for visual image classification. In: International conference on advanced machine learning technologies and applications. Springer, pp 23–31Google Scholar
  5. 5.
    Gad R, Talha M, El-Latif AAA, Zorkany M, Ayman ES, Nawal EF, Muhammad G (2018) Iris recognition using multi-algorithmic approaches for cognitive internet of things (ciot) framework. Futur Gener Comput Syst 89:178–191CrossRefGoogle Scholar
  6. 6.
    Goodfellow IJ, Warde-Farley D, Mirza M, Courville A, Bengio Y (2013) Maxout networks. arXiv preprint arXiv:1302.4389
  7. 7.
    Han D, Kim J, Kim J (2017) Deep pyramidal residual networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 5927–5935Google Scholar
  8. 8.
    He K, Zhang X, Ren S, Sun J (2016) Identity mappings in deep residual networks. In: European conference on computer vision. Springer, pp 630–645Google Scholar
  9. 9.
    He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 770–778Google Scholar
  10. 10.
    Huang G, Li Y, Pleiss G, Liu Z, Hopcroft JE, Weinberger KQ (2017) Snapshot ensembles: Train 1, get m for free. arXiv preprint arXiv:1704.00109
  11. 11.
    Huang G, Liu Z, Van Der Maaten L, Weinberger KQ (2017) Densely connected convolutional networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 4700–4708Google Scholar
  12. 12.
    Iandola FN, Han S, Moskewicz MW, Ashraf K, Dally WJ, Keutzer K (2016) Squeezenet: Alexnet-level accuracy with 50x fewer parameters and 0.5 mb model size. arXiv preprint arXiv:1602.07360
  13. 13.
    Ioffe S, Szegedy C (2015) Batch normalization: Accelerating deep network training by reducing internal covariate shift. arXiv preprint arXiv:1502.03167
  14. 14.
    Kim Y, Hwang I, Cho NI (2017) A new convolutional network-in-network structure and its applications in skin detection, semantic segmentation, and artifact reduction. arXiv preprint arXiv:1701.06190
  15. 15.
    Krizhevsky A, Hinton G, et al. (2009) Learning multiple layers of features from tiny images. Tech. rep., CiteseerGoogle Scholar
  16. 16.
    Krizhevsky A, Sutskever I, Hinton GE (2012) Imagenet classification with deep convolutional neural networks. In: Advances in neural information processing systems, pp 1097–1105Google Scholar
  17. 17.
    Lee CY, Xie S, Gallagher P, Zhang Z, Tu Z (2015) Deeply-supervised nets. In: Artificial intelligence and statistics, pp 562–570Google Scholar
  18. 18.
    Li K, Cheng G, Bu S, You X (2017) Rotation-insensitive and context augmented object detection in remote sensing images. IEEE Trans Geosci Remote Sens 56(4):2337–2348CrossRefGoogle Scholar
  19. 19.
    Lin M, Chen Q, Yan S (2013) Network in network. arXiv preprint arXiv:1312.4400
  20. 20.
    Loshchilov I, Hutter F (2016) Sgdr: Stochastic gradient descent with warm restarts. arXiv preprint arXiv:1608.03983
  21. 21.
    Peng J, Li Q, El-Latif AAA, Wang N, Niu X (2013) Finger vein recognition with gabor wavelets and local binary patterns. IEICE Trans Inf Sys 96 (8):1886–1889CrossRefGoogle Scholar
  22. 22.
    Powers DM (2011) Evaluation: from precision, recall and f-measure to roc, informedness, markedness and correlationGoogle Scholar
  23. 23.
    Romero A, Ballas N, Kahou SE, Chassang A, Gatta C, Bengio Y (2014) Fitnets: Hints for thin deep nets. arXiv preprint arXiv:1412.6550
  24. 24.
    Shin HC, Roth HR, Gao M, Lu L, Xu Z, Nogues I, Yao J, Mollura D, Summers RM (2016) Deep convolutional neural networks for computeraided detection: Cnn architectures, dataset characteristics and transfer learning. IEEE Trans Med Imaging 35(5):1285–1298CrossRefGoogle Scholar
  25. 25.
    Srivastava RK, Greff K, Schmidhuber J (2015) Highway networks. arXiv preprint arXiv:1505.00387
  26. 26.
    Szegedy C, Ioffe S, Vanhoucke V, Alemi AA (2017) Inception-v4, inception-resnet and the impact of residual connections on learning. In: Thirty-first AAAI conference on artificial intelligenceGoogle Scholar
  27. 27.
    Szegedy C, Liu W, Jia Y, Sermanet P, Reed S, Anguelov D, Erhan D, Vanhoucke V, Rabinovich A (2015) Going deeper with convolutions. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 1–9Google Scholar
  28. 28.
    Szegedy C, Vanhoucke V, Ioffe S, Shlens J, Wojna Z (2016) .. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 2818–2826Google Scholar
  29. 29.
    Termritthikun C, Jamtsho Y, Muneesawang P (2019) On-device facial verification using NUF-Net model of deep learning. Engineering Applications of Artificial Intelligence 85:579–589CrossRefGoogle Scholar
  30. 30.
    Termritthikun C, Kanprachar S (2017) Accuracy improvement of thai food image recognition using deep convolutional neural networks. In: 2017 International electrical engineering congress (iEECON). IEEE, pp 1–4Google Scholar
  31. 31.
    Termritthikun C, Kanprachar S (2018) Nu-resnet: Deep residual networks for thai food image recognition. Journal of Telecommunication, Electronic and Computer Engineering (JTEC) 10(1–4):29–33Google Scholar
  32. 32.
    Termritthikun C, Kanprachar S, Muneesawang P (2019) NU-LiteNet: Mobile Landmark Recognition using Convolutional Neural Networks. ECTI Transactions on Computer and Information Technology (ECTI-CIT) 13(1):21–28Google Scholar
  33. 33.
    Termritthikun C, Muneesawang P, Kanprachar S (2017) Nu-innet: Thai food image recognition using convolutional neural networks on smartphone. Journal of Telecommunication, Electronic and Computer Engineering (JTEC) 9(2-6):63–67Google Scholar
  34. 34.
    Wang N, Li Q, El-Latif AAA, Peng J, Niu X (2014) An enhanced thermal face recognition method based on multiscale complex fusion for gabor coefficients. Multimed Tools Appl 72(3):2339–2358CrossRefGoogle Scholar
  35. 35.
    Wang N, Li Q, El-Latif AAA, Zhang T, Niu X (2014) Toward accurate localization and high recognition performance for noisy iris images. Multimed Tools Appl 71(3):1411–1430CrossRefGoogle Scholar
  36. 36.
    Xie S, Girshick R, Dollár P, Tu Z, He K (2017) Aggregated residual transformations for deep neural networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 1492–1500Google Scholar
  37. 37.
    Zagoruyko S, Komodakis N (2016) Wide residual networks. arXiv preprint arXiv:1605.07146
  38. 38.
    Zagoruyko S, Komodakis N (2017) Diracnets: Training very deep neural networks without skipconnections. arXiv preprint arXiv:1706.00388
  39. 39.
    Zhang K, Zhang Z, Li Z, Qiao Y (2016) Joint face detection and alignment using multitask cascaded convolutional networks. IEEE Signal Process Lett 23(10):1499–1503CrossRefGoogle Scholar
  40. 40.
    Zhong Z, Zheng L, Kang G, Li S, Yang Y (2017) Random erasing data augmentation. arXiv preprint arXiv:1708.04896

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2019

Authors and Affiliations

  1. 1.Faculty of Engineering, Department of Electrical and Computer EngineeringNaresuan UniversityPhitsanulokThailand

Personalised recommendations