Skip to main content
SpringerLink
Log in

Enhancing Image Recognition with Pre-Defined Convolutional Layers Based on PDEs

  • Published:
Programming and Computer Software Aims and scope Submit manuscript

Abstract

In this paper, we propose the use of 2D pre-defined convolutional layers based on parabolic and hyperbolic Partial Differential Equations (PDEs) to learn low-level abstractions. We demonstrate their effectiveness by adding them on top of ResNet, ResNext, DenseNet, and VGG architectures and performing numerical experiments on the CIFAR-10 dataset.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1.
Fig. 2.
Fig. 3.

Similar content being viewed by others

REFERENCES

  1. Bengio, Y., Learning deep architectures for AI, Found. Trends Mach. Learn., 2009, vol. 2, no. 1, pp. 1–155.

    Article  MATH  Google Scholar 

  2. LeCun, Y., Bengio, Y., and Hinton, G., Deep learning, Nature, 2015, vol. 521, p. 436.

    Article  Google Scholar 

  3. Aubert, G. and Kornprobst, P., Mathematical Problems in Image Processing: Partial Differential Equations and the Calculus of Variations, New York: Springer Verlag, 2002.

    Book  MATH  Google Scholar 

  4. He, K., Zhang, X., Ren, S., and Sun, J., Deep residual learning for image recognition, 2015. arXiv: 1512.03385v1 [cs.CV].

  5. Xie, S., Girshick, R., Dollár, P., Tu, Z., and He, K., Aggregated residual transformations for deep neural networks, Proc. Conf. on Computer Vision and Pattern Recognition, CVPR, Honolulu, 2017.

  6. Huang, G., Liu, Z., Maaten, L., and Weinberger, K.Q., Densely connected convolutional networks, Proc. Conf. on Computer Vision and Pattern Recognition, CVPR, Honolulu, 2017.

  7. Ruthotto, L. and Haber, E., Deep neural networks motivated by partial differential equations, 2019. arXiv:1804.04272v2 [cs.LG].

  8. Kag, A. and Saligrama, V., Condensing CNNs with partial differential equation, Proc. Computer Vision and Pattern Recognition Conf., New Orleans, 2022.

  9. Guo, P., Huang, K., and Xu, Z., Partial differential equations is all you need for generating neural architectures, 2021. arXiv:2103.08313v1 [cs.LG].

  10. Kundu, S., Nazemi, M., Pendram, M., Chugg, K.M., and Beerel, P.A., Pre-defined sparsity for low-complexity convolutional neural networks, IEEE Trans. Comput., 2020, vol. 99, p. 1–1.

    Article  MATH  Google Scholar 

  11. Ma, Y., Luo, Y., and Yang, Z., PCFNet: Deep neural network with predefined convolutional filters, Neurocomputing, 2020, vol. 382, no. 7.

  12. Richtmyer, R.D. and Morton, K.W., Difference Methods for Initial-Value Problems, 2nd ed., New York, London, Sydney: Interscience Publ. John Wiley & Sons, 1967, pp. 1–30.

    MATH  Google Scholar 

  13. Dong, S., Finite Difference Methods for the Hyperbolic Wave Partial Differential Equations, Princeton Plasma Physics Laboratory, 2006, pp. 1–17.

    Google Scholar 

  14. Krizhevsky, A., Learning multiple layers of features from tiny images, Sci. Tech. Rep., Univ. of Toronto, Dep. of Computer, 2009.

  15. Lee, C.Y., Xie, S., Gallagher, P., Zhang, Z., and Tu, Z., Deeply-supervised nets, 2014. arXiv:1409.5185.

  16. Simonyan, K. and Zisserman, A., Very deep convolutional networks for large-scale image recognition, Proc. Int. Conf. on Learning Representations, ICLR, San Diego, 2015.

  17. Gabor, D., Very theory of communication. Part 1: the analysis of information, J. Inst. Electr. Eng.-Part III: Radio Commun. Eng., 1946, vol. 93, no. 26, pp. 429–441.

    Google Scholar 

  18. Schmid, C., Constructing models for content-based image retrieval, Proc. IEEE Computer Society Conf. on Computer Vision and Pattern Recognition, CVPR, Kauai, HI, 2001.

  19. Gao, W., Zhang, X., Yang, L., and Liu, H., An improved sobel edge detection, Proc. 3rd IEEE Int. Conf. on Conference: Computer Science and Information Technology (ICCSIT), Chengdu, 2010, vol. 5, pp. 67–71.

Download references

ACKNOWLEDGMENTS

The work was supported by the Science Committee of RA, in the frames of the research project no. 23DP-1B017. Also, the authors wish to thank Professor Armenak Babayan and Tsolak Ghukasyan for their assistance and advice.

Author information

Authors and Affiliations

  1. Russian-Armenian University, Yerevan, Armenia

    V. R. Sahakyan & V. G. Melkonyan

  2. National Polytechnic University of Armenia, Yerevan, Armenia

    G. A. Gharagyozyan & A. S. Avetisyan

Authors
  1. V. R. Sahakyan
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. V. G. Melkonyan
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. G. A. Gharagyozyan
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. A. S. Avetisyan
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding authors

Correspondence to V. R. Sahakyan, V. G. Melkonyan, G. A. Gharagyozyan or A. S. Avetisyan.

Ethics declarations

The authors declare that they have no conflicts of interest.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Sahakyan, V.R., Melkonyan, V.G., Gharagyozyan, G.A. et al. Enhancing Image Recognition with Pre-Defined Convolutional Layers Based on PDEs. Program Comput Soft 49, 192–197 (2023). https://doi.org/10.1134/S0361768823030088

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1134/S0361768823030088

Search

Navigation