Skip to main content
SpringerLink
Log in

Canonical Retina-to-Cortex Vision Model Ready for Automatic Differentiation

  • Conference paper
  • First Online:
Brain Informatics (BI 2020)

Part of the book series: Lecture Notes in Computer Science ((LNAI,volume 12241))

Included in the following conference series:

  • 1268 Accesses

Abstract

Canonical vision models of the retina-to-V1 cortex pathway consist of cascades of several Linear+Nonlinear layers. In this setting, parameter tuning is the key to obtain a sensible behavior when putting all these multiple layers to work together. Conventional tuning of these neural models very much depends on the explicit computation of the derivatives of the response with regard to the parameters. And, in general, this is not an easy task. Automatic differentiation is a tool developed by the deep learning community to solve similar problems without the need of explicit computation of the analytic derivatives. Therefore, implementations of canonical visual neuroscience models that are ready to be used in an automatic differentiation environment are extremely needed nowadays. In this work we introduce a Python implementation of a standard multi-layer model for the retina-to-V1 pathway. Results show that the proposed default parameters reproduce image distortion psychophysics. More interestingly, given the python implementation, the parameters of this visual model are ready to be optimized with automatic differentiation tools for alternative goals.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    https://github.com/sinodanish/BioMulti-L-NL-Model.

References

  1. Ballé, J., Laparra, V., Simoncelli, E.P.: End-to-end optimized image compression. In: 5th International Conference on Learning Representations, ICLR 2017 (2017)

    Google Scholar 

  2. Baydin, A., Pearlmutter, B., Radul, A., Siskind, J.: Automatic differentiation in machine learning: a survey. CoRR abs/1502.05767 (2015). http://arxiv.org/abs/1502.05767

  3. Cai, D., DeAngelis, G., Freeman, R.: Spatiotemporal receptive field organization in the LGN of cats and kittens. J. Neurophysiol. 78(2), 1045–1061 (1997)

    Article  Google Scholar 

  4. Campbell, F., Robson, J.: Application of Fourier analysis to the visibility of gratings. J. Physiol. 197, 551–566 (1968)

    Article  Google Scholar 

  5. Capilla, P., Malo, J., Luque, M., Artigas, J.: Colour representation spaces at different physiological levels: a comparative analysis. J. Opt. 29(5), 324 (1998)

    Article  Google Scholar 

  6. Carandini, M., Heeger, D.: Summation and division by neurons in visual cortex. Science 264(5163), 1333–1336 (1994)

    Article  Google Scholar 

  7. Carandini, M., Heeger, D.J.: Normalization as a canonical neural computation. Nat. Rev. Neurosci. 13(1), 51–62 (2012)

    Article  Google Scholar 

  8. Fairchild, M.: Color Appearance Models. The Wiley-IS&T Series in Imaging Science and Technology, Wiley, Sussex, UK (2013)

    Google Scholar 

  9. Gardner, J., Sun, P., Waggoner, R., Ueno, K., Tanaka, K., Cheng, K.: Contrast adaptation and representation in human early visual cortex. Neuron 47, 607–20 (2005). https://doi.org/10.1016/j.neuron.2005.07.016

  10. Gatys, L.A., Ecker, A.S., Bethge, M.: A neural algorithm of artistic style (2015). http://arxiv.org/abs/1508.06576

  11. Gomez-Villa, A., Bertalmio, M., Malo, J.: Visual information flow in Wilson-Cowan networks. J. Neurophysiol. (2020). https://doi.org/10.1152/jn.0487.2019

    Article  Google Scholar 

  12. Goodfellow, I., Bengio, Y., Courville, A.: Deep Learning. MIT Press, Cambridge, MA (2016). http://www.deeplearningbook.org

  13. Günthner, M.F., et al.: Learning divisive normalization in primary visual cortex. bioRxiv (2019). https://doi.org/10.1101/767285

  14. Heeger, D.J.: Normalization of cell responses in cat striate cortex. Vis. Neurosci. 9(2), 181–197 (1992)

    Article  MathSciNet  Google Scholar 

  15. Hepburn, A., Laparra, V., Malo, J., McConville, R., Santos, R.: PerceptNet: a human visual system inspired neural net for estimating perceptual distance. In: Proceedings of IEEE ICIP (2020). https://arxiv.org/abs/1910.12548

  16. Krizhevsky, A., Sutskever, I., Hinton, G.: ImageNet classification with deep convolutional neural networks. In: 25th Neural Information Processing System, NIPS 2012, pp. 1097–1105, Curran Associates Inc., USA (2012)

    Google Scholar 

  17. Laparra, V., Berardino, A., Balle, J., Simoncelli, E.: Perceptually optimized image rendering. JOSA A 34(9), 1511–1525 (2017)

    Article  Google Scholar 

  18. Laparra, V., Muñoz-Marí, J., Malo, J.: Divisive normalization image quality metric revisited. JOSA A 27(4), 852–864 (2010)

    Article  Google Scholar 

  19. Malo, J., Simoncelli, E.: Geometrical and statistical properties of vision models obtained via maximum differentiation. In: SPIE Electronic Imaging, pp. 93940L–93940L. International Society for Optics and Photonics (2015)

    Google Scholar 

  20. Martinez, M., Bertalmío, M., Malo, J.: In praise of artifice reloaded: caution with natural image databases in modeling vision. Front. Neurosci. (2019). https://doi.org/10.3389/fnins.2019.00008

    Article  Google Scholar 

  21. Martinez-Garcia, M., Cyriac, P., Batard, T., Bertalmío, M., Malo, J.: Derivatives and inverse of cascaded linear+nonlinear neural models. PLOS ONE 13(10), 1–49 (2018). https://doi.org/10.1371/journal.pone.0201326

  22. Mullen, K.T.: The CSF of human colour vision to red-green and yellow-blue chromatic gratings. J. Physiol. 359, 381–400 (1985)

    Article  Google Scholar 

  23. Pestilli, F., Carrasco, M., Heeger, D., Gardner, J.: Attentional enhancement via selection and pooling of early sensory responses in human visual cortex. Neuron 72, 832–46 (2011). https://doi.org/10.1016/j.neuron.2011.09.025

  24. Ponomarenko, N., Carli, M., Lukin, V., Egiazarian, K., Astola, J., Battisti, F.: Color image database for evaluation of image quality metrics. In: Proceedings of International Workshop on Multimedia Signal Processing, pp. 403–408 (2008)

    Google Scholar 

  25. Ringach, D.: Spatial structure and symmetry of simple-cell receptive fields in macaque primary visual cortex. J. Neurophysiol. 88(1), 455–463 (2002)

    Article  Google Scholar 

  26. Schutt, H.H., Wichmann, F.A.: An image-computable psychophysical spatial vision model. J. Vis. 17(12), 12 (2017). https://doi.org/10.1167/17.12.12

  27. Shapley, R., Hawken, M.: Color in the cortex: single- and double-opponent cells. Vis. Res. 51(7), 701–717 (2011)

    Article  Google Scholar 

  28. Simoncelli, E.P., Freeman, W.T., Adelson, E.H., Heeger, D.J.: Shiftable multi-scale transforms. IEEE Trans. Inf. Theory 38(2), 587–607 (1992). https://doi.org/10.1109/18.119725. Special Issue on Wavelets

    Article  Google Scholar 

  29. Stockman, A., Brainard, D.: Color vision mechanisms. In: OSA Handbook of Optics, 3rd edn., pp. 147–152. McGraw-Hill, NY (2010)

    Google Scholar 

  30. Stockman, A., Sharpe, L.: The spectral sensitivities of the middle- and long-wavelength-sensitive cones derived from measurements in observers of known genotype. Vis. Res. 40(13), 1711–1737 (2000)

    Article  Google Scholar 

  31. Wang, Z., Bovik, A.C., Sheikh, H.R., Simoncelli, E.P.: Image quality assessment: from error visibility to structural similarity. IEEE Trans. Image Proc. 13(4), 600–612 (2004)

    Article  Google Scholar 

  32. Wang, Z., Simoncelli, E.: SSIM results in TID2008 (2011). http://cns.nyu.edu/ lcv/ssim

  33. Wang, Z., Simoncelli, E.P.: Maximum differentiation (MAD) competition: a methodology for comparing computational models of perceptual quantities. J. Vis. 8(12), 8 (2008)

    Article  Google Scholar 

  34. Watson, A.B., Malo, J.: Video quality measures based on the standard spatial observer. In: 2002 International Conference on Image Processing, Proceedings, vol. 3, pp. III-41. IEEE (2002)

    Google Scholar 

  35. Watson, A.B., Ramirez, C.: A standard observer for spatial vision based on modelfest dataset (1999)

    Google Scholar 

  36. Watson, A.B., Solomon, J.A.: Model of visual contrast gain control and pattern masking. JOSA A 14(9), 2379–2391 (1997)

    Article  Google Scholar 

  37. Whitman, A., Obituary, van der Rohe, M.: Leader of Modern Architecture. The New York Times (1969)

    Google Scholar 

  38. Wilson, H.R., Cowan, J.D.: A mathematical theory of the functional dynamics of cortical and thalamic nervous tissue. Kybernetik 13(2), 55–80 (1973)

    Article  Google Scholar 

  39. Zenger-Landolt, B., Heeger, D.: Response suppression in v1 agrees with psychophysics of surround masking. J. Neurosci. Off. J. Soc. Neurosci. 23, 6884–6893 (2003). https://doi.org/10.1523/JNEUROSCI.23-17-06884.2003

  40. Zhang, R., Isola, P., Efros, A., Shechtman, E., Wang, O.: The unreasonable effectiveness of deep features as a perceptual metric. In: Proceedings of IEEE CVPR, pp. 586–595 (2018)

    Google Scholar 

Download references

Acknowledgment

Thanks to all of the open-source contributors in the open-source community. We would like to thank the reviewers for their thoughtful comments and efforts towards improving our manuscript. Finally, we also want to thank all of the doctors in the world who keep all of us safe during the COVID 19 epidemic.

This work was partially funded by the Spanish Government through the grant MINECO DPI2017-89867 and by the Generalitat Valenciana through the grant GrisoliaP/2019/035.

Author information

Authors and Affiliations

  1. Image and Signal Processing Lab, University of Valencia, 46980, Valencia, Spain

    Qiang Li & Jesus Malo

Authors
  1. Qiang Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jesus Malo
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Qiang Li .

Editor information

Editors and Affiliations

  1. Nottingham Trent University, Nottingham, UK

    Mufti Mahmud

  2. University of Padua, Padua, Italy

    Stefano Vassanelli

  3. Jahangirnagar University, Dhaka, Bangladesh

    M. Shamim Kaiser

  4. Maebashi Institute of Technology, Maebashi, Japan

    Ning Zhong

Rights and permissions

Reprints and permissions

Copyright information

© 2020 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Li, Q., Malo, J. (2020). Canonical Retina-to-Cortex Vision Model Ready for Automatic Differentiation. In: Mahmud, M., Vassanelli, S., Kaiser, M.S., Zhong, N. (eds) Brain Informatics. BI 2020. Lecture Notes in Computer Science(), vol 12241. Springer, Cham. https://doi.org/10.1007/978-3-030-59277-6_30

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-59277-6_30

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-59276-9

  • Online ISBN: 978-3-030-59277-6

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation