Skip to main content

Grounding Psychological Shape Space in Convolutional Neural Networks

  • Conference paper
  • First Online:
Software Engineering and Formal Methods. SEFM 2021 Collocated Workshops (SEFM 2021)

Abstract

Shape information is crucial for human perception and cognition, and should therefore also play a role in cognitive AI systems. We employ the interdisciplinary framework of conceptual spaces, which proposes a geometric representation of conceptual knowledge through low-dimensional interpretable similarity spaces. These similarity spaces are often based on psychological dissimilarity ratings for a small set of stimuli, which are then transformed into a spatial representation by a technique called multidimensional scaling. Unfortunately, this approach is incapable of generalizing to novel stimuli. In this paper, we use convolutional neural networks to learn a generalizable mapping between perceptual inputs (pixels of grayscale line drawings) and a recently proposed psychological similarity space for the shape domain. We investigate different network architectures (classification network vs. autoencoder) and different training regimes (transfer learning vs. multi-task learning). Our results indicate that a classification-based multi-task learning scenario yields the best results, but that its performance is relatively sensitive to the dimensionality of the similarity space.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Subscribe and save

Springer+ Basic
EUR 32.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or Ebook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

Notes

  1. 1.

    See https://github.com/lbechberger/LearningPsychologicalSpaces/.

  2. 2.

    Since our autoencoder receives a corrupted image, but needs to reconstruct the uncorrupted original, it is a so-called denoising autoencoder [67].

References

  1. Attarian, I.M., Roads, B.D., Mozer, M.C.: Transforming neural network visual representations to predict human judgments of similarity. In: NeurIPS 2020 Workshop SVRHM (2020). https://openreview.net/forum?id=8wNMPXWK5VX

  2. Attneave, F.: Dimensions of similarity. Am. J. Psychol. 63(4), 516–556 (1950). https://doi.org/10.2307/1418869

    Article  Google Scholar 

  3. Baker, N., Lu, H., Erlikhman, G., Kellman, P.J.: Deep convolutional networks do not classify based on global object shape. PLOS Comput. Biol. 14(12), 1–43 (2018). https://doi.org/10.1371/journal.pcbi.1006613

    Article  Google Scholar 

  4. Bar, M.: A cortical mechanism for triggering top-down facilitation in visual object recognition. J. Cogn. Neurosci. 15(4), 600–609 (2003). https://doi.org/10.1162/089892903321662976

    Article  Google Scholar 

  5. Battleday, R.M., Peterson, J.C., Griffiths, T.L.: Capturing human categorization of natural images by combining deep networks and cognitive models. Nat. Commun. 11(1), 1–14 (2020)

    Article  Google Scholar 

  6. Battleday, R.M., Peterson, J.C., Griffiths, T.L.: From convolutional neural networks to models of higher-level cognition (and back again). Ann. N. Y. Acad. Sci. (2021). https://doi.org/10.1111/nyas.14593

  7. Bechberger, L.: lbechberger/LearningPsychologicalSpaces v1.5: machine learning study with CNNs on shapes data, September 2021. https://doi.org/10.5281/zenodo.5524374

  8. Bechberger, L., Kühnberger, K.-U.: Generalizing psychological similarity spaces to unseen stimuli – combining multidimensional scaling with artificial neural networks. In: Bechberger, L., Kühnberger, K.-U., Liu, M. (eds.) Concepts in Action. LCM, vol. 9, pp. 11–36. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-69823-2_2

    Chapter  Google Scholar 

  9. Bechberger, L., Scheibel, M.: Analyzing psychological similarity spaces for shapes. In: Alam, M., Braun, T., Yun, B. (eds.) ICCS 2020. LNCS (LNAI), vol. 12277, pp. 204–207. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-57855-8_16

    Chapter  Google Scholar 

  10. Bechberger, L., Scheibel, M.: Representing complex shapes with conceptual spaces. In: Second International Workshop ‘Concepts in Action: Representation, Learning, and Application’ (CARLA 2020) (2020). https://openreview.net/forum?id=OhFQNQicgXy

  11. Bechberger, L., Scheibel, M.: Modeling the holistic perception of everyday object shapes with conceptual spaces (in preparation)

    Google Scholar 

  12. Op de Beeck, H.P., Torfs, K., Wagemans, J.: Perceived shape similarity among unfamiliar objects and the organization of the human object vision pathway. J. Neurosci. 28(40), 10111–10123 (2008). https://doi.org/10.1523/JNEUROSCI.2511-08.2008

  13. Biederman, I.: Recognition-by-components: a theory of human image understanding. Psychol. Rev. 94(2), 115–147 (1987)

    Article  Google Scholar 

  14. Borg, I., Groenen, J.F.: Modern Multidimensional Scaling: Theory and Applications. Springer Series in Statistics, 2nd edn. Springer, New York (2005). https://doi.org/10.1007/0-387-28981-X

    Book  MATH  Google Scholar 

  15. Chella, A., Frixione, M., Gaglio, S.: Conceptual spaces for computer vision representations. Artif. Intell. Rev. 16(2), 137–152 (2001). https://doi.org/10.1023/a:1011658027344

    Article  MATH  Google Scholar 

  16. Deng, J., Dong, W., Socher, R., Li, L.J., Li, K., Fei-Fei, L.: ImageNet: a large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). https://doi.org/10.1109/CVPR.2009.5206848

  17. Derrac, J., Schockaert, S.: Inducing semantic relations from conceptual spaces: a data-driven approach to plausible reasoning. Artif. Intell. 228, 66–94 (2015). https://doi.org/10.1016/j.artint.2015.07.002

    Article  MathSciNet  MATH  Google Scholar 

  18. Diesendruck, G., Bloom, P.: How specific is the shape bias? Child Dev. 74(1), 168–178 (2003). https://doi.org/10.1111/1467-8624.00528

    Article  Google Scholar 

  19. Dosovitskiy, A., Brox, T.: Inverting visual representations with convolutional networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2016)

    Google Scholar 

  20. Dosovitskiy, A., Tobias Springenberg, J., Brox, T.: Learning to generate chairs with convolutional neural networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2015)

    Google Scholar 

  21. Eitz, M., Hays, J., Alexa, M.: How do humans sketch objects? ACM Trans. Graph. 31(4), 1–10 (2012). https://doi.org/10.1145/2185520.2185540

    Article  Google Scholar 

  22. Erdogan, G., Jacobs, R.A.: Visual shape perception as Bayesian inference of 3D object-centered shape representations. Psychol. Rev. 124(6), 740–761 (2017)

    Article  Google Scholar 

  23. Garcez, A.D., et al.: Neural-symbolic learning and reasoning: contributions and challenges. In: AAAI 2015 Spring Symposium on Knowledge Representation and Reasoning: Integrating Symbolic and Neural Approaches (2015)

    Google Scholar 

  24. Gärdenfors, P.: Conceptual Spaces: The Geometry of Thought. MIT Press, Cambridge (2000)

    Book  Google Scholar 

  25. Geirhos, R., Rubisch, P., Michaelis, C., Bethge, M., Wichmann, F.A., Brendel, W.: ImageNet-trained CNNs are biased towards texture; increasing shape bias improves accuracy and robustness. In: International Conference on Learning Representations (2019). https://openreview.net/forum?id=Bygh9j09KX

  26. Goodfellow, I., Bengio, Y., Courville, A.: Deep Learning. MIT Press (2016). http://www.deeplearningbook.org

  27. Harnad, S.: The symbol grounding problem. Phys. D 42(1–3), 335–346 (1990). https://doi.org/10.1016/0167-2789(90)90087-6

    Article  Google Scholar 

  28. Hebart, M.N., Zheng, C.Y., Pereira, F., Baker, C.I.: Revealing the multidimensional mental representations of natural objects underlying human similarity judgements. Nat. Hum. Behav. (2020). https://doi.org/10.1038/s41562-020-00951-3

    Article  Google Scholar 

  29. Horst, J.S., Hout, M.C.: The novel object and unusual name (NOUN) database: a collection of novel images for use in experimental research. Behav. Res. Methods 48(4), 1393–1409 (2015). https://doi.org/10.3758/s13428-015-0647-3

    Article  Google Scholar 

  30. Huang, L.: Space of preattentive shape features. J. Vis. 20(4), 10–10 (2020). https://doi.org/10.1167/jov.20.4.10

    Article  Google Scholar 

  31. Hubel, D.H., Wiesel, T.N.: Receptive fields of single neurones in the cat’s striate cortex. J. Physiol. 148(3), 574–591 (1959). https://doi.org/10.1113/jphysiol.1959.sp006308

    Article  Google Scholar 

  32. Jha, A., Peterson, J., Griffiths, T.: Extracting low-dimensional psychological representations from convolutional neural networks. In: Proceedings for the 42nd Annual Meeting of the Cognitive Science Society (2020)

    Google Scholar 

  33. Jones, S.S., Smith, L.B.: The place of perception in children’s concepts. Cogn. Dev. 8(2), 113–139 (1993). https://doi.org/10.1016/0885-2014(93)90008-S

    Article  Google Scholar 

  34. Kaipainen, M., Zenker, F., Hautamäki, A., Gärdenfors, P. (eds.): Conceptual Spaces: Elaborations and Applications. SL, vol. 405. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-12800-5

    Book  Google Scholar 

  35. Kendall, M.G.: A new measure of rank correlation. Biometrika 30(1–2), 81–93 (1938). https://doi.org/10.1093/biomet/30.1-2.81

    Article  MATH  Google Scholar 

  36. Kingma, D.P., Ba, J.: Adam: a method for stochastic optimization. arXiV (2014). https://arxiv.org/abs/1412.6980

  37. Krizhevsky, A., Sutskever, I., Hinton, G.E.: ImageNet classification with deep convolutional neural networks. In: Pereira, F., Burges, C.J.C., Bottou, L., Weinberger, K.Q. (eds.) Advances in Neural Information Processing Systems, vol. 25, pp. 1097–1105. Curran Associates, Inc. (2012). https://proceedings.neurips.cc/paper/2012/file/c399862d3b9d6b76c8436e924a68c45b-Paper.pdf

  38. Kubilius, J., Bracci, S., Op de Beeck, H.P.: Deep neural networks as a computational model for human shape sensitivity. PLOS Comput. Biol. 12(4), 1–26 (2016). https://doi.org/10.1371/journal.pcbi.1004896

  39. Lake, B., Zaremba, W., Fergus, R., Gureckis, T.: Deep neural networks predict category typicality ratings for images. In: Noelle, D.C., et al. (eds.) Proceedings of the 37th Annual Conference of the Cognitive Science Society (2015)

    Google Scholar 

  40. Landau, B., Smith, L., Jones, S.: Object perception and object naming in early development. Trends Cogn. Sci. 2(1), 19–24 (1998). https://doi.org/10.1016/S1364-6613(97)01111-X

    Article  Google Scholar 

  41. Li, A.Y., Liang, J.C., Lee, A.C.H., Barense, M.D.: The validated circular shape space: quantifying the visual similarity of shape. J. Exp. Psychol. Gen. 149(5), 949–966 (2019)

    Article  Google Scholar 

  42. Lieto, A.: Cognitive Design for Artificial Minds. Routledge (2021)

    Google Scholar 

  43. Lieto, A., Chella, A., Frixione, M.: Conceptual spaces for cognitive architectures: a lingua franca for different levels of representation. Biolog. Inspired Cogn. Archit. (2016). https://doi.org/10.1016/j.bica.2016.10.005

  44. Marcus, G., Davis, E.: Rebooting AI: Building Artificial Intelligence We Can Trust. Pantheon (2019)

    Google Scholar 

  45. Marr, D., Nishihara, H.K.: Representation and recognition of the spatial organization of three-dimensional shapes. Proc. Roy. Soc. London Ser. B Biol. Sci. 200(1140), 269–294 (1978)

    Google Scholar 

  46. Maruyama, Y.: Symbolic and statistical theories of cognition: towards integrated artificial intelligence. In: Cleophas, L., Massink, M. (eds.) SEFM 2020. LNCS, vol. 12524, pp. 129–146. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-67220-1_11

    Chapter  Google Scholar 

  47. Mingqiang, Y., Kidiyo, K., Joseph, R.: A survey of shape feature extraction techniques. Pattern Recogn. 15(7), 43–90 (2008)

    Google Scholar 

  48. Mitchell, T.M.: Machine Learning. McGraw Hill, New York (1997)

    MATH  Google Scholar 

  49. Morgenstern, Y., et al.: An image-computable model of human visual shape similarity. PLoS Comput. Biol. 17(6), 1–34 (2021). https://doi.org/10.1371/journal.pcbi.1008981

    Article  Google Scholar 

  50. Ons, B., Baene, W.D., Wagemans, J.: Subjectively interpreted shape dimensions as privileged and orthogonal axes in mental shape space. J. Exp. Psychol. Hum. Percept. Perform. 37(2), 422–441 (2011)

    Article  Google Scholar 

  51. Peterson, J.C., Abbott, J.T., Griffiths, T.L.: Adapting deep network features to capture psychological representations: an abridged report. In: Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence, IJCAI-17, pp. 4934–4938 (2017). https://doi.org/10.24963/ijcai.2017/697

  52. Peterson, J.C., Abbott, J.T., Griffiths, T.L.: Evaluating (and improving) the correspondence between deep neural networks and human representations. Cogn. Sci. 42(8), 2648–2669 (2018)

    Article  Google Scholar 

  53. Peterson, J.C., Battleday, R.M., Griffiths, T.L., Russakovsky, O.: Human uncertainty makes classification more robust. In: Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV) (2019)

    Google Scholar 

  54. Riesenhuber, M., Poggio, T.: Hierarchical models of object recognition in cortex. Nat. Neurosci. 2(11), 1019–1025 (1999)

    Article  Google Scholar 

  55. Roads, B.D., Love, B.C.: Enriching ImageNet with human similarity judgments and psychological embeddings. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 3547–3557 (2021)

    Google Scholar 

  56. Rosch, E., Mervis, C.B., Gray, W.D., Johnson, D.M., Boyes-Braem, P.: Basic objects in natural categories. Cogn. Psychol. 8(3), 382–439 (1976). https://doi.org/10.1016/0010-0285(76)90013-x

    Article  Google Scholar 

  57. Rousseeuw, P.J.: Silhouettes: a graphical aid to the interpretation and validation of cluster analysis. J. Comput. Appl. Math. 20, 53–65 (1987). https://doi.org/10.1016/0377-0427(87)90125-7

    Article  MATH  Google Scholar 

  58. Sanders, C.A., Nosofsky, R.M.: Using deep-learning representations of complex natural stimuli as input to psychological models of classification. In: Proceedings of the 2018 Conference of the Cognitive Science Society, Madison (2018)

    Google Scholar 

  59. Sanders, C.A., Nosofsky, R.M.: Training deep networks to construct a psychological feature space for a natural-object category domain. Comput. Brain Behav. 3, 229–251 (2020)

    Article  Google Scholar 

  60. Sangkloy, P., Burnell, N., Ham, C., Hays, J.: The sketchy database: learning to retrieve badly drawn bunnies. ACM Trans. Graph. 35(4), 1–12 (2016). https://doi.org/10.1145/2897824.2925954

    Article  Google Scholar 

  61. Schockaert, S., Prade, H.: Interpolation and extrapolation in conceptual spaces: a case study in the music domain. In: Rudolph, S., Gutierrez, C. (eds.) RR 2011. LNCS, vol. 6902, pp. 217–231. Springer, Heidelberg (2011). https://doi.org/10.1007/978-3-642-23580-1_16

    Chapter  Google Scholar 

  62. Shepard, R.N.: Attention and the metric structure of the stimulus space. J. Math. Psychol. 1(1), 54–87 (1964). https://doi.org/10.1016/0022-2496(64)90017-3

    Article  Google Scholar 

  63. Singer, J., Hebart, M.N., Seeliger, K.: The representation of object drawings and sketches in deep convolutional neural networks. In: NeurIPS 2020 Workshop SVRHM (2020). https://openreview.net/forum?id=wXv6gtWnDO2

  64. Singh, P., Peterson, J., Battleday, R., Griffiths, T.: End-to-end deep prototype and exemplar models for predicting human behavior. In: Proceedings for the 42nd Annual Meeting of the Cognitive Science Society (2020)

    Google Scholar 

  65. Sorscher, B., Ganguli, S., Sompolinsky, H.: The geometry of concept learning. bioRxiv (2021). https://doi.org/10.1101/2021.03.21.436284

  66. Treisman, A., Gormican, S.: Feature analysis in early vision: evidence from search asymmetries. Psychol. Rev. 95(1), 15–48 (1988)

    Article  Google Scholar 

  67. Vincent, P., Larochelle, H., Bengio, Y., Manzagol, P.A.: Extracting and composing robust features with denoising autoencoders. In: Proceedings of the 25th International Conference on Machine Learning - ICML 2008 (2008). https://doi.org/10.1145/1390156.1390294

  68. Yu, Q., Yang, Y., Liu, F., Song, Y.Z., Xiang, T., Hospedales, T.M.: Sketch-a-net: a deep neural network that beats humans. Int. J. Comput. Vis. 122(3), 411–425 (2017)

    Article  MathSciNet  Google Scholar 

  69. Yu, Q., Yang, Y., Song, Y.Z., Xiang, T., Hospedales, T.: Sketch-a-net that beats humans. In: Xie, X., Jones, M.W., Tam, G.K.L. (eds.) Proceedings of the British Machine Vision Conference (BMVC), pp. 7.1–7.12. BMVA Press (2015). https://doi.org/10.5244/C.29.7

  70. Zenker, F., Gärdenfors, P. (eds.): Applications of Conceptual Spaces. Springer, Heidelberg (2015). https://doi.org/10.1007/978-3-319-15021-5

    Book  MATH  Google Scholar 

  71. Zhang, D., Lu, G.: Review of shape representation and description techniques. Pattern Recogn. 37(1), 1–19 (2004). https://doi.org/10.1016/j.patcog.2003.07.008

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Lucas Bechberger .

Editor information

Editors and Affiliations

Rights and permissions

Reprints and permissions

Copyright information

© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Bechberger, L., Kühnberger, KU. (2022). Grounding Psychological Shape Space in Convolutional Neural Networks. In: Cerone, A., et al. Software Engineering and Formal Methods. SEFM 2021 Collocated Workshops. SEFM 2021. Lecture Notes in Computer Science, vol 13230. Springer, Cham. https://doi.org/10.1007/978-3-031-12429-7_7

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-12429-7_7

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-12428-0

  • Online ISBN: 978-3-031-12429-7

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics