Investigating Non-classical Correlations Between Decision Fused Multi-modal Documents

  • Dimitris GkoumasEmail author
  • Sagar Uprety
  • Dawei Song
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11690)


Correlation has been widely used to facilitate various information retrieval methods such as query expansion, relevance feedback, document clustering, and multi-modal fusion. Especially, correlation and independence are important issues when fusing different modalities that influence a multi-modal information retrieval process. The basic idea of correlation is that an observable can help predict or enhance another observable. In quantum mechanics, quantum correlation, called entanglement, is a sort of correlation between the observables measured in atomic-size particles when these particles are not necessarily collected in ensembles. In this paper, we examine a multimodal fusion scenario that might be similar to that encountered in physics by firstly measuring two observables (i.e., text-based relevance and image-based relevance) of a multi-modal document without counting on an ensemble of multi-modal documents already labeled in terms of these two variables. Then, we investigate the existence of non-classical correlations between pairs of multi-modal documents. Despite there are some basic differences between entanglement and classical correlation encountered in the macroscopic world, we investigate the existence of this kind of non-classical correlation through the Bell inequality violation. Here, we experimentally test several novel association methods in a small-scale experiment. However, in the current experiment we did not find any violation of the Bell inequality. Finally, we present a series of interesting discussions, which may provide theoretical and empirical insights and inspirations for future development of this direction.


Multi-modal information retrieval Non-classical correlations Decision fused multi-modal documents CHSH inequality 



This work is funded by the European Union’s Horizon 2020 research and innovation programme under the Marie Sklodowska-Curie grant agreement No 721321.


  1. 1.
    Aerts, D., Sozzo, S.: Quantum structure in cognition: why and how concepts are entangled. In: Song, D., Melucci, M., Frommholz, I., Zhang, P., Wang, L., Arafat, S. (eds.) QI 2011. LNCS, vol. 7052, pp. 116–127. Springer, Heidelberg (2011). Scholar
  2. 2.
    Aerts, D., Sozzo, S.: Quantum entanglement in concept combinations. Int. J. Theor. Phys. 53(10), 3587–3603 (2014)MathSciNetCrossRefGoogle Scholar
  3. 3.
    Aspect, A., Grangier, P., Roger, G.: Experimental realization of Einstein-Podolsky-Rosen-Bohm Gedankenexperiment: a new violation of Bell’s inequalities. Phys. Rev. Lett. 49(2), 91 (1982)CrossRefGoogle Scholar
  4. 4.
    Atrey, P.K., Hossain, M.A., El Saddik, A., Kankanhalli, M.S.: Multimodal fusion for multimedia analysis: a survey. Multimedia Syst. 16(6), 345–379 (2010)CrossRefGoogle Scholar
  5. 5.
    Baltrušaitis, T., Ahuja, C., Morency, L.P.: Multimodal machine learning: a survey and taxonomy. IEEE Trans. Pattern Anal. Mach. Intell. 41, 423–443 (2018)CrossRefGoogle Scholar
  6. 6.
    Bruza, P.D., Kitto, K., Ramm, B., Sitbon, L., Song, D., Blomberg, S.: Quantum-like non-separability of concept combinations, emergent associates and abduction. Logic J. IGPL 20(2), 445–457 (2011)MathSciNetCrossRefGoogle Scholar
  7. 7.
    Bruza, P.D., Kitto, K., Ramm, B.J., Sitbon, L.: A probabilistic framework for analysing the compositionality of conceptual combinations. J. Math. Psychol. 67, 26–38 (2015)MathSciNetCrossRefGoogle Scholar
  8. 8.
    Cirel’son, B.S.: Quantum generalizations of Bell’s inequality. Lett. Math. Phys. 4(2), 93–100 (1980)MathSciNetCrossRefGoogle Scholar
  9. 9.
    Clauser, J.F., Horne, M.A., Shimony, A., Holt, R.A.: Proposed experiment to test local hidden-variable theories. Phys. Rev. Lett. 23(15), 880 (1969)CrossRefGoogle Scholar
  10. 10.
    Gisin, N.: Bell inequality for arbitrary many settings of the analyzers. Phys. Lett. A 260(1–2), 1–3 (1999)MathSciNetCrossRefGoogle Scholar
  11. 11.
    Gleason, A.M.: Measures on the closed subspaces of a Hilbert space. J. Math. Mech. 6, 885–893 (1957)MathSciNetzbMATHGoogle Scholar
  12. 12.
    Grubinger, M., Clough, P., Hanbury, A., Müller, H.: Overview of the ImageCLEFphoto 2007 photographic retrieval task. In: Peters, C., et al. (eds.) CLEF 2007. LNCS, vol. 5152, pp. 433–444. Springer, Heidelberg (2008). Scholar
  13. 13.
    Hou, Y., Song, D.: Characterizing pure high-order entanglements in lexical semantic spaces via information geometry. In: Bruza, P., Sofge, D., Lawless, W., van Rijsbergen, K., Klusch, M. (eds.) QI 2009. LNCS (LNAI), vol. 5494, pp. 237–250. Springer, Heidelberg (2009). Scholar
  14. 14.
    Hou, Y., Zhao, X., Song, D., Li, W.: Mining pure high-order word associations via information geometry for information retrieval. ACM Trans. Inf. Syst. (TOIS) 31(3), 12 (2013)CrossRefGoogle Scholar
  15. 15.
    Melucci, M.: Introduction to Information Retrieval and Quantum Mechanics, pp. 156–158, 176–181, 212–213, 217–221. Springer, Berlin (2015). Scholar
  16. 16.
    Nielsen, M.A., Chuang, I.: Quantum computation and quantum information (2002)Google Scholar
  17. 17.
    Pathak, A.: Elements of Quantum Computation and Quantum Communication, pp. 92–98. Taylor & Francis, Abingdon (2013)zbMATHGoogle Scholar
  18. 18.
    Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556 (2014)
  19. 19.
    Stenger, V.J.: Timeless Reality: Symmetry, Simplicity and Multiple Universes. (Chap. 12)Google Scholar
  20. 20.
    Tversky, A., Kahneman, D.: Extensional versus intuitive reasoning: the conjunction fallacy in probability judgment. Psychol. Rev. 90(4), 293 (1983)CrossRefGoogle Scholar
  21. 21.
    Van Rijsbergen, C.J.: The Geometry of Information Retrieval. Cambridge University Press, Cambridge (2004)CrossRefGoogle Scholar
  22. 22.
    Veloz, T., Zhao, X., Aerts, D.: Measuring conceptual entanglement in collections of documents. In: Atmanspacher, H., Haven, E., Kitto, K., Raine, D. (eds.) QI 2013. LNCS, vol. 8369, pp. 134–146. Springer, Heidelberg (2014). Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  1. 1.The Open UniversityMilton KeynesUK
  2. 2.Beijing Institute of TechnologyBeijingChina

Personalised recommendations