Introducing Scalable Quantum Approaches in Language Representation

  • Peter Wittek
  • Sándor Darányi
Part of the Lecture Notes in Computer Science book series (LNCS, volume 7052)

Abstract

High-performance computational resources and distributed systems are crucial for the success of real-world language technology applications. The novel paradigm of general-purpose computing on graphics processors (GPGPU) offers a feasible and economical alternative: it has already become a common phenomenon in scientific computation, with many algorithms adapted to the new paradigm. However, applications in language technology do not readily adapt to this approach. Recent advances show the applicability of quantum metaphors in language representation, and many algorithms in quantum mechanics have already been adapted to GPGPU computing. SQUALAR aims to match quantum algorithms with heterogeneous computing to develop new formalisms of information representation for natural language processing in quantum environments.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Dean, J., Ghemawat, S.: MapReduce: Simplified data processing on large clusters. In: Proceedings of OSDI 2004, 6th International Symposium on Operating Systems Design & Implementation, San Francisco, CA, USA. ACM Press, New York (2004)Google Scholar
  2. 2.
    Lin, J., Dyer, C.: Data-Intensive Text Processing with MapReduce. Morgan & Claypool (2010)Google Scholar
  3. 3.
    Cavanagh, J., Potok, T., Cui, X.: Parallel latent semantic analysis using a graphics processing unit. In: Proceedings of GECCO 2009, 11th Annual Conference Companion on Genetic and Evolutionary Computation Conference: Late Breaking Papers, Montreal, QC, Canada, pp. 2505–2510. ACM Press, New York (2009)Google Scholar
  4. 4.
    Ding, S., He, J., Yan, H., Suel, T.: Using graphics processors for high performance IR query processing. In: Proceedings of WWW 2009, 18th International Conference on World Wide Web, Spain, Madrid, pp. 421–430. ACM Press, New York (2009)Google Scholar
  5. 5.
    Zhang, Y., Mueller, F., Cui, X., Potok, T.: Large-scale multi-dimensional document clustering on GPU clusters. In: Proceedings of IDPDS 2010, 24th International Parallel and Distributed Computing Symposium, Atlanta, GA, USA. IEEE Computer Society Press, Los Alamitos (2010)Google Scholar
  6. 6.
    Byna, S., Meng, J., Raghunathan, A., Chakradhar, S., Cadambi, S.: Best-effort semantic document search on GPUs. In: Proceedings of GPGPU 2010, 3rd Workshop on General-Purpose Computation on Graphics Processing Units, pp. 86–93. ACM, New York (2010)Google Scholar
  7. 7.
    Wei, Z., JaJa, J.: A fast algorithm for constructing inverted files on heterogeneous platforms. In: Proceedings of IPDPS 2011, 25th International Parallel and Distributed Computing Symposium, Anchorage, AK, USA (2011)Google Scholar
  8. 8.
    Krüger, J., Westermann, R.: Linear algebra operators for GPU implementation of numerical algorithms. In: Proceedings of SIGGRAPH 2005, 32nd International Conference on Computer Graphics and Interactive Techniques, Los Angeles, CA, USA, pp. 234–242. ACM Press, New York (2005)Google Scholar
  9. 9.
    Galoppo, N., Govindaraju, N., Henson, M., Bondhugula, V., Larsen, S., Manocha, D.: Efficient numerical algorithms on graphics hardware. In: Proceedings of EDGE 2006, Workshop on Edge Computing Using New Commodity Architectures, Chapel Hill, NC, USA (2006)Google Scholar
  10. 10.
    Barrachina, S., Castillo, M., Igual, F., Mayo, R., Quintana-Orti, E.: Evaluation and tuning of the level 3 CUBLAS for graphics processors. In: Proceedings of IPDPS 2008, 22nd International Symposium on Parallel and Distributed Processing, Miami, FL, USA, pp. 1–8. IEEE, Los Alamitos (2008)Google Scholar
  11. 11.
    Lahabar, S., Narayanan, P.: Singular value decomposition on GPU using CUDA. In: Proceedings of IPDPS 2009, 23rd International Symposium on Parallel and Distributed Processing, Rome, Italy, IEEE, Los Alamitos (2009)Google Scholar
  12. 12.
    Brodtkorb, A., Dyken, C., Hagen, T., Hjelmervik, J., Storaasli, O.: State-of-the-art in heterogeneous computing. Scientific Programming 18(1), 1–33 (2010)CrossRefGoogle Scholar
  13. 13.
    Kirk, D., Hwu, W.: Programming massively parallel processors: A hands-on approach (2009)Google Scholar
  14. 14.
    Jiménez, V., Vilanova, L., Gelado, I., Gil, M., Fursin, G., Navarro, N.: Predictive runtime code scheduling for heterogeneous architectures. High Performance Embedded Architectures and Compilers, 19–33 (2009)Google Scholar
  15. 15.
    Lee, S., Min, S.J., Eigenmann, R.: OpenMP to GPGPU: a compiler framework for automatic translation and optimization. In: Proceedings of PPOPP 2009, 14th Symposium on Principles and Practice of Parallel Programming, pp. 101–110. ACM Press, New York (2009)Google Scholar
  16. 16.
    Luk, C., Hong, S., Kim, H.: Qilin: Exploiting parallelism on heterogeneous multiprocessors with adaptive mapping. In: MICRO-42, 42nd Annual IEEE/ACM International Symposium on Microarchitecture, New York, NY, USA, pp. 45–55. IEEE, Los Alamitos (2009)Google Scholar
  17. 17.
    Phillips, J., Stone, J., Schulten, K.: Adapting a message-driven parallel application to GPU-accelerated clusters. In: Proceedings of SC 2008, 21st Conference on Supercomputing, Austin, TX, USA, pp. 1–9. IEEE Press, Los Alamitos (2008)Google Scholar
  18. 18.
    Kuhn, B., Petersen, P., O’Toole, E.: OpenMP versus threading in C/C++. Concurrency: Practice and Experience 12(12), 1165–1176 (2000)CrossRefMATHGoogle Scholar
  19. 19.
    Koop, M., Sur, S., Gao, Q., Panda, D.: High performance MPI design using unreliable datagram for ultra-scale InfiniBand clusters. In: Proceedings of ISC-06, 21st Annual International Conference on Supercomputing, Dresden, Germany, pp. 180–189. ACM, New York (2006)Google Scholar
  20. 20.
    NVida Compute Unified Device Architecture Best Practices Guide 3.2 (2010)Google Scholar
  21. 21.
    Shirahata, K., Sato, H., Matsuoka, S.: Hybrid map task scheduling on GPU-based heterogeneous clusters. In: Proceedings of CloudCom 2010, The 2nd International Conference on Cloud Computing, Indianapolis, IN, USA (2010)Google Scholar
  22. 22.
    Stuart, J., Owens, J.: Multi-GPU MapReduce on GPU clusters. In: Proceedings of IPDPS 2011, 25th International Parallel and Distributed Computing Symposium, Anchorage, AK, USA (2011)Google Scholar
  23. 23.
    Aerts, D., Aerts, S., Broekaert, J., Gabora, L.: The violation of bell inequalities in the macroworld. Foundations of Physics 30(9), 1387–1414 (2000)MathSciNetCrossRefGoogle Scholar
  24. 24.
    Widdows, D., Peters, S.: Word vectors and quantum logic: Experiments with negation and disjunction. In: Proceedings of MoL 2003, 8th Mathematics of Language Conference, Bloomington, IN, USA, vol. 8, pp. 141–154 (2003)Google Scholar
  25. 25.
    Aerts, D., Czachor, M.: Quantum aspects of semantic analysis and symbolic artificial intelligence. Journal of Physics A: Mathematical and General 37, L123–L132 (2004)MathSciNetCrossRefMATHGoogle Scholar
  26. 26.
    van Rijsbergen, C.J.: The Geometry of Information Retrieval. Cambridge University Press, New York (2004)CrossRefMATHGoogle Scholar
  27. 27.
    Widdows, D.: Geometry and meaning (2004)Google Scholar
  28. 28.
    Bruza, P., Widdows, D., Woods, J.: A quantum logic of down below. In: Engesser, K., Gabbay, D., Lehmann, D. (eds.) Handbook of Quantum Logic and Quantum Structures, vol. 2, Elsevier, Amsterdam (2009)Google Scholar
  29. 29.
    Kitto, K.: Why quantum theory? In: Proceedings of QI 2008, 2nd International Symposium on Quantum Interaction, Oxford, UK, pp. 11–18 (2008)Google Scholar
  30. 30.
    Lyons, J.: Semantics. Cambridge University Press, New York (1977)CrossRefGoogle Scholar
  31. 31.
    Jones, M., Mewhort, D.: Representing word meaning and order information in a composite holographic lexicon. Psychological Review 114(1), 1–37 (2007)CrossRefGoogle Scholar
  32. 32.
    Bruza, P., Woods, J.: Quantum collapse in semantic space: interpreting natural language argumentation. In: Proceedings of QI 2008, 2nd International Symposium on Quantum Interaction, Oxford, UK. College Publications (2008)Google Scholar
  33. 33.
    Widdows, D.: Semantic vector products: Some initial investigations. In: Proceedings of QI 2008, 2nd International Symposium on Quantum Interaction. College Publications, Oxford (2008)Google Scholar
  34. 34.
    Kanerva, P., Kristofersson, J., Holst, A.: Random indexing of text samples for latent semantic analysis. In: Proceedings of CogSci 2000, 22nd Annual Conference of the Cognitive Science Society, Philadelphia, PA, USA, vol. 1036 (2000)Google Scholar
  35. 35.
    Sahlgren, M.: An introduction to random indexing. In: Proceedings of TKE 2005, Methods and Applications of Semantic Indexing Workshop at the 7th International Conference on Terminology and Knowledge Engineering, Copenhagen, Denmark, Citeseer (2005)Google Scholar
  36. 36.
    Sahlgren, M., Holst, A., Kanerva, P.: Permutations as a means to encode order in word space. In: Proceedings of CogSci 2008, 30th Annual Meeting of the Cognitive Science Society, Washington, DC, USA (2008)Google Scholar
  37. 37.
    De Vine, L., Bruza, P.: Semantic oscillations: Encoding context and structure in complex valued holographic vectors. In: Proceedings of QI 2010, 4th Symposium on Quantum Informatics for Cognitive, Social, and Semantic Processes, Arlington, VA, USA, pp. 11–13 (2010)Google Scholar
  38. 38.
    Mitchell, J., Lapata, M.: Vector-based models of semantic composition. In: Proceedings of ACL 2008, 46th Annual Meeting of the Association for Computational Linguistics, Columbus, Ohio, pp. 236–244. ACL, Morristown (2008)Google Scholar
  39. 39.
    Song, D., Lalmas, M., van Rijsbergen, C., Frommholz, I., Piwowarski, B., Wang, J., Zhang, P., Zuccon, G., Bruza, P., Arafat, S., et al.: How quantum theory is developing the field of Information Retrieval. In: Proceedings of QI 2010, 4th Symposium on Quantum Informatics for Cognitive, Social, and Semantic Processes, Arlington, VA, USA, pp. 105–108 (2010)Google Scholar
  40. 40.
    Humphreys, M., Bain, J., Pike, R.: Different ways to cue a coherent memory system: A theory for episodic, semantic, and procedural tasks. Psychological Review 96(2), 208–233 (1989)CrossRefGoogle Scholar
  41. 41.
    Wiles, J., Halford, G., Stewart, J., Humphreys, M., Bain, J., Wilson, W.: Tensor models: A creative basis for memory retrieval and analogical mapping. In: Dartnall, T. (ed.) Artificial Intelligence and Creativity, pp. 145–159. Kluwer Academic, Dordrecht (1994)CrossRefGoogle Scholar
  42. 42.
    Plate, T.: Holographic reduced representations: Convolution algebra for compositional distributed representations. In: Proceedings of IJCAI 1991, 12th International Joint Conference on Artificial Intelligence, Syndey, Australia, Citeseer, pp. 30–35 (1991)Google Scholar
  43. 43.
    Plate, T.: Holographic reduced representations. IEEE Transactions on Neural Networks 6(3), 623–641 (1995)CrossRefGoogle Scholar
  44. 44.
    Antonellis, I., Gallopoulos, E.: Exploring term-document matrices from matrix models in text mining. In: Proceedings of SDM 2006, Text Mining Workshop in Conjuction with the 6th SIAM International Conference on Data Mining, Bethesda, MD, USA (2006)Google Scholar
  45. 45.
    Rudolph, S., Giesbrecht, E.: Compositional matrix-space models of language. In: Proceedings of ACL 2010, 48th Annual Meeting of the Association for Computational Linguistics, Uppsala, Sweden, pp. 907–916. Association for Computational Linguistics (2010)Google Scholar
  46. 46.
    Rölleke, T., Tsikrika, T., Kazai, G.: A general matrix framework for modelling information retrieval. Information Processing & Management 42(1), 4–30 (2006)CrossRefMATHGoogle Scholar
  47. 47.
    Swen, B.: A sense matrix model for information retrieval. Technical report TR-2004-2 of ICL-PK (2004)Google Scholar
  48. 48.
    Novakovitch, D., Bruza, P., Sitbon, L.: Inducing shades of meaning by matrix methods: a first step towards thematic analysis of opinion. In: Proceedings of SEMAPRO 2009, 3rd International Conference on Advances in Semantic Processing, Sliema, Malta, pp. 86–91. IEEE, Los Alamitos (2009)CrossRefGoogle Scholar
  49. 49.
    Jones, M., Kintsch, W., Mewhort, D.: High-dimensional semantic space accounts of priming. Journal of Memory and Language 55(4), 534–552 (2006)CrossRefGoogle Scholar
  50. 50.
    Zuccon, G., Azzopardi, L.A., van Rijsbergen, C.J.: Semantic spaces: Measuring the distance between different subspaces. In: Bruza, P., Sofge, D., Lawless, W., van Rijsbergen, K., Klusch, M. (eds.) QI 2009. LNCS, vol. 5494, pp. 225–236. Springer, Heidelberg (2009)CrossRefGoogle Scholar
  51. 51.
    Deerwester, S., Dumais, S., Furnas, G., Landauer, T., Harshman, R.: Indexing by latent semantic analysis. Journal of the American Society for Information Science 41(6), 391–407 (1990)CrossRefGoogle Scholar
  52. 52.
    Wittgenstein, L.: Philosophical Investigations. Blackwell Publishing, Oxford (1967)MATHGoogle Scholar
  53. 53.
    Harris, Z.: Distributional structure. In: Harris, Z. (ed.) Papers in Structural and Transformational Linguistics. Formal Linguistics, pp. 775–794. Humanities Press, New York (1970)CrossRefGoogle Scholar
  54. 54.
    Peirce, C.: Logic as semiotic: The theory of signs. In: Peirce, C.S., Buchler, J. (eds.) Philosophical Writings of Peirce, pp. 98–119. Dover Publications, Mineola (1955)Google Scholar
  55. 55.
    Frege, G.: Sense and reference. The Philosophical Review 57(3), 209–230 (1948)CrossRefGoogle Scholar
  56. 56.
    Lund, K., Burgess, C.: Producing high-dimensional semantic spaces from lexical co-occurrence. Behavior Research Methods Instruments and Computers 28, 203–208 (1996)CrossRefGoogle Scholar
  57. 57.
    Govindaraju, N., Lloyd, B., Dotsenko, Y., Smith, B., Manferdelli, J.: High performance discrete Fourier transforms on graphics processors. In: Proceedings of SC 2008, 21st Conference on Supercomputing, Austin, TX, USA. IEEE, Los Alamitos (2008)Google Scholar
  58. 58.
    Ufimtsev, I., Martínez, T.: Graphical processing units for quantum chemistry. Computing in Science & Engineering 10(6), 26–34 (2008)CrossRefGoogle Scholar
  59. 59.
    Watson, M., Olivares-Amaya, R., Edgar, R., Aspuru-Guzik, A.: Accelerating correlated quantum chemistry calculations using graphical processing units. Computing in Science & Engineering 12(4), 40–51 (2010)CrossRefGoogle Scholar
  60. 60.
    Stone, J., Saam, J., Hardy, D., Vandivort, K., Hwu, W., Schulten, K.: High performance computation and interactive display of molecular orbitals on GPUs and multi-core CPUs. In: Proceedings of GPGPU 2009, 2nd Workshop on General Purpose Processing on Graphics Processing Units, Washington, DC, USA, pp. 9–18. ACM, New York (2009)CrossRefGoogle Scholar
  61. 61.
    Catanzaro, B., Sundaram, N., Keutzer, K.: Fast support vector machine training and classification on graphics processors. In: McCallum, A., Roweis, S. (eds.) Proceedings of ICML 2008, 25th Annual International Conference on Machine Learning, Helsinki, Finland, pp. 104–111. Omnipress (2008)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2011

Authors and Affiliations

  • Peter Wittek
    • 1
  • Sándor Darányi
    • 1
  1. 1.Swedish School of Library and Information ScienceGöteborg University & University of BoråsBoråsSweden

Personalised recommendations