Skip to main content
SpringerLink
Log in

Introducing Scalable Quantum Approaches in Language Representation

  • Conference paper
Quantum Interaction (QI 2011)

Part of the book series: Lecture Notes in Computer Science ((LNTCS,volume 7052))

Included in the following conference series:

Abstract

High-performance computational resources and distributed systems are crucial for the success of real-world language technology applications. The novel paradigm of general-purpose computing on graphics processors (GPGPU) offers a feasible and economical alternative: it has already become a common phenomenon in scientific computation, with many algorithms adapted to the new paradigm. However, applications in language technology do not readily adapt to this approach. Recent advances show the applicability of quantum metaphors in language representation, and many algorithms in quantum mechanics have already been adapted to GPGPU computing. SQUALAR aims to match quantum algorithms with heterogeneous computing to develop new formalisms of information representation for natural language processing in quantum environments.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Dean, J., Ghemawat, S.: MapReduce: Simplified data processing on large clusters. In: Proceedings of OSDI 2004, 6th International Symposium on Operating Systems Design & Implementation, San Francisco, CA, USA. ACM Press, New York (2004)

    Google Scholar 

  2. Lin, J., Dyer, C.: Data-Intensive Text Processing with MapReduce. Morgan & Claypool (2010)

    Google Scholar 

  3. Cavanagh, J., Potok, T., Cui, X.: Parallel latent semantic analysis using a graphics processing unit. In: Proceedings of GECCO 2009, 11th Annual Conference Companion on Genetic and Evolutionary Computation Conference: Late Breaking Papers, Montreal, QC, Canada, pp. 2505–2510. ACM Press, New York (2009)

    Google Scholar 

  4. Ding, S., He, J., Yan, H., Suel, T.: Using graphics processors for high performance IR query processing. In: Proceedings of WWW 2009, 18th International Conference on World Wide Web, Spain, Madrid, pp. 421–430. ACM Press, New York (2009)

    Google Scholar 

  5. Zhang, Y., Mueller, F., Cui, X., Potok, T.: Large-scale multi-dimensional document clustering on GPU clusters. In: Proceedings of IDPDS 2010, 24th International Parallel and Distributed Computing Symposium, Atlanta, GA, USA. IEEE Computer Society Press, Los Alamitos (2010)

    Google Scholar 

  6. Byna, S., Meng, J., Raghunathan, A., Chakradhar, S., Cadambi, S.: Best-effort semantic document search on GPUs. In: Proceedings of GPGPU 2010, 3rd Workshop on General-Purpose Computation on Graphics Processing Units, pp. 86–93. ACM, New York (2010)

    Google Scholar 

  7. Wei, Z., JaJa, J.: A fast algorithm for constructing inverted files on heterogeneous platforms. In: Proceedings of IPDPS 2011, 25th International Parallel and Distributed Computing Symposium, Anchorage, AK, USA (2011)

    Google Scholar 

  8. Krüger, J., Westermann, R.: Linear algebra operators for GPU implementation of numerical algorithms. In: Proceedings of SIGGRAPH 2005, 32nd International Conference on Computer Graphics and Interactive Techniques, Los Angeles, CA, USA, pp. 234–242. ACM Press, New York (2005)

    Google Scholar 

  9. Galoppo, N., Govindaraju, N., Henson, M., Bondhugula, V., Larsen, S., Manocha, D.: Efficient numerical algorithms on graphics hardware. In: Proceedings of EDGE 2006, Workshop on Edge Computing Using New Commodity Architectures, Chapel Hill, NC, USA (2006)

    Google Scholar 

  10. Barrachina, S., Castillo, M., Igual, F., Mayo, R., Quintana-Orti, E.: Evaluation and tuning of the level 3 CUBLAS for graphics processors. In: Proceedings of IPDPS 2008, 22nd International Symposium on Parallel and Distributed Processing, Miami, FL, USA, pp. 1–8. IEEE, Los Alamitos (2008)

    Google Scholar 

  11. Lahabar, S., Narayanan, P.: Singular value decomposition on GPU using CUDA. In: Proceedings of IPDPS 2009, 23rd International Symposium on Parallel and Distributed Processing, Rome, Italy, IEEE, Los Alamitos (2009)

    Google Scholar 

  12. Brodtkorb, A., Dyken, C., Hagen, T., Hjelmervik, J., Storaasli, O.: State-of-the-art in heterogeneous computing. Scientific Programming 18(1), 1–33 (2010)

    Article  Google Scholar 

  13. Kirk, D., Hwu, W.: Programming massively parallel processors: A hands-on approach (2009)

    Google Scholar 

  14. Jiménez, V., Vilanova, L., Gelado, I., Gil, M., Fursin, G., Navarro, N.: Predictive runtime code scheduling for heterogeneous architectures. High Performance Embedded Architectures and Compilers, 19–33 (2009)

    Google Scholar 

  15. Lee, S., Min, S.J., Eigenmann, R.: OpenMP to GPGPU: a compiler framework for automatic translation and optimization. In: Proceedings of PPOPP 2009, 14th Symposium on Principles and Practice of Parallel Programming, pp. 101–110. ACM Press, New York (2009)

    Google Scholar 

  16. Luk, C., Hong, S., Kim, H.: Qilin: Exploiting parallelism on heterogeneous multiprocessors with adaptive mapping. In: MICRO-42, 42nd Annual IEEE/ACM International Symposium on Microarchitecture, New York, NY, USA, pp. 45–55. IEEE, Los Alamitos (2009)

    Google Scholar 

  17. Phillips, J., Stone, J., Schulten, K.: Adapting a message-driven parallel application to GPU-accelerated clusters. In: Proceedings of SC 2008, 21st Conference on Supercomputing, Austin, TX, USA, pp. 1–9. IEEE Press, Los Alamitos (2008)

    Google Scholar 

  18. Kuhn, B., Petersen, P., O’Toole, E.: OpenMP versus threading in C/C++. Concurrency: Practice and Experience 12(12), 1165–1176 (2000)

    Article  MATH  Google Scholar 

  19. Koop, M., Sur, S., Gao, Q., Panda, D.: High performance MPI design using unreliable datagram for ultra-scale InfiniBand clusters. In: Proceedings of ISC-06, 21st Annual International Conference on Supercomputing, Dresden, Germany, pp. 180–189. ACM, New York (2006)

    Google Scholar 

  20. NVida Compute Unified Device Architecture Best Practices Guide 3.2 (2010)

    Google Scholar 

  21. Shirahata, K., Sato, H., Matsuoka, S.: Hybrid map task scheduling on GPU-based heterogeneous clusters. In: Proceedings of CloudCom 2010, The 2nd International Conference on Cloud Computing, Indianapolis, IN, USA (2010)

    Google Scholar 

  22. Stuart, J., Owens, J.: Multi-GPU MapReduce on GPU clusters. In: Proceedings of IPDPS 2011, 25th International Parallel and Distributed Computing Symposium, Anchorage, AK, USA (2011)

    Google Scholar 

  23. Aerts, D., Aerts, S., Broekaert, J., Gabora, L.: The violation of bell inequalities in the macroworld. Foundations of Physics 30(9), 1387–1414 (2000)

    Article  MathSciNet  Google Scholar 

  24. Widdows, D., Peters, S.: Word vectors and quantum logic: Experiments with negation and disjunction. In: Proceedings of MoL 2003, 8th Mathematics of Language Conference, Bloomington, IN, USA, vol. 8, pp. 141–154 (2003)

    Google Scholar 

  25. Aerts, D., Czachor, M.: Quantum aspects of semantic analysis and symbolic artificial intelligence. Journal of Physics A: Mathematical and General 37, L123–L132 (2004)

    Article  MathSciNet  MATH  Google Scholar 

  26. van Rijsbergen, C.J.: The Geometry of Information Retrieval. Cambridge University Press, New York (2004)

    Book  MATH  Google Scholar 

  27. Widdows, D.: Geometry and meaning (2004)

    Google Scholar 

  28. Bruza, P., Widdows, D., Woods, J.: A quantum logic of down below. In: Engesser, K., Gabbay, D., Lehmann, D. (eds.) Handbook of Quantum Logic and Quantum Structures, vol. 2, Elsevier, Amsterdam (2009)

    Google Scholar 

  29. Kitto, K.: Why quantum theory? In: Proceedings of QI 2008, 2nd International Symposium on Quantum Interaction, Oxford, UK, pp. 11–18 (2008)

    Google Scholar 

  30. Lyons, J.: Semantics. Cambridge University Press, New York (1977)

    Book  Google Scholar 

  31. Jones, M., Mewhort, D.: Representing word meaning and order information in a composite holographic lexicon. Psychological Review 114(1), 1–37 (2007)

    Article  Google Scholar 

  32. Bruza, P., Woods, J.: Quantum collapse in semantic space: interpreting natural language argumentation. In: Proceedings of QI 2008, 2nd International Symposium on Quantum Interaction, Oxford, UK. College Publications (2008)

    Google Scholar 

  33. Widdows, D.: Semantic vector products: Some initial investigations. In: Proceedings of QI 2008, 2nd International Symposium on Quantum Interaction. College Publications, Oxford (2008)

    Google Scholar 

  34. Kanerva, P., Kristofersson, J., Holst, A.: Random indexing of text samples for latent semantic analysis. In: Proceedings of CogSci 2000, 22nd Annual Conference of the Cognitive Science Society, Philadelphia, PA, USA, vol. 1036 (2000)

    Google Scholar 

  35. Sahlgren, M.: An introduction to random indexing. In: Proceedings of TKE 2005, Methods and Applications of Semantic Indexing Workshop at the 7th International Conference on Terminology and Knowledge Engineering, Copenhagen, Denmark, Citeseer (2005)

    Google Scholar 

  36. Sahlgren, M., Holst, A., Kanerva, P.: Permutations as a means to encode order in word space. In: Proceedings of CogSci 2008, 30th Annual Meeting of the Cognitive Science Society, Washington, DC, USA (2008)

    Google Scholar 

  37. De Vine, L., Bruza, P.: Semantic oscillations: Encoding context and structure in complex valued holographic vectors. In: Proceedings of QI 2010, 4th Symposium on Quantum Informatics for Cognitive, Social, and Semantic Processes, Arlington, VA, USA, pp. 11–13 (2010)

    Google Scholar 

  38. Mitchell, J., Lapata, M.: Vector-based models of semantic composition. In: Proceedings of ACL 2008, 46th Annual Meeting of the Association for Computational Linguistics, Columbus, Ohio, pp. 236–244. ACL, Morristown (2008)

    Google Scholar 

  39. Song, D., Lalmas, M., van Rijsbergen, C., Frommholz, I., Piwowarski, B., Wang, J., Zhang, P., Zuccon, G., Bruza, P., Arafat, S., et al.: How quantum theory is developing the field of Information Retrieval. In: Proceedings of QI 2010, 4th Symposium on Quantum Informatics for Cognitive, Social, and Semantic Processes, Arlington, VA, USA, pp. 105–108 (2010)

    Google Scholar 

  40. Humphreys, M., Bain, J., Pike, R.: Different ways to cue a coherent memory system: A theory for episodic, semantic, and procedural tasks. Psychological Review 96(2), 208–233 (1989)

    Article  Google Scholar 

  41. Wiles, J., Halford, G., Stewart, J., Humphreys, M., Bain, J., Wilson, W.: Tensor models: A creative basis for memory retrieval and analogical mapping. In: Dartnall, T. (ed.) Artificial Intelligence and Creativity, pp. 145–159. Kluwer Academic, Dordrecht (1994)

    Chapter  Google Scholar 

  42. Plate, T.: Holographic reduced representations: Convolution algebra for compositional distributed representations. In: Proceedings of IJCAI 1991, 12th International Joint Conference on Artificial Intelligence, Syndey, Australia, Citeseer, pp. 30–35 (1991)

    Google Scholar 

  43. Plate, T.: Holographic reduced representations. IEEE Transactions on Neural Networks 6(3), 623–641 (1995)

    Article  Google Scholar 

  44. Antonellis, I., Gallopoulos, E.: Exploring term-document matrices from matrix models in text mining. In: Proceedings of SDM 2006, Text Mining Workshop in Conjuction with the 6th SIAM International Conference on Data Mining, Bethesda, MD, USA (2006)

    Google Scholar 

  45. Rudolph, S., Giesbrecht, E.: Compositional matrix-space models of language. In: Proceedings of ACL 2010, 48th Annual Meeting of the Association for Computational Linguistics, Uppsala, Sweden, pp. 907–916. Association for Computational Linguistics (2010)

    Google Scholar 

  46. Rölleke, T., Tsikrika, T., Kazai, G.: A general matrix framework for modelling information retrieval. Information Processing & Management 42(1), 4–30 (2006)

    Article  MATH  Google Scholar 

  47. Swen, B.: A sense matrix model for information retrieval. Technical report TR-2004-2 of ICL-PK (2004)

    Google Scholar 

  48. Novakovitch, D., Bruza, P., Sitbon, L.: Inducing shades of meaning by matrix methods: a first step towards thematic analysis of opinion. In: Proceedings of SEMAPRO 2009, 3rd International Conference on Advances in Semantic Processing, Sliema, Malta, pp. 86–91. IEEE, Los Alamitos (2009)

    Chapter  Google Scholar 

  49. Jones, M., Kintsch, W., Mewhort, D.: High-dimensional semantic space accounts of priming. Journal of Memory and Language 55(4), 534–552 (2006)

    Article  Google Scholar 

  50. Zuccon, G., Azzopardi, L.A., van Rijsbergen, C.J.: Semantic spaces: Measuring the distance between different subspaces. In: Bruza, P., Sofge, D., Lawless, W., van Rijsbergen, K., Klusch, M. (eds.) QI 2009. LNCS, vol. 5494, pp. 225–236. Springer, Heidelberg (2009)

    Chapter  Google Scholar 

  51. Deerwester, S., Dumais, S., Furnas, G., Landauer, T., Harshman, R.: Indexing by latent semantic analysis. Journal of the American Society for Information Science 41(6), 391–407 (1990)

    Article  Google Scholar 

  52. Wittgenstein, L.: Philosophical Investigations. Blackwell Publishing, Oxford (1967)

    MATH  Google Scholar 

  53. Harris, Z.: Distributional structure. In: Harris, Z. (ed.) Papers in Structural and Transformational Linguistics. Formal Linguistics, pp. 775–794. Humanities Press, New York (1970)

    Chapter  Google Scholar 

  54. Peirce, C.: Logic as semiotic: The theory of signs. In: Peirce, C.S., Buchler, J. (eds.) Philosophical Writings of Peirce, pp. 98–119. Dover Publications, Mineola (1955)

    Google Scholar 

  55. Frege, G.: Sense and reference. The Philosophical Review 57(3), 209–230 (1948)

    Article  Google Scholar 

  56. Lund, K., Burgess, C.: Producing high-dimensional semantic spaces from lexical co-occurrence. Behavior Research Methods Instruments and Computers 28, 203–208 (1996)

    Article  Google Scholar 

  57. Govindaraju, N., Lloyd, B., Dotsenko, Y., Smith, B., Manferdelli, J.: High performance discrete Fourier transforms on graphics processors. In: Proceedings of SC 2008, 21st Conference on Supercomputing, Austin, TX, USA. IEEE, Los Alamitos (2008)

    Google Scholar 

  58. Ufimtsev, I., Martínez, T.: Graphical processing units for quantum chemistry. Computing in Science & Engineering 10(6), 26–34 (2008)

    Article  Google Scholar 

  59. Watson, M., Olivares-Amaya, R., Edgar, R., Aspuru-Guzik, A.: Accelerating correlated quantum chemistry calculations using graphical processing units. Computing in Science & Engineering 12(4), 40–51 (2010)

    Article  Google Scholar 

  60. Stone, J., Saam, J., Hardy, D., Vandivort, K., Hwu, W., Schulten, K.: High performance computation and interactive display of molecular orbitals on GPUs and multi-core CPUs. In: Proceedings of GPGPU 2009, 2nd Workshop on General Purpose Processing on Graphics Processing Units, Washington, DC, USA, pp. 9–18. ACM, New York (2009)

    Chapter  Google Scholar 

  61. Catanzaro, B., Sundaram, N., Keutzer, K.: Fast support vector machine training and classification on graphics processors. In: McCallum, A., Roweis, S. (eds.) Proceedings of ICML 2008, 25th Annual International Conference on Machine Learning, Helsinki, Finland, pp. 104–111. Omnipress (2008)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Swedish School of Library and Information Science, Göteborg University & University of Borås, Allégatan 1, 50190, Borås, Sweden

    Peter Wittek & Sándor Darányi

Authors
  1. Peter Wittek
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Sándor Darányi
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. School of Computing, The Robert Gordon University, St. Andrew Street, AB25 1HG, Aberdeen, U.K.

    Dawei Song

  2. Department of Information Engineering, University of Padua, Via Gradenigo 6/B, 35131, Padova, Italy

    Massimo Melucci

  3. Department of Computer Science and Technology, University of Bedfordshire, Park Square, LU1 3JU, Luton, UK

    Ingo Frommholz

  4. School of Computing, The Robert Gordon University, St. Andrew Street, AB25 1HG, Aberdeen, UK

    Peng Zhang  & Lei Wang  & 

  5. School of Computing, University of Glasgow, 18 Lilybank Gardens, G 128 QQ, Glasgow, UK

    Sachi Arafat

Rights and permissions

Reprints and permissions

Copyright information

© 2011 Springer-Verlag Berlin Heidelberg

About this paper

Cite this paper

Wittek, P., Darányi, S. (2011). Introducing Scalable Quantum Approaches in Language Representation. In: Song, D., Melucci, M., Frommholz, I., Zhang, P., Wang, L., Arafat, S. (eds) Quantum Interaction. QI 2011. Lecture Notes in Computer Science, vol 7052. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-24971-6_2

Download citation

  • DOI: https://doi.org/10.1007/978-3-642-24971-6_2

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-642-24970-9

  • Online ISBN: 978-3-642-24971-6

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation