Distributed Query Processing Using Suffix Arrays

  • Mauricio Marín
  • Gonzalo Navarro
Part of the Lecture Notes in Computer Science book series (LNCS, volume 2857)

Abstract

Suffix arrays are more efficient than inverted files for solving complex queries in a number of applications related to text databases. Examples arise when dealing with biological or musical data or with texts written in oriental languages, and when searching for phrases, approximate patterns and, in general, regular expressions involving separators. In this paper we propose algorithms for processing in parallel batches of queries upon distributed text databases. We present efficient alternatives for speeding up query processing using distributed realizations of suffix arrays. Empirical results obtained from natural language text on a cluster of PCs show that the proposed algorithms are efficient in practice.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    MacFarlane, A.A., McCann, J.A., Robertson, S.E.: Parallel search using partitioned inverted files. In: 7th International Symposium on String Processing and Information Retrieval, pp. 209–220 (2000)Google Scholar
  2. 2.
    Santos Badue, C., Baeza-Yates, R., Ribeiro-Neto, B., Ziviani, N.: Concurrent query processing using distributed inverted files. In: 8th International Symposium on String Processing and Information Retrieval, pp. 10–20 (2001)Google Scholar
  3. 3.
    Baeza, R., Ribeiro, B.: Modern Information Retrieval. Addison-Wesley, Reading (1999)Google Scholar
  4. 4.
    Chung, S.H., Kwon, H.C., Ryu, K.R., Jang, H.K., Kim, J.H., Choi, C.A.: Parallel information retrieval on a SCI-based PC-NOW. In: Workshop on Personal Computers based Networks of Workstations (PC-NOW 2000), Springer, Heidelberg (2000)Google Scholar
  5. 5.
    Ferragina, P., Luccio, F.: String search in coarse-grained parallel computers. Algorithmica 24, 177–194 (1999)MATHCrossRefMathSciNetGoogle Scholar
  6. 6.
    Kitajima, J., Navarro, G.: A fast distributed suffix array generation algorithm. In: 6th International Symposium on String Processing and Information Retrieval, pp. 97–104 (1999)Google Scholar
  7. 7.
    McColl, W.F.: General purpose parallel computing. In: Gibbons, A.M., Spirakis, P. (eds.) Lectures on Parallel Computation, pp. 337–391. Cambridge University Press, Cambridge (1993)Google Scholar
  8. 8.
    Navarro, G., Kitajima, J., Ribeiro, B., Ziviani, N.: Distributed generation of suffix arrays. In: Hein, J., Apostolico, A. (eds.) CPM 1997. LNCS, vol. 1264, pp. 102–115. Springer, Heidelberg (1997)Google Scholar
  9. 9.
    Ribeiro, B., Kitajima, J., Navarro, G., Santana, C., Ziviani, N.: Parallel generation of inverted lists for distributed text collections. In: XVIII Conference of the Chilean Computer Science Society, pp. 149–157 (1998)Google Scholar
  10. 10.
    Skillicorn, D.B., Hill, J.M.D., McColl, W.F.: Questions and answers about BSP. Technical Report PRG-TR-15-96, Computing Laboratory, Oxford University (1996); Also in Journal of Scientific Programming 6(3) (1997)Google Scholar
  11. 11.
    Skillicorn, D.B., Talia, D.: Models and languages for parallel computation. ACM Computing Surveys 20(2) (1998)Google Scholar
  12. 12.
    Valiant, L.G.: A bridging model for parallel computation. Comm. ACM 33, 103–111 (1990)CrossRefGoogle Scholar
  13. 13.
    BSP World-wide Standard, http://www.bsp-worldwide.org
  14. 14.
    BSP PUB Library at Paderborn University, http://www.uni-paderborn.de/bsp

Copyright information

© Springer-Verlag Berlin Heidelberg 2003

Authors and Affiliations

  • Mauricio Marín
    • 1
    • 3
  • Gonzalo Navarro
    • 2
    • 3
  1. 1.Dept. of Computer ScienceUniversity of Magallanes 
  2. 2.Dept. of Computer ScienceUniversity of Chile 
  3. 3.Center for Web Research 

Personalised recommendations