Advertisement

DAPSS: Exact Subsequence Matching for Data Streams

  • Yasuhiro Fujiwara
  • Yasushi Sakurai
  • Masashi Yamamuro
Part of the Lecture Notes in Computer Science book series (LNCS, volume 3882)

Abstract

There is much interest in the processing of data streams for applications in the fields such as financial analysis, network monitoring, mobile services, and sensor network management. The key characteristic of stream data, that it continues to arrive, demands a new approach. This paper focuses on the problem of detecting, exactly, similar pairs of subsequences of arbitrary length in streaming fashion. We propose DAPSS (DAta stream Processing for Store and Search), an efficient and effective method to detect the similar pairs, which keeps (1) the feature data of each sequence in the memory space and (2) the compressed data of the original sequences in the disk space. Experiments on synthetic and real data sets show that DAPSS is significantly (up to 35 times) faster than the naive method while it guarantees the correctness of query results.

Keywords

Data Stream Query Processing Memory Space Feature Matrix Lossless Compression 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Agrawal, R., Faloutsos, C., Swami, A.N.: Efficient similarity search in sequence databases. In: Lomet, D.B. (ed.) FODO 1993. LNCS, vol. 730, pp. 69–84. Springer, Heidelberg (1993)CrossRefGoogle Scholar
  2. 2.
    Beckmann, N., Kriegel, H.P., Schneider, R., Seeger, B.: The r*-tree: An efficient and robust access method for points and rectangles. In: SIGMOD Conference, pp. 322–331 (1990)Google Scholar
  3. 3.
    Faloutsos, C., Ranganathan, M., Manolopoulos, Y.: Fast subsequence matching in time-series databases. In: SIGMOD Conference, pp. 419–429 (1994)Google Scholar
  4. 4.
    Moon, Y.S., Whang, K.Y., Han, W.S.: General match: a subsequence matching method in time-series databases based on generalized windows. In: SIGMOD Conference, pp. 382–393 (2002)Google Scholar
  5. 5.
    Indyk, P., Koudas, N., Muthukrishnan, S.: Identifying representative trends in massive time series data sets using sketches. In: VLDB, pp. 363–372 (2000)Google Scholar
  6. 6.
    Sakurai, Y., Yoshikawa, M., Faloutsos, C.: Ftw: Fast similarity search under the time warping distance. In: PODS, pp. 326–337 (2005)Google Scholar
  7. 7.
    Golab, L., Özsu, M.T.: Issues in data stream management. SIGMOD Record 32(2), 5–14 (2003)CrossRefGoogle Scholar
  8. 8.
    Babcock, B., Babu, S., Datar, M., Motwani, R., Widom, J.: Models and issues in data stream systems. In: PODS, pp. 1–16 (2002)Google Scholar
  9. 9.
    Law, Y.N., Wang, H., Zaniolo, C.: Query languages and data models for database sequences and data streams. In: VLDB, pp. 492–503 (2004)Google Scholar
  10. 10.
    Balakrishnan, H., Balazinska, M., Carney, D., Çetintemel, U., Cherniack, M., Convey, C., Galvez, E.F., Salz, J., Stonebraker, M., Tatbul, N., Tibbetts, R., Zdonik, S.B.: Retrospective on aurora. VLDB J. 13, 370–383 (2004)CrossRefGoogle Scholar
  11. 11.
    Johnson, T., Muthukrishnan, S., Rozenbaum, I.: Sampling algorithms in a stream operator. In: SIGMOD Conference, pp. 1–12 (2005)Google Scholar
  12. 12.
    Chandrasekaran, S., Franklin, M.J.: Remembrance of streams past: Overload-sensitive management of archived streams. In: VLDB, pp. 348–359 (2004)Google Scholar
  13. 13.
    Yu, J.X., Chong, Z., Lu, H., Zhou, A.: False positive or false negative: Mining frequent itemsets from high speed transactional data streams. In: VLDB, pp. 204–215 (2004)Google Scholar
  14. 14.
    Aggarwal, C.C., Han, J., Wang, J., Yu, P.S.: A framework for projected clustering of high dimensional data streams. In: VLDB, pp. 852–863 (2004)Google Scholar
  15. 15.
    Sakurai, Y., Papadimitriou, S., Faloutsos, C.: Braid: Stream mining through group lag correlations. In: SIGMOD Conference, pp. 599–610 (2005)Google Scholar
  16. 16.
    Zhu, Y., Shasha, D.: Statstream: Statistical monitoring of thousands of data streams in real time. In: Bressan, S., Chaudhri, A.B., Li Lee, M., Yu, J.X., Lacroix, Z. (eds.) CAiSE 2002 and VLDB 2002. LNCS, vol. 2590, pp. 358–369. Springer, Heidelberg (2003)Google Scholar
  17. 17.
    Bulut, A., Singh, A.K.: A unified framework for monitoring data streams in real time. In: ICDE, pp. 44–55 (2005)Google Scholar
  18. 18.
    Pong Chan, K., Fu, A.W.C.: Efficient time series matching by wavelets. In: ICDE, pp. 126–133 (1999)Google Scholar
  19. 19.
    Katayama, N., Satoh, S.: The sr-tree: An index structure for high-dimensional nearest neighbor queries. In: SIGMOD Conference, pp. 369–380 (1997)Google Scholar
  20. 20.
    Sakurai, Y., Yoshikawa, M., Uemura, S., Kojima, H.: The a-tree: An index structure for high-dimensional spaces using relative approximation. In: VLDB, pp. 516–526 (2000)Google Scholar
  21. 21.
    Kollios, G., Papadopoulos, D., Gunopulos, D., Tsotras, V.J.: Indexing mobile objects using dual transformations. VLDB J. 14, 238–256 (2005)CrossRefGoogle Scholar
  22. 22.
    Keogh, E.J., Chakrabarti, K., Mehrotra, S., Pazzani, M.J.: Locally adaptive dimensionality reduction for indexing large time series databases. In: SIGMOD Conference, pp. 188–228 (2001)Google Scholar
  23. 23.
    Korn, F., Jagadish, H.V., Faloutsos, C.: Efficiently supporting ad hoc queries in large datasets of time sequences. In: SIGMOD Conference, pp. 289–300 (1997)Google Scholar
  24. 24.
    Burrows, M., Wheeler, D.J.: A block-sorting lossless data compression algorithm. Technical Report 124, SRC Research Report (1994)Google Scholar
  25. 25.
    Huffman, D.A.: A method for the construction of minimum redundancy codes. Proc. IRE 40, 1098–1101 (1952)CrossRefMATHGoogle Scholar
  26. 26.
    Ziv, J., Lempel, A.: A universal algorithm for sequential data compression. IEEE Transactions on Information Theory 23, 337–343 (1977)MathSciNetCrossRefMATHGoogle Scholar
  27. 27.
    Ciaccia, P., Patella, M., Zezula, P.: M-tree: An efficient access method for similarity search in metric spaces. In: VLDB, pp. 426–435 (1997)Google Scholar
  28. 28.
    Bozkaya, T., Özsoyoglu, Z.M.: Distance-based indexing for high-dimensional metric spaces. In: SIGMOD Conference, pp. 357–368 (1997)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2006

Authors and Affiliations

  • Yasuhiro Fujiwara
    • 1
  • Yasushi Sakurai
    • 1
  • Masashi Yamamuro
    • 1
  1. 1.NTT Cyber Space LaboratoriesNTT CorporationKanagawaJapan

Personalised recommendations