Management of Highly Dynamic Multidimensional Data in a Cluster of Workstations

  • Vassil Kriakov
  • Alex Delis
  • George Kollios
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 2992)

Abstract

Due to the proliferation and widespread use of mobile devices and satellite based sensors there has been increased interest in storing and managing spatio-temporal and sensory data. It has been recognized that centralized and monolithic index structures are not scalable enough to address the highly dynamic nature (high update rates) and the unpredictable access patterns in such datasets. In this paper, we propose an adaptive networked index method designed to address the above challenges. Our method not only facilitates fast query and update response times via dynamic data partitioning but is also able to self-tune highly loaded sites. Our contributions consist of techniques that offer dynamic load balancing of computing sites, non-disruptive on-the-fly addition/removal of storing sites, distributed collaborative decision making for the self-administering of the manager, and statistics-based data reorganization. These features are incorporated into a distributed software layer prototype used to evaluate the design choices made. Our experimentation compares the performance of a baseline configuration with our multi-site system, examines the attained speed-up as a function of the sites participating, investigates the effect of data reorganization on query/update response times, asserts the effectiveness of our proposed dynamic load balancing method, and examines the behavior of the system under diverse types of multi-dimensional data.

Keywords

Data Management in Cluster of Workstations Networked Storage Manager Self-tuning Storage Nodes and Multi-dimensional Data 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Abello, J., Korn, J.: MGV: A System for Visualizing Massive Multidigraphs. IEEE Trans. on Visualization and Computer Graphics 8(1) (January-March 2002)Google Scholar
  2. 2.
    Barclay, T., Slutz, D., Gray, J.: TerraServer: A Spatial Data Warehouse. In: Proc. of ACM SIGMOD, pp. 307–318 (2000)Google Scholar
  3. 3.
    Bastani, F., Iyengar, S., Yen, I.: Concurrent Maintenance of Data Structures in a Distributed Environment. The Computer Journal 31(12), 165–174 (1988)MATHCrossRefGoogle Scholar
  4. 4.
    Beckmann, N., Kriegel, H., Schneider, R., Seeger, B.: The R*-Tree: An Efficient and Robust Access Method for Points and Rectangles. In: Proc. of ACM SIGMOD 1990, pp. 322–331. ACM Press, New York (1990)CrossRefGoogle Scholar
  5. 5.
    Calhoun, V., Adali, T., Pearlson, G. (Non)stationarity of Temporal Dynamics in fMRI. In: 21st Annual Conference of Engineering in Medicine and Biology, Atlanta, GA, October 1999, vol. 2 (1999)Google Scholar
  6. 6.
    Ellis, C.: Distributed Data Structures: A Case Study. IEEE Transactions on Computers 34(12), 1178–1185 (1985)Google Scholar
  7. 7.
    The Earth Observing System Data and Information System, http://spsosun.gsfc.nasa.gov/eosinfo/EOSDIS_Site/index.html
  8. 8.
    Gray, J., Reuter, A.: Transaction Processing: Concepts and Techniques. Morgan-Kaufman, San Mateo (1992)Google Scholar
  9. 9.
    Guttman, A.: R-Trees: A Dynamic Index Structure for Spatial Searching. In: SIGMOD 1984, Proceedings of Annual Meeting, Boston, Massachusetts, June 18-21, pp. 47–57. ACM Press, New York (1984)CrossRefGoogle Scholar
  10. 10.
    Honishi, T., Satoh, T., Inoue, U.: An Index Structure for Parallel Database Processing. In: IEEE Second International Workshop on Research Issues on Data Engineering, pp. 224–225 (1992)Google Scholar
  11. 11.
    Johnson, T., Krishna, P., Colbrook, A.: Distributed Indices for Accessing Distributed Data. In: IEEE Symposium on Mass Storage Systems (MSS 1993), Los Alamitos, Ca., USA, April 1993, pp. 199–208. IEEE Computer Society Press, Los Alamitos (1993)CrossRefGoogle Scholar
  12. 12.
    Kamel, I., Faloutsos, C.: Parallel R-trees. In: Proceedings of the 1992 ACM SIGMOD International Conference on Management of Data, San Diego, California, June 2-5, pp. 195–204. ACM Press, New York (1992)CrossRefGoogle Scholar
  13. 13.
    Koudas, N., Faloutsos, C., Kamel, I.: Declustering Spatial Databases on a Multi-Computer Architecture. In: Apers, P.M.G., Bouzeghoub, M., Gardarin, G. (eds.) EDBT 1996. LNCS, vol. 1057, Springer, Heidelberg (1996)CrossRefGoogle Scholar
  14. 14.
    Kroll, B., Widmayer, P.: Distributing a Search Structure Among a Growing Number of Processors. In: Proceedings of the 1994 ACM SIGMOD Conference, pp. 265–276 (1994)Google Scholar
  15. 15.
    Lee, M., Kitsuregawa, M., Ooi, B., Tan, K., Mondal, A.: Towards Self-Tuning Data Placement in Parallel Database Systems. In: Proc. of ACM SIGMOD 2000, pp. 225–236 (2000)Google Scholar
  16. 16.
    Litwin, W., Neimat, M.A., Schneider, D.: Linear Hashing for Distributed Files. In: Proceedinsg of the 1993 SIGMOD Conference, Washington D.C. (May 1993)Google Scholar
  17. 17.
    Matsliach, G., Shmueli, O.: An Efficient Method for Distributing Search Structures. In: First International Conference on Parallel and Distributed Information Systems, pp. 159–166 (1991)Google Scholar
  18. 18.
    Mondal, A., Kitsuregawa, M., Ooi, B.C., Tan, K.L.: R-tree-based Data Migration and Self-tuning Strategies in Shared-nothing Spatial Databases. In: Proceedings of ACM Geographic Information Systems, pp. 28–33. ACM Press, New York (2001)Google Scholar
  19. 19.
    Ousterhout, J.K., Hamachi, G.T., Mayo, R.N., Scott, W.S., Taylor, G.S.: Magic: A VLSI Layout System. In: 21st Design Automation Conference, June 1984, pp. 152–159 (1984)Google Scholar
  20. 20.
    Panagos, E., Biliris, A.: Synchronization and Recovery in a Client-Server Storage System. The VLDB Journal 6(3), 209–223 (1997)CrossRefGoogle Scholar
  21. 21.
    Patel, J., Yu, J.-B., Kabra, N., Tufte, K.: Building a Scaleable Geo-Spatial DBMS: Technology, Implementation, and Evaluation. In: Proc. of the ACM SIGMOD, pp. 336–347 (1997)Google Scholar
  22. 22.
    Porkaew, K., Lazaridis, I., Mehrotra, S.: Querying Mobile Objects in Spatio-Temporal Databases. In: Jensen, C.S., Schneider, M., Seeger, B., Tsotras, V.J. (eds.) SSTD 2001. LNCS, vol. 2121, p. 59. Springer, Heidelberg (2001)CrossRefGoogle Scholar
  23. 23.
    Prado, M., Roa, L., Reina-Tosina, J., Palma, A., Milan, J.A.: Virtual Center for Renal Support: Technological Approach to Patient Physiological Image. IEEE Transactions on Biomedical Engineering 49(12), 1420–1430 (2002)CrossRefGoogle Scholar
  24. 24.
    Reina-Tosina, J., Roa, L.M., Caceres, J., Gomez-Cia, T.: New Approaches Toward the Fully Digital Integrated Management of a Burn Unit. IEEE Transactions on Biomedical Engineering 49(12), 1470–1476 (2002)CrossRefGoogle Scholar
  25. 25.
    Saltenis, S., Jensen, C., Leutenegger, S., Lopez, M.A.: Indexing the Positions of Continuously Moving Objects. In: Proc. of the ACM SIGMOD, May 2000, pp. 331–342 (2000)Google Scholar
  26. 26.
    Salzberg, B., Tsotras, V.J.: Comparison of Access Methods for Time-Evolving Data. ACM Computing Surveys 31(2), 158–221 (1999)CrossRefGoogle Scholar
  27. 27.
    Scheuermann, P., Weikum, G., Zabback, P.: Data Partitioning and Load Balancing in Parallel Disk Systems. VLDB Journal 7(1) (1998)Google Scholar
  28. 28.
    Schnitzer, B., Leutenegger, S.: Master-Client R-Trees: A New Parallel R-Tree Architecture. In: Statistical and Scientific Database Management, pp. 68–77 (1999)Google Scholar
  29. 29.
    Szalay, A., Gray, J., van den Berg, J.: Petabyte Scale Data Mining: Dream or Reality. In: Proc. of SIPE Astronmy Telescopes and Instruments (August 2002)Google Scholar
  30. 30.
    Zeiler, T.L.: LANDSAT Program Report 2002, Technical report, U.S. Geological Survey - U.S. Department of Interior, Sioux Falls, SD, EROS Data Center (2002)Google Scholar
  31. 31.
    Zou, C., Salzberg, B.: Safely and Efficiently Updating References During On-line Reorganization. In: Proc. of VLDB, pp. 512–522 (1998)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2004

Authors and Affiliations

  • Vassil Kriakov
    • 1
  • Alex Delis
    • 2
  • George Kollios
    • 3
  1. 1.Polytechnic UniversityBrooklyn
  2. 2.The Univ. of AthensAthensGreece
  3. 3.Boston UniversityBoston

Personalised recommendations