Intelligent Data Granulation on Load: Improving Infobright’s Knowledge Grid

  • Dominik Ślęzak
  • Marcin Kowalski
Conference paper

DOI: 10.1007/978-3-642-10509-8_3

Volume 5899 of the book series Lecture Notes in Computer Science (LNCS)
Cite this paper as:
Ślęzak D., Kowalski M. (2009) Intelligent Data Granulation on Load: Improving Infobright’s Knowledge Grid. In: Lee Y., Kim T., Fang W., Ślęzak D. (eds) Future Generation Information Technology. FGIT 2009. Lecture Notes in Computer Science, vol 5899. Springer, Berlin, Heidelberg

Abstract

One of the major aspects of Infobright’s relational database technology is automatic decomposition of each of data tables onto Rough Rows, each consisting of 64K of original rows. Rough Rows are automatically annotated by Knowledge Nodes that represent compact information about the rows’ values. Query performance depends on the quality of Knowledge Nodes, i.e., their efficiency in minimizing the access to the compressed portions of data stored on disk, according to the specific query optimization procedures. We show how to implement the mechanism of organizing the incoming data into such Rough Rows that maximize the quality of the corresponding Knowledge Nodes. Given clear business-driven requirements, the implemented mechanism needs to be fully integrated with the data load process, causing no decrease in the data load speed. The performance gain resulting from better data organization is illustrated by some tests over our benchmark data. The differences between the proposed mechanism and some well-known procedures of database clustering or partitioning are discussed. The paper is a continuation of our patent application [22].

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

Copyright information

© Springer-Verlag Berlin Heidelberg 2009

Authors and Affiliations

  • Dominik Ślęzak
    • 1
  • Marcin Kowalski
    • 2
  1. 1.Institute of MathematicsUniversity of WarsawWarsawPoland
  2. 2.Infobright Inc., PolandWarsawPoland