How to Analyze 1 Billion CDRs per Sec on $200K Hardware

  • Ian Pattison
  • Russ Green
Part of the Lecture Notes in Computer Science book series (LNCS, volume 1819)

Abstract

Modern telecommunication systems generate large amounts of data such as details of network traffic and service usage (call detail records). Amongst other information, these huge databases contain the behavioral patterns of the company’s customers. By extracting this data, a telecommunications company (Telco) can better understand the needs of its customers.

Traditional database technology scales to hold vast amounts of data but has severe performance limitations when it comes to analyzing this data. Data mining tools, which often store data in a private representation, offer fast analysis on small data sets but generally do not scale beyond a few million rows.

This paper presents a scalable, parallel data analysis engine capable of processing tens of millions of rows per second per CPU. This technology enables knowledge workers to get sub-second responses to queries that would previously have taken minutes or even hours.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Berry, M.J.A., Linoff, G.: Data Mining Techniques. John Wiley & Sons, Chichester (1997)Google Scholar
  2. 2.
    Boncz, A., Ruhl, T., Kwakkel, K.: The Drill Down Benchmark. In: Proc. of 24th VLDB Conf. (1998)Google Scholar
  3. 3.
    Chattratichat, J., et al.: Large Scale Data Mining: Challenges and Responses. In: Proceedings KDD 1997 (1997)Google Scholar
  4. 4.
    Inmon, W.H.: Building the Data Warehouse. John Wiley & Sons, Chichester (1996)Google Scholar
  5. 5.
    TPC-D benchmark for ProLiant 65000 6/200-1, report data september 19 (1997) Google Scholar
  6. 6.
    Triangle Database Marketing: Report produced for internal use of TANTAU Software Inc. June 03 (1999)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2000

Authors and Affiliations

  • Ian Pattison
    • 1
  • Russ Green
    • 1
  1. 1.TANTAU Software IncFriedrichsdorfGermany

Personalised recommendations