Investigating the Integration of Supercomputers and Data-Warehouse Appliances

  • Ron A. Oldfield
  • George Davidson
  • Craig Ulmer
  • Andrew Wilson
Part of the Lecture Notes in Computer Science book series (LNCS, volume 8374)


Two decades of experience with massively parallel supercomputing has given insight into the problem domains where these architectures are cost effective. Likewise experience with database machines and more recently massively parallel database appliances has shown where these architectures are valuable. Combining both architectures to simultaneously solve problems has received much less attention. In this paper, we describe a motivating application for economic modeling that requires both HPC and database capabilities. Then we discuss hardware and software integration issues related to a direct integration of a Cray XT supercomputer and a Netezza database appliance.


Message Passing Interface Latent Semantic Analysis Sandia National Laboratory Service Node Database Machine 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Alverson, R., Roweth, D., Kaplan, L.: The Gemini system interconnect. In: Proceedings of the 18th Annual Symposium on High Performance Interconnects (HOTI), Mountain View, CA, pp. 83–87. IEEE Computer Society Press (August 2010)Google Scholar
  2. 2.
    Brightwell, R., Pedretti, K., Underwood, K., Hudson, T.: SeaStar interconnect: Balanced bandwidth for scalable performance. IEEE Micro 26(3), 41–57 (2006)CrossRefGoogle Scholar
  3. 3.
    Brightwell, R., Riesen, R., Lawry, B., Maccabe, A.B.: Portals 3.0: protocol building blocks for low overhead communication. In: Proceedings of the International Parallel and Distributed Processing Symposium, Fort Lauderdale, FL, p. 268 (April 2002)Google Scholar
  4. 4.
    Bruaset, A.M., Tveito, A. (eds.): Numerical Solution of Partial Differential Equations on Parallel Computers. Lecture Notes in Computational Science and Engineering, vol. 51. Springer (2006)Google Scholar
  5. 5.
    Carns, P., et al.: Understanding and improving computational science storage access through continuous characterization. In: IEEE Conference on Mass Storage Systems and Technologies, pp. 1–14 (2011)Google Scholar
  6. 6.
    Davidson, G.S., et al.: Data-centric computing with the Netezza architecture. Technical Report SAND2006-3640, Sandia National Laboratories (2006)Google Scholar
  7. 7.
    Deerwester, S.C.: et al. Indexing by latent semantic analysis. Journal of the American Society for Information Science 41(6), 391–407 (1990)CrossRefGoogle Scholar
  8. 8.
    DeWitt, D.J., Hawthorn, P.B.: A performance evaluation of data base machine architectures. In: Proceedings of the Seventh International Conference on Very Large Data Bases, VLDB 1981, Cannes, France, pp. 199–214 (1981) (invited paper)Google Scholar
  9. 9.
    Dongarra, J., Luszczek, P., Petitet, A.: The LINPACK benchmark: past, present, and future. Concurrency - Practice and Experience 15(9), 803–820 (2003)CrossRefGoogle Scholar
  10. 10.
    Eidson, E.D., Ehlen, M.A.: NISAC agent-based laboratory for economics (N-ABLETM): Overview of agent and simulation architectures. Technical Report SAND2005-0263, Sandia National Laboratories (2005)Google Scholar
  11. 11.
    Francisco, P.: The Netezza data appliance architecture: A platform for high performance data warehousing and analytics. IBM Redguide (2011),
  12. 12.
    Graves, S.: HPC databases: The data ingest challenge. HPCWire (June 2007)Google Scholar
  13. 13.
    Greenberg, D.S., et al.: A system software architecture for high-end computing. In: Proceedings of SC 1997: High Performance Networking and Computing, San Jose, California, pp. 1–15. ACM Press (November 1997)Google Scholar
  14. 14.
    Heroux, M., et al.: An overview of Trilinos. Technical Report SAND2003-2927, Sandia National Laboratories (2003)Google Scholar
  15. 15.
    Kelly, S.M., Brightwell, R.: Software architecture of the Light Weight Kernel, Catamount. In: Proceedings of the Cray User Group Meeting, Albuquerque, NM (May 2005)Google Scholar
  16. 16.
    Maccabe, A.B., Wheat, S.R.: Message passing in PUMA. Technical Report SAND-93-0935C, Sandia National Labs (1993)Google Scholar
  17. 17.
    Negash, S.: Business intelligence. Communications of the Association for Information Systems 13(15) (2004)Google Scholar
  18. 18.
    Oldfield, R.A., et al.: Trilinos I/O Support (Trios). Scientific Programming (August 2012)Google Scholar
  19. 19.
    Oldfield, R.A., Kordenbrock, T., Lofstead, J.: Developing integrated data services for Cray systems with a Gemini interconnect. In: Cray User Group Meeting (April 2012)Google Scholar
  20. 20.
    Plale, B., Schwan, K.: Dynamic querying of streaming data with the dQUOB system. IEEE Transactions on Parallel and Distributed Systems 14(4), 422–432 (2003)CrossRefGoogle Scholar
  21. 21.
    Ralston, A., Reilly, E.D., Hemmendinger, D. (eds.): Encyclopedia of Computer Science, 4th edn. Wiley (2003)Google Scholar
  22. 22.
    Riedel, E., Faloutsos, C., Gibson, G.A., Nagle, D.: Active disks for large-scale data processing. IEEE Computer 34(6), 68–74 (2001)CrossRefGoogle Scholar
  23. 23.
    Sloan, R.D.: A practical implementation of the data base machine – Teradata DBC/1012. In: Proceedings of the Twenty-Fifth Hawaii International Conference on System Sciences, pp. 320–327 (January 1992)Google Scholar
  24. 24.
    The BlueGene/L Team: An overview of the BlueGene/L supercomputer. In: Proceedings of SC2002: High Performance Networking and Computing, Baltimore, MD (November 2002)Google Scholar
  25. 25.
    Thuraisingham, B.M.: Web data mining and applications in business intelligence and counter-terrorism. CRC Press (2005)Google Scholar
  26. 26.
    Ulmer, C., Bayer, G., Choe, Y.R., Roe, D.: Exploring data warehouse appliances for mesh analysis applications. In: Proceedings of the 43rd International Conference on System Sciences, Koloa, Kauai, Hawai, pp. 1–10. IEEE Press (January 2010)Google Scholar
  27. 27.
    Wallace, D.: Compute Node Linux: Overview, progress to date & roadmap. In: Proceedings of the Cray User Group Meeting, Helsinki, Finland (May 2007)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2014

Authors and Affiliations

  • Ron A. Oldfield
    • 1
  • George Davidson
    • 1
  • Craig Ulmer
    • 1
  • Andrew Wilson
    • 1
  1. 1.Sandia National LaboratoriesUSA

Personalised recommendations