Impact of Recent Hardware and Software Trends on High Performance Transaction Processing and Analytics

  • C. Mohan
Part of the Lecture Notes in Computer Science book series (LNCS, volume 6417)


In this paper, I survey briefly some of the recent and emerging trends in hardware and software features which impact high performance transaction processing and data analytics applications. These features include multicore processor chips, ultra large main memories, flash storage, storage class memories, database appliances, field programmable gate arrays, transactional memory, key-value stores, and cloud computing. While some applications, e.g., Web 2.0 ones, were initially built without traditional transaction processing functionality in mind, slowly system architects and designers are beginning to address such previously ignored issues. The availability, analytics and response time requirements of these applications were initially given more importance than ACID transaction semantics and resource consumption characteristics. A project at IBM Almaden is studying the implications of phase change memory on transaction processing, in the context of a key-value store. Bitemporal data management has also become an important requirement, especially for financial applications. Power consumption and heat dissipation properties are also major considerations in the emergence of modern software and hardware architectural features. Considerations relating to ease of configuration, installation, maintenance and monitoring, and improvement of total cost of ownership have resulted in database appliances becoming very popular. The MapReduce paradigm is now quite popular for large scale data analysis, in spite of the major inefficiencies associated with it.


Analytics Appliances Cloud Computing Databases FPGAs Hardware Key-Value Stores Multicore Performance Software Storage Class Memories Transaction Processing 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Fang, R., Hsiao, H., He, B., Mohan, C., Wang, Y.: High Performance Database Logging using Storage Class Memory (2010) (submitted for publication)Google Scholar
  2. 2.
    Freitas, R., Wilcke, W.: Storage-Class Memory: The Next Storage System Technology. IBM Journal of Research and Development 52(4), 439–447 (2008)CrossRefGoogle Scholar
  3. 3.
    Lee, B., Ipek, E., Mutlu, O., Burger, D.: Phase Change Memory Architecture and the Quest for Scalability. Communications of the ACM 51(7) (July 2010)Google Scholar
  4. 4.
    Johnson, R., Pandis, I., Hardavellas, N., Ailamaki, A., Falsafi, B.: Shore-MT: A Scalable Storage Manager for the Multicore Era. In: Proc. 12th International Conference on Extending Data Base Technology (EDBT), St. Petersburg, Russia (March 2009)Google Scholar
  5. 5.
    DeCandia, G., Hastorun, D., Jampani, M., Kakulapati, G., Lakshman, A., Pilchin, A., Sivasubramanian, S., Vosshall, P., Vogels, W.: Dynamo: Amazon’s Highly Available Key-Value Store. In: Proc. 21st ACM Symposium on Operating Systems Principles (SOSP), Stevenson, USA (October 2007)Google Scholar
  6. 6.
    Lee, R., Ding, X., Chen, F., Lu, Q., Zhang, X.: MCC-DB: Minimizing Cache Conflicts in Muli-Core Processors for Databases. In: Proc. 35th International Conference on Very Large Data Bases (VLDB), Lyon, France (August 2009)Google Scholar
  7. 7.
    Shriraman, A., Dwarkadas, S., Scott, M.: Tapping into Parallelism with Transactional Memory. The USENIX Magazine 34(2), 12–23 (2009)Google Scholar
  8. 8.
    Porter, D., Hofmann, O., Witchel, E.: Is the Optimism in Optimistic Concurrency Warranted? In: Proc. 11th Workshop on Hot Topics in Operating Systems (HotOS), San Diego, USA (May 2007)Google Scholar
  9. 9.
    Porter, D., Witchel, E.: Understanding Transactional Memory Performance. In: Proc. IEEE International Symposium on Performance Analysis of Systems and Software (ISPASS), White Plains, USA (March 2010)Google Scholar
  10. 10.
    Pankratius, V., Adl-Tabatabai, A.-R., Otto, F.: Does Transactional Memory Keep Its Promises? Results from an Empirical Study, Technical Report 2009-12, IPD, University of Karlsruhe, Karlsruhe, Germany (September 2009)Google Scholar
  11. 11.
    Dias, R., Lourenco, J.: Unifying Memory and Database Transactions. In: Sips, H., Epema, D., Lin, H.-X. (eds.) Euro-Par 2009 Parallel Processing. LNCS, vol. 5704, pp. 349–360. Springer, Heidelberg (2009)CrossRefGoogle Scholar
  12. 12.
    Rossbach, C., Hofmann, O., Witchel, E.: Is Transactional Programming Actually Easier? In: Proc. 15th ACM SIGPLAN Annual Symposium on Principles and Practice of Parallel Programming (PPoPP), Bangalore, India (January 2010)Google Scholar
  13. 13.
    Mohan, C., Haderle, D., Lindsay, B., Pirahesh, H., Schwarz, P.: ARIES: A Transaction Recovery Method Supporting Fine-Granularity Locking and Partial Rollbacks Using Write-Ahead Logging. ACM Transactions on Database Systems 17(1), 94–162 (1992)CrossRefGoogle Scholar
  14. 14.
    Condit, J., Nightingale, E., Frost, C., Ipek, E., Lee, B., Burger, D., Coetzee, D.: Better I/O Through Byte-Addressable, Persistent Memory. In: Proc. 23rd ACM Symposium on Operating Systems Principles (SOSP), Big Sky, USA (October 2009)Google Scholar
  15. 15.
    Ousterhout, J., Agrawal, P., Erickson, D., Kozyrakis, C., Leverich, J., Mazières, D., Mitra, S., Narayanan, A., Parulkar, G., Rosenblum, M., Rumble, S., Stratmann, E., Stutsman, R.: The case for RAMClouds: Scalable High-Performance Storage Entirely in DRAM. ACM SIGOPS Operating Systems Review 43(4), 92–105 (2010)CrossRefGoogle Scholar
  16. 16.
    IBM SolidDB In-memory Database,
  17. 17.
    Oracle TimesTen In-memory Database,
  18. 18.
    Mohan, C.: Implications of Storage Class Memories (SCMs) on Software Architectures. Presentation at 2nd TPC Technology Conference on Performance Evaluation & Benchmarking (TPC TC), Collocated with 36th International Conference on Very Large Data Bases (VLDB), Singapore (September 2010),
  19. 19.
  20. 20.
    Mueller, R., Teubner, J.: FPGA: What’s in it for a Database? Tutorial. In: Proc. ACM SIGMOD International Conference on Management of Data, Providence, USA (June 2009)Google Scholar
  21. 21.
    Anselmi, G., Blanchard, B., Cho, Y., Hales, C., Quezada, M.: IBM Power 770 and 780 Technical Overview and Introduction, IBM Redpaper REDP-4639-00 (March 2010),
  22. 22.
    Sears, R., van Ingen, C., Gray, J.: To BLOB or Not To BLOB: Large Object Storage in a Database or a Filesystem? Microsoft Research Technical Report MSR-TR-2006-45 (April 2006)Google Scholar
  23. 23.
    Mohan, C., Haderle, D.: Algorithms for Flexible Space Management in Transaction Systems Supporting Fine-Granularity Locking. In: Proc. 4th International Conference on Extending Database Technology, Cambridge, UK (March 1994); A longer version of this paper is available as IBM Research Report RJ9732, IBM Almaden Research Center, San Jose, USA (March 1994)Google Scholar
  24. 24.
    Yen, S.: Memcached Slab Allocator (June 2009),
  25. 25.
    Wikipedia: Multi-core Processor,
  26. 26.
    Wikipedia: Symmetric Multiprocessing,
  27. 27.
    Dewitt, D., Gray, J.: Parallel Database Systems: The Future of High Performance Database Systems. Communications of the ACM 35(6), 85–98 (1992)CrossRefGoogle Scholar
  28. 28.
    DIAS: Data-Intensive Applications and Systems Laboratory, EPFL Lausanne, Switzerland,
  29. 29.
    Database Group. MIT, Cambridge,
  30. 30.
    Database Research at Yale. Yale University, New Haven,
  31. 31.
    Information Management, IBM Almaden Research Center, San Jose, USA,
  32. 32.
    The Database Research Group, ETH Zurich, Switzerland,
  33. 33.
    Distributed Systems Lab. University of California at Santa Barbara, Santa Barbara, USA,
  34. 34.
    RAMCloud Project. Stanford University, Stanford, USA,
  35. 35.
    Cooper, B.F., Silberstein, A., Tam, E., Ramakrishnan, R., Sears, R.: Benchmarking Cloud Serving Systems with YCSB. In: ACM Symposium on Cloud Computing (SOCC), Indianapolis, USA (June 2010)Google Scholar
  36. 36.
  37. 37.
    Web Information Management, Yahoo! Research,
  38. 38.
    Apache Cassandra Project,
  39. 39.
  40. 40.
    Boike, N.: Leading Edge - IBM’s Workload-Optimized Approach Puts Your Business in Front of the Competition. IBM Systems Magazine (October 2010),
  41. 41.
    Wikipedia: Temporal Database,
  42. 42.
    Beulke, D., and Associates: IBM DB2 10 for z/OS Beta,
  43. 43.
  44. 44.
    Atwood, G.: The Evolution of Phase Change Memory, Micron White Paper (July 2010),
  45. 45.
    Cloud Computing Futures, Microsoft Research, Redmond, USA,
  46. 46.
    Ford, D., Labelle, F., Popovici, F., Stokely, M., Truong, V.-A., Barroso, L., Grimes, C., Quinlan, S.: Availability in Globally Distributed Storage Systems. In: Proc. 9th USENIX Symposium on Operating Systems Design and Implementation (OSDI), Vancouver, Canada (October 2010)Google Scholar
  47. 47.
    Portal for Multicore Processor News and Information,
  48. 48.
    Hoelzle, U.: Brawny Cores Still Beat Wimpy Cores, Most of the Time. To Appear in IEEE MICRO (2010)Google Scholar
  49. 49.
  50. 50.
    Chen, A., Cruickshank, J., Costantini, C., Haug, V., Maciel, C., Schmidt, J.: IBM Power 795 Technical Overview and Introduction - A Draft IBM Redpaper Publication, REDP-4640-00 (September 2010),
  51. 51.
  52. 52.
    Redis – A Persistent Key-Value Database with Built-in Net Interface Written in ANSI-C for Posix Systems,
  53. 53.
    Project Voldemort – A Distributed Database,

Copyright information

© Springer-Verlag Berlin Heidelberg 2011

Authors and Affiliations

  • C. Mohan
    • 1
  1. 1.IBM Almaden Research CenterSan JoseUSA

Personalised recommendations