Abstract
The advancement of multicore processors and database technologies have enable database insertion to be implemented concurrently via multithreading programming. In this work, we evaluate the performance of using multithreading technique to perform database insertion of large data set with known size. The performance evaluation includes techniques such as using single database connection, multithreads the insertion process with respective database connections, single threaded bulk insertion and multithreaded bulk insertion. MySQL 5.2 and SQL Server 2008 were used and the experimental results show that larger datasets bulk insertion of both databases can drastically be improved with multithreading.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsPreview
Unable to display preview. Download preview PDF.
References
Intel® CoreTM i7 Processor Extreme Edition, http://www.intel.com/products/processor/corei7EE/index.html
Intel® Core TM i7 – 920 Desktop Processor Series Product Specifications, http://ark.intel.com/Product.aspx?id=37147
Johnson, R., Ippokratis, P., Hardavellas, N., Ailamaki, A., Falsafi, B.: Shore-MT: A Scalable Storage Manager for the Multicore Era. In: Proceedings of the 12th International Conference on Extending Database Technology: Advances in Database Technology, pp. 24–35. ACM, New York (2009)
Verenker, A.: Using.NET4 Parallel Programming Model to Achieve Data Parallelism in Multi-tier Applications, MSIT, Microsoft Corporation (2010)
Seagate Barracude 7200.10 SATA 3.0Gb/s 320-GB Hard Drive, http://www.seagate.com/ww/v/index.jsp?vgnextoid=2d1099f4fa74c010VgnVCM100000dd04090aRCRD
TransactionScope Class, http://msdn.microsoft.com/en-us/library/system.transactions.transactionscope.aspx
Using the Bulk Loader, http://dev.mysql.com/doc/refman/5.1/en/connector-net-programming-bulk-loader.html
SqlBulkCopy Class, http://msdn.microsoft.com/en-us/library/system.data.sqlclient.sqlbulkcopy.aspx
Performance Counter Constructor, http://msdn.microsoft.com/en-us/library/xx7e9t8e.aspx
Bunn, J.J., Holtman, K., Newman, H.B.: Object Database Scalability for Scientific Workloads. Technical report, California Institute of Technology (2000)
Thread Class, http://msdn.microsoft.com/en-us/library/system.threading.thread.aspx
Lui, D., Wang, S.: Analysis of Database Workloads on Modern Processors. In: Proceedings of the 1st SIGMOD PhD Workshop on Innovation Database Research 2007, pp. 63–68. ACM, New York (2007)
Performance Monitoring, http://www.csharphelp.com/2006/05/performance-monitoring/
How to Create and Terminate Thread (C# Programming Guide), http://msdn.microsoft.com/en-US/library/7a2f3ay4v=VS.80.spx
Zhou, J., Cieslewicz, J., Ross, K.A., Shah, M.: Improving Database Performance on Simultaneous Multithreading Processors. In: Proceedings of the 31st International Conference on Very Large Data Bases 2005, pp. 49–60. VLDB Endowment, Norway (2005)
Haggander, D., Lundberg, L.: Multiprocessor Performance Evaluation of a Telecommunication Fraud Detection Application. In: ARTES Graduate Student Conference, Sweden (1999)
DeWitt, D., Gray, J.: Parallel Database Systems: The Future of High Performance Database Processing. Commun. ACM 35, 85–98 (1992)
Valduriez, P.: Parallel Database Systems: Open Problems and New Issues. J. Distributed and Parallel Databases 1, 137–165 (1993)
Özsu, M.T., Valduriez, P.: Distributed and Parallel Database System. J. ACM Computing Surveys 28, 125–128 (1991)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2011 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Low, B.W., Ooi, B.Y., Wong, C.S. (2011). Scalability of Database Bulk Insertion with Multi-threading. In: Zain, J.M., Wan Mohd, W.M.b., El-Qawasmeh, E. (eds) Software Engineering and Computer Systems. ICSECS 2011. Communications in Computer and Information Science, vol 181. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-22203-0_14
Download citation
DOI: https://doi.org/10.1007/978-3-642-22203-0_14
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-22202-3
Online ISBN: 978-3-642-22203-0
eBook Packages: Computer ScienceComputer Science (R0)