Reordering Transaction Execution to Boost High Frequency Trading Applications

  • Ningnan Zhou
  • Xuan Zhou
  • Xiao Zhang
  • Xiaoyong Du
  • Shan Wang
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10367)


High frequency trading (HFT) has always been welcomed because it benefits not only personal interests but also the whole social welfare. While the recent advance of portfolio selection in HFT market generates more profit, it yields much contended OLTP workloads. Featuring in exploiting the abundant parallelism, transaction pipeline, the state-of-the-art concurrency control (CC) mechanism, however suffers from limited concurrency confronted with HFT workloads. Its variants that enable more parallel execution by leveraging find-grained contention information also take little effect. To solve this problem, we for the first time observe and formulate the source of restricted concurrency as harmful ordering of transaction statements. To resolve harmful ordering, we propose PARE, a pipeline-aware reordered execution, to improve application performance by rearranging statements in order of their degrees of contention. In concrete, two mechanisms are devised to ensure the correctness of statement rearrangement and identify the degrees of contention of statements respectively. Experiment results show that PARE can improve transaction throughput and reduce transaction latency on HFT applications by up to an order of magnitude than the state-of-the-art CC mechanism.



This work is supported by Nature Science foundation of China Key Project No. 61432006 and The National Key Research and Development Program of China, No. 2016YFB1000702.


  1. 1.
    Bailis, P., Fekete, A., Franklin, M.J., Ghodsi, A., Hellerstein, J.M., Stoica, I.: Coordination avoidance in database systems. PVLDB 8(3), 185–196 (2014)Google Scholar
  2. 2.
    Cheung, A., Madden, S., Arden, O., Myers, A.C.: Automatic partitioning of database applications. PVLDB 5(11), 1471–1482 (2012)Google Scholar
  3. 3.
    Cheung, A., Madden, S., Solar-Lezama, A.: Sloth: being lazy is a virtue (when issuing database queries). In: SIGMOD 2014, pp. 931–942 (2014)Google Scholar
  4. 4.
    Creamer, G.G., Freund, Y.: Automated trading with boosting and expert weighting. Quant. Financ. 4(10), 401–420 (2013)MathSciNetzbMATHGoogle Scholar
  5. 5.
    Dashti, M., John, S.B., Shaikhha, A., Koch, C.: Repairing conflicts among MVCC transactions. CoRR, abs/1603.00542 (2016)Google Scholar
  6. 6.
    Eswaran, K.P., Gray, J.N., Lorie, R.A., Traiger, I.L.: The notions of consistency and predicate locks in a database system. Commun. ACM 19(11), 624–633 (1976)MathSciNetCrossRefzbMATHGoogle Scholar
  7. 7.
    Faleiro, J.M., Abadi, D., Hellerstein, J.M.: High performance transactions via early write visibility. PVLDB 10(5), 613–624 (2017)Google Scholar
  8. 8.
    Han, J., Wen, J.-R.: Mining frequent neighborhood patterns in a large labeled graph. In: CIKM 2013, pp. 259–268 (2013)Google Scholar
  9. 9.
    Han, J., Wen, J.-R., Pei, J.: Within-network classification using radius-constrained neighborhood patterns. In: CIKM 2014, pp. 1539–1548 (2014)Google Scholar
  10. 10.
    Han, J., Zheng, K., Sun, A., Shang, S., Wen, J.R.: Discovering neighborhood pattern queries by sample answers in knowledge base. In: ICDE 2016, pp. 1014–1025 (2016)Google Scholar
  11. 11.
    Kung, H.T., Robinson, J.T.: On optimistic methods for concurrency control. ACM Trans. Database Syst. 6(2), 213–226 (1981)CrossRefGoogle Scholar
  12. 12.
    Larson, P.-A., Blanas, S., Diaconu, C., Freedman, C., Patel, J.M., Zwilling, M.: High-performance concurrency control mechanisms for main-memory databases. PVLDB 5(4), 298–309 (2011)Google Scholar
  13. 13.
    Marlowe, T.J., Ryder, B.G.: Properties of data flow frameworks: a unified model. Acta Inf. 28(2), 121–163 (1990)MathSciNetCrossRefzbMATHGoogle Scholar
  14. 14.
    Neumann, T., Mühlbauer, T., Kemper, A.: Fast serializable multi-version concurrency control for main-memory database systems. In: SIGMOD 2015, pp. 677–689 (2015)Google Scholar
  15. 15.
    Pandis, I., Johnson, R., Hardavellas, N., Ailamaki, A.: Data-oriented transaction execution. PVLDB 3(1–2), 928–939 (2010)Google Scholar
  16. 16.
    Shen, W., Wang, J., Jiang, Y.-G., Zha, H.: Portfolio choices with orthogonal bandit learning. In: IJCAI 2015, pp. 974–980 (2015)Google Scholar
  17. 17.
    Wang, Z., Mu, S., Cui, Y., Yi, H., Chen, H., Li, J.: Scaling multicore databases via constrained parallel execution. In: Proceedings of the 2016 International Conference on Management of Data, SIGMOD 2016, pp. 1643–1658 (2016)Google Scholar
  18. 18.
    Weikum, G., Vossen, G.: Transactional Information Systems: Theory, Algorithms, and the Practice of Concurrency Control and Recovery. Elsevier (2001)Google Scholar
  19. 19.
    Wolfe, M.J.: High Performance Compilers for Parallel Computing (1995)Google Scholar
  20. 20.
    Wu, Y., Chan, C.-Y., Tan, K.-L.: Transaction healing: scaling optimistic concurrency control on multicores. In: SIGMOD 2016, pp. 1689–1704 (2016)Google Scholar
  21. 21.
    Yan, C., Cheung, A.: Leveraging lock contention to improve oltp application performance. PVLDB 9(5), 444–455 (2016)Google Scholar

Copyright information

© Springer International Publishing AG 2017

Authors and Affiliations

  • Ningnan Zhou
    • 1
    • 2
  • Xuan Zhou
    • 3
  • Xiao Zhang
    • 1
    • 2
  • Xiaoyong Du
    • 1
    • 2
  • Shan Wang
    • 1
    • 2
  1. 1.School of InformationRenmin University of ChinaBeijingChina
  2. 2.MOE Key Laboratory of DEKERenmin University of ChinaBeijingChina
  3. 3.School of Data Science and EngineeringEast China Normal UniversityShanghaiChina

Personalised recommendations