Big Data Framework for Finding Patterns in Multi-market Trading Data

  • Daya Ram Budhathoki
  • Dipankar DasguptaEmail author
  • Pankaj Jain
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10968)


In the United States, multimarket trading is becoming very popular for investors, professionals and high-frequency traders. This research focuses on 13 exchanges and applies data mining algorithm, an unsupervised machine learning technique for discovering the relationships between stock exchanges. In this work, we used an association rule (FP-growth) algorithm for finding trading pattern in exchanges. Thirty days NYSE Trade and Quote (TAQ) data were used for these experiments. We implemented a big data framework of Spark clusters on the top of Hadoop to conduct the experiment. The rules and co-relations found in this work seems promising and can be used by the investors and traders to make a decision.


Multimarket Exchanges Association rules FP-Growth Hadoop Spark TAQ Clusters 


  1. 1.
  2. 2.
  3. 3.
    Apache spark (2018).
  4. 4.
    Nyse daily taq (trade and quote) (2018).
  5. 5.
    Agarwal, R.C., Aggarwal, C.C., Prasad, V.: A tree projection algorithm for generation of frequent item sets. J. Parall. Distrib. Comput. 61(3), 350–371 (2001)CrossRefGoogle Scholar
  6. 6.
    Agrawal, R., Imieliński, T., Swami, A.: Mining association rules between sets of items in large databases. In: ACM SIGMOD Record, vol. 22, pp. 207–216. ACM (1993)CrossRefGoogle Scholar
  7. 7.
    Agrawal, R., Srikant, R., et al.: Fast algorithms for mining association rules. In: Proceedings 20th International Conference on Very Large Data Bases, VLDB, vol. 1215, pp. 487–499 (1994)Google Scholar
  8. 8.
    Aldridge, I.: High-Frequency Trading: A Practical Guide to Algorithmic Strategies and Trading Systems, vol. 459. Wiley, Hoboken (2009)Google Scholar
  9. 9.
    Asadifar, S., Kahani, M.: Semantic association rule mining: a new approach for stock market prediction. In: 2017 2nd Conference on Swarm Intelligence and Evolutionary Computation (CSIEC), pp. 106–111. IEEE (2017)Google Scholar
  10. 10.
    Chowdhry, B., Nanda, V.: Multimarket trading and market liquidity. Rev. Financ. Stud. 4(3), 483–511 (1991)CrossRefGoogle Scholar
  11. 11.
    Fang, B., Zhang, P.: Big data in finance. In: Yu, S., Guo, S. (eds.) Big Data Concepts, Theories, and Applications, pp. 391–412. Springer, Cham (2016). Scholar
  12. 12.
    Han, J., Cheng, H., Xin, D., Yan, X.: Frequent pattern mining: current status and future directions. Data Mining Knowl. Discov. 15(1), 55–86 (2007)MathSciNetCrossRefGoogle Scholar
  13. 13.
    Han, J., Kamber, M., Pei, J.: Data Mining: Concepts and Techniques. The Morgan Kaufmann Series in Data Management Systems. Morgan Kaufmann, Burlington (2000)zbMATHGoogle Scholar
  14. 14.
    Han, J., Pei, J., Yin, Y.: Mining frequent patterns without candidate generation. In: ACM SIGMOD Record, vol. 29, pp. 1–12. ACM (2000)CrossRefGoogle Scholar
  15. 15.
    Holden, C.W., Jacobsen, S.: Liquidity measurement problems in fast, competitive markets: expensive and cheap solutions. J. Financ. 69(4), 1747–1785 (2014)CrossRefGoogle Scholar
  16. 16.
    Luhr, S., Venkatesh, S., West, G.: Emergent intertransaction association rules for abnormality detection in intelligent environments. In: Proceedings of the 2005 International Conference on Intelligent Sensors, Sensor Networks and Information Processing Conference, pp. 343–347. IEEE (2005)Google Scholar
  17. 17.
    Mythili, M., Shanavas, A.M.: Performance evaluation of apriori and fp-growth algorithms. Int. J. Comput. Appl. 79(10), 279–293 (2013)Google Scholar
  18. 18.
    Preis, T., Moat, H.S., Stanley, H.E.: Quantifying trading behavior in financial markets using google trends. Sci. Rep. 3, 1684 (2013). Scholar
  19. 19.
    Shvachko, K., Kuang, H., Radia, S., Chansler, R.: The hadoop distributed file system. In: Proceedings of the 2010 IEEE 26th Symposium on Mass Storage Systems and Technologies (MSST), MSST 2010, pp. 1–10. IEEE Computer Society, Washington, DC (2010).
  20. 20.
    Sohangir, S., Wang, D., Pomeranets, A., Khoshgoftaar, T.M.: Big data: deep learning for financial sentiment analysis. J. Big Data 5(1), 3 (2018)CrossRefGoogle Scholar
  21. 21.
    Tan, P.N., et al.: Introduction to Data Mining. Pearson Education, India (2006)Google Scholar
  22. 22.
    Wu, X., Zhu, X., Wu, G.Q., Ding, W.: Data mining with big data. IEEE Trans. Knowl. Data Eng. 26(1), 97–107 (2014)CrossRefGoogle Scholar
  23. 23.
    Zaharia, M., Chowdhury, M., Franklin, M.J., Shenker, S., Stoica, I.: Spark: cluster computing with working sets. HotCloud 10(10–10), 95 (2010)Google Scholar
  24. 24.
    Zaki, M.J., Parthasarathy, S., Ogihara, M., Li, W., et al.: New algorithms for fast discovery of association rules. In: KDD, vol. 97, pp. 283–286 (1997)Google Scholar

Copyright information

© Springer International Publishing AG, part of Springer Nature 2018

Authors and Affiliations

  • Daya Ram Budhathoki
    • 1
  • Dipankar Dasgupta
    • 1
    Email author
  • Pankaj Jain
    • 1
  1. 1.University of MemphisMemphisUSA

Personalised recommendations