Script Based Migration Toolkit for Cloud Computing Architecture in Building Scalable Investment Platforms

  • Rao CasturiEmail author
  • Rajshekhar SunderramanEmail author
Conference paper
Part of the Communications in Computer and Information Science book series (CCIS, volume 903)


The 2008 Financial Crisis which created a global financial market meltdown is mainly due to badly structured mortgage loans with poor or subpar credit quality and lack of proper tools to measure portfolio risks by the lenders. Even though several problems led to this crisis, we looked at this from a Big Data. Had the infrastructure and analytical analysis tools were present to the lenders, they would have found the various early warning signs on these mortgage loans and could have better prepared for the crisis. Aftermath of the crisis, all the big financial institutions took a fresh look and embarked onto build various tools and frameworks to address this Big Data in their portfolios with data driven analysis. The 3Vs (Velocity, Volume and Variety) of the Big Data in our Mortgage Loan Analysis System challenges our traditional approach in collecting, processing and presenting the individual and aggregated loan level data in a meaningful format to facilitate our portfolio managers in decision making. The traditional methods are implemented on a standalone on-premises SQL server. Our Framework creates the foundation of migrating from traditional standalone database architecture (on-premises) to Cloud Computing environment using “Script Based Implementation”. The methods we present are simple but effective and saves resources in terms of Hardware, Software and on-going maintenance costs. Big Data “Capture, Transform, Calculate and Visualize” (CTCV) implementation takes a phased approach rather than a big bang model. Our implementation helps the Big Data Management to be part of organizational tool kit. This saves hard dollars and brings us in line with the overall firm strategic vision of moving to Cloud Computing for Investment Management Services.


Big Data Financial applications Cloud Computing 


  1. 1.
    Fabozzi, F.: Fixed Income Analysis. CFA Institute Investment Series, 2nd edn. Wiley, Hoboken (2007)Google Scholar
  2. 2.
    Fabozzi, F.: The Hand book of Fixed Income Securities, 7th edn. McGraw-Hill, New York (2005)Google Scholar
  3. 3.
    Ozur, M., Tuna, H., Coffin, C., Sampaio, T.: Azure Virtual Datacenter. Microsoft, November 2017Google Scholar
  4. 4.
    Ghemawat, S., Gobioff, H., Leug, S.-T.: The Google file system. In: SOSP 2003, Bolton Landing, New York, USA, 19–22 October 2003Google Scholar
  5. 5.
    Gemayel, N.: Analyzing Google file system and hadoop distributed file system. Research Journal of Information Technology 8(3), 66–74 (2016)CrossRefGoogle Scholar
  6. 6.
    Weber, N.: Big Data: can we prevent the next financial crisis? King’s College London Economics & Finance Society (EFS)Google Scholar
  7. 7.
    Eaton, C., Deroos, D., Deutsch, T., Lapis, G., Zikopoulos, P.: Understanding Big Data: Analytics for Enterprise Class Hadoop and Streaming Data. McGraw-Hill, New York (2011). ISBN 978-0-07-179053-6Google Scholar
  8. 8.
    Microsoft Azure Documentation: Overview of Azure Data Lake Store, MicrosoftGoogle Scholar
  9. 9.
    Microsoft Azure Documentation: Create an Azure SQL Database in Azure portal and Linked Databases MicrosoftGoogle Scholar
  10. 10.
    Microsoft Azure Documentation: Load data from flat files into Azure SQL database. MicrosoftGoogle Scholar
  11. 11.
    Elmasri, R., Navathe, S.: Fundamentals of Data Base Systems, 7th edn. Pearson, London (2015)zbMATHGoogle Scholar
  12. 12.
    Esling, P., Agon, C.: Time series data mining. ACM Comput. Surv. 45, Article no. 12 (2012)Google Scholar
  13. 13.
    Zaharia, M., Wendell, P., Konwinski, A., Karau, H.: Working with key/value Pairs. In: Learning Spark. O’Relly Media, Inc., February 2015Google Scholar
  14. 14.
    Codd, E.F.: A relational model of data for large shared data banks. Commun. ACM 13(6), 377–387 (1970)CrossRefGoogle Scholar
  15. 15.
    Grossman, R., Gu, Y.: Data mining using high performance data clouds: experimental studies using sector and sphere. In: AMC KDD 2008, Las Vegas, Nevada, USA, 24–27 August 2008Google Scholar
  16. 16.
  17. 17.
    White, T.: Hadoop The Definitive Guide, 3rd edn. O’Reilly Media Inc, Sebastopol (2012)Google Scholar
  18. 18.
    Han, J., Kamber, M., Pei, J.: Data Mining Concepts and Techniques, 3rd edn. Morgan Kaufmann, Burlington (2012)zbMATHGoogle Scholar
  19. 19.
    Microsoft Press: Cloud Application Architecture GuideGoogle Scholar
  20. 20.
    Sobati Moghadam, S., Darmont, J., Gavin, G.: Enforcing privacy in cloud databases. In: Bellatreche, L., Chakravarthy, S. (eds.) DaWaK 2017. LNCS, vol. 10440, pp. 53–73. Springer, Cham (2017). Scholar

Copyright information

© Springer Nature Switzerland AG 2018

Authors and Affiliations

  1. 1.V.P. Risk ManagementVoya Investment ManagementAtlantaUSA
  2. 2.Department of Computer ScienceGeorgia State UniversityAtlantaUSA

Personalised recommendations