Facilitating the Exploration of Open Health-Care Data Through BOAT: A Big Data Open Source Analytics Tool

  • A. Ravishankar RaoEmail author
  • Daniel Clarke
Conference paper
Part of the Springer Proceedings in Business and Economics book series (SPBE)


Big Data Analytics is the use of advanced analytic techniques on very large data sets to discover hidden patterns and useful information. Many governments publicly release significant amounts of health data, including hospital ratings and patient outcomes. We propose applying Big Data Analytics to understand open health data. Ideally, citizens would use this data to choose hospitals or evaluate care options. There are major challenges, including merging data from disparate sources and applying interpretive analytics.

We are building an open-source tool to facilitate analytical exploration. Such a tool could enable researchers, hospitals, insurers, and citizens to obtain integrated global-to-local perspectives on health-care expenditures, procedure costs, and emerging trends. Our tool is based on the Python ecosystem, and contains a variety of modules from database analytics to machine learning and visualization. We analyzed data from the New York Statewide Planning and Research Cooperative System and determined the distribution of costs for hip replacement surgery across the state. The mean cost over 168,676 patients is $22,700, the standard deviation is $20,900, and 88% of these patients had hip replacement costs of less than $30,000. This provides the background to understand why in a state with similar demographics, The California Public Employees’ Retirement System capped hip replacement reimbursements at $30,000, which resulted in significant medical savings. Obtaining transparency of medical costs is important to control expenditure. Even though such information is available, consumers have trouble utilizing it effectively. Our tool could be truly transformative, allowing consumers to fully use available data, and to perhaps demand access to data that ought to be made public.


Big data BOAT Open source Health-care 


  1. Baigent C, Blackwell L, Collins R, Emberson J, Godwin J, Peto R et al (2009) Aspirin in the primary and secondary prevention of vascular disease: collaborative meta-analysis of individual participant data from randomized trials. Lancet 373:1849–1860CrossRefGoogle Scholar
  2. Bibbins-Domingo K (2016) Aspirin use for the primary prevention of cardiovascular disease and colorectal cancer: US preventive services task force recommendation. Ann Intern Med 164:836–845CrossRefGoogle Scholar
  3. Borgman CL (2012) The conundrum of sharing research data. J Assoc Inf Sci Technol 63:1059–1078CrossRefGoogle Scholar
  4. Boulton G, Rawlins M, Vallance P, Walport M (2011) Science as a public enterprise: the case for open data. Lancet 377:1633–1635CrossRefGoogle Scholar
  5. Bridgewater B, Grayson AD, Brooks N, Grotte G, Fabri BM, Au J et al (2007) Has the publication of cardiac surgery outcome data been associated with changes in practice in northwest England: an analysis of 25,730 patients undergoing CABG surgery under 30 surgeons over eight years. Heart 93:744–748CrossRefGoogle Scholar
  6. Bullinger AC, Rass M, Adamczyk S, Moeslein KM, Sohn S (2012) Open innovation in health care: analysis of an open health platform. Health Policy 105:165–175CrossRefGoogle Scholar
  7. Centre for disease control and prevention. http://wwwcdcgov/DataStatistics/
  8. Choudhry NK, Fletcher RH, Soumerai SB (2005) Systematic review: the relationship between clinical experience and quality of health care. Ann Intern Med 142:260–273CrossRefGoogle Scholar
  9. CMS.GOV, Private correspondence with the data vendor for CMS.Google Scholar
  10. Conradie P, Choenni S (2014) On the barriers for local government releasing open data. Gov Inf Q 31:S10–S17CrossRefGoogle Scholar
  11. Davies T, Perini F (2016) Researching the emerging impacts of open data: revisiting the ODDC conceptual framework. J Commun Inform 12Google Scholar
  12. DeJong C, Aguilar T, Tseng C-W, Lin GA, Boscardin WJ, Dudley RA (2016) Pharmaceutical industry–sponsored meals and physician prescribing patterns for Medicare beneficiaries. JAMA Intern Med 176:1114–1110CrossRefGoogle Scholar
  13. Department of Health and Human Services. Secondary analyses of existing data sets and stored biospecimens to address clinical aging research questions (R01)Google Scholar
  14. Department of Health and Human Services. Secondary dataset analyses in heart, lung, and blood diseases and sleep disorders (R21)Google Scholar
  15. Estrin D, Sim I (2010) Health care delivery open mHealth architecture: an engine for health care innovation. Science 330:759–760CrossRefGoogle Scholar
  16. Fontenot SF (2013) Understanding the affordable care act bit by bit: will transparency and sunshine shrink costs? Physician Exec 39:86Google Scholar
  17. Frakt A (2016) How common procedures became 20 percent cheaper for many Californians. The New York Times 2016Google Scholar
  18. Base Frame Work for Data MiningGoogle Scholar
  19. Harding AJ, Pritchard C (2016) UK and twenty comparable countries GDP-expenditure-on-health 1980-2013: the historic and continued low priority of UK health-related expenditure. Int J Health Policy Manag 5:519CrossRefGoogle Scholar
  20. Hardy Q (2013) Technology workers are young (really young). The New York TimesGoogle Scholar
  21. Hunter JD (2007) Matplotlib: a 2D graphics environment. Comput Sci Eng 9:90–95CrossRefGoogle Scholar
  22. Jena AB, Romley J (2015) Changes in hospitalizations, treatment patterns, and outcomes during major cardiovascular meetings—reply. JAMA Intern Med 175:1420–1421CrossRefGoogle Scholar
  23. Jetzek T (2015) The sustainable value of open government data: uncovering the generative mechanisms of open data through a mixed methods approach. Copenhagen Business School, Institute for IT-Ledelse Department of IT ManagementGoogle Scholar
  24. Krumholz HM (2014) Big data and new knowledge in medicine: the thinking, training, and tools needed for a learning health system. Health Aff (Millwood) 33:1163–1170CrossRefGoogle Scholar
  25. Kupfer JM (2016) The graying of US physicians: implications for quality and the future supply of physicians. JAMA 315:341–342CrossRefGoogle Scholar
  26. Le ME, Kurd MF (2014) A review and analysis of the YODA trials: what can we glean clinically? Curr Rev Musculoskelet Med 7:189–192CrossRefGoogle Scholar
  27. Martin EG, Helbig N, Shah NR (2014) Liberating data to transform health care: New york’s open data experience. JAMA 311:2481–2482CrossRefGoogle Scholar
  28. Martin AB, Hartman M, Benson J, Catlin A, Team NHEA (2016) National health spending in 2014: faster growth driven by coverage expansion and prescription drug spending. Health Aff 35:150–160CrossRefGoogle Scholar
  29. Martínez-Torres MR, Diaz-Fernandez MDC (2014) Current issues and research trends on open-source software communities. Tech Anal Strat Manag 26:55–68CrossRefGoogle Scholar
  30. McAlister FA, Youngson E, Bakal JA, Holroyd-Leduc J, Kassam N (2015) Physician experience and outcomes among patients admitted to general internal medicine teaching wards. CMAJ 187:1041–1048CrossRefGoogle Scholar
  31. McKinney W (2010) Data structures for statistical computing in python. In: Proceedings of the 9th python in science conference, pp 51–56Google Scholar
  32. McKinney W (2012) Python for data analysis: data wrangling with Pandas, NumPy, and IPython. O’Reilly Media, IncGoogle Scholar
  33. Miller RH, Sim I (2004) Physicians’ use of electronic medical records: barriers and solutions. Health Aff 23:116–126CrossRefGoogle Scholar
  34. New York State Department of Health, Statewide Planning and Research Cooperative System (SPARCS).
  35. Nisen P, Rockhold F (2013) Access to patient-level data from GlaxoSmithKline clinical trials. N Engl J Med 369:475–478CrossRefGoogle Scholar
  36. Pedregosa F, Varoquaux G, Gramfort A, Michel V, Thirion B, Grisel O et al (2011) Scikit-learn: machine learning in python. J Mach Learn Res 12:2825–2830Google Scholar
  37. Pérez F, Granger BE (2007) IPython: a system for interactive scientific computing. Comput Sci Eng 9:21–29CrossRefGoogle Scholar
  38. Physician specialty data book (2014). Accessed 5 Jul 2017.
  39. Rao AR, Clarke D (2016) A fully integrated open-source toolkit for mining healthcare big-data: architecture and applications. In: IEEE international conference on healthcare informatics ICHI, Chicago, vol 2016, pp 255–261Google Scholar
  40. Rao AR, Chhabra A, Das R, Ruhil V (2015) A framework for analyzing publicly available healthcare data. In 17th international conference on E-health networking. Appl Services (IEEE HealthCom) vol 2015, pp 653–656Google Scholar
  41. Roshan Paul AC (2011) An open source approach to medical research. Stanford Social Innovation ReviewGoogle Scholar
  42. Russell SAGE Foundation Funding opportunity: the social, economic, and political effects of the affordable care actGoogle Scholar
  43. Safran C, Bloomrosen M, Hammond WE, Labkoff S, Markel-Fox S, Tang PC et al (2007) Toward a national framework for the secondary use of health data: an American Medical Informatics Association White Paper. J Am Med Inform Assoc 14:1–9CrossRefGoogle Scholar
  44. Salthouse TA (2009) When does age-related cognitive decline begin? Neurobiol Aging 30:507–514CrossRefGoogle Scholar
  45. Schenarts PJ, Cemaj S (2016) The aging surgeon: implications for the workforce, the surgeon, and the patient. Surg Clin North Am 96:129–138CrossRefGoogle Scholar
  46. Schwartz AL, Landon BE, Elshaug AG, Chernew ME, McWilliams JM (2014) Measuring low-value care in Medicare. JAMA Intern Med 174:1067–1076CrossRefGoogle Scholar
  47. Shadbolt N, O’Hara K, Berners-Lee T, Gibbins N, Glaser H, Hall W (2012) Linked open government data: lessons from IEEE Intell Syst 27:16–24CrossRefGoogle Scholar
  48. Specks U, Merkel PA, Seo P, Spiera R, Langford CA, Hoffman GS et al (2013) Efficacy of remission-induction regimens for ANCA-associated vasculitis. N Engl J Med 369:417–427CrossRefGoogle Scholar
  49. Stilgoe J, Lock SJ, Wilsdon J (2014) Why should we promote public engagement with science? Public Underst Sci 23:4–15CrossRefGoogle Scholar
  50. Sudlow C, Gallacher J, Allen N, Beral V, Burton P, Danesh J et al (2015) UK biobank: an open access resource for identifying the causes of a wide range of complex diseases of middle and old age. PLoS Med 12:e1001779CrossRefGoogle Scholar
  51. Thibodeau P (2015) Median age at Google is 29, says age discrimination lawsuit. ComputerworldGoogle Scholar
  52. Thomas K (2016) Furor over drug prices puts patient advocacy groups in bind. The New York TimesGoogle Scholar
  53. Walker J, Darer JD, Elmore JG, Delbanco T (2014) The road toward fully transparent medical records. N Engl J Med 370:6–8CrossRefGoogle Scholar
  54. Zarin DA (2013) Participant-level data and the new frontier in trial transparency. N Engl J Med 369:468–469CrossRefGoogle Scholar

Copyright information

© Springer International Publishing AG 2018

Authors and Affiliations

  1. 1.School of Computer Science and EngineeringFairleigh Dickinson UniversityTeaneckUSA

Personalised recommendations