The Impact of Big Data on the Physician

  • Elizabeth Le
  • Sowmya Iyer
  • Teja Patil
  • Ron Li
  • Jonathan H. Chen
  • Michael Wang
  • Erica Sobel
Part of the Studies in Big Data book series (SBD, volume 26)


Over the past few years technology has evolved to integrate itself into a myriad of aspects of every day life, including healthcare access and delivery. One way in which technology has begun to truly transform healthcare is with big data and big data analytics. Using sophisticated tools to capture, aggregate, and translate data across multiple sources, ranging from traditional electronic health records to non-traditional consumer devices, big data has the potential to transform the practice of medicine at both the level of the patient-physician relationship as well as in clinical decision-making and treatment. Already apparent are intriguing examples of increased patient engagement, improved evidence-based physician decision-making, and multi-faceted tools for more individualized patient care. As big data becomes more and more incorporated into the practice of medicine, issues regarding systems interoperability, privacy and security, and legal and regulatory boundaries will need to be resolved. This chapter is intended to be an overview of big data applications and potential challenges as they relate to the patient and physician.


Healthcare Medicine Physician Data science Big data Electronic health record 


  1. Agaku, I. T., Adisa, A. O., Ayo-Yusuf, O. A., & Connolly, G. N. (2014). Concern about security and privacy, and perceived control over collection and use of health information are related to withholding of health information from healthcare providers. Journal of the American Medical Informatics Association, 21(2), 374–378. doi: 10.1136/amiajnl-2013-002079.CrossRefGoogle Scholar
  2. Agboola, S., Jethwani, K., Khateeb, K., Moore, S., & Kvedar, J. (2015). Heart failure remote monitoring: Evidence from the retrospective evaluation of a real-world remote monitoring program. Journal of Medical Internet Research, 17(4), e101. doi: 10.2196/jmir.4417.CrossRefGoogle Scholar
  3. Agency for Healthcare Research and Quality. (2014). Total expenses and percent distribution for selected conditions by type of service. Retrieved from
  4. Ball, M., Weaver, C., Kim, G., & Kiel, J. (2016). Healthcare information management systems. Journal of Information Systems Education, 14. doi: 10.1007/978-3-319-20765-0.
  5. Bushey, R. (2015, March 5). Meet thomas goetz, The co-founder of iodine. Retrieved October 31, 2016, from
  6. Caouette, H. (2015a). eClinicalWorks announces integration between wearable devicesand healow platform. Retrieved from
  7. Caouette, H. (2015b). healow Announces Internet of Things Integration. Retrieved from
  8. Chaterjee, P. (2015, October 09). Delivering value by focusing on patient experience—AJMC. Retrieved October 25, 2016, from
  9. Chen, J. H., Goldstein, M. K., Asch, S. M., Mackey, L., & Altman, R. B. (2016). Predicting inpatient clinical order patterns with probabilistic topic models vs. conventional order sets. Journal of the American Medical Informatics Association. doi: 10.1093/jamia/ocw136 [Epub ahead of print].
  10. Clark, N. M. (2003). Management of chronic disease by patients. Annual Review of Public Health, 24, 289–313. doi: 10.1146/annurev.publhealth.24.100901.141021.CrossRefGoogle Scholar
  11. Collins, F. S., & Varmus, H. (2015). A new initiative on precision medicine. New England Journal of Medicine, 372(9), 793–795. doi: 10.1056/NEJMp1500523.CrossRefGoogle Scholar
  12. Cunningham, P. J. (2009). Beyond parity: Primary care physicians’ perspectives on access to mental health care. Health Affairs, 28(3), w490–w501.CrossRefGoogle Scholar
  13. Darkins, A., Ryan, P., Kobb, R., Foster, L., Edmonson, E., Wakefield, B., & Lancaster, A. E. (2008). Care coordination/home telehealth: The systematic implementation of health informatics, home telehealth, and disease management to support the care of veteran patients with chronic conditions. Telemedicine and E-Health, 14(10), 1118–1126. doi: 10.1089/tmj.2008.0021.CrossRefGoogle Scholar
  14. Dentzer, S. (2013). Rx for the “Blockbuster Drug” of patient engagement. Health Affairs, 32(2), 202–202. doi: 10.1377/hlthaff.2013.0037.CrossRefGoogle Scholar
  15. Downing, N. L., Adler-Milstein, J., Palma, J. P., Lane, S., Eisenberg, M., Sharp, C., et al. (2016). Health information exchange policies of 11 diverse health systems and the associated impact on volume of exchange. Journal of the American Medical Informatics Association, 34(13), 150–160. doi: 10.1093/jamia/ocw063.Google Scholar
  16. Duke, J. (2015, June 14). An open science approach to medical evidence generation. Retrieved October 31, 2016, from
  17. Feldman, B. (2014, January 30). Using big data and game play to improve mental fitness. Retrieved September 20, 2016, from
  18. Fisher, M. (2014, June 12). Apple’s health app and healthkit and HIPAA. Retrieved from
  19. Food and Drug Administration. (2006). Guidance: Useful written consumer medication information (CMI). Rockville, MD: US Food and Drug Administration.Google Scholar
  20. Fowles, J. B., Terry, P., Xi, M., Hibbard, J., Bloom, C. T., & Harvey, L. (2009). Measuring self-management of patients’ and employees’ health: Further validation of the Patient Activation Measure (PAM) based on its relation to employee characteristics. Patient Education and Counseling, 77(1), 116–122.CrossRefGoogle Scholar
  21. Fox, S. (2013). After Dr Google: Peer-to-peer health care. Pediatrics, 131(Suppl. 4), S224–S225.CrossRefGoogle Scholar
  22. Fox S. (2014). The social life of health information. Washington, DC: Pew Research Center’s Internet and American Life Project. January 15, 2014, Available at
  23. Freese, N. (2016, May 11). It’s high time for change — And health care data is paving the way. Retrieved 25 October, 2016, from
  24. Frost, J., & Massagli, M. (2008). Social uses of personal health information within PatientsLikeMe, an online patient community: What can happen when patients have access to one another’s data. Journal of Medical Internet Research, 10(3), e15.CrossRefGoogle Scholar
  25. Gabai-Kapara, E., Lahad, A., Kaufman, B., Friedman, E., Segev, S., Renbaum, P., et al. (2014). Population-based screening for breast and ovarian cancer risk due to BRCA1 and BRCA2. Proceedings of the National Academy of Sciences, USA, 111(39), 14205–14210. doi: 10.1073/pnas.1415979111.CrossRefGoogle Scholar
  26. Gartner. (2016, August 16). Gartner’s 2016 hype cycle for emerging technologies identifies three key trends that organizations must track to gain competitive advantage. Retrieved October 10, 2016 from
  27. Evidence (n.d.) Retrieved from
  28. Grand Rounds (n.d.). Retrieved October 25, 2016, from
  29. Greene, J., & Hibbard, J. H. (2012). Why does patient activation matter? An examination of the relationships between patient activation and health-related outcomes. Journal of General Internal Medicine, 27(5), 520–526.CrossRefGoogle Scholar
  30. Healthgrades. (2016a). Methodology: Mortality and complications outcomes. Retrieved October 16, 2016, from
  31. Healthgrades. (2016b, April 05). Patient experience measures data source. Retrieved October 25, 2016, from
  32. Healthgrades (2016c). Patient safety ratings and patient safety excellence award 2016 methodology. Retrieved October 25, 2016, from
  33. Healthgrades (n.d.). Healthgrades find a doctor doctor reviews hospital … (n.d.). Retrieved 25 October, 2016, from
  34. Hibbard, J. H. (2008). Using systematic measurement to target consumer activation strategies. Medical Care Research and Review, 66, 9s–27s.CrossRefGoogle Scholar
  35. Hibbard, J. H., & Greene, J. (2013). What the evidence shows about patient activation: Better health outcomes and care experiences; fewer data on costs. Health Affairs, 32(2), 207–214.CrossRefGoogle Scholar
  36. Hibbard, J. H., Stockard, J., Mahoney, E. R., & Tusler, M. (2004). Development of the patient activation measure (PAM): Conceptualizing and measuring activation in patients and consumers. Health Services Research, 39(4 Pt. 1), 1005–1026. doi: 10.1111/j.1475-6773.2004.00269.x.CrossRefGoogle Scholar
  37. Higgins, T. (2015, April 26). Apple’s healthkit linked to patients at big los angeles hospital. Retrieved from
  38. Hood, L., & Flores, M. (2012). A personal view on systems medicine and the emergence of proactive P4 medicine: Predictive, preventive, personalized and participatory. New Biotechnology, 29(6), 613–624. doi: 10.1016/j.nbt.2012.03.004.CrossRefGoogle Scholar
  39. Humer, C., & Finkle, J. (2014, September 24). Your medical record is worth more to hackers than your credit card. Reuters. Retrieved from
  40. Institute of Medicine. (2012). Best care at lower cost. Washington, DC: The National Academies Press. doi: 10.17226/13444.Google Scholar
  41. Iodine Data. n.d.. Retrieved from
  42. Jang, S., & Atkins, M. B. (2013). Which drug, and when, for patients with BRAF-mutant melanoma? The Lancet Oncology, 14(2), e60–e69. doi: 10.1016/S1470-2045(12)70539-9.CrossRefGoogle Scholar
  43. Japsen, B. (2013, December 22). ObamaCare, doctor shortage to Spur $2 billion telehealth market. Retrieved October 10, 2016, from
  44. Kayyali, B., Knott, D., & Van Kuiken, S. (2013). The big-data revolution in US Health Care: Accelerating value and innovation (pp. 1–13). New York City, NY: Mc Kinsey & Company.Google Scholar
  45. Kumar, R. B., Goren, N. D., Stark, D. E., Wall, D. P., & Longhurst, C. A. (2016). Automated integration of continuous glucose monitor data in the electronic health record using consumer technology. Journal of the American Medical Informatics Association, 23(3), 532–537. doi: 10.1093/jamia/ocv206.CrossRefGoogle Scholar
  46. Kummar, S., Williams, P. M., Lih, C.-J., Polley, E. C., Chen, A. P., Rubinstein, L. V., et al. (2015). Application of molecular profiling in clinical trials for advanced metastatic cancers. Journal of the National Cancer Institute, 107(4), djv003. doi: 10.1093/jnci/djv003.CrossRefGoogle Scholar
  47. Lander, E. S., Linton, L. M., Birren, B., Nusbaum, C., Zody, M. C., Baldwin, J., et al. (2001). Initial sequencing and analysis of the human genome. Nature, 409(6822), 860–921. doi: 10.1038/35057062.CrossRefGoogle Scholar
  48. Leventhal, R. (2015, April 21). How duke is using HealthKit to get patient-generated data into the EHR. Retrieved October 5, 2016, from
  49. Liao, K. P., Cai, T., Savova, G. K., Murphy, S. N., Karlson, E. W., Ananthakrishnan, A. N., et al. (2015). Development of phenotype algorithms using electronic medical records and incorporating natural language processing. BMJ, 350(apr24_11), h1885. doi: 10.1136/bmj.h1885.CrossRefGoogle Scholar
  50. Longhurst, C. A., Harrington, R. A., & Shah, N. H. (2014). A “green button” for using aggregate patient data at the point of care. Health Affairs, 33(7), 1229–1235. doi: 10.1377/hlthaff.2014.0099.CrossRefGoogle Scholar
  51. Mann, D. M., Quintiliani, L. M., Reddy, S., Kitos, N. R., & Weng, M. (2014). Dietary approaches to stop hypertension: Lessons learned from a case study on the development of an mHealth behavior change system. JMIR mHealth and uHealth, 2(4), e41.CrossRefGoogle Scholar
  52. Metcalf, D., Milliard, S., Gomez, M., & Schwartz, M. (2016, October 3). Wearables and the internet of things for health. Retrieved from
  53. Might, M., & Wilsey, M. (2014). The shifting model in clinical diagnostics: How next-generation sequencing and families are altering the way rare diseases are discovered, studied, and treated. Genetics in Medicine, 16(10), 736–737.CrossRefGoogle Scholar
  54. Morris, Z., Wooding, S., & Grant, J. (2011, December). The answer is 17 years, what is the question: Understanding time lags in translational research. Retrieved October 30, 2016, from
  55. National Academies Press. (2011). Toward precision medicine: Building a knowledge network for biomedical research and a new taxonomy of disease. Washington, DC: National Academies Press. Retrieved from
  56. National eHealth Collaborative. (2013, December). Patient-generated health information technical expert panel final report. Retrieved October 21, 2016, from
  57. National Institutes of Health (2016). About the Precision Medicine Initiative Cohort Program. Retrieved October 30, 2016, from
  58. Neff, G. (2013). Why big data won’t cure us. Big Data, 1(3), 117–123.CrossRefGoogle Scholar
  59. Nielsen. (2014, April 16). Hacking health: How consumers use smartphones and wearable tech to track their health. Retrieved October 3, 2016, from
  60. Panahiazar, M., Taslimitehrani, V., Jadhav, A., & Pathak, J. (2014). Empowering personalized medicine with big data and semantic web technology: Promises, challenges, and use cases. Proceedings: IEEE International Conference on Big Data, 2014, 790–795. doi: 10.1109/BigData.2014.7004307.Google Scholar
  61. Parodi, S., Riccardi, G., Castagnino, N., Tortolina, L., Maffei, M., Zoppoli, G., et al. (2016). Systems medicine in oncology: Signaling network modeling and new-generation decision-support systems. In U. Schmitz & O. Wolkenhauer (Eds.), Systems medicine (pp. 181–219). New York: Springer.CrossRefGoogle Scholar
  62. PatientsLikeMe Services(n.d.) Retrieved from
  63. PwC. (2014, October 21). Wearable technology future is ripe for growth—Most notably among millennials, says PwC US. Retrieved October 5, 2016, from
  64. Rask, K. J., Ziemer, D. C., Kohler, S. A., Hawley, J. N., Arinde, F. J., & Barnes, C. S. (2009). Patient activation is associated with healthy behaviors and ease in managing diabetes in an indigent population. The Diabetes Educator, 35(4), 622–630. doi: 10.1177/0145721709335004.CrossRefGoogle Scholar
  65. Robert, C., Karaszewska, B., Schachter, J., Rutkowski, P., Mackiewicz, A., Stroiakovski, D., et al. (2015). Improved overall survival in melanoma with combined dabrafenib and trametinib. New England Journal of Medicine, 372(1), 30–39. doi: 10.1056/NEJMoa1412690.CrossRefGoogle Scholar
  66. Robinson, R. (2016, February). Patients and patient organizations power rare disease therapies—Pharmavoice. Retrieved October 1, 2016, from
  67. Rolnick, J., Downing, N. L., Shepard, J., Chu, W., Tam, J., Wessels, A., et al. (2016). Validation of test performance and clinical time zero for an electronic health record embedded severe sepsis alert. Applied Clinical Informatics, 7(2), 560–572. doi: 10.4338/ACI-2015-11-RA-0159.CrossRefGoogle Scholar
  68. Sarasohn-Kahn, J. (2008). The Wisdom of Patients: Health Care Meets Online Social Media. Retrieved from
  69. Schueller, S. M., Tomasino, K. N., Lattie, E. G., & Mohr, D. C. (2016). Human support for behavioral intervention technologies for mental health: The efficiency model. Management, 21, 22.Google Scholar
  70. Sepah, S. C., Jiang, L., & Peters, A. L. (2015). Long-term outcomes of a web-based diabetes prevention program: 2-year results of a single-arm longitudinal study. Journal of medical Internet research, 17(4), e92.Google Scholar
  71. Shapiro, M., Johnston, D., Wald, J., & Mon, D. (2012). Patient-generated health data. White Paper: Prepared for Office of Policy and Planning. Retrieved from
  72. Shay, L. A., & Lafata, J. E. (2015). Where is the evidence? A systematic review of shared decision making and patient outcomes. Medical Decision Making: An International Journal of the Society for Medical Decision Making, 35(1), 114–131. doi: 10.1177/0272989X14551638.CrossRefGoogle Scholar
  73. Shortliffe, E., & Cimino, J. (2014). Biomedical informatics: Computer applications in healthcare and medicine (4th ed.). London: Springer.CrossRefGoogle Scholar
  74. Sifferlin, A. (2014, September 24). Look up your meds on this massive new drug database.Retrieved November 1, 2016, from
  75. Sittig, D. F., Wright, A., Osheroff, J. A., Middleton, B., Teich, J. M., Ash, J. S., et al. (2008). Grand challenges in clinical decision support. Journal of Biomedical Informatics, 41(2), 387–392. doi: 10.1016/j.jbi.2007.09.00.CrossRefGoogle Scholar
  76. Smith, A. (2015, April 1). U.S. smartphone use in 2015. Retrieved from
  77. Sunyaev, A., Dehling, T., Taylor, P. L., & Mandl, K. D. (2015). Availability and quality of mobile health app privacy policies. Journal of the American Medical Informatics Association, 22(e1), e28–e33. doi: 10.1136/amiajnl-2013-002605.Google Scholar
  78. The White House. (2015, January 30). FACT SHEET: President Obama’s Precision Medicine Initiative. Retrieved October 20, 2016, from
  79. Trainer, A. H., Lewis, C. R., Tucker, K., Meiser, B., Friedlander, M., & Ward, R. L. (2010). The role of BRCA mutation testing in determining breast cancer therapy. Nature Reviews Clinical Oncology, 7(12), 708–717. doi: 10.1038/nrclinonc.2010.175.CrossRefGoogle Scholar
  80. Tricoci, P., Allen, J. M., Kramer, J. M., Califf, R. M., & Smith, Jr. S. C. (2009). Scientific evidence underlying the ACC/AHA clinical practice guidelines. Journal of the American Medical Association, 301(8), 831--841.Google Scholar
  81. Tu, H. T., & Lauer, J. (2008, December). Word of mouth and physician referrals still drive health care provider choice. Retrieved November 1, 2016, from
  82. U.S. Department of Health and Human Services. (2013, January). Realizing the full potential of health information technology to improve healthcare for americans: The path forward. Retrieved October 4, 2016, from
  83. U.S. Department of Health and Human Services. (2015, February 9). Mobile medical applications: Guidance for industry and food and drug administration staff. Retrieved from
  84. U.S. Department of Health and Human Services. (2009, October 28). HITECH Act enforcement interim final rule. Retrieved from
  85. U.S. Department of Health and Human Services. (2012, June 26). Alaska DHSS settles HIPAA security case for $1,700,000. Retrieved October 18, 2016, from
  86. U.S. Department of Health and Human Services. (2013, March 26). HIPAA administrative simplification. Retrieved from
  87. U.S. Preventative Services Task Force. (2014, December 24). BRCA-related cancer: risk assessment, genetic counseling, and genetic testing. Retrieved from
  88. US Department of Veterans Affairs Office of Research and Development. (n.d.) Million veterans program. Retrieved October 30, 2016, from
  89. Wicks, P. (2007, July 7). Excessive yawning in ALS/MND. Retrieved October 1, 2016, from
  90. Winner, J. G., Carhart, J. M., Altar, A., Allen, J. D., & Dechairo, B. M. (2013). A prospective, randomized, double-blind study assessing the clinical impact of integrated pharmacogenomic testing for major depressive disorder. Discovery Medicine, 16(89), 219–227.Google Scholar
  91. Winner, J. G., Carhart, J. M., Altar, C. A., Goldfarb, S., Allen, J. D., Lavezzari, G., et al. (2015). Combinatorial pharmacogenomic guidance for psychiatric medications reduces overall pharmacy costs in a 1 year prospective evaluation. Current Medical Research and Opinion, 31(9), 1633–1643. doi: 10.1185/03007995.2015.1063483.CrossRefGoogle Scholar
  92. Witters, D., & Agrawal, S. (2014, November 3). How mobile technology can improve employees’ well-being. Retrieved October 10, 2016, from
  93. Xu, J., Rasmussen, L. V, Shaw, P. L., Jiang, G., Kiefer, R. C., Mo, H., et al. (2015). Review and evaluation of electronic health records-driven phenotype algorithm authoring tools for clinical and translational. Journal of the American Medical Informatics Association, 22 (6), 1251–1260. doi: 10.1093/jamia/ocv070
  94. Yang, Y. T., & Silverman, R. D. (2014). Mobile health applications: The patchwork of legal and liability issues suggests strategies to improve oversight. Health Affairs, 33(2), 222–227. doi: 10.1377/hlthaff.2013.0958.CrossRefGoogle Scholar

Copyright information

© Springer International Publishing AG 2018

Authors and Affiliations

  • Elizabeth Le
    • 1
  • Sowmya Iyer
    • 1
  • Teja Patil
    • 1
  • Ron Li
    • 2
  • Jonathan H. Chen
    • 2
  • Michael Wang
    • 3
  • Erica Sobel
    • 4
  1. 1.Veterans Affairs Palo Alto Healthcare SystemPalo AltoUSA
  2. 2.Stanford UniversityPalo AltoUSA
  3. 3.University of CaliforniaSan FranciscoUSA
  4. 4.Kaiser Permanente Santa Clara Medical CenterSanta ClaraUSA

Personalised recommendations