Recommender Systems for Health Informatics: State-of-the-Art and Future Perspectives

  • André Calero ValdezEmail author
  • Martina Ziefle
  • Katrien Verbert
  • Alexander Felfernig
  • Andreas Holzinger
Part of the Lecture Notes in Computer Science book series (LNCS, volume 9605)


Recommender systems are a classical example for machine learning applications, however, they have not yet been used extensively in health informatics and medical scenarios. We argue that this is due to the specifics of benchmarking criteria in medical scenarios and the multitude of drastically differing end-user groups and the enormous context-complexity of the medical domain. Here both risk perceptions towards data security and privacy as well as trust in safe technical systems play a central and specific role, particularly in the clinical context. These aspects dominate acceptance of such systems. By using a Doctor-in-the-Loop approach some of these difficulties could be mitigated by combining both human expertise with computer efficiency. We provide a three-part research framework to access health recommender systems, suggesting to incorporate domain understanding, evaluation and specific methodology into the development process.


Health recommender systems Human-computer interaction Evaluation framework Uncertainty Trust Risk Privacy 



The authors thank the German Research Council DFG for the friendly support of the research in the excellence cluster “Integrative Production Technology in High Wage Countries”, and the anonymous reviewers for their constructive comments.

Part of the work of Katrien Verbert has been supported by the Research Foundation Flanders (FWO), grant agreement no. G0C9515N, and the KU Leuven Research Council, grant agreement no. STG/14/019.


  1. 1.
    Bruns, S., Valdez, A.C., Greven, C., Ziefle, M., Schroeder, U.: What should i read next? a personalized visual publication recommender system. In: Yamamoto, S. (ed.) HCI 2015. LNCS, vol. 9173, pp. 89–100. Springer, Heidelberg (2015). doi: 10.1007/978-3-319-20618-9_9 CrossRefGoogle Scholar
  2. 2.
    Jordan, M.I., Mitchell, T.M.: Machine learning: trends, perspectives, and prospects. Science 349(6245), 255–260 (2015)MathSciNetCrossRefGoogle Scholar
  3. 3.
    Shahriari, B., Swersky, K., Wang, Z., Adams, R.P., de Freitas, N.: Taking the human out of the loop: a review of bayesian optimization. Proc. IEEE 104(1), 148–175 (2016)CrossRefGoogle Scholar
  4. 4.
    Holzinger, A.: Interactive machine learning for health informatics: when do we need the human-in-the-loop? Springer Brain Inform. (BRIN) 3(2), 119–131 (2016)CrossRefGoogle Scholar
  5. 5.
    Holzinger, A., Plass, M., Holzinger, K., Crişan, G.C., Pintea, C.-M., Palade, V.: Towards interactive Machine Learning (iML): applying ant colony algorithms to solve the traveling salesman problem with the human-in-the-loop approach. In: Buccafurri, F., Holzinger, A., Kieseberg, P., Tjoa, A.M., Weippl, E. (eds.) CD-ARES 2016. LNCS, vol. 9817, pp. 81–95. Springer, Heidelberg (2016). doi: 10.1007/978-3-319-45507-5_6 CrossRefGoogle Scholar
  6. 6.
    Puthiya Parambath, S.A., Usunier, N., Grandvalet, Y.: A coverage-based approach to recommendation diversity on similarity graph. In: Proceedings of the 10th ACM Conference on Recommender Systems, pp. 15–22. ACM (2016)Google Scholar
  7. 7.
    Goldberg, D., Nichols, D., Oki, B.M., Terry, D.: Using collaborative filtering to weave an information tapestry. Commun. ACM 35(12), 61–70 (1992)CrossRefGoogle Scholar
  8. 8.
    Resnick, P., Varian, H.R.: Recommender systems. Commun. ACM 40(3), 56–58 (1997)CrossRefGoogle Scholar
  9. 9.
    Pazzani, M.J.: A framework for collaborative, content-based and demographic filtering. Artif. Intell. Rev. 13(5–6), 393–408 (1999)CrossRefGoogle Scholar
  10. 10.
    Burke, R.: Hybrid recommender systems: survey and experiments. User Model. User-Adap. Inter. 12(4), 331–370 (2002)CrossRefzbMATHGoogle Scholar
  11. 11.
    Sill, J., Takács, G., Mackey, L., Lin, D.: Feature-weighted linear stacking. arXiv preprint arXiv:0911.0460 (2009)
  12. 12.
    Tinghuai, M., Jinjuan, Z., Meili, T., Yuan, T., Abdullah, A.D., Mznah, A.R., Sungyoung, L.: Social network and tag sources based augmenting collaborative recommender system. IEICE Trans. Inf. Syst. 98(4), 902–910 (2015)Google Scholar
  13. 13.
    O’Donovan, J., Smyth, B.: Trust in recommender systems. In: Proceedings of the 10th International Conference on Intelligent User Interfaces, pp. 167–174. ACM (2005)Google Scholar
  14. 14.
    Gao, P., Miao, H., Baras, J.S., Golbeck, J.: Star: semiring trust inference for trust-aware social recommenders. In: Proceedings of the 10th ACM Conference on Recommender Systems, pp. 301–308. ACM (2016)Google Scholar
  15. 15.
    Bansal, T., Belanger, D., McCallum, A.: Ask the gru: Multi-task learning for deep text recommendations. arXiv preprint arXiv:1609.02116 (2016)
  16. 16.
    Covington, P., Adams, J., Sargin, E.: Deep neural networks for youtube recommendations. In: Proceedings of the 10th ACM Conference on Recommender Systems, pp. 191–198. ACM (2016)Google Scholar
  17. 17.
    Koren, Y., Bell, R., Volinsky, C., et al.: Matrix factorization techniques for recommender systems. Computer 42(8), 30–37 (2009)CrossRefGoogle Scholar
  18. 18.
    Ge, H., Caverlee, J., Lu, H.: Taper: a contextual tensor-based approach for personalized expert recommendation. In: Proceedings of the 10th ACM Conference on Recommender Systems, pp. 261–268. ACM (2016)Google Scholar
  19. 19.
    Kim, D., Park, C., Oh, J., Lee, S., Yu, H.: Convolutional matrix factorization for document context-aware recommendation. In: Proceedings of the 10th ACM Conference on Recommender Systems, pp. 233–240. ACM (2016)Google Scholar
  20. 20.
    Karatzoglou, A., Amatriain, X., Baltrunas, L., Oliver, N.: Multiverse recommendation: n-dimensional tensor factorization for context-aware collaborative filtering. In: Proceedings of the Fourth ACM Conference on Recommender Systems, pp. 79–86. ACM (2010)Google Scholar
  21. 21.
    Koren, Y.: The bellkor solution to the netflix grand prize. Netflix Prize Documentation 81, 1–10 (2009)Google Scholar
  22. 22.
    Powers, D.M.: Evaluation: from precision, recall and f-measure to roc, informedness, markedness and correlation (2011)Google Scholar
  23. 23.
    Ge, M., Delgado-Battenfeld, C., Jannach, D.: Beyond accuracy: evaluating recommender systems by coverage and serendipity. In: Proceedings of the Fourth ACM Conference on Recommender Systems, pp. 257–260. ACM (2010)Google Scholar
  24. 24.
    Swearingen, K., Sinha, R.: Beyond algorithms: An hci perspective on recommender systems. In: ACM SIGIR 2001 Workshop on Recommender Systems, vol. 13, Citeseer, pp. 1–11 (2001)Google Scholar
  25. 25.
    Knijnenburg, B.P., Reijmer, N.J., Willemsen, M.C.: Each to his own: How different users call for different interaction methods in recommender systems. In: Proceedings of the Fifth ACM Conference on Recommender Systems, RecSys 2011, pp. 141–148. ACM, New York (2011)Google Scholar
  26. 26.
    Pu, P., Chen, L., Hu, R.: Evaluating recommender systems from the user?s perspective: survey of the state of the art. User Model. User-Adap. Inter. 22(4–5), 317–355 (2012)CrossRefGoogle Scholar
  27. 27.
    He, C., Parra, D., Verbert, K.: Interactive recommender systems: a survey of the state of the art and future research challenges and opportunities. Expert Syst. Appl. 56, 9–27 (2016)CrossRefGoogle Scholar
  28. 28.
    Calero Valdez, A., Özdemir, D., Yazdi, M.A., Schaar, A.K., Ziefle, M.: Orchestrating collaboration-using visual collaboration suggestion for steering of research clusters. Procedia Manufact. 3, 363–370 (2015)Google Scholar
  29. 29.
    Calero Valdez, A., Ziefle, M., Verbert, K.: HCI for recommender systems: the past, the present and the future. In: Proceedings of the 10th ACM Conference on Recommender Systems, RecSys 2016, pp. 123–126. ACM, New York (2016)Google Scholar
  30. 30.
    Wiesner, M., Pfeifer, D.: Health recommender systems: concepts, requirements, technical basics and challenges. Int. J. Environ. Res. Public Health 11(3), 2580–2607 (2014)CrossRefGoogle Scholar
  31. 31.
    Farrell, R.G., Danis, C.M., Ramakrishnan, S., Kellogg, W.A.: Intrapersonal retrospective recommendation: lifestyle change recommendations using stable patterns of personal behavior. In: Proceedings of the First International Workshop on Recommendation Technologies for Lifestyle Change (LIFESTYLE 2012), p. 24. Ireland, Citeseer, Dublin (2012)Google Scholar
  32. 32.
    Roitman, H., Messika, Y., Tsimerman, Y., Maman, Y.: Increasing patient safety using explanation-driven personalized content recommendation. In: Proceedings of the 1st ACM International Health Informatics Symposium, pp. 430–434. ACM (2010)Google Scholar
  33. 33.
    Rokicki, M., Herder, E., Demidova, E.: Whats on my plate: towards recommending recipe variations for diabetes patients. In: Proceedings of UMAP 2015 (2015)Google Scholar
  34. 34.
    Elsweiler, D., Harvey, M., Ludwig, B., Said, A.: Bringing the healthy into food recommenders. In: DRMS Workshop (2015)Google Scholar
  35. 35.
    Fernandez-Luque, L., Karlsen, R., Vognild, L.K.: Challenges and opportunities of using recommender systems for personalized health education. In: MIE, pp. 903–907 (2009)Google Scholar
  36. 36.
    Sezgin, E., Özkan, S.: A systematic literature review on health recommender systems. In: E-Health and Bioengineering Conference (EHB), pp. 1–4. IEEE (2013)Google Scholar
  37. 37.
    Achananuparp, P., Weber, I.: Extracting food substitutes from food diary via distributional similarity. arXiv preprint arXiv:1607.08807 (2016)
  38. 38.
    Freyne, J., Berkovsky, S.: Recommending food: reasoning on recipes and ingredients. In: Bra, P., Kobsa, A., Chin, D. (eds.) UMAP 2010. LNCS, vol. 6075, pp. 381–386. Springer, Heidelberg (2010). doi: 10.1007/978-3-642-13470-8_36 CrossRefGoogle Scholar
  39. 39.
    Ge, M., Ricci, F., Massimo, D.: Health-aware food recommender system. In: Proceedings of the 9th ACM Conference on Recommender Systems, pp. 333–334. ACM (2015)Google Scholar
  40. 40.
    Harvey, M., Ludwig, B., Elsweiler, D.: Learning user tastes: a first step to generating healthy meal plans. In: First International Workshop on Recommendation Technologies for Lifestyle Change (lifestyle 2012), Citeseer, p. 18 (2012)Google Scholar
  41. 41.
    Berkovsky, S., Freyne, J.: Group-based recipe recommendations: analysis of data aggregation strategies. In: Proceedings of the Fourth ACM Conference on Recommender Systems, pp. 111–118. ACM (2010)Google Scholar
  42. 42.
    Said, A., Tikk, D., Shi, Y., Larson, M., Stumpf, K., Cremonesi, P.: Recommender systems evaluation: A 3d benchmark. In: ACM RecSys 2012 Workshop on Recommendation Utility Evaluation: Beyond RMSE, Dublin, Ireland, pp. 21–23 (2012)Google Scholar
  43. 43.
    Herlocker, J.L., Konstan, J.A., Terveen, L.G., Riedl, J.T.: Evaluating collaborative filtering recommender systems. ACM Trans. Inf. Syst. (TOIS) 22(1), 5–53 (2004)CrossRefGoogle Scholar
  44. 44.
    Ziefle, M., Rocker, C., Holzinger, A.: Medical technology in smart homes: exploring the user’s perspective on privacy, intimacy and trust. In: 2011 IEEE 35th Annual Computer Software and Applications Conference Workshops (COMPSACW), pp. 410–415. IEEE (2011)Google Scholar
  45. 45.
    Zhou, T., Kuscsik, Z., Liu, J.G., Medo, M., Wakeling, J.R., Zhang, Y.C.: Solving the apparent diversity-accuracy dilemma of recommender systems. Proc. Natl. Acad. Sci. 107(10), 4511–4515 (2010)CrossRefGoogle Scholar
  46. 46.
    Beale, R.: Supporting serendipity: Using ambient intelligence to augment user exploration for data mining and web browsing. Int. J. Hum. Comput. Stud. 65(5), 421–433 (2007)CrossRefGoogle Scholar
  47. 47.
    Hors-Fraile, S., Benjumea, F.J.N., Hernández, L.C., Ruiz, F.O., Fernandez-Luque, L.: Design of two combined health recommender systems for tailoring messages in a smoking cessation app. arXiv preprint arXiv:1608.07192 (2016)
  48. 48.
    Ekstrand, J.D., Ekstrand, M.D.: First do no harm: Considering and minimizing harm in recommender systems designed for engendering health. In: Engendering Health Workshop at the RecSys 2016 Conference (2016)Google Scholar
  49. 49.
    Mobasher, B., Burke, R., Bhaumik, R., Williams, C.: Toward trustworthy recommender systems: an analysis of attack models and algorithm robustness. ACM Trans. Internet Technol. (TOIT) 7(4), 23 (2007)CrossRefGoogle Scholar
  50. 50.
    Grasgruber, P., Sebera, M., Hrazdira, E., Hrebickova, S., Cacek, J.: Food consumption and the actual statistics of cardiovascular diseases: an epidemiological comparison of 42 European countries. Food Nutr. Res. 60 (2016). doi: 10.3402/fnr.v60.31694 Google Scholar
  51. 51.
    Spinuzzi, C.: The methodology of participatory design. Tech. Commun. 52(2), 163–174 (2005)Google Scholar
  52. 52.
    Ekstrand, M.D., Willemsen, M.C.: Behaviorism is not enough: better recommendations through listening to users. In: Proceedings of the 10th ACM Conference on Recommender Systems. RecSys 2016, pp. 221–224. ACM, New York (2016)Google Scholar
  53. 53.
    Barnes, S.B.: A privacy paradox: Social networking in the united states. First Monday 11(9) (2006)Google Scholar
  54. 54.
    Martin, R.L.: The design of business: why design thinking is the next competitive advantage. Harvard Business Press (2009)Google Scholar
  55. 55.
    Dwork, C.: Differential privacy: a survey of results. In: Agrawal, M., Du, D., Duan, Z., Li, A. (eds.) TAMC 2008. LNCS, vol. 4978, pp. 1–19. Springer, Heidelberg (2008). doi: 10.1007/978-3-540-79228-4_1 CrossRefGoogle Scholar
  56. 56.
    Aggarwal, C.C., Philip, S.Y.: A general survey of privacy-preserving data mining models and algorithms. In: Aggarwal, C.C., Yu, P.S. (eds.) Privacy-Preserving Data Mining, pp. 11–52. Springer, US (2008)CrossRefGoogle Scholar
  57. 57.
    Ziefle, M., Halbey, J., Kowalewski, S.: Users willingness to share data on the internet: Perceived benefits and caveats. In: Proceedings of the International Conference on Internet of Things and Big Data (IoTBD 2016), pp. 255–265 (2016)Google Scholar
  58. 58.
    Kowalewski, S., Ziefle, M., Ziegeldorf, H., Wehrle, K.: Like us on facebook!-analyzing user preferences regarding privacy settings in germany. Procedia Manuf. 3, 815–822 (2015)CrossRefGoogle Scholar
  59. 59.
    Akhter, H.S.: Privacy concern and online transactions: the impact of internet self-efficacy and internet involvement. J. Consum. Mark. 31(2), 118–125 (2014)MathSciNetCrossRefGoogle Scholar
  60. 60.
    Fogel, J., Nehmad, E.: Internet social network communities: risk taking, trust, and privacy concerns. Comput. Hum. Behav. 25(1), 153–160 (2009)CrossRefGoogle Scholar
  61. 61.
    Freestone, O., Mitchell, V.: Generation y attitudes towards e-ethics and internet-related misbehaviours. J. Bus. Ethics 54(2), 121–128 (2004)CrossRefGoogle Scholar
  62. 62.
    Nissenbaum, H.: A contextual approach to privacy online. Daedalus 140(4), 32–48 (2011)CrossRefGoogle Scholar
  63. 63.
    Wilkowska, W., Ziefle, M.: Privacy and data security in e-health: requirements from the users perspective. Health Inf. J. 18(3), 191–201 (2012)CrossRefGoogle Scholar
  64. 64.
    Ziefle, M., Himmel, S., Wilkowska, W.: When your living space knows what you do: acceptance of medical home monitoring by different technologies. In: Holzinger, A., Simonic, K.-M. (eds.) USAB 2011. LNCS, vol. 7058, pp. 607–624. Springer, Heidelberg (2011). doi: 10.1007/978-3-642-25364-5_43 CrossRefGoogle Scholar
  65. 65.
    Schmidt, T., Philipsen, R., Ziefle, M.: From V2X to Control2Trust. In: Tryfonas, T., Askoxylakis, I. (eds.) HAS 2015. LNCS, vol. 9190, pp. 570–581. Springer, Heidelberg (2015). doi: 10.1007/978-3-319-20376-8_51 CrossRefGoogle Scholar
  66. 66.
    Lipkus, I.M., Hollands, J.: The visual communication of risk. J. National Cancer Inst. Monogr. 25, 149–163 (1998)Google Scholar
  67. 67.
    Seipp, K., Ochoa, X., Gutiérrez, F., Verbert, K.: A research agenda for managing uncertainty in visual analytics. Mensch und Computer 2016-Workshopband (2016)Google Scholar
  68. 68.
    Gigerenzer, G., Edwards, A.: Simple tools for understanding risks: from innumeracy to insight. BMJ. Brit. Med. J. 327, 741–744 (2003)CrossRefGoogle Scholar
  69. 69.
    Sedlmair, M., Meyer, M., Munzner, T.: Design study methodology: reflections from the trenches and the stacks. IEEE Trans. Vis. Comput. Graph. 18(12), 2431–2440 (2012)CrossRefGoogle Scholar
  70. 70.
    Calero Valdez, A., Bruns, S., Greven, C., Schroeder, U., Ziefle, M.: What do my colleagues know? dealing with cognitive complexity in organizations through visualizations. In: Zaphiris, P., Ioannou, A. (eds.) LCT 2015. LNCS, vol. 9192, pp. 449–459. Springer, Heidelberg (2015). doi: 10.1007/978-3-319-20609-7_42 CrossRefGoogle Scholar
  71. 71.
    Parra, D., Brusilovsky, P., Trattner, C.: See what you want to see: visual user-driven approach for hybrid recommendation. In: Proceedings of the 19th International Conference on Intelligent User Interfaces, pp. 235–240. ACM (2014)Google Scholar
  72. 72.
    Calero Valdez, A., Brauner, P., Ziefle, M., Kuhlen, T.W., Sedlmair, M.: Human factors in information visualization and decision support systems. Mensch und Computer 2016-Workshopband (2016)Google Scholar
  73. 73.
    Holzinger, A.: Interactive machine learning (iml). Informatik-Spektrum 39(1), 64–68 (2016)CrossRefGoogle Scholar
  74. 74.
    Kieseberg, P., Malle, B., Frühwirt, P., et al.: Brain Inf. 3, 269 (2016). doi: 10.1007/s40708-016-0046-2 Google Scholar
  75. 75.
    Malle, B., Kieseberg, P., Weippl, E., Holzinger, A.: The right to be forgotten: towards machine learning on perturbed knowledge bases. In: Buccafurri, F., Holzinger, A., Kieseberg, P., Tjoa, A.M., Weippl, E. (eds.) CD-ARES 2016. LNCS, vol. 9817, pp. 251–266. Springer, Heidelberg (2016). doi: 10.1007/978-3-319-45507-5_17 CrossRefGoogle Scholar
  76. 76.
    Kieseberg, P., Weippl, E., Holzinger, A.: Trust for the doctor-in-the-loop. European Research Consortium for Informatics and Mathematics (ERCIM) News: Tackling Big Data in the Life Sciences 104(1), 32–33Google Scholar
  77. 77.
    Felfernig, A., Jeran, M., Ninaus, G., Reinfrank, F., Reiterer, S.: Toward the next generation of recommender systems: applications and research challenges. In: Tsihrintzis, G.A. (ed.) Multimedia Services in Intelligent Environments, pp. 81–98. Springer, Switzerland (2013)CrossRefGoogle Scholar
  78. 78.
    Rossetti, M., Stella, F., Zanker, M.: Contrasting offline and online results when evaluating recommendation algorithms. In: Proceedings of the 10th ACM Conference on Recommender Systems, pp. 31–34. ACM (2016)Google Scholar
  79. 79.
    Karumur, R.P., Nguyen, T.T., Konstan, J.A.: Exploring the value of personality in predicting rating behaviors: a study of category preferences on movielens. In: Proceedings of the 10th ACM Conference on Recommender Systems, pp. 139–142. ACM (2016)Google Scholar
  80. 80.
    Azaria, A., Hong, J.: Recommender systems with personality. In: Proceedings of the 10th ACM Conference on Recommender Systems, RecSys 2016, pp. 207–210. ACM, New York (2016)Google Scholar
  81. 81.
    Shiv, B., Carmon, Z., Ariely, D.: Placebo effects of marketing actions: consumers may get what they pay for. J. Mark. Res. 42(4), 383–393 (2005)CrossRefGoogle Scholar
  82. 82.
    Laufer, P., Wagner, C., Flöck, F., Strohmaier, M.: Mining cross-cultural relations from wikipedia: a study of 31 european food cultures. In: Proceedings of the ACM Web Science Conference, p. 3. ACM (2015)Google Scholar
  83. 83.
    Babitsch, B., Braun, T., Borde, T., David, M.: Doctor’s perception of doctor-patient relationships in emergency departments: what roles do gender and ethnicity play? BMC Health Serv. Res. 8(1), 1 (2008)CrossRefGoogle Scholar
  84. 84.
    Knijnenburg, B.P., Sivakumar, S., Wilkinson, D.: Recommender systems for self-actualization. In: Proceedings of the 10th ACM Conference on Recommender Systems, RecSys 2016, pp. 11–14. ACM, New York (2016)Google Scholar

Copyright information

© Springer International Publishing AG 2016

Authors and Affiliations

  • André Calero Valdez
    • 1
    • 4
    Email author
  • Martina Ziefle
    • 1
  • Katrien Verbert
    • 2
  • Alexander Felfernig
    • 3
  • Andreas Holzinger
    • 4
  1. 1.Human-Computer Interaction CenterRWTH-Aachen UniversityAachenGermany
  2. 2.KU LeuvenLeuvenBelgium
  3. 3.Institute for Software TechnologyGraz University of TechnologyGrazAustria
  4. 4.Holzinger Group HCI-KDD, Institute for Medical Informatics, Statistics & DocumentationMedical University GrazGrazAustria

Personalised recommendations