Architecting dietary intake monitoring as a service combining NLP and IoT

  • Antonio Benítez-Guijarro
  • Zoraida Callejas
  • Manuel NogueraEmail author
  • Kawtar Benghazi
Original Research


Currently there exist many tools that support monitoring and encouragement of healthy nutrition habits in the context of wellness promotion. In this domain, interfaces based on natural language provide more flexibility for nutritional self-reporting than traditional form-based applications, allowing the users to provide richer and spontaneous descriptions. Nonetheless, in certain circumstances, natural language records may miss some important aspects, such as the quantity of food eaten, which results in incomplete recordings. In the Internet-of-Things (IoT) paradigm, smart home appliances can support and complement the recording process so as to make it more accurate. However, in order to build systems that support the semantic analysis of nutritional self-reports, it is necessary to integrate multiple inter-related components, possibly within complex e-health platforms. For this reason, these components should be designed and encapsulated avoiding monolithic approaches that derive in rigidity and dependency of particular technologies. Currently, there are no models or architectures that serve as a reference for developers towards this objective. In this paper, we present a service-based architecture that helps to contrast and complement the descriptions of food intakes by means of connected smart home devices, coordinating all the stages during the process of recognizing food records provided in natural language. Additionally, we aim to identify and design the essential services that are required to automate the recording and subsequent processing of natural language descriptions of nutritional intakes in association with smart home devices. The functionalities provided by each of these services are ready to work in isolation, just out of the box, or in downstream pipeline processes, bypassing the inconveniences of monolithic architectures.


Natural language processing NLP Services IoT Smart devices Nutrition Semantic processing Well-being E-health 



This research has been supported by the project DEP2015-70980-R of the Spanish Ministry of Economy and Competi- tiveness (MINECO) and European Regional Development Fund (ERDF), the European Union’s Horizon 2020 research and innovation programme under grant agreement No 823907, as well as, received inputs from the COST Action IC1303 AAPELE.


  1. Axelsson J, Skoglund M (2016) Quality assurance in software ecosystems: a systematic literature mapping and research agenda. J Syst Softw 114:69–81CrossRefGoogle Scholar
  2. Ayoola I, Wetzels M, Peters P, van Berlo S, Feijs L (2018) Do change platform: a service-based architecture for secure aggregation and distribution of health and wellbeing data. Int J Med Inform 117:103–111CrossRefGoogle Scholar
  3. Azar KM, Lesser LI, Laing BY, Stephens J, Aurora MS, Burke LE, Palaniappan LP (2013) Mobile applications for weight management: theory-based content analysis. Am J Prev Med 45(5):583–589CrossRefGoogle Scholar
  4. Becker M (2008) Software architecture trends and promising technology for ambient assisted living systems. In: Dagstuhl seminar proceedings, Schloss Dagstuhl-Leibniz-Zentrum für InformatikGoogle Scholar
  5. Benítez-Guijarro A, Callejas Z, Noguera M, Benghazi K (2018) Introducing computational semantics for natural language understanding in conversational nutrition coaches for healthy eating. Multidiscip Digit Publ Inst Proc 2:506Google Scholar
  6. Benítez-Guijarro A, Ruiz-Zafra A, Callejas Z, Medina-Medina N, Noguera M, Benghazi K (2019) General architecture for development of virtual coaches for healthy habits monitoring and encouragement. Sensors 19(1):108CrossRefGoogle Scholar
  7. Blasco R, Marco Á, Casas R, Cirujano D, Picking R (2014) A smart kitchen for ambient assisted living. Sensors 14(1):1629–1653CrossRefGoogle Scholar
  8. Byambasuren O, Sanders S, Beller E, Glasziou P (2018) Prescribable mHealth apps identified from an overview of systematic reviews. NPJ Digit Med 1:2398–6352CrossRefGoogle Scholar
  9. Chen C, Karvela M, Sohbati M, Shinawatra T, Toumazou C (2018a) Person—personalized expert recommen- dation system for optimized nutrition. IEEE Trans Biomed Circuits Syst 12(1):151–160CrossRefGoogle Scholar
  10. Chen J, Gemming L, Hanning R, Allman-Farinelli M (2018b) Smartphone apps and the nutrition care process: current perspectives and future considerations. Patient Educ Couns 101(4):750–757CrossRefGoogle Scholar
  11. Choi A, Shin H (2018) Longitudinal healthcare data management platform of healthcare iot devices for personalized services. J Univ Comput Sci 24(9):1153–1169.
  12. Djeddar A, Bendjenna H, Amirat A, Roose P, Chung L (2018) An emerging multi-paradigm for representing mobile applications’ architectures using heterogeneous conceptual bricks. IJCAT 57(1):1–13CrossRefGoogle Scholar
  13. Dolejsova M (2018) Edible speculations in the parlour of food futures. In: Extended abstracts of the 2018 CHI conference on human factors in computing systems, ACM, p alt13Google Scholar
  14. Dragoni M, Bailoni T, Eccher C, Guerini M, Maimone R (2017) A semantic-enabled platform for supporting healthy lifestyles. In: Proceedings of the symposium on applied computing, SAC’17, ACM, New York, pp 315–322Google Scholar
  15. Fielding RT, Taylor RN (2000) Architectural styles and the design of network-based software architectures, vol 7. University of California, Irvine Doctoral dissertationGoogle Scholar
  16. Fuchsberger V (2008) Ambient assisted living: elderly people’s needs and how to face them. In: Proceedings of the 1st ACM international workshop on semantic ambient media experiences, ACM, pp 21–24Google Scholar
  17. Gochhayat SP, Kaliyar P, Conti M, Tiwari P, Prasath V, Gupta D, Khanna A (2019) Lisa: lightweight context-aware iot service architecture. J Clean Prod 212:1345–1356CrossRefGoogle Scholar
  18. Gu H, Wang D (2009) A content-aware fridge based on rfid in smart home for home-healthcare. In: 2009 11th international conference on advanced communication technology, vol 2, IEEE, pp 987–990Google Scholar
  19. Hezarjaribi N, Mazrouee S, Ghasemzadeh H (2018) Speech2health: a mobile framework for monitoring dietary composition from spoken data. IEEE J Biomed Health Inform 22(1):252–264CrossRefGoogle Scholar
  20. Hirschberg J, Manning CD (2015) Advances in natural language processing. Science 349(6245):261–266MathSciNetCrossRefGoogle Scholar
  21. Hornik K (2014) Opennlp: apache opennlp tools interface, 2014. R package version 0.2-3. Accessed 1 Nov 2019
  22. Kolodziejczyk JK, Merchant G, Norman GJ (2012) Reliability and validity of child/adolescent food frequency questionnaires that assess foods and/or food groups. J Pediatr Gastroenterol Nutr 55(1):4–13CrossRefGoogle Scholar
  23. Kumar A, Gupta A, Chan J, Tucker S, Hoffmeister B, Dreyer M, Peshterliev S, Gandhe A, Filiminov D, Rastrow A, Monson C, Kumar A (2017) Just ask: building an architecture for extensible self-service spoken language understanding. arXiv preprint arXiv:1711.00549
  24. Lacson R, Long W (2006) Natural language processing of spoken diet records (sdrs). In: AMIA annual symposium proceedings, vol 2006, American Medical Informatics Association, p 454Google Scholar
  25. Leijdekkers P, Gay V, Lawrence E (2007) Smart homecare system for health tele-monitoring. In: First international conference on the digital society (ICDS’07), IEEE, p 3Google Scholar
  26. Lieffers JR, Arocha JF, Grindrod K, Hanning RM (2018) Experiences and perceptions of adults accessing publicly available nutrition behavior-change mobile apps for weight management. J Acad Nutr Diet 118(2):229–239CrossRefGoogle Scholar
  27. Lin C-KE, Wang T-H, Yang J-FK (2012) Touch doctor—a nutrition control service system developed under living lab methodology. Int J Autom Smart Technol 2(3):253–263CrossRefGoogle Scholar
  28. Loke S (2006) Context-aware pervasive systems: architectures for a new breed of applications. Auerbach Publications, Boca RatonCrossRefGoogle Scholar
  29. Loper E, Bird S (2002) NLTK: the natural language toolkit. arXiv preprint arXiv:cs/0205028
  30. Macias C, Panch T, Hicks YM, Scolnick JS, Weene DL, O¨ngu¨r D, Cohen BM (2015) Using smartphone apps to promote psychiatric and physical well-being. Psychiatr Q 86(4):505–519CrossRefGoogle Scholar
  31. Manning C, Surdeanu M, Bauer J, Finkel J, Bethard S, McClosky D (2014) The stanford corenlp natural language processing toolkit. In: Proceedings of 52nd annual meeting of the association for computational linguistics: system demonstrations, pp 55–60Google Scholar
  32. Minker W, López-Cózar R, Mctear M (2009) The role of spoken language dialogue interaction in intelligent environments. J Ambient Intell Smart Environ 1(1):31–36Google Scholar
  33. Moguel E, Berrocal J, Murillo JM, Garcia-Alonso J, Mendes D, Fonseca C, Lopes M (2018) Monitoring food intake in an aging population: a survey on technological solutions. Multidiscip Digit Publ Inst Proc 2(19):445Google Scholar
  34. Montenegro JLZ, da Costa CA, da Rosa R (2019) Survey of conversational agents in health. Expert Syst Appl 129:56–67CrossRefGoogle Scholar
  35. Ochoa SF, Gutierrez FJ (2018) Architecting e-coaching systems: a first step for dealing with their intrinsic design complexity. Computer 51(3):16–23CrossRefGoogle Scholar
  36. Rangan AM, Tieleman L, Louie JCY, Tang LM, Hebden L, Roy R, Kay J, Allman-Farinelli M (2016) Electronic dietary intake assessment (e-dia): relative validity of a mobile phone application to measure intake of food groups. Br J Nutr 115(12):2219–2226CrossRefGoogle Scholar
  37. Roda C, Navarro E, Zdun U, López-Jaquero V, Simhandl G (2018) Past and future of software architectures for context-aware systems: a systematic mapping study. J Syst Softw 146:310–355CrossRefGoogle Scholar
  38. Ruiz-Zafra Á, Benghazi K, Noguera M, Garrido JL (2013) Zappa: an open mobile platform to build cloud-based m-health systems. In: Ambient intelligence-software and applications. Springer, Heidelberg, pp 87–94CrossRefGoogle Scholar
  39. Ruiz-Zafra Á, Noguera M, Benghazi K, Ochoa SF (2015) A model-driven approach for wearable systems developments. Int J Distrib Sens Netw 11(10):637130Google Scholar
  40. Ruiz-Zafra Á, Benghazi K, Mavromoustakis CX, Noguera M (2018) An iot-aware architectural model for smart habitats. In: 16th IEEE international conference on embedded and ubiquitous computing, EUC 2018, Bucharest, Romania, 29–31 Oct 2018, pp 103–110Google Scholar
  41. Souza R, Lopes J, Geyer C, Cardozo A, Yamin A, Barbosa J (2018) An architecture for iot management targeted to context awareness of ubiquitous applications. J Univ Comput Sci 24(10):1452–1471.
  42. Stone AA, Shiffman S, Schwartz JE, Broderick JE, Hufford MR (2003) Patient compliance with paper and electronic diaries. Controll Clin Trials 24(2):182–199CrossRefGoogle Scholar
  43. Villarreal V, Fontecha J, Hervas R, Bravo J (2014) Mobile and ubiquitous architecture for the medical control of chronic diseases through the use of intelligent devices: using the architecture for patients with diabetes. Future Gener Comput Syst 34:161–175CrossRefGoogle Scholar
  44. Yu Z, Sealey-Potts C, Rodriguez J (2015) Dietary self-monitoring in weight management: current evidence on efficacy and adherence. J Acad Nutr Diet 115(12):1931–1938CrossRefGoogle Scholar
  45. Zgheib R, Conchon E, Bastide R (2017) Engineering iot healthcare applications: towards a semantic data driven sustainable architecture. In: Giokas K, Bokor L, Hopfgartner F (eds) eHealth 360. Springer International Publishing, Cham, pp 407–418Google Scholar
  46. Zhu L, Aurum A, Gorton I, Jeffery R (2005) Tradeoff and sensitivity analysis in software architecture evaluation using analytic hierarchy process. Softw Qual J 13(4):357–375CrossRefGoogle Scholar

Copyright information

© Springer-Verlag GmbH Germany, part of Springer Nature 2019

Authors and Affiliations

  1. 1.University of GranadaGranadaSpain

Personalised recommendations