Architecting dietary intake monitoring as a service combining NLP and IoT


Currently there exist many tools that support monitoring and encouragement of healthy nutrition habits in the context of wellness promotion. In this domain, interfaces based on natural language provide more flexibility for nutritional self-reporting than traditional form-based applications, allowing the users to provide richer and spontaneous descriptions. Nonetheless, in certain circumstances, natural language records may miss some important aspects, such as the quantity of food eaten, which results in incomplete recordings. In the Internet-of-Things (IoT) paradigm, smart home appliances can support and complement the recording process so as to make it more accurate. However, in order to build systems that support the semantic analysis of nutritional self-reports, it is necessary to integrate multiple inter-related components, possibly within complex e-health platforms. For this reason, these components should be designed and encapsulated avoiding monolithic approaches that derive in rigidity and dependency of particular technologies. Currently, there are no models or architectures that serve as a reference for developers towards this objective. In this paper, we present a service-based architecture that helps to contrast and complement the descriptions of food intakes by means of connected smart home devices, coordinating all the stages during the process of recognizing food records provided in natural language. Additionally, we aim to identify and design the essential services that are required to automate the recording and subsequent processing of natural language descriptions of nutritional intakes in association with smart home devices. The functionalities provided by each of these services are ready to work in isolation, just out of the box, or in downstream pipeline processes, bypassing the inconveniences of monolithic architectures.

This is a preview of subscription content, access via your institution.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5


  1. Axelsson J, Skoglund M (2016) Quality assurance in software ecosystems: a systematic literature mapping and research agenda. J Syst Softw 114:69–81

    Article  Google Scholar 

  2. Ayoola I, Wetzels M, Peters P, van Berlo S, Feijs L (2018) Do change platform: a service-based architecture for secure aggregation and distribution of health and wellbeing data. Int J Med Inform 117:103–111

    Article  Google Scholar 

  3. Azar KM, Lesser LI, Laing BY, Stephens J, Aurora MS, Burke LE, Palaniappan LP (2013) Mobile applications for weight management: theory-based content analysis. Am J Prev Med 45(5):583–589

    Article  Google Scholar 

  4. Becker M (2008) Software architecture trends and promising technology for ambient assisted living systems. In: Dagstuhl seminar proceedings, Schloss Dagstuhl-Leibniz-Zentrum für Informatik

  5. Benítez-Guijarro A, Callejas Z, Noguera M, Benghazi K (2018) Introducing computational semantics for natural language understanding in conversational nutrition coaches for healthy eating. Multidiscip Digit Publ Inst Proc 2:506

    Google Scholar 

  6. Benítez-Guijarro A, Ruiz-Zafra A, Callejas Z, Medina-Medina N, Noguera M, Benghazi K (2019) General architecture for development of virtual coaches for healthy habits monitoring and encouragement. Sensors 19(1):108

    Article  Google Scholar 

  7. Blasco R, Marco Á, Casas R, Cirujano D, Picking R (2014) A smart kitchen for ambient assisted living. Sensors 14(1):1629–1653

    Article  Google Scholar 

  8. Byambasuren O, Sanders S, Beller E, Glasziou P (2018) Prescribable mHealth apps identified from an overview of systematic reviews. NPJ Digit Med 1:2398–6352

    Article  Google Scholar 

  9. Chen C, Karvela M, Sohbati M, Shinawatra T, Toumazou C (2018a) Person—personalized expert recommen- dation system for optimized nutrition. IEEE Trans Biomed Circuits Syst 12(1):151–160

    Article  Google Scholar 

  10. Chen J, Gemming L, Hanning R, Allman-Farinelli M (2018b) Smartphone apps and the nutrition care process: current perspectives and future considerations. Patient Educ Couns 101(4):750–757

    Article  Google Scholar 

  11. Choi A, Shin H (2018) Longitudinal healthcare data management platform of healthcare iot devices for personalized services. J Univ Comput Sci 24(9):1153–1169.

  12. Djeddar A, Bendjenna H, Amirat A, Roose P, Chung L (2018) An emerging multi-paradigm for representing mobile applications’ architectures using heterogeneous conceptual bricks. IJCAT 57(1):1–13

    Article  Google Scholar 

  13. Dolejsova M (2018) Edible speculations in the parlour of food futures. In: Extended abstracts of the 2018 CHI conference on human factors in computing systems, ACM, p alt13

  14. Dragoni M, Bailoni T, Eccher C, Guerini M, Maimone R (2017) A semantic-enabled platform for supporting healthy lifestyles. In: Proceedings of the symposium on applied computing, SAC’17, ACM, New York, pp 315–322

  15. Fielding RT, Taylor RN (2000) Architectural styles and the design of network-based software architectures, vol 7. University of California, Irvine Doctoral dissertation

  16. Fuchsberger V (2008) Ambient assisted living: elderly people’s needs and how to face them. In: Proceedings of the 1st ACM international workshop on semantic ambient media experiences, ACM, pp 21–24

  17. Gochhayat SP, Kaliyar P, Conti M, Tiwari P, Prasath V, Gupta D, Khanna A (2019) Lisa: lightweight context-aware iot service architecture. J Clean Prod 212:1345–1356

    Article  Google Scholar 

  18. Gu H, Wang D (2009) A content-aware fridge based on rfid in smart home for home-healthcare. In: 2009 11th international conference on advanced communication technology, vol 2, IEEE, pp 987–990

  19. Hezarjaribi N, Mazrouee S, Ghasemzadeh H (2018) Speech2health: a mobile framework for monitoring dietary composition from spoken data. IEEE J Biomed Health Inform 22(1):252–264

    Article  Google Scholar 

  20. Hirschberg J, Manning CD (2015) Advances in natural language processing. Science 349(6245):261–266

    MathSciNet  Article  Google Scholar 

  21. Hornik K (2014) Opennlp: apache opennlp tools interface, 2014. R package version 0.2-3. Accessed 1 Nov 2019

  22. Kolodziejczyk JK, Merchant G, Norman GJ (2012) Reliability and validity of child/adolescent food frequency questionnaires that assess foods and/or food groups. J Pediatr Gastroenterol Nutr 55(1):4–13

    Article  Google Scholar 

  23. Kumar A, Gupta A, Chan J, Tucker S, Hoffmeister B, Dreyer M, Peshterliev S, Gandhe A, Filiminov D, Rastrow A, Monson C, Kumar A (2017) Just ask: building an architecture for extensible self-service spoken language understanding. arXiv preprint arXiv:1711.00549

  24. Lacson R, Long W (2006) Natural language processing of spoken diet records (sdrs). In: AMIA annual symposium proceedings, vol 2006, American Medical Informatics Association, p 454

  25. Leijdekkers P, Gay V, Lawrence E (2007) Smart homecare system for health tele-monitoring. In: First international conference on the digital society (ICDS’07), IEEE, p 3

  26. Lieffers JR, Arocha JF, Grindrod K, Hanning RM (2018) Experiences and perceptions of adults accessing publicly available nutrition behavior-change mobile apps for weight management. J Acad Nutr Diet 118(2):229–239

    Article  Google Scholar 

  27. Lin C-KE, Wang T-H, Yang J-FK (2012) Touch doctor—a nutrition control service system developed under living lab methodology. Int J Autom Smart Technol 2(3):253–263

    Article  Google Scholar 

  28. Loke S (2006) Context-aware pervasive systems: architectures for a new breed of applications. Auerbach Publications, Boca Raton

    Book  Google Scholar 

  29. Loper E, Bird S (2002) NLTK: the natural language toolkit. arXiv preprint arXiv:cs/0205028

  30. Macias C, Panch T, Hicks YM, Scolnick JS, Weene DL, O¨ngu¨r D, Cohen BM (2015) Using smartphone apps to promote psychiatric and physical well-being. Psychiatr Q 86(4):505–519

    Article  Google Scholar 

  31. Manning C, Surdeanu M, Bauer J, Finkel J, Bethard S, McClosky D (2014) The stanford corenlp natural language processing toolkit. In: Proceedings of 52nd annual meeting of the association for computational linguistics: system demonstrations, pp 55–60

  32. Minker W, López-Cózar R, Mctear M (2009) The role of spoken language dialogue interaction in intelligent environments. J Ambient Intell Smart Environ 1(1):31–36

    Google Scholar 

  33. Moguel E, Berrocal J, Murillo JM, Garcia-Alonso J, Mendes D, Fonseca C, Lopes M (2018) Monitoring food intake in an aging population: a survey on technological solutions. Multidiscip Digit Publ Inst Proc 2(19):445

    Google Scholar 

  34. Montenegro JLZ, da Costa CA, da Rosa R (2019) Survey of conversational agents in health. Expert Syst Appl 129:56–67

    Article  Google Scholar 

  35. Ochoa SF, Gutierrez FJ (2018) Architecting e-coaching systems: a first step for dealing with their intrinsic design complexity. Computer 51(3):16–23

    Article  Google Scholar 

  36. Rangan AM, Tieleman L, Louie JCY, Tang LM, Hebden L, Roy R, Kay J, Allman-Farinelli M (2016) Electronic dietary intake assessment (e-dia): relative validity of a mobile phone application to measure intake of food groups. Br J Nutr 115(12):2219–2226

    Article  Google Scholar 

  37. Roda C, Navarro E, Zdun U, López-Jaquero V, Simhandl G (2018) Past and future of software architectures for context-aware systems: a systematic mapping study. J Syst Softw 146:310–355

    Article  Google Scholar 

  38. Ruiz-Zafra Á, Benghazi K, Noguera M, Garrido JL (2013) Zappa: an open mobile platform to build cloud-based m-health systems. In: Ambient intelligence-software and applications. Springer, Heidelberg, pp 87–94

    Chapter  Google Scholar 

  39. Ruiz-Zafra Á, Noguera M, Benghazi K, Ochoa SF (2015) A model-driven approach for wearable systems developments. Int J Distrib Sens Netw 11(10):637130

    Google Scholar 

  40. Ruiz-Zafra Á, Benghazi K, Mavromoustakis CX, Noguera M (2018) An iot-aware architectural model for smart habitats. In: 16th IEEE international conference on embedded and ubiquitous computing, EUC 2018, Bucharest, Romania, 29–31 Oct 2018, pp 103–110

  41. Souza R, Lopes J, Geyer C, Cardozo A, Yamin A, Barbosa J (2018) An architecture for iot management targeted to context awareness of ubiquitous applications. J Univ Comput Sci 24(10):1452–1471.

  42. Stone AA, Shiffman S, Schwartz JE, Broderick JE, Hufford MR (2003) Patient compliance with paper and electronic diaries. Controll Clin Trials 24(2):182–199

    Article  Google Scholar 

  43. Villarreal V, Fontecha J, Hervas R, Bravo J (2014) Mobile and ubiquitous architecture for the medical control of chronic diseases through the use of intelligent devices: using the architecture for patients with diabetes. Future Gener Comput Syst 34:161–175

    Article  Google Scholar 

  44. Yu Z, Sealey-Potts C, Rodriguez J (2015) Dietary self-monitoring in weight management: current evidence on efficacy and adherence. J Acad Nutr Diet 115(12):1931–1938

    Article  Google Scholar 

  45. Zgheib R, Conchon E, Bastide R (2017) Engineering iot healthcare applications: towards a semantic data driven sustainable architecture. In: Giokas K, Bokor L, Hopfgartner F (eds) eHealth 360. Springer International Publishing, Cham, pp 407–418

    Google Scholar 

  46. Zhu L, Aurum A, Gorton I, Jeffery R (2005) Tradeoff and sensitivity analysis in software architecture evaluation using analytic hierarchy process. Softw Qual J 13(4):357–375

    Article  Google Scholar 

Download references


This research has been supported by the project DEP2015-70980-R of the Spanish Ministry of Economy and Competi- tiveness (MINECO) and European Regional Development Fund (ERDF), the European Union’s Horizon 2020 research and innovation programme under grant agreement No 823907, as well as, received inputs from the COST Action IC1303 AAPELE.

Author information



Corresponding author

Correspondence to Manuel Noguera.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Benítez-Guijarro, A., Callejas, Z., Noguera, M. et al. Architecting dietary intake monitoring as a service combining NLP and IoT. J Ambient Intell Human Comput (2019).

Download citation


  • Natural language processing
  • NLP
  • Services
  • IoT
  • Smart devices
  • Nutrition
  • Semantic processing
  • Well-being
  • E-health