Food Intake Detection from Inertial Sensors Using LSTM Networks

  • Konstantinos KyritsisEmail author
  • Christos Diou
  • Anastasios Delopoulos
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10590)


Unobtrusive analysis of eating behavior based on Inertial Measurement Unit (IMU) sensors (e.g. accelerometer) is a topic that has attracted the interest of both the industry and the research community over the past years. This work presents a method for detecting food intake moments that occur during a meal session using the accelerometer and gyroscope signals of an off-the-shelf smartwatch. We propose a two step approach. First, we model the hand micro-movements that take place while eating using an array of binary Support Vector Machines (SVMs); then the detection of intake moments is achieved by processing the sequence of SVM score vectors by a Long Short Term Memory (LSTM) network. Evaluation is performed on a publicly available dataset with 10 subjects, where the proposed method outperforms similar approaches by achieving an F1 score of 0.892.


Food intake Eating monitoring Wearable sensors LSTM 



The work leading to these results has received funding from the European Community’s Health, demographic change and well-being Programme under Grant Agreement No. 727688 (, 01/12/2016–30/11/2020. We gratefully acknowledge the support of NVIDIA Corporation with the donation of the Tesla K40 GPU used for this research.


  1. 1.
    Amft, O., Junker, H., Troster, G.: Detection of eating and drinking arm gestures using inertial body-worn sensors. In: Ninth IEEE International Symposium on Wearable Computers, pp. 160–163 (2005)Google Scholar
  2. 2.
    Dong, Y., Hoover, A., Scisco, J., Muth, E.: A new method for measuring meal intake in humans via automated wrist motion tracking. Appl. Psychophysiol. Biofeedback 37(3), 205–215 (2012)CrossRefGoogle Scholar
  3. 3.
    Karpathy, A., Johnson, J., Li, F.: Visualizing and understanding recurrent networks. CoRR abs/1506.02078 (2015).
  4. 4.
    Kyritsis, K., Tatli, C.L., Diou, C., Delopoulos, A.: Automated analysis of in meal eating behavior using a commercial wristband IMU sensor. In: 2017 39th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC) (2017)Google Scholar
  5. 5.
    Madgwick, S.O.H., Harrison, A.J.L., Vaidyanathan, R.: Estimation of IMU and MARG orientation using a gradient descent algorithm. In: 2011 IEEE International Conference on Rehabilitation Robotics, pp. 1–7 (2011)Google Scholar
  6. 6.
    Mirtchouk, M., Merck, C., Kleinberg, S.: Automated estimation of food type and amount consumed from body-worn audio and motion sensors. In: Proceedings of the 2016 ACM International Joint Conference on Pervasive and Ubiquitous Computing, pp. 451–462 (2016)Google Scholar
  7. 7.
    Ordez, F.J., Roggen, D.: Deep convolutional and LSTM recurrent neural networks for multimodal wearable activity recognition. Sensors 16(1), 115 (2016)CrossRefGoogle Scholar
  8. 8.
    Papapanagiotou, V., Diou, C., Langlet, B., Ioakimidis, I., Delopoulos, A.: A parametric probabilistic context-free grammar for food intake analysis based on continuous meal weight measurements. In: 2015 37th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC) (2015)Google Scholar
  9. 9.
    Papapanagiotou, V., Diou, C., Zhou, L., van den Boer, J., Mars, M., Delopoulos, A.: A novel chewing detection system based on PPG, audio, and accelerometry. IEEE J. Biomedical Health Inf. 21(3), 607–618 (2017)CrossRefGoogle Scholar
  10. 10.
    Ramos-Garcia, R.I., et al.: Improving the recognition of eating gestures using intergesture sequential dependencies. IEEE J. Biomed. Health Inf. 19(3), 825–831 (2015)CrossRefGoogle Scholar
  11. 11.
    World Health Organization: Global health risks: mortality and burden of disease attributable to selected major risks. World Health Organization (2009)Google Scholar
  12. 12.
    Zhang, S., et al.: Food watch: detecting and characterizing eating episodes through feeding gestures. In: Proceedings of the 11th EAI International Conference on Body Area Networks, pp. 91–96 (2016)Google Scholar

Copyright information

© Springer International Publishing AG 2017

Authors and Affiliations

  • Konstantinos Kyritsis
    • 1
    Email author
  • Christos Diou
    • 1
  • Anastasios Delopoulos
    • 1
  1. 1.Information Processing Laboratory, Multimedia Understanding GroupAristotle University of ThessalonikiThessalonikiGreece

Personalised recommendations