Skip to main content

Classification of Eating Behaviors in Unconstrained Environments

  • 482 Accesses

Part of the Communications in Computer and Information Science book series (CCIS,volume 1400)


Obesity and its numerous devastating consequences are on the rise globally. While widespread tactics to fight against obesity often focus on healthy eating, how the food is consumed is oftentimes overlooked even though convincing evidence attests that merely eating slowly and properly chewing one’s meal significantly reduces obesity. This research introduces a method that recognizes common human actions during mealtime—namely, food chewing, food swallowing, drink swallowing, and talking. The proposed system is unobtrusive. It uses a cheap and small bone conduction microphone to collect intra-body sound and a smartphone that provides feedback in real-time. Our proposed approach achieves similar performances (Accuracy = 97.5%, Specificity = 98.0%, Precision = 83.8%, Recall = 91.7%, \(F_1\) score = 87.2%, and MCC = 0.85) as those achieved by the most recent state of the art models even though our system uses modest machine learning models.


  • Eating quantification
  • Chewing
  • Swallowing
  • Sound analysis
  • Activity recognition
  • Free-living conditions

This is a preview of subscription content, access via your institution.

Buying options

USD   29.95
Price excludes VAT (USA)
  • DOI: 10.1007/978-3-030-72379-8_29
  • Chapter length: 18 pages
  • Instant PDF download
  • Readable on all devices
  • Own it forever
  • Exclusive offer for individuals only
  • Tax calculation will be finalised during checkout
USD   109.00
Price excludes VAT (USA)
  • ISBN: 978-3-030-72379-8
  • Instant PDF download
  • Readable on all devices
  • Own it forever
  • Exclusive offer for individuals only
  • Tax calculation will be finalised during checkout
Softcover Book
USD   139.99
Price excludes VAT (USA)
Fig. 1.
Fig. 2.
Fig. 3.
Fig. 4.
Fig. 5.
Fig. 6.
Fig. 7.


  1. 1.

  2. 2.

    \(XGB(colsample\_bytree\,=\,0.5, \gamma \,=\,0.7, depth\,=\,4, subsample\,=\,0.4)\).

  3. 3.

    \(KNeighbors(k\_neighbors\,=\,14, weights\) = “distance”).

  4. 4.

    \(RandomForest(depth\,=\,24, max\_features\) = “\(log_2\)”).

  5. 5.

    \(LGBM(colsample\_bytree\,=\,0.7, depth\,=\,32,num\_leaves\,=\,70, \alpha \,=\,0.5)\).


  1. Alías, F., Socoró, J., Sevillano, X.: A Review of physical and perceptual feature extraction techniques for speech, music and environmental sounds. Appl. Sci. 6(5), 143 (2016).

    CrossRef  Google Scholar 

  2. Amft, O., Troster, G.: On-body sensing solutions for automatic dietary monitoring. IEEE Pervasive Comput. 8(2), 62–70 (2009).

    CrossRef  Google Scholar 

  3. Bi, S., et al.: Auracle: detecting eating episodes with an ear-mounted sensor. In: Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies, vol. 2, no. 3, pp. 1–27, September 2018.

  4. Box, G.E.P., Cox, D.R.: An analysis of transformations. J. Roy. Stat. Soc.: Series B (Methodol.) 26(2), 211–243 (1964).

  5. Chakroborty, S., Roy, A., Saha, G.: Fusion of a complementary feature set with MFCC for improved closed set text-independent speaker identification. In: 2006 IEEE International Conference on Industrial Technology, pp. 387–390. IEEE (2006).

  6. Chawla, N.V., Bowyer, K.W., Hall, L.O., Kegelmeyer, W.P.: SMOTE: synthetic minority over-sampling technique. J. Artif. Intell. Res. 16, 321–357 (2002).

    CrossRef  MATH  Google Scholar 

  7. Chicco, D., Jurman, G.: The advantages of the Matthews correlation coefficient (MCC) over F1 score and accuracy in binary classification evaluation. BMC Genomics 21(1), 6 (2020).

    CrossRef  Google Scholar 

  8. Goto, M.: SmartMusicKIOSK: music listening station with chorus-search function. In: Proceedings of the 16th Annual ACM Symposium on User Interface Software and Technology - UIST 2003, vol. 5, pp. 31–40. ACM Press, New York (2003).,

  9. Guyon, I., Weston, J., Barnhill, S., Vapnik, V.: Gene selection for cancer classification using support vector machines. Mach. Learn. 46(1–3), 389–422 (2002).

    CrossRef  MATH  Google Scholar 

  10. Han, H., Wang, W.Y., Mao, B.H.: Borderline-SMOTE: a new over-sampling method in imbalanced data sets learning. Adv. Intell. Syst. Comput. 683, 878–887 (2005).

    CrossRef  Google Scholar 

  11. He, H., Ma, Y.: Imbalanced Learning: Foundations, Algorithms, and Applications. Wiley-IEEE Press, July 2013

    Google Scholar 

  12. Hurst, Y., Fukuda, H.: Effects of changes in eating speed on obesity in patients with diabetes: a secondary analysis of longitudinal health check-up data. BMJ Open 8(1), e019589 (2018).

    CrossRef  Google Scholar 

  13. Hussain, G., Javed, K., Cho, J., Yi, J.: Food intake detection and classification using a necklace-type piezoelectric wearable sensor system. IEICE Trans. Inf. Syst. 101(11), 2795–2807 (2018).

    CrossRef  Google Scholar 

  14. Jain, A., Kondo, T., Kamachi, H., Yokokubo, A., Lopez, G.: Detailed classification of meal-related activities from eating sound collected in free living conditions. In: Proceedings of the 13th International Joint Conference on Biomedical Engineering Systems and Technologies - Vol. 5: HEALTHINF, pp. 284–291. INSTICC, SCITEPRESS, February 2020.

  15. Kondo, T., Kamachi, H., Ishii, S., Yokokubo, A., Lopez, G.: Robust classification of eating sound collected in natural meal environment. In: Proceedings of the 2019 ACM International Joint Conference on Pervasive and Ubiquitous Computing and Proceedings of the 2019 ACM International Symposium on Wearable Computers - UbiComp/ISWC 2019, pp. 105–108. ACM Press, New York, September 2019.

  16. Kondo, T., Shiro, H., Yokokubo, A., Lopez, G.: Optimized classification model for efficient recognition of meal-related activities in daily life meal environment. In: 2019 Joint 8th International Conference on Informatics, Electronics & Vision (ICIEV) and 2019 3rd International Conference on Imaging, Vision & Pattern Recognition (icIVPR), pp. 146–151. IEEE, May 2019.,

  17. Kyritsis, K., Diou, C., Delopoulos, A.: A data driven end-to-end approach for in-the-wild monitoring of eating behavior using smartwatches. IEEE J. Biomed. Health Inform. 1 (2020).

  18. de Lara, J.R.C., et al.: A method of automatic speaker recognition using cepstral features and vectorial quantization. In: Sanfeliu, A., Cortés, M.L. (eds.) CIARP 2005. LNCS, vol. 3773, pp. 146–153. Springer, Heidelberg (2005).

    CrossRef  Google Scholar 

  19. Le, T.T., Fu, W., Moore, J.H.: Scaling tree-based automated machine learning to biomedical big data with a feature set selector. Bioinformatics 36(1), 250–256 (2020).

    CrossRef  Google Scholar 

  20. Lopez, G., Mitsui, H., Ohara, J., Yokokubo, A.: Effect of feedback medium for real-time mastication awareness increase using wearable sensors. In: Proceedings of the 12th International Joint Conference on Biomedical Engineering Systems and Technologies, pp. 442–449, no. Biostec. SCITEPRESS - Science and Technology Publications (2019).

  21. Mellina, A., Sentinelli, A., Marfia, G., Roccetti, M.: AREEB: automatic refrain extraction for thumbnail. In: 2012 IEEE Consumer Communications and Networking Conference (CCNC), no. Mi, pp. 472–476. IEEE, January 2012.

  22. Scheirer, E., Slaney, M.: Construction and evaluation of a robust multifeature speech/music discriminator. In: 1997 IEEE International Conference on Acoustics, Speech, and Signal Processing, vol. 2, pp. 1331–1334. IEEE Comput. Soc. Press (1801).

  23. Schoeller, D.A.: Limitations in the assessment of dietary energy intake by self-report. Metabolism 44(SUPPL. 2), 18–22 (1995).

    CrossRef  Google Scholar 

  24. Selamat, N.A., Ali, S.H.M.: Automatic food intake monitoring based on chewing activity: a survey. IEEE Access 8, 48846–48869 (2020).

    CrossRef  Google Scholar 

  25. Tada, A., Miura, H.: Association of mastication and factors affecting masticatory function with obesity in adults: a systematic review. BMC Oral Health 18(1), 76 (2018).

    CrossRef  Google Scholar 

  26. Vu, T., Lin, F., Alshurafa, N., Xu, W.: Wearable food intake monitoring technologies: a comprehensive review. Computers 6(1), 1–28 (2017).

    CrossRef  Google Scholar 

  27. Westerterp, K.R., Goris, A.H.: Validity of the assessment of dietary intake: problems of misreporting. Curr. Opinion Clin. Nutr. Metab. Care 5(5), 489–493 (2002).

    CrossRef  Google Scholar 

  28. Yeo, I.K.: A new family of power transformations to improve normality or symmetry. Biometrika 87(4), 954–959 (2000).

    MathSciNet  CrossRef  MATH  Google Scholar 

Download references

Author information

Authors and Affiliations


Corresponding author

Correspondence to Kizito Nkurikiyeyezu .

Editor information

Editors and Affiliations

Rights and permissions

Reprints and Permissions

Copyright information

© 2021 Springer Nature Switzerland AG

About this paper

Verify currency and authenticity via CrossMark

Cite this paper

Nkurikiyeyezu, K., Kamachi, H., Kondo, T., Jain, A., Yokokubo, A., Lopez, G. (2021). Classification of Eating Behaviors in Unconstrained Environments. In: , et al. Biomedical Engineering Systems and Technologies. BIOSTEC 2020. Communications in Computer and Information Science, vol 1400. Springer, Cham.

Download citation

  • DOI:

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-72378-1

  • Online ISBN: 978-3-030-72379-8

  • eBook Packages: Computer ScienceComputer Science (R0)