Personal and Ubiquitous Computing

, Volume 16, Issue 6, pp 729–739 | Cite as

Exploiting visual quasi-periodicity for real-time chewing event detection using active appearance models and support vector machines

  • Steven Cadavid
  • Mohamed Abdel-Mottaleb
  • Abdelsalam Helal
Original Article


Steady increases in healthcare costs and obesity have inspired recent studies into cost-effective, assistive systems capable of monitoring dietary habits. Few researchers, though, have investigated the use of video as a means of monitoring dietary activities. Video possesses several inherent qualities, such as passive acquisition, that merits its analysis as an input modality for such an application. To this end, we propose a method to automatically detect chewing events in surveillance video of a subject. Firstly, an Active Appearance Model (AAM) is used to track a subject’s face across the video sequence. It is observed that the variations in the AAM parameters across chewing events demonstrate a distinct periodicity. We utilize this property to discriminate between chewing and non-chewing facial actions such as talking. A feature representation is constructed by applying spectral analysis to a temporal window of model parameter values. The estimated power spectra subsequently undergo non-linear dimensionality reduction. The low-dimensional embedding of the power spectra are employed to train a binary Support Vector Machine classifier to detect chewing events. To emulate the gradual onset and offset of chewing, smoothness is imposed over the class predictions of neighboring video frames in order to deter abrupt changes in the class labels. Experiments are conducted on a dataset consisting of 37 subjects performing each of five actions, namely, open- and closed-mouth chewing, clutter faces, talking, and still face. Experimental results yielded a cross-validated percentage agreement of 93.0%, indicating that the proposed system provides an efficient approach to automated chewing detection.


Dietary monitoring Behavior detection Active appearance models Manifold learning Support vector machines 



This research was supported in part by a National Institute of Health (NIH) grant number 5R21DA024294.


  1. 1.
    Amft O, Tröster G (2008) Recognition of dietary activity events using on-body sensors. Artif Intell Med 42(2):121–136CrossRefGoogle Scholar
  2. 2.
    Cai D, He X, Han J (2005) Document clustering using locality preserving indexing. IEEE Trans Knowl Data Eng 17(12):1624–1637CrossRefGoogle Scholar
  3. 3.
    Cai D, He X, Han J, Zhang H (2006) Orthogonal laplacianfaces for face recognition. IEEE Trans Image Process 15(11):3608–3614CrossRefGoogle Scholar
  4. 4.
    Cai D, He X, Zhang WV, Han J (2007) Regularized locality preserving indexing via spectral regression. In: Proceeding of the ACM conference on conference on information and knowledge management, pp 741–750Google Scholar
  5. 5.
    Cootes TF, Edwards GJ, Taylor CJ (1998) Active appearance models. In: Proceedings of the European conference on computer vision 2, pp 484–498Google Scholar
  6. 6.
    Guo G, Fu Y, Dyer CR, Huang TS (2008) Image-based human age estimation by manifold learning and locally adjusted robust regression. IEEE Trans Image Process 17(7):1178–1188MathSciNetCrossRefGoogle Scholar
  7. 7.
    Hayes MH (1996) Statistical digital signal processing and modeling. Wiley, New YorkGoogle Scholar
  8. 8.
    He X, Yan S, Hu Y, Niyogi P, Zhang H (2005) Face recognition using laplacianfaces. IEEE Trans Pattern Anal Mach Intell 27(3):328–340CrossRefGoogle Scholar
  9. 9.
    Mahoor MH, Cadavid S, Messinger DS, Cohn JF (2009) A framework for automated measurement of the intensity of non-posed facial action units. In: CVPR workshop on human communicative behavior analysis (CVPR4HB)Google Scholar
  10. 10.
    Nishimura J, Kuroda T (2008) Eating habits monitoring using wireless wearable in-ear microphone. In: Third international symposium on wireless pervasive computing. ISWPC 2008, Santorini, pp 130–132. doi: 10.1109/ISWPC.2008.4556181. ISBN 978-1-4244-1652-3
  11. 11.
    Niyogi P (2003) Laplacian eigenmaps for dimensionality reduction and data representation. Neural Comput Appl 15:1373–1396zbMATHCrossRefGoogle Scholar
  12. 12.
    Pogalin E, Smeulders A, Thean A (2008) Visual quasi-periodicity. In: Proceedings of the conference on computer vision and pattern recognition, pp 1–8Google Scholar
  13. 13.
    Reis HT, Judd CM (2000) Handbook of research methods in social and personality psychology. Cambridge University Press, CambridgeGoogle Scholar
  14. 14.
    Roweis S, Saul L (2000) Nonlinear dimensionality reduction by locally linear embedding. Sci Agric 290(5500):2323–2326CrossRefGoogle Scholar
  15. 15.
    Sazonov E, Schuckers S, Lopez-Meyer P, Makeyev O, Sazonova N, Melanson EL, Neuman M (2008) Non-invasive monitoring of chewing and swallowing for objective quantification of ingestive behavior. Physiol Meas 29(5):525CrossRefGoogle Scholar
  16. 16.
    Schmalz M, Mendez-Vazquez A, Helal A (2009) Algorithms for the detection of chewing behavior in dietary monitoring applications. In: Proceedings of SPIE technical symposium: mathematics of data/image coding, compression, and encryption with applications XII, vol 7444AGoogle Scholar
  17. 17.
    Tanigawa S, Nishihara H, Kaneda S, Haga H (2008) Detecting mastication by using microwave doppler sensor. In: PETRA ’08: Proceedings of the 1st international conference on PErvasive Technologies Related to Assistive Environments, New York, pp 1–7Google Scholar
  18. 18.
    Tenenbaum J, de Silva V, Langford J (2000) A global geometric framework for nonlinear dimensionality reduction. Sci Agric 290(5500):2319–2323CrossRefGoogle Scholar
  19. 19.
    Tolstikov A, Biswas J, Tham CK, Yap P (2008) Eating activity primitives detection—a step towards adl recognition. In: e-health networking, applications and services, 2008. HealthCom 2008. 10th International Conference on, pp 35–41Google Scholar
  20. 20.
    Turk M, Pentland A (1991) Eigenfaces for recognition. J Cogn Neurosci 3(1):71–86Google Scholar
  21. 21.
    Vapnik VN (1995) The nature of statistical learning theory. Springer, BerlinzbMATHGoogle Scholar
  22. 22.
    Viola P, Jones MJ (2004) Robust real-time face detection. Int J Comput Vis 57(2):137–154CrossRefGoogle Scholar
  23. 23.
    Wang Y, Lucey S, Cohn J (2008) Enforcing convexity for improved alignment with constrained local models. In: IEEE international conference on computer vision and pattern recognitionGoogle Scholar
  24. 24.
    Zhang S, Ang MH, Xiao W, Tham CK (2009) Detection of activities by wireless sensors for daily life surveillance: eating and drinking. Sensors 9(3):1499–1517CrossRefGoogle Scholar

Copyright information

© Springer-Verlag London Limited 2011

Authors and Affiliations

  • Steven Cadavid
    • 1
  • Mohamed Abdel-Mottaleb
    • 1
  • Abdelsalam Helal
    • 2
  1. 1.Department of Electrical and Computer EngineeringUniversity of MiamiCoral GablesUSA
  2. 2.Department of Computer and Information Science and EngineeringUniversity of FloridaGainesvilleUSA

Personalised recommendations