Steady increases in healthcare costs and obesity have inspired recent studies into cost-effective, assistive systems capable of monitoring dietary habits. Few researchers, though, have investigated the use of video as a means of monitoring dietary activities. Video possesses several inherent qualities, such as passive acquisition, that merits its analysis as an input modality for such an application. To this end, we propose a method to automatically detect chewing events in surveillance video of a subject. Firstly, an Active Appearance Model (AAM) is used to track a subject’s face across the video sequence. It is observed that the variations in the AAM parameters across chewing events demonstrate a distinct periodicity. We utilize this property to discriminate between chewing and non-chewing facial actions such as talking. A feature representation is constructed by applying spectral analysis to a temporal window of model parameter values. The estimated power spectra subsequently undergo non-linear dimensionality reduction. The low-dimensional embedding of the power spectra are employed to train a binary Support Vector Machine classifier to detect chewing events. To emulate the gradual onset and offset of chewing, smoothness is imposed over the class predictions of neighboring video frames in order to deter abrupt changes in the class labels. Experiments are conducted on a dataset consisting of 37 subjects performing each of five actions, namely, open- and closed-mouth chewing, clutter faces, talking, and still face. Experimental results yielded a cross-validated percentage agreement of 93.0%, indicating that the proposed system provides an efficient approach to automated chewing detection.
This is a preview of subscription content, log in to check access.
Buy single article
Instant access to the full article PDF.
Price includes VAT for USA
Subscribe to journal
Immediate online access to all issues from 2019. Subscription will auto renew annually.
This is the net price. Taxes to be calculated in checkout.
Amft O, Tröster G (2008) Recognition of dietary activity events using on-body sensors. Artif Intell Med 42(2):121–136
Cai D, He X, Han J (2005) Document clustering using locality preserving indexing. IEEE Trans Knowl Data Eng 17(12):1624–1637
Cai D, He X, Han J, Zhang H (2006) Orthogonal laplacianfaces for face recognition. IEEE Trans Image Process 15(11):3608–3614
Cai D, He X, Zhang WV, Han J (2007) Regularized locality preserving indexing via spectral regression. In: Proceeding of the ACM conference on conference on information and knowledge management, pp 741–750
Cootes TF, Edwards GJ, Taylor CJ (1998) Active appearance models. In: Proceedings of the European conference on computer vision 2, pp 484–498
Guo G, Fu Y, Dyer CR, Huang TS (2008) Image-based human age estimation by manifold learning and locally adjusted robust regression. IEEE Trans Image Process 17(7):1178–1188
Hayes MH (1996) Statistical digital signal processing and modeling. Wiley, New York
He X, Yan S, Hu Y, Niyogi P, Zhang H (2005) Face recognition using laplacianfaces. IEEE Trans Pattern Anal Mach Intell 27(3):328–340
Mahoor MH, Cadavid S, Messinger DS, Cohn JF (2009) A framework for automated measurement of the intensity of non-posed facial action units. In: CVPR workshop on human communicative behavior analysis (CVPR4HB)
Nishimura J, Kuroda T (2008) Eating habits monitoring using wireless wearable in-ear microphone. In: Third international symposium on wireless pervasive computing. ISWPC 2008, Santorini, pp 130–132. doi:10.1109/ISWPC.2008.4556181. ISBN 978-1-4244-1652-3
Niyogi P (2003) Laplacian eigenmaps for dimensionality reduction and data representation. Neural Comput Appl 15:1373–1396
Pogalin E, Smeulders A, Thean A (2008) Visual quasi-periodicity. In: Proceedings of the conference on computer vision and pattern recognition, pp 1–8
Reis HT, Judd CM (2000) Handbook of research methods in social and personality psychology. Cambridge University Press, Cambridge
Roweis S, Saul L (2000) Nonlinear dimensionality reduction by locally linear embedding. Sci Agric 290(5500):2323–2326
Sazonov E, Schuckers S, Lopez-Meyer P, Makeyev O, Sazonova N, Melanson EL, Neuman M (2008) Non-invasive monitoring of chewing and swallowing for objective quantification of ingestive behavior. Physiol Meas 29(5):525
Schmalz M, Mendez-Vazquez A, Helal A (2009) Algorithms for the detection of chewing behavior in dietary monitoring applications. In: Proceedings of SPIE technical symposium: mathematics of data/image coding, compression, and encryption with applications XII, vol 7444A
Tanigawa S, Nishihara H, Kaneda S, Haga H (2008) Detecting mastication by using microwave doppler sensor. In: PETRA ’08: Proceedings of the 1st international conference on PErvasive Technologies Related to Assistive Environments, New York, pp 1–7
Tenenbaum J, de Silva V, Langford J (2000) A global geometric framework for nonlinear dimensionality reduction. Sci Agric 290(5500):2319–2323
Tolstikov A, Biswas J, Tham CK, Yap P (2008) Eating activity primitives detection—a step towards adl recognition. In: e-health networking, applications and services, 2008. HealthCom 2008. 10th International Conference on, pp 35–41
Turk M, Pentland A (1991) Eigenfaces for recognition. J Cogn Neurosci 3(1):71–86
Vapnik VN (1995) The nature of statistical learning theory. Springer, Berlin
Viola P, Jones MJ (2004) Robust real-time face detection. Int J Comput Vis 57(2):137–154
Wang Y, Lucey S, Cohn J (2008) Enforcing convexity for improved alignment with constrained local models. In: IEEE international conference on computer vision and pattern recognition
Zhang S, Ang MH, Xiao W, Tham CK (2009) Detection of activities by wireless sensors for daily life surveillance: eating and drinking. Sensors 9(3):1499–1517
This research was supported in part by a National Institute of Health (NIH) grant number 5R21DA024294.
About this article
Cite this article
Cadavid, S., Abdel-Mottaleb, M. & Helal, A. Exploiting visual quasi-periodicity for real-time chewing event detection using active appearance models and support vector machines. Pers Ubiquit Comput 16, 729–739 (2012). https://doi.org/10.1007/s00779-011-0425-x
- Dietary monitoring
- Behavior detection
- Active appearance models
- Manifold learning
- Support vector machines