Image and video processing techniques are being frequently used in medical science applications. Computer vision-based systems have successfully replaced various manual medical processes such as analyzing physical and biomechanical parameters, physical examination of patients. These systems are gaining popularity because of their robustness and the objectivity they bring to various medical procedures. Hammersmith Infant Neurological Examinations (HINE) is a set of physical tests that are carried out on infants in the age group of 3–24 months with neurological disorders. However, these tests are graded through visual observations, which can be highly subjective. Therefore, computer vision-aided approach can be used to assist the experts in the grading process. In this paper, we present a method of automatic exercise classification through visual analysis of the HINE videos recorded at hospitals. We have used scale-invariant-feature-transform features to generate a bag-of-words from the image frames of the video sequences. Frequency of these visual words is then used to classify the video sequences using HMM. We also present a method of event segmentation in long videos containing more than two exercises. Event segmentation coupled with a classifier can help in automatic indexing of long and continuous video sequences of the HINE set. Our proposed framework is a step forward in the process of automation of HINE tests through computer vision-based methods. We conducted tests on a dataset comprising of 70 HINE video sequences. It has been found that the proposed method can successfully classify exercises with accuracy as high as 84%. The proposed work has direct applications in automatic or semiautomatic analysis of “vertical suspension” and “ventral suspension” tests of HINE. Though some of the critical tests such as “pulled-to-sit,” “lateral tilting,” or “adductor’s angle measurement” have already been addressed using image- and video-guided techniques, scopes are there for further improvement.
This is a preview of subscription content, access via your institution.
Buy single article
Instant access to the full article PDF.
Tax calculation will be finalised during checkout.
Subscribe to journal
Immediate online access to all issues from 2019. Subscription will auto renew annually.
Tax calculation will be finalised during checkout.
Zhang, R., Vogler, C., Metaxas, D.: Human gait recognition at sagittal plane. Image Vis. Comput. 25(3), 321–330 (2007)
Liu, Q., Sclabassi, R., Sun, M.: Change detection in epilepsy monitoring video based on Markov random field theory. In: Proceedings of International Symposium on Intelligent Signal Processing and Communication Systems, pp. 63–66 (2004)
Viola, P., Jones., M.J., Snow, D.: Detecting pedestrians using patterns of motion and appearance. In: Ninth IEEE International Conference on Computer Vision, pp. 734–741 (2003)
Singh, S., Hsiao, H.: Infant telemonitoring system. In: Proceedings of the 25th Annual International Conference of the IEEE on Engineering in Medicine and Biology Society, vol. 2, pp. 1354–1357 (2003)
Nishida, Y., Motomura, Y., Kitamura, K., Mizoguchi, H.: Infant behavior simulation based on an environmental model and a developmental behavior model. In: Proceedings of IEEE International Conference on Systems, Man and Cybernetics, vol. 2, pp. 1555–1560 (2004)
Dubowitz, L., Dubowitz, V., Mercuri, E.: The Neurological Assessment of the Preterm and Full Term Infant, Clinics in Developmental Medicine. Heinemann, London (2000)
Romeo, D., Guzzetta, A., Scoto, M., Cioni, M., Patusi, P., Mazzone, D., Romeo, M.: Early neurologic assessment in preterm infants: integration of traditional neurologic examination and observation of general movements. Eur. J. Paediatr. Neurol. 12(3), 183–189 (2008)
Dogra, D., Majumdar, A., Sural, S., Mukherjee, J., Mukherjee, S., Singh, A.: Analysis of adductors angle measurement in hammersmith infant neurological examinations using mean shift segmentation and feature point based object tracking. Comput. Biol. Med. 42(9), 925–934 (2012)
Dogra, D.P., Badri, V., Majumdar, A.K., Sural, S., Mukherjee, J., Mukherjee, S., Singh, A.: Video analysis of Hammersmith lateral tilting examination using Kalman filter guided multi-path tracking. Med. Biol. Eng. Comput. 52(9), 759–772 (2014)
Dogra, D.P., Majumdar, A.K., Sural, S., Mukherjee, J., Mukherjee, S., Singh, A.: Automatic adductors angle measurement for neurological assessment of post-neonatal infants during follow up. Pattern Recognit. Mach. Intell. Lect. Notes Comput. Sci. 6744, 160–166 (2011)
Dogra, D.P., Majumdar, A.K., Sural, S., Mukherjee, J., Mukherjee, S., Singh, A.: Toward automating Hammersmith pulled-to-sit examination of infants using feature point based video object tracking. IEEE Trans. Neural Syst. Rehabil. Eng. 20(1), 38–47 (2012)
Romeo, D., Cioni, M., Scoto, M., Mazzone, L., Palermo, F., Romeo, M.: Neuromotor development in infants with cerebral palsy investigated by the Hammersmith Infant Neurological Examination during the first year of age. Eur. J. Paediatr. Neurol. 12(1), 24–31 (2008)
Dey, P., Dogra, D.P., Roy, P.P., Bhaskar, H.: Autonomous vision-guided approach for the analysis and grading of vertical suspension tests during HINE. In: Proceedings of the 38th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), pp. 863–866 (2016)
Roy, S., Dogra, D.P., Bhattacharya, S., Saha, B., Biswas, A., Majumdar, A.K., Mukhopadhyay, J., Majumdar, B., Singh, A., Paria, A., Mukherjee, S.: A web enabled health information system for Neonatal Intensive Care Unit (NICU). In: Proceedings of the 7th IEEE World Congress on Services (SERVICES), pp. 451–458 (2011)
Dogra, D.P., Nandam, K., Majumdar, A.K., Sural, S., Mukhopadhyay, J., Majumdar, B., Mukherjee, S., Singh, A.: A tool for automatic Hammersmith infant neurological examination. E-Health Med. Commun. 2(2), 1–13 (2011)
Ansari, A.F., Roy, P.P., Dogra, D.P.: Posture recognition in HINE exercises. In: Proceedings of International Conference on Computer Vision and Image Processing (CVIP) (2016)
Poleg, Y., Arora, C., Peleg, S.: Temporal segmentation of egocentric videos. In: CVPR, pp. 2537–2544 (2014)
Ballan, L., Bertini, M., Bimbo, A.D., Serra, G.: Action categorization in soccer videos using string kernels. In: Seventh International Workshop on Content-Based Multimedia Indexing, pp. 13–18 (2009)
Lowe, D.G.: Distinctive image features from scale-invariant keypoints. Int. J. Comput. Vis. 60(2), 91–110 (2004)
Luhn, H.P.: A statistical approach to mechanized encoding and searching of literary information. IBM J. Res. Dev. 1(4), 315 (1957)
Jones, K.S.: A statistical interpretation of term specificity and its application in retrieval. J. Doc. 28, 11–21 (1972)
Chai, J., Ngan, K.: Face segmentation using skin-color map in videophone applications. IEEE Trans. Circuits Syst. Video Technol. 9(4), 551–564 (1999)
Hu, J., Lim, S.G., Brown, M.K.: Writer independent on-line handwriting recognition using an HMM approach. Pattern Recogn. 33(1), 133–147 (2000)
Huang, X.D., Arikiand, Y., Jack, M.A.: Hidden Markov Models for Speech Recognition. Edinburgh University Press, Edinburgh (1990)
Yang, J., Xu, Y.: Hidden Markov model for gesture recognition. Technical Report, Robotics Institute, Carnegie Mellon University, 10 (1994)
Barron, J.L., Fleet, D.J., Beauchemin, S.S.: Performance of optical flow techniques. Int. J. Comput. Vision. 12(1), 43–77 (1994)
Baccouche, M., Mamalet, F., Wolf, C., Garcia, C., Baskurt, A.: Action classification in soccer videos with long short-term memory recurrent neural networks. In: 20th International Conference on Artificial Neural Networks, pp. 154–159 (2010)
Graves, A., Schmidhuber, J.: Framewise phoneme classification with bidirectional LSTM and other neural network architectures. Neural Networks. 18(5), 602–610 (2005)
Guo, Z., Hall, R.W.: Parallel thinning with two-subiteration algorithms. Commun. ACM 32(3), 359–373 (1989)
About this article
Cite this article
Ansari, A.F., Roy, P.P. & Dogra, D.P. Exercise classification and event segmentation in Hammersmith Infant Neurological Examination videos. Machine Vision and Applications 29, 233–245 (2018). https://doi.org/10.1007/s00138-017-0896-5
- HINE tests
- Exercise classification
- Video segmentation
- Event segmentation