Abstract
Patient Performance Status (PS) is used in cancer medicine to predict prognosis and prescribe treatment. Today, PS assessments rely on assessor’s observation, which is susceptible to biases. A motion tracking system can be used to supplement PS assessments, by recording and analyzing patient’s movement as they perform a standardized mobility task e.g. getting up from office chair to sit on examination table. A temporal alignment of the extracted motion skeleton time series is then needed to enable comparison of corresponding motions in mobility task across recordings. In this paper, we apply existing state-of-the-art temporal alignment algorithms to the extracted time series and evaluate their performance in aligning the keyframes that separate corresponding motions. We then identify key characteristics of these time series that the existing algorithms are not able to exploit correctly: task left-right invariance and vertical-horizontal relative importance. We thus propose Invariant Weighted Dynamic Time Warping (IW-DTW), which takes advantage of these key characteristics. In an evaluation against state-of-the-art algorithms, IW-DTW outperforms them in aligning the keyframes where these key characteristics are present.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Abid, A., Zou, J.: Autowarp: learning a warping distance from unlabeled time series using sequence autoencoders. In: Advances in Neural Information Processing Systems (NeurIPS), October 2018
Broderick, J.E., May, M., Schwartz, J.E., Li, M., Mejia, A., Nocera, L., Kolatkar, A., Ueno, N.T., Yennu, S., Lee, J.S.H., Hanlon, S.E., Cozzens Philips, F.A., Shahabi, C., Kuhn, P., Nieva, J.: Patient reported outcomes can improve performance status assessment: a pilot study. J. Patient-Reported Outcomes 3(1), 41 (2019)
Hasnain, Z., Li, M., Dorff, T., Quinn, D., Ueno, N.T., Yennu, S., Kolatkar, A., Shahabi, C., Nocera, L., Nieva, J., Kuhn, P., Newton, P.K.: Low-dimensional dynamical characterization of human performance of cancer patients using motion data. Clinical Biomech. 56(December 2017), 61–69 (2018)
Kao, J.Y., Nguyen, M., Nocera, L., Shahabi, C., Ortega, A., Winstein, C., Sorkhoh, I., Chung, Y.C., Chen, Y.A., Bacon, H.: Validation of Automated Mobility Assessment Using a Single 3D Sensor. In: Hua, G., Jégou, H. (eds.) European Conference on Computer Vision (ECCV) Workshops, vol. 3, pp. 162–177. Springer, Cham (2016)
Nguyen, M.N.B., Hasnain, Z., Li, M., Dorff, T., Quinn, D., Purushotham, S., Nocera, L., Newton, P.K., Kuhn, P., Nieva, J., Shahabi, C.: Mining Human Mobility to Quantify Performance Status. In: IEEE International Conference on Data Mining (ICDM) Workshop (2017)
Ofli, F., Chaudhry, R., Kurillo, G.: Berkeley Multimodal Human Action Database (MHAD). In: IEEE Workshop on Applications of Computer Vision (WACV), pp. 53–60 (2013)
Oken, M.M., Creech, R.H., Tormey, D.C., Horton, J., Davis, T.E., McFadden, E.T., Carbone, P.P.: Toxicity and response criteria of the Eastern Cooperative Oncology Group. Am. J. Clin. Oncol. 5(6), 649–656 (1982)
Rabiner, L., Juang, B.H.: Fundamentals of Speech Recognition. Prentice-Hall Inc., Upper Saddle River (1993)
Schnadig, I.D., Fromme, E.K., Loprinzi, C.L., Sloan, J.A., Mori, M., Li, H., Beer, T.M.: Patient-physician disagreement regarding performance status is associated with worse survivorship in patients with advanced cancer. Cancer 113(8), 2205–2214 (2008)
Trigeorgis, G., Nicolaou, M.A., Zafeiriou, S., Schuller, B.W.: Deep canonical time warping. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 5110–5118 (2016)
Vu, H.T., Carey, C.J., Mahadevan, S.: Manifold warping: Manifold alignment over time. In: Proceedings of the National Conference on Artificial Intelligence, vol. 2, pp. 1155–1161 (2012)
Wang, R., Medioni, G., Winstein, C.J., Blanco, C.: Home monitoring musculo-skeletal disorders with a single 3D sensor. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops, pp. 521–528 (2013)
Zhou, F., De La Torre, F.: Generalized canonical time warping. IEEE Trans. Pattern Anal. Mach. Intell. 38(2), 279–294 (2016)
Zhou, F., de la Torre, F.: Canonical time warping for alignment of human behavior. In: Advances in Neural Information Processing Systems (NeurIPS), pp. 1–9 (2009)
Acknowledgements
This research has been funded in part by US National Cancer Institute under award number P30CA014089, USC Integrated Media Systems Center (IMSC), and unrestricted cash gifts from Oracle, Microsoft, and Google. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the authors and do not necessarily reflect the views of any of the sponsors. T. Nilanon was also supported in part by DPST, IPST, Thailand.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 The Editor(s) (if applicable) and The Author(s), under exclusive license to Springer Nature Switzerland AG
About this chapter
Cite this chapter
Nilanon, T., Nocera, L.P., Nieva, J.J., Shahabi, C. (2021). Towards Automated Performance Status Assessment: Temporal Alignment of Motion Skeleton Time Series. In: Shaban-Nejad, A., Michalowski, M., Buckeridge, D.L. (eds) Explainable AI in Healthcare and Medicine. Studies in Computational Intelligence, vol 914. Springer, Cham. https://doi.org/10.1007/978-3-030-53352-6_32
Download citation
DOI: https://doi.org/10.1007/978-3-030-53352-6_32
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-53351-9
Online ISBN: 978-3-030-53352-6
eBook Packages: Intelligent Technologies and RoboticsIntelligent Technologies and Robotics (R0)