Abstract
Action recognition is a hot topic in computer vision and machine learning. The related applications are widely used in daily life. This paper conducts research on pedestrian fall detection methods from escalators based on TSN to recognize falling actions on the escalator and improve its performance. The recognition accuracy is closely related to the network architecture, the input data and parameter tuning. The popular input data of the network is mainly RGB images and optical flow and the classic network is two-stream and C3D. Instead of the traditional method, this paper uses limited samples and a feature fusion approach to find out falling pedestrians in class imbalance. The key and non-key frames are extracted from the video segments and sent to spatial networks and temporal ones relatively. With feature fusion and a series of optimization, the results showed that this method achieved higher accuracy, lower FAR and better performance effectively on both UCF101/HMDB51 datasets and pedestrian fall action recognition task compared with other popular action recognition methods.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Yao, G., Lei, T., Zhong, J.: A review of convolutional-neural-network-based action recognition. Pattern Recogn. Lett. 118, 14–22 (2019)
Ren, S., He, K., Girshick, R., et al.: Faster R-CNN: towards real-time object detection with region proposal networks. IEEE Trans. Pattern Anal. Mach. Intell. 39, 1137–1149 (2017)
Karpathy, A., Toderici, G., Shetty, S., et al.: Large-scale video classification with convolutional neural networks. In: Proceeding of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1725–1732 (2014)
Simonyan, K., Zisserman, A.: Two-stream convolutional networks for action recognition in videos. In: Proceeding of Advances in Neural Information Processing Systems, pp. 568–576 (2014)
Tran, D., Bourdev, L., Fergus, R., et al.: Learning spatiotemporal features with 3D convolutional networks. In: Proceeding of the IEEE International Conference on Computer Vision, pp. 4489–4497 (2015)
Wang, L., Xiong, Y., Wang, Z., et al.: Temporal segment networks: towards good practices for deep action recognition. In: Proceeding of European Conference on Computer Vision, pp. 20–36 (2016)
Zhou, B., Andonian, A., Torralba, A.: A Temporal relational reasoning in videos. In: Proceeding of European Conference on Computer Vision, pp. 831–846 (2018)
Xu, H., Das, A., Saenko, K.: R-C3D: region convolutional 3D network for temporal activity detection. In: Proceeding of the IEEE International Conference on Computer Vision, pp. 5794–5803 (2017)
Gers, F., Schmidhuber, J.: Recurrent nets that time and count. In: Proceeding of the IEEE International Conference on Neural Networks, pp. 189–194 (2000)
Wang, L., Qiao, Y., Tang, X.: MoFAP: a multi-level representation for action recognition. Int. J. Comput. Vis. 119, 254–271 (2016)
Samitha, H., Mehrtash, H., Fatih, P.: Going deeper into action recognition: a survey. Image Vis. Comput. 60, 4–21 (2017)
Wang., X., Farhadi, A., Gupta, A.: Actions transformations. In: Proceeding of IEEE Conference on Computer Vision and Pattern Recognition, pp. 2658–2667 (2016)
Wang, L., Qiao, Y., Tang, X.: Action recognition with trajectory-pooled deep-convolutional descriptors. In: Proceeding of IEEE Conference on Computer Vision and Pattern Recognition, pp. 4305–4314 (2015)
Qiu, Z., Yao, T., Mei, T.: Learning spatio-temporal representation with pseudo-3D residual networks. In: Proceeding of IEEE International Conference on Computer Vision, pp. 5534–5542 (2017)
Lan, Z., Zhu, Y., Hauptmann, A., et al.: Deep local video feature for action recognition. In: Proceeding of IEEE Conference on Computer Vision and Pattern Recognition Workshops, pp. 1219–1225 (2017)
Du, W., Wang, Y., Qiao, Y.: RPAN: an end-to-end recurrent pose-attention network for action recognition in videos. In: Proceeding of IEEE Conference on Computer Vision and Pattern Recognition, pp. 3745–3754. IEEE (2017)
Long, X., Gan, C., Gerard, D.: Multimodal keyless attention fusion for video classification. In: Proceeding of Association for the Advancement of Artificial Intelligence, pp. 7203–7209 (2018)
Zhu, X., Xiong, Y., Dai, J., et al.: Deep feature flow for video recognition. In: Proceeding of Conference on Computer Vision and Pattern Recognition, pp. 4141–4150 (2017)
He, K., Zhang, X., Ren, S., et al.: Deep residual learning for image recognition. In: Proceeding of Conference on Computer Vision and Pattern Recognition, pp. 770–778 (2016)
Simonyan, K., Zisserman, A., et al.: Going deep convolutional networks for large-scale image recognitions. In: Proceeding of Conference on Computer Vision and Pattern Recognition, pp. 1–9 (2015)
Zhao, Y., Xiong, Y., Wang, L., et al.: Temporal Action detection with structured segment networks. Int. J. Comput. Vis. 128, 74–85 (2020)
Bai, X., Chen, E., Tinega, H.: Real-time action recognition based on enhanced motion vector temporal segment network. In: Proceeding of Eleventh International Conference on Digital Image Processing, pp. 391–401 (2019)
Zhang, B., Wang, L., Wang, Z., et al.: Real-time action recognition with enhanced motion vector CNNs. In: Proceeding of Conference on Computer Vision and Pattern Recognition, pp. 141–145 (2016)
Video augmentation techniques for deep learning. https://github.com/okankop/vidaug
Compressed video action recognition. https://github.com/chaoyuaw/pytorch-coviar
Acknowledgements
This work is sponsored by the National Key Research and Development Program of China (Project No. 2018YFC0809704).
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Wu, Y., Wu, S., Yan, Z. (2021). Research on Pedestrian Fall Action Recognition from Escalators. In: Sugumaran, V., Xu, Z., Zhou, H. (eds) Application of Intelligent Systems in Multi-modal Information Analytics. MMIA 2021. Advances in Intelligent Systems and Computing, vol 1385. Springer, Cham. https://doi.org/10.1007/978-3-030-74814-2_40
Download citation
DOI: https://doi.org/10.1007/978-3-030-74814-2_40
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-74813-5
Online ISBN: 978-3-030-74814-2
eBook Packages: Intelligent Technologies and RoboticsIntelligent Technologies and Robotics (R0)