Abstract
This paper introduces our methodology to estimate sidewalk accessibilities from wheelchair behavior via a triaxial accelerometer in a smartphone installed under a wheelchair seat. Our method recognizes sidewalk accessibilities from environmental factors, e.g. gradient, curbs, and gaps, which influence wheelchair bodies and become a burden for people with mobility difficulties. This paper developed and evaluated a prototype system that visualizes sidewalk accessibility information by extracting knowledge from wheelchair acceleration using deep neural networks. Firstly, we created a supervised convolutional neural network model to classify road surface conditions using wheelchair acceleration data. Secondly, we applied a weakly supervised method to extract representations of road surface conditions without manual annotations. Finally, we developed a self-supervised variational autoencoder to assess sidewalk barriers for wheelchair users. The results show that the proposed method estimates sidewalk accessibilities from wheelchair accelerations and extracts knowledge of accessibilities by weakly supervised and self-supervised approaches.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Laakso, M., Sarjakoski, T., Sarjakoski, L.T.: Improving accessibility information in pedestrian maps and databases. Cartogr. Int. J. Geogr. Inf. Geovisualization 46(2), 101–108 (2011). https://doi.org/10.3138/carto.46.2.101
Karimi, H.A., Zhang, L., Benner, J.G.: Personalized accessibility map (PAM): a novel assisted wayfinding approach for people with disabilities. Ann. GIS 20(2), 99–108 (2014). https://doi.org/10.1080/19475683.2014.904438
Ponsard, C., Snoeck, V.: Objective accessibility assessment of public infrastructures. In: Miesenberger, K., Klaus, J., Zagler, W.L., Karshmer, A.I. (eds.) ICCHP 2006. LNCS, vol. 4061, pp. 314–321. Springer, Heidelberg (2006). https://doi.org/10.1007/11788713_47
Hara, K.: Scalable methods to collect and visualize sidewalk accessibility data for people with mobility impairments. In: UIST, pp. 1–4 (2014). https://doi.org/10.1145/2658779.2661163
Swan, M.: The quantified self: fundamental disruption in big data science and biological discovery. Big Data 1(2), 85–99 (2013). https://doi.org/10.1089/big.2012.0002
Wang, J., Chen, Y., Hao, S., Peng, X., Hu, L.: Deep learning for sensor-based activity recognition: a survey. Pattern Recogn. Lett. 119, 3–11 (2019). https://doi.org/10.1016/j.patrec.2018.02.010
Plötz, T., Hammerla, N.Y., Olivier, P.L.: Feature learning for activity recognition in ubiquitous computing. In: IJCAI, pp. 1729–1734 (2011). https://doi.org/10.5591/978-1-57735-516-8/IJCAI11-290
Yang, J., Nguyen, M.N., San, P.P., Li, X., Krishnaswamy, S.: Deep convolutional neural networks on multichannel time series for human activity recognition. In: IJCAI, pp. 25–31 (2015)
Edel, M., Köppe, E.: Binarized-BLSTM-RNN based human activity recognition. In: IPIN, pp. 1–7 (2016). https://doi.org/10.1109/IPIN.2016.7743581
Yao, S., Hu, S., Zhao, Y., Zhang, A., Abdelzaher, T.: Deepsense: a unified deep learning framework for time-series mobile sensing data processing. In: WWW, pp. 351–360 (2017). https://doi.org/10.1145/3038912.3052577
Mora, H., Gilart-Iglesias, V., Pérez-del Hoyo, R., Andújar-Montoya, M.D.: A comprehensive system for monitoring urban accessibility in smart cities. Sensors 17(8), 1834 (2017). https://doi.org/10.3390/s17081834
Lara, O.D., Labrador, M.A.: A survey on human activity recognition using wearable sensors. IEEE Commun. Surv. Tutor. 15(3), 1192–1209 (2012). https://doi.org/10.1109/SURV.2012.110112.00192
Yairi, E.I., Igi, S.: Research on ubiquitous system for mobility support of the elderly and disabled and its technology transfer to industry. J. Inf. Process. 48, 770–779 (2007). (in Japanese)
Zimmermann-Janschitz, S.: Geographic information systems in the context of disabilities. J. Access. Des. All 8(2), 161–192 (2018). https://doi.org/10.17411/jacces.v8i2.171
Koga, M., Izumi, S., Matsubara, S., Morishita, K., Yoshioka, D.: Development and verification of navigation system to support wheelchair user activity in urban areas. IADIS Int. J. WWW/Internet 13(1), 43–56 (2015)
Allouch, A., Koubâa, A., Abbes, T., Ammar, A.: Roadsense: smartphone application to estimate road conditions using accelerometer and gyroscope. IEEE Sens. J. 17(13), 4231–4238 (2017). https://doi.org/10.1109/JSEN.2017.2702739
Mohan, P., Padmanabhan, V.N., Ramjee, R.: Nericell: rich monitoring of road and traffic conditions using mobile smartphones. In: SENSYS, pp. 323–336 (2008). https://doi.org/10.1145/1460412.1460444
Yu, J., et al.: Senspeed: sensing driving conditions to estimate vehicle speed in urban environments. IEEE Trans. Mob. Comput 15(1), 202–216 (2016). https://doi.org/10.1109/TMC.2015.2411270
Zhou, Z.-H.: A brief introduction to weakly supervised learning. Natl. Sci. Rev. 5(1), 44–53 (2018). https://doi.org/10.1093/nsr/nwx106
Längkvist, M., Karlsson, L., Loutfi, A.: A review of unsupervised feature learning and deep learning for time-series modeling. Pattern Recogn. Lett. 42, 11–24 (2014). https://doi.org/10.1016/j.patrec.2014.01.008
Sargano, A.B., Angelov, P., Habib, Z.: A comprehensive review on handcrafted and learning-based action representation approaches for human activity recognition. Appl. Sci. 7(1), 110 (2017). https://doi.org/10.3390/app7010110
Iwasawa, Y., Yairi, I.E., Matsuo, Y.: Combining human action sensing of wheelchair users and machine learning for autonomous accessibility data collection. IEICE Trans. Inf. Syst. 99(4), 115–124 (2016). https://doi.org/10.1587/transinf.2015EDP7278
Takahashi, H., Nagamine, K., Iwasawa, Y., Matsuo, Y., Yairi, I.E.: Quantification of road condition from wheelchair sensing data using deep convolutional neural network. IEICE Trans. Inf. Syst. 101(6), 2502–2505 (2018). https://doi.org/10.14923/transinfj.2017jdp7067. (in Japanese)
Cao, H., Nguyen, M.N., Phua, C., Krishnaswamy, S., Li, X.-L.: An integrated framework for human activity classification. In: UbiComp, pp. 331–340 (2012). https://doi.org/10.1145/2370216.2370334
Watanabe, T., Takahashi, H., Iwasawa, Y., Matsuo, Y., Yairi, I.E.: Weakly supervised learning for evaluating road surface condition from wheelchair driving data. Information 11(1), 2 (2020). https://doi.org/10.3390/info11010002
Ji, X., Henriques, J.F., Vedaldi, A.: Invariant information clustering for unsupervised image classification and segmentation. In: ICCV, pp. 9865–9874 (2019). https://doi.org/10.1109/ICCV.2019.00996
Acknowledgments
We would like to show our best gratitude to all participants in data collection. This research was supported by a Grant-in-Aid for Scientific Research (B), 17H01946 and 20H04476, Japan Society for the Promotion of Science.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Watanabe, T., Takahashi, H., Sato, G., Iwasawa, Y., Matsuo, Y., Yairi, I.E. (2021). Wheelchair Behavior Recognition for Visualizing Sidewalk Accessibility by Deep Neural Networks. In: Li, X., Wu, M., Chen, Z., Zhang, L. (eds) Deep Learning for Human Activity Recognition. DL-HAR 2021. Communications in Computer and Information Science, vol 1370. Springer, Singapore. https://doi.org/10.1007/978-981-16-0575-8_2
Download citation
DOI: https://doi.org/10.1007/978-981-16-0575-8_2
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-16-0574-1
Online ISBN: 978-981-16-0575-8
eBook Packages: Computer ScienceComputer Science (R0)