Skip to main content

Sensor-Based Transportation Mode Recognition Using Variational Autoencoder

Journal of Big Data Analytics in Transportation volume 3, pages 15–26 (2021)Cite this article

Abstract

We present a data augmentation technique that can improve the classification of transportation modes when the training data is insufficient. The proposed method uses a variational autoencoder (VAE) based synthetic data generation algorithm for smartphone data. Often the data collected by individuals for research is limited due to practical constraints. The algorithm discussed would aid in generating similar data from a handful of collected data to give a substantial dataset for any machine learning models. We propose a VAE, the decoder of which can help generate this synthetic data. We show that the synthetic data closely follows the pattern of the real data. We also show that classification accuracy is improved with the use of this type of data. Our method would also be a useful tool to boost the samples of an underrepresented class in a dataset. The detection of activity recognition using smartphone sensors could be applied to multiple aspects of vehicle to pedestrian P2V systems and smart mobility.

This is a preview of subscription content, access via your institution.

Access options

Buy single article

Instant access to the full article PDF.

USD 39.95

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

References

  • Alzantot M, Chakraborty S, Srivastava M (2017) Sensegen: a deep learning architecture for synthetic sensor data generation. IEEE Int Conf Pervasive Comput Commun Workshops (PerCom Workshops) 2017:188–193

    Article  Google Scholar 

  • An J, Cho S (2015) Variational autoencoder based anomaly detection using reconstruction probability. Spec Lect IE 2(1):1–18

    Google Scholar 

  • Bhattacharya S, Lane ND (2016) From smart to deep: Robust activity recognition on smartwatches using deep learning. IEEE Int Conf Pervasive Comput Commun Workshops (PerCom Workshops) 2016:1–6

    Google Scholar 

  • Bulling A, Blanke U, Schiele B (2014) A tutorial on human activity recognition using body-worn inertial sensors. ACM Comput Surv 46(3):1–33. https://doi.org/10.1145/2499621

    Article  Google Scholar 

  • Chavarriaga R, Sagha H, Calatroni A, Digumarti ST, Tröster G, Millán JDR, Roggen D (2013) The opportunity challenge: a benchmark database for on-body sensor-based activity recognition. Pattern Recogn Lett 34(15):2033–2042. https://doi.org/10.1016/j.patrec.2012.12.014

    Article  Google Scholar 

  • Chawla NV, Bowyer KW, Hall LO, Kegelmeyer WP (2002) SMOTE: synthetic minority over-sampling technique. J Artif Intell Res 16:321–357

    Article  Google Scholar 

  • Chen C, Jafari R, Kehtarnavaz N (2015) UTD-MHAD: a multimodal dataset for human action recognition utilizing a depth camera and a wearable inertial sensor. IEEE Int Conf Image Process (ICIP) 2015:168–172

    Google Scholar 

  • Cichy RM, Khosla A, Pantazis D, Torralba A, Oliva A (2016) Deep neural networks predict hierarchical spatio-temporal cortical dynamics of human visual object recognition. ArXiv Preprint arXiv: 1601.02970

  • DeVries T, Taylor GW (2017) Dataset augmentation in feature space. ArXiv Preprint arXiv: 1702.05538

  • Foerster F, Smeja M, Fahrenberg J (1999) Detection of posture and motion by accelerometry: a validation study in ambulatory monitoring. Comput Hum Behav 15(5):571–583

    Article  Google Scholar 

  • Forestier G, Petitjean F, Dau HA, Webb GI, Keogh E (2017) Generating synthetic time series to augment sparse datasets. IEEE Int Conf Data Min (ICDM) 2017:865–870

    Google Scholar 

  • Frid-Adar M, Diamant I, Klang E, Amitai M, Goldberger J, Greenspan H (2018) GAN-based synthetic medical image augmentation for increased CNN performance in liver lesion classification. Neurocomputing 321:321–331

    Article  Google Scholar 

  • Gjoreski H, Ciliberto M, Wang L, Morales FJO, Mekki S, Valentin S, Roggen D (2018) The university of Sussex-Huawei locomotion and transportation dataset for multimodal analytics with mobile devices. IEEE Access 6:42592–42604

    Article  Google Scholar 

  • Gu F, Khoshelham K, Valaee S (2017) Locomotion activity recognition: a deep learning approach. In: 2017 IEEE 28th annual international symposium on personal, indoor, and mobile radio communications (PIMRC), pp 1–5

  • Gu F, Khoshelham K, Valaee S, Shang J, Zhang R (2018) Locomotion activity recognition using stacked denoising autoencoders. IEEE Int Things J 5(3):2085–2093. https://doi.org/10.1111/acer.13391

    Article  Google Scholar 

  • Hassan MM, Uddin MZ, Mohamed A, Almogren A (2018) A robust human activity recognition system using smartphone sensors and deep learning. Future Gen Comput Syst 81:307–313. https://doi.org/10.1016/j.future.2017.11.029

    Article  Google Scholar 

  • Islam Z, Abdel-Aty M, Cai Q, Yuan J (2020) Crash data augmentation using variational autoencoder. Accid Anal Prev 151:105950

    Article  Google Scholar 

  • Javier Ordóñez F, de Toledo P, Sanchis A (2013) Activity recognition using hybrid generative/discriminative models on home environments using binary sensors. Sensors (Switzerland) 13(5):5460–5477. https://doi.org/10.3390/s130505460

    Article  Google Scholar 

  • Kakihara M (2014) Grasping a global view of smartphone diffusion: an analysis from a global smartphone study. ICMB 11

  • Kingma DP, Welling M (2014) Auto-encoding variational bayes. In: 2nd international conference on learning representations, ICLR 2014—conference track proceedings, Ml, pp 1–14

  • Kusner MJ, Paige B, Hernández-Lobato JM (2017) Grammar variational autoencoder. In: Proceedings of the 34th international conference on machine learning, vol 70, pp 1945–1954

  • Lara OD, Labrador MA (2012) A survey on human activity recognition using wearable sensors. IEEE Commun Surv Tutor 15(3):1192–1209

    Article  Google Scholar 

  • Lau SL, König I, David K, Parandian B, Carius-Düssel C, Schultz M (2010) Supporting patient monitoring using activity recognition with a smartphone. In: 2010 7th international symposium on wireless communication systems, pp 810–814

  • Le Guennec A, Malinowski S, Tavenard R, Cui Z, Chen W, Chen Y (2016) Data augmentation for time series classification using convolutional neural networks. ArXiv Preprint arXiv: 1603.06995

  • Lin F, Song C, Xu X, Cavuoto L, Xu W (2016) Sensing from the bottom: Smart insole enabled patient handling activity recognition through manifold learning. In: 2016 IEEE first international conference on connected health: applications, systems and engineering technologies (CHASE), pp 254–263

  • Madabhushi A, Aggarwal JK (1999) A bayesian approach to human activity recognition. In: Proceedings second IEEE workshop on visual surveillance (VS’99) (Cat. No. 98-89223), pp 25–32

  • Nweke HF, Teh YW, Al-Garadi MA, Alo UR (2018) Deep learning algorithms for human activity recognition using mobile and wearable sensor networks: state of the art and research challenges. Expert Syst Appl 105:233–261

    Article  Google Scholar 

  • Pedregosa F, Varoquaux G, Gramfort A, Michel V, Thirion B, Grisel O, Blondel M, Prettenhofer P, Weiss R, Dubourg V et al (2011) Scikit-learn: machine learning in python. J Mach Learn Res 12:2825–2830

    MathSciNet  MATH  Google Scholar 

  • Perez L, Wang J (2017) The effectiveness of data augmentation in image classification using deep learning. ArXiv Preprint arXiv:1712.04621

  • Pu Y, Gan Z, Henao R, Yuan X, Li C, Stevens A, Carin L (2016) Variational autoencoder for deep learning of images, labels and captions. arXiv preprint. arXiv:1609.08976

  • Sønderby CK, Raiko T, Maaløe L, Sønderby SK, Winther O (2016) Ladder variational autoencoders. arXiv preprint. arXiv:1602.02282

  • Song B, Kamal AT, Soto C, Ding C, Farrell JA, Roy-Chowdhury AK (2010) Tracking and activity recognition through consensus in distributed camera networks. IEEE Trans Image Process 19(10):2564–2579

    MathSciNet  Article  Google Scholar 

  • Steven Eyobu O, Han DS (2018) Feature representation and data augmentation for human activity classification based on wearable IMU sensor data using a deep LSTM neural network. Sensors 18(9):2892

    Article  Google Scholar 

  • Um TT, Pfister FMJ, Pichler D, Endo S, Lang M, Hirche S, Fietzek U, Kulić D (2017) Data augmentation of wearable sensor data for parkinson’s disease monitoring using convolutional neural networks. In: Proceedings of the 19th ACM international conference on multimodal interaction, pp 216–220

  • Vavoulas G, Chatzaki C, Malliotakis T, Pediaditis M, Tsiknakis M (2016) The MobiAct Dataset: recognition of activities of daily living using smartphones. ICT4AgeingWell 143–151

  • Wang J, Chen Y, Gu Y, Xiao Y, Pan H (2018) SensoryGANs: an effective generative adversarial framework for sensor-based human activity recognition. In: Proceedings of the international joint conference on neural networks, 2018-July, pp 1–8. https://doi.org/10.1109/IJCNN.2018.8489106

  • Wang J, Chen Y, Hao S, Peng X, Hu L (2019a) Deep learning for sensor-based activity recognition: a survey. Pattern Recogn Lett 119:3–11

    Article  Google Scholar 

  • Wang L, Gjoreski H, Ciliberto M, Mekki S, Valentin S, Roggen D (2019b) Enabling reproducible research in sensor-based transportation mode recognition with the Sussex-Huawei dataset. IEEE Access 7:10870–10891

    Article  Google Scholar 

  • Wong SC, Gatt A, Stamatescu V, McDonnell MD (2016) Understanding data augmentation for classification: when to warp? Int Conf Digit Image Comput Tech Appl (DICTA) 2016:1–6

    Google Scholar 

  • Xu W, Sun H, Deng C, Tan Y (2017) Variational autoencoder for semi-supervised text classification. In: Thirty-first AAAI conference on artificial intelligence

  • Yin J, Yang Q, Member S, Pan JJ (n.d.) Sensor-based abnormal human-activity detection. IEEE Transactions on Knowledge and Data Engineering, 20(8):1082–1090

  • Zhang M, Sawchuk AA (2011) A feature selection-based framework for human activity recognition using wearable multimodal sensors. BodyNets 92–98

Download references

Author information

Authors and Affiliations

  1. Department of Civil, Environmental and Construction Engineering, University of Central Florida, Orlando, FL, 32816, USA

    Zubayer Islam & Mohamed Abdel-Aty

Authors
  1. Zubayer Islam
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Mohamed Abdel-Aty
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

The authors confirm contribution to the paper as follows: study conception and design: ZI, MA-A; data collection: ZI; analysis and interpretation of results: ZI, MA-A; draft manuscript preparation: ZI, MA-A. All authors reviewed the results and approved the final version of the manuscript.

Corresponding author

Correspondence to Zubayer Islam.

Ethics declarations

Conflict of interest

On behalf of all authors, the corresponding author states that there is no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Islam, Z., Abdel-Aty, M. Sensor-Based Transportation Mode Recognition Using Variational Autoencoder. J. Big Data Anal. Transp. 3, 15–26 (2021). https://doi.org/10.1007/s42421-021-00035-2

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s42421-021-00035-2

Keywords

  • Variational autoencoder
  • Data augmentation
  • Smart phone data
  • Transportation mode
  • Pedestrian to vehicles communication (P2V)