Skip to main content
SpringerLink
Log in

Improving Smartphone-Based Transport Mode Recognition Using Generative Adversarial Networks

  • Chapter
  • First Online:
Activity and Behavior Computing

Part of the book series: Smart Innovation, Systems and Technologies ((SIST,volume 204))

Abstract

Wearable devices such as smartphones and smartwatches are widely used and record a significant amount of data. Labelling this data for human activity recognition is a time-consuming task, therefore methods which reduce the amount of labelled data required to train accurate classifiers are important. Generative Adversarial Networks (GANs) can be used to model the implicit distribution of a dataset. Traditional GANs, which only consist of a generator and a discriminator, result in networks able to generate synthetic data and distinguish real from fake samples. This adversarial game can be extended to include a classifier, which allows the training of the classification network to be enhanced with synthetic and unlabelled data. The network architecture presented in this paper is inspired by SenseGAN [1], but instead of generating and classifying sensor-recorded time-series data, our approach operates with extracted features, which drastically reduces the amount of stored and processed data and enables deployment on less powerful and potentially wearable devices. We show that this technique can be used to improve the classification performance of a classifier trained to recognise locomotion modes based on recorded acceleration data and that it reduces the amount of labelled training data necessary to achieve a similar performance compared to a baseline classifier. Specifically, our approach reached the same accuracy as the baseline classifier up to 50% faster and was able to achieve a 10% higher accuracy in the same number of epochs.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 129.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 169.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 169.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    x is a sample originating from a given dataset.

  2. 2.

    G(z) is a synthetic sample originating from the generator that ought to mimic the distribution of the given dataset.

  3. 3.

    In this work, \(X_U\) also originates from the SHL dataset (see Sect. 5.3), but we simply omitted the associated label for the purpose of experimentation.

  4. 4.

    http://www.shl-dataset.org/download/.

References

  1. Yao, S., Abdelzaher, T., Zhao, Y., Shao, H., Zhang, C., Zhang, A., Hu, S., Liu, D., Liu, S., Su, L., et al.: Sensegan: Enabling deep learning for internet of things with a semi-supervised framework. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 2(3), 1–21 (2018)

    Article  Google Scholar 

  2. Vaizman, Y., Ellis, K., Lanckriet, G.: Recognizing detailed human context in the wild from smartphones and smartwatches. IEEE Pervasive Computing 16(4), 62–74 (2017)

    Article  Google Scholar 

  3. Richoz, S., Birch, P., Ciliberto, M., Wang, L., Gjoreski, H., Perez-Uribe, A., Roggen, D.: Human and machine recognition of transportation modes from body-worn camera images. In: Joint 8th International Conference on Informatics, Electronics & Vision (ICIEV) and 3rd International Conference on Imaging, Vision & Pattern Recognition (icIVPR), IEEE (2019)

    Google Scholar 

  4. Ward, J.A., Lukowicz, P., Troster, G., Starner, T.E.: Activity recognition of assembly tasks using body-worn microphones and accelerometers. IEEE Transactions on Pattern Analysis and Machine Intelligence 28(10), 1553–1567 (2006)

    Article  Google Scholar 

  5. Kautz, H.: A formal theory of plan recognition. PhD thesis, University of Rochester (1987)

    Google Scholar 

  6. Tao Gu, Zhanqing Wu, Xianping Tao, Pung, H.K., Jian Lu: epsicar: An emerging patterns based approach to sequential, interleaved and concurrent activity recognition. In: 2009 IEEE International Conference on Pervasive Computing and Communications. (2009) 1–9

    Google Scholar 

  7. Charniak, E., Goldman, R.P.: A bayesian model of plan recognition. Artificial Intelligence 64(1), 53–79 (1993)

    Article  Google Scholar 

  8. Bulling, A., Blanke, U., Schiele, B.: A tutorial on human activity recognition using body-worn inertial sensors. ACM Computing Surveys 46(3), (2014)

    Google Scholar 

  9. Roggen, D., Calatroni, A., Rossi, M., Holleczek, T., Förster, K., Tröster, G., Lukowicz, P., Bannach, D., Pirkl, G., Ferscha, A., Doppler, J., Holzmann, C., Kurz, M., Holl, G., Chavarriaga, R., Sagha, H., Bayati, H., Creatura, M., d. R. Millán, J.: Collecting complex activity datasets in highly rich networked sensor environments. In: 2010 Seventh International Conference on Networked Sensing Systems (INSS). (2010) 233–240

    Google Scholar 

  10. Miu, T., Missier, P., Plötz, T.: Bootstrapping personalised human activity recognition models using online active learning. In: 2015 IEEE International Conference on Computer and Information Technology; Ubiquitous Computing and Communications; Dependable, Autonomic and Secure Computing; Pervasive Intelligence and Computing. (2015) 1138–1147

    Google Scholar 

  11. Zeng, M., Yu, T., Wang, X., Nguyen, L.T., Mengshoel, O.J., Lane, I.: Semi-supervised convolutional neural networks for human activity recognition. In: 2017 IEEE International Conference on Big Data (Big Data). (2017) 522–529

    Google Scholar 

  12. Goodfellow, I.J., Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley, D., Ozair, S., Courville, A., Bengio, Y.: Generative adversarial networks. arXiv preprint (2014) arXiv:1406.2661

  13. Salimans, T., Goodfellow, I., Zaremba, W., Cheung, V., Radford, A., Chen, X., Chen, X.: Improved techniques for training gans. In: Lee, D.D., Sugiyama, M., Luxburg, U.V., Guyon, I., Garnett, R. (eds.) Advances in Neural Information Processing Systems 29, pp. 2234–2242. Curran Associates, Inc. (2016)

    Google Scholar 

  14. Radford, A., Metz, L., Chintala, S.: Unsupervised representation learning with deep convolutional generative adversarial networks. arXiv preprint (2015)

    Google Scholar 

  15. Jolicoeur-Martineau, A.: The relativistic discriminator: a key element missing from standard gan. arXiv preprint (2018) arXiv:1807.00734

  16. Karras, T., Aila, T., Laine, S., Lehtinen, J.: Progressive Growing of GANs for Improved Quality, Stability, and Variation. arXiv e-prints (2017) arXiv:1710.10196

  17. Brock, A., Donahue, J., Simonyan, K.: Large Scale GAN Training for High Fidelity Natural Image Synthesis. arXiv e-prints (2018) arXiv:1809.11096

  18. Isola, P., Zhu, J.Y., Zhou, T., Efros, A.A.: Image-to-Image Translation with Conditional Adversarial Networks. arXiv e-prints (2016) arXiv:1611.07004

  19. Wang, T.C., Liu, M.Y., Zhu, J.Y., Tao, A., Kautz, J., Catanzaro, B.: High-Resolution Image Synthesis and Semantic Manipulation with Conditional GANs. arXiv e-prints (2017) arXiv:1711.11585

  20. Wu, H., Zheng, S., Zhang, J., Huang, K.: GP-GAN: Towards Realistic High-Resolution Image Blending. arXiv e-prints (2017) arXiv:1703.07195

  21. Ledig, C., Theis, L., Huszar, F., Caballero, J., Cunningham, A., Acosta, A., Aitken, A., Tejani, A., Totz, J., Wang, Z., Shi, W.: Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network. arXiv e-prints (2016) arXiv:1609.04802

  22. Soleimani, E., Nazerfard, E.: Cross-Subject Transfer Learning in Human Activity Recognition Systems using Generative Adversarial Networks. arXiv e-prints (2019) arXiv:1903.12489

  23. Wang, D., Yuan, Y., Wang, Q.: Early action prediction with generative adversarial networks. IEEE Access 7, 35795–35804 (2019)

    Article  Google Scholar 

  24. Wang, J., Chen, Y., Gu, Y., Xiao, Y., Pan, H.: SensoryGANs: An effective generative adversarial framework for sensor-based human activity recognition. In: 2018 International Joint Conference on Neural Networks (IJCNN). (2018) 1–8

    Google Scholar 

  25. Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization (2014)

    Google Scholar 

  26. Figo, D., Diniz, P.C., Ferreira, D.R., Cardoso, J.M.P.: Preprocessing techniques for context recognition from accelerometer data. Personal and Ubiquitous Computing 14(7), 645–662 (2010)

    Article  Google Scholar 

  27. Chawla, N.V., Bowyer, K.W., Hall, L.O., Kegelmeyer, W.P.: Smote: Synthetic minority over-sampling technique. J. Artif. Int. Res. 16(1), 321–357 (2002)

    MATH  Google Scholar 

  28. Gjoreski, H., Ciliberto, M., Wang, L., Ordonez Morales, F.J., Mekki, S., Valentin, S., Roggen, D.: The University of Sussex-Huawei Locomotion and Transportation Dataset for Multimodal Analytics With Mobile Devices. IEEE Access 6, 42592–42604 (2018)

    Article  Google Scholar 

  29. Engelbrecht, J., Booysen, M.J., van Rooyen, G., Bruwer, F.J.: Survey of smartphone-based sensing in vehicles for intelligent transportation system applications. IET Intelligent Transport Sys. 9(10), 924–935 (2015)

    Article  Google Scholar 

  30. Gjoreski, H., Kaluza, B., Gams, M., Milic, R., Lustrek, M.: Context-based ensemble method for human energy expenditure estimation. Appl. Soft Comput. 37, 96–970 (2015)

    Article  Google Scholar 

  31. Anagnostopoulou, E., Urbancic, J., Bothos, E., Magoutas, B., Bradesko, L., Schrammel, J., Mentzas, G.: From mobility patterns to behavioural change: leveraging travel behaviour and personality profiles to nudge for sustainable transportation. J. Intelligent Information Sys. 2018, 1–22 (2018)

    Google Scholar 

  32. Wang, L., Gjoreski, H., Ciliberto, M., Mekki, S., Valentin, S., Roggen, D.: Enabling reproducible research in sensor-based transportation mode recognition with the Sussex-Huawei dataset. IEEE Access 7, 10870–10891 (2019)

    Article  Google Scholar 

  33. Wang, L., Gjoreski, H., Ciliberto, M., Mekki, S., Valentin, S., Roggen, D.: Benchmarking the SHL recognition challenge with classical and deep-learning pipelines. In: Proc. ACM Int Joint Conf and 2018 Int Symp on Pervasive and Ubiquitous Computing and Wearable Computers, ACM (2018) 1626–1635

    Google Scholar 

  34. Wang, L., Gjoreski, H., Murao, K., Okita, T., Roggen, D.: Summary of the Sussex-Huawei Locomotion-Transportation Recognition Challenge. In: Proc. ACM Int Joint Conf and 2018 Int Symp on Pervasive and Ubiquitous Computing and Wearable Computers, ACM (2018) 1521–1530

    Google Scholar 

  35. Wang, L., Gjoreski, H., Ciliberto, M., Lago, P., Murao, K., Okita, T., Roggen, D.: Summary of the Sussex-Huawei locomotion-transportation recognition challenge 2019. In: Adjunct Proceedings of the 2019 ACM International Joint Conference on Pervasive and Ubiquitous Computing and Proceedings of the 2019 ACM International Symposium on Wearable Computers, ACM (2019) 849–856

    Google Scholar 

  36. Hawkins, D.: The problem of overfitting. Journal of chemical information and computer sciences 44, 1–12 (2004)

    Article  Google Scholar 

  37. Bergstra, J., Bengio, Y.: Random search for hyper-parameter optimization. Journal of Machine Learning Research 13(10), 281–305 (2012)

    MathSciNet  MATH  Google Scholar 

  38. Bergstra, J.S., Bardenet, R., Bengio, Y., Kégl, B.: Algorithms for hyper-parameter optimization. In: Shawe-Taylor, J., Zemel, R.S., Bartlett, P.L., Pereira, F., Weinberger, K.Q. (eds.) Advances in Neural Information Processing Systems 24, pp. 2546–2554. Curran Associates, Inc. (2011)

    Google Scholar 

  39. Stang, M., Meier, C., Rau, V., Sax, E.: An evolutionary approach to hyper-parameter optimization of neural networks. In: Ahram, T., Taiar, R., Colson, S., Choplin, A. (eds.) Human Interaction and Emerging Technologies, pp. 713–718. Springer International Publishing, Cham (2020)

    Google Scholar 

Download references

Acknowledgements

We acknowledge NVIDIA for their donation of a TITAN XP.

Author information

Authors and Affiliations

  1. Sensor Technology Research Centre, University of Sussex, Brighton, UK

    Lukas Günthermann & Daniel Roggen

  2. Centre for Computational Neuroscience and Robotics, University of Sussex, Brighton, UK

    Andrew Philippides

Authors
  1. Lukas Günthermann
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Andrew Philippides
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Daniel Roggen
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Lukas Günthermann .

Editor information

Editors and Affiliations

  1. University of Dhaka, Dhaka, Bangladesh

    Md Atiqur Rahman Ahad

  2. Kyushu Institute of Technology, Fukuoka, Japan

    Sozo Inoue

  3. Sussex University, Brighton, UK

    Daniel Roggen

  4. Tokyo University of Agriculture and Technology, Tokyo, Japan

    Kaori Fujinami

Rights and permissions

Reprints and permissions

Copyright information

© 2021 The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.

About this chapter

Check for updates. Verify currency and authenticity via CrossMark

Cite this chapter

Günthermann, L., Philippides, A., Roggen, D. (2021). Improving Smartphone-Based Transport Mode Recognition Using Generative Adversarial Networks. In: Ahad, M.A.R., Inoue, S., Roggen, D., Fujinami, K. (eds) Activity and Behavior Computing. Smart Innovation, Systems and Technologies, vol 204. Springer, Singapore. https://doi.org/10.1007/978-981-15-8944-7_5

Download citation

Publish with us

Policies and ethics

Search

Navigation