Skip to main content
SpringerLink
Log in

Trainable Gaussian-based activation functions for sensor-based human activity recognition

  • Original Article
  • Published:
Journal of Reliable Intelligent Environments Aims and scope Submit manuscript

Abstract

Neural networks’ capability to model non-linear relationships strongly depends on their activation functions (AFs). This dependency makes the search for AFs with better performance a relevant open research subject. This research work proposes two Trainable Gaussian-based AFs for Multilayer Perceptron neural networks on sensor-based human activity recognition (HAR), namely, the Four-Parameter Activation Gaussian Radial Basis Function (T4GRBF) and the Weighted Gaussian Radial Basis Function (WGRBF). The T4GRBF considers the training of four parameters. Two of these parameters regulate the center and spread of the Gaussian shape separately. Another parameter regulates the center and shape simultaneously, and the last parameter regulates the output range of the AF. The WGRBF adjusts the Gaussian shape with only two trainable parameters, and it is then scaled by the AF input. The proposals are validated through experiments on the Opportunity and UniMiB SHAR benchmark datasets. The results found for the Opportunity dataset in this research work evidence that the sliding window segmentation method has a high impact on the AFs’ performance, and the results in the UniMiB dataset show that the trainable GRBF-based AFs improve the HAR models performance. Furthermore, in both datasets, the Trainable Gaussian-based AFs fit the training data better than the standard AFs, regardless of the specific time window setup used in the experiments.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Algorithm 1
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14

Similar content being viewed by others

References

  1. Scardapane S, Van Vaerenbergh S, Totaro S, Uncini A (2019) Kafnets: Kernel-based non-parametric activation functions for neural networks. Neural Netw 110:19–32. https://doi.org/10.1016/j.neunet.2018.11.002

    Article  Google Scholar 

  2. Paduano I, Mileto A, Lofrano E (2023) A perspective on ai-based image analysis and utilization technologies in building engineering: recent developments and new directions. Buildings. https://doi.org/10.3390/buildings13051198

    Article  Google Scholar 

  3. Wang C, Song L-h, Yuan Z, Fan J-s (2023) State-of-the-art ai-based computational analysis in civil engineering. J Ind Inf Integr 33:100470. https://doi.org/10.1016/j.jii.2023.100470

    Article  Google Scholar 

  4. Ren Y, Jin C, Fang S, Yang L, Wu Z, Wang Z, Peng R, Gao K (2023) A comprehensive review of key technologies for enhancing the reliability of lithium-ion power batteries. Energies. https://doi.org/10.3390/en16176144

    Article  Google Scholar 

  5. Cui S, Gao Y, Huang Y, Shen L, Zhao Q, Pan Y, Zhuang S (2023) Advances and applications of machine learning and deep learning in environmental ecology and health. Environ Pollut 335:122358. https://doi.org/10.1016/j.envpol.2023.122358

    Article  Google Scholar 

  6. Amin AA, Sajid Iqbal M, Hamza Shahbaz M (2024) Development of intelligent fault-tolerant control systems with machine learning, deep learning, and transfer learning algorithms: A review. Expert Syst Appl 238:121956. https://doi.org/10.1016/j.eswa.2023.121956

    Article  Google Scholar 

  7. Jieyang P, Kimmig A, Dongkun W, Niu Z, Zhi F, Jiahai W, Liu X, Ovtcharova J (2023) A systematic review of data-driven approaches to fault diagnosis and early warning. J Intell Manuf 34(8):3277–3304. https://doi.org/10.1007/s10845-022-02020-0

    Article  Google Scholar 

  8. Yang D, Zhang W, Jiang Y (2023) Mechanical fault diagnosis based on deep transfer learning: a review. Meas Sci Technol 34(11):112001. https://doi.org/10.1088/1361-6501/ace7e6

    Article  Google Scholar 

  9. Guo Y, Zhang J, Sun B, Wang Y (2023) Adversarial deep transfer learning in fault diagnosis: progress, challenges, and future prospects. Sensors. https://doi.org/10.3390/s23167263

    Article  Google Scholar 

  10. Mishra BK, Thakker D, Mazumdar S, Neagu D, Gheorghe M, Simpson S (2020) A novel application of deep learning with image cropping: a smart city use case for flood monitoring. J Reliab Intell Environ 6(1):51–61. https://doi.org/10.1007/s40860-020-00099-x

    Article  Google Scholar 

  11. Schmitt M (2023) Securing the digital world: protecting smart infrastructures and digital industries with artificial intelligence (ai)-enabled malware and intrusion detection. J Ind Inf Integr 36:100520. https://doi.org/10.1016/j.jii.2023.100520

    Article  Google Scholar 

  12. Gil Pereira P, Vogelgesang K, Miodek M, Schmidt A, Herfet T (2023) Deepsharq: hybrid error coding using deep learning. J Reliab Intell Environ 9(3):283–301. https://doi.org/10.1007/s40860-023-00207-7

    Article  Google Scholar 

  13. Bingham G, Miikkulainen R (2022) Discovering parametric activation functions. Neural Netw 148:48–65. https://doi.org/10.1016/j.neunet.2022.01.001

    Article  Google Scholar 

  14. Hu H, Liu A, Guan Q, Qian H, Li X, Chen S, Zhou Q (2022) Adaptively customizing activation functions for various layers. IEEE Trans Neural Netw Learn Syst. https://doi.org/10.1109/TNNLS.2021.3133263

    Article  Google Scholar 

  15. Tavakoli M, Agostinelli F, Baldi P (2021) Splash: Learnable activation functions for improving accuracy and adversarial robustness. Neural Netw 140:1–12. https://doi.org/10.1016/j.neunet.2021.02.023

    Article  Google Scholar 

  16. Wang X, Ren H, Wang A (2022) Smish: A novel activation function for deep learning methods. Electronics. https://doi.org/10.3390/electronics11040540

    Article  Google Scholar 

  17. Shen S-L, Zhang N, Zhou A, Yin Z-Y (2022) Enhancement of neural networks with an alternative activation function tanhlu. Exp Syst Appl. https://doi.org/10.1016/j.eswa.2022.117181

    Article  Google Scholar 

  18. Li J, Ding F, Hayat T (2022) A novel nonlinear optimization method for fitting a noisy gaussian activation function. Int J Adapt Control Signal Process 36:690–707. https://doi.org/10.1002/acs.3367

    Article  Google Scholar 

  19. Kiliçarslan S, Celik M (2022) Kaf + rsigelu: a nonlinear and kernel-based activation function for deep neural networks. Neural Comput Appl. https://doi.org/10.1007/s00521-022-07211-7

    Article  Google Scholar 

  20. Apicella A, Donnarumma F, Isgrò F, Prevete R (2021) A survey on modern trainable activation functions. Neural Netw 138:14–32. https://doi.org/10.1016/j.neunet.2021.01.026

    Article  Google Scholar 

  21. Essa E, Abdelmaksoud IR (2023) Temporal-channel convolution with self-attention network for human activity recognition using wearable sensors. Knowl-Based Syst 278:110867. https://doi.org/10.1016/j.knosys.2023.110867

    Article  Google Scholar 

  22. Raja Sekaran S, Han PY, Yin OS (2023) Smartphone-based human activity recognition using lightweight multiheaded temporal convolutional network. Expert Syst Appl 227:120132. https://doi.org/10.1016/j.eswa.2023.120132

    Article  Google Scholar 

  23. Choudhury NA, Soni B (2023) An adaptive batch size-based-cnn-lstm framework for human activity recognition in uncontrolled environment. IEEE Trans Industr Inf 19(10):10379–10387. https://doi.org/10.1109/TII.2022.3229522

    Article  Google Scholar 

  24. Jiang Q, Zhu L, Shu C, Sekar V (2022) Multilayer perceptron neural network activated by adaptive gaussian radial basis function and its application to predict lid-driven cavity flow. Acta Mech Sin. https://doi.org/10.1007/s10409-021-01144-5

    Article  Google Scholar 

  25. Augusto JC (2022) Contexts and context-awareness revisited from an intelligent environments perspective. Appl Artif Intell. https://doi.org/10.1080/08839514.2021.2008644

    Article  Google Scholar 

  26. Ferrari A, Micucci D, Mobilio M, Napoletano P (2021) Trends in human activity recognition using smartphones. J Reliab Intell Environ 7(3):189–213. https://doi.org/10.1007/s40860-021-00147-0

    Article  Google Scholar 

  27. Feng H, Shen Q, Song R, Shi L, Xu H (2024) Atfa: adversarial time-frequency attention network for sensor-based multimodal human activity recognition. Expert Syst Appl 236:121296. https://doi.org/10.1016/j.eswa.2023.121296

    Article  Google Scholar 

  28. Sarkar A, Hossain SKS, Sarkar R (2023) Human activity recognition from sensor data using spatial attention-aided cnn with genetic algorithm. Neural Comput Appl 35(7):5165–5191. https://doi.org/10.1007/s00521-022-07911-0

    Article  Google Scholar 

  29. Chen C-T, Chang W-D (1996) A feedforward neural network with function shape autotuning. Neural Netw 9(4):627–641. https://doi.org/10.1016/0893-6080(96)00006-8

    Article  Google Scholar 

  30. Ramachandran P, Zoph B, Le QV (2017) Searching for activation functions

  31. Apicella A, Isgrò F, Prevete R (2019) A simple and efficient architecture for trainable activation functions. Neurocomputing 370:1–15. https://doi.org/10.1016/j.neucom.2019.08.065

    Article  Google Scholar 

  32. Hornik K, Stinchcombe M, White H (1989) Multilayer feedforward networks are universal approximators. Neural Netw 2(5):359–366. https://doi.org/10.1016/0893-6080(89)90020-8

    Article  Google Scholar 

  33. Ferrari A, Micucci D, Mobilio M, Napoletano P (2023) Deep learning and model personalization in sensor-based human activity recognition. J Reliab Intell Environ 9(1):27–39. https://doi.org/10.1007/s40860-021-00167-w

    Article  Google Scholar 

  34. Sanchez S, Machacuay J, Quinde M (2023) Federated learning for human activity recognition on the mhealth dataset. In: Rutkowski L, Scherer R, Korytkowski M, Pedrycz W, Tadeusiewicz R, Zurada JM (eds) Artificial intelligence and soft computing. Springer, Cham, pp 215–225

    Chapter  Google Scholar 

  35. Gao W, Zhang L, Teng Q, He J, Wu H (2021) Danhar: Dual attention network for multimodal human activity recognition using wearable sensors. Appl Soft Comput 111:107728. https://doi.org/10.1016/j.asoc.2021.107728

    Article  Google Scholar 

  36. Cheng X, Zhang L, Tang Y, Liu Y, Wu H, He J (2022) Real-time human activity recognition using conditionally parametrized convolutions on mobile and wearable devices. IEEE Sens J 22(6):5889–5901. https://doi.org/10.1109/JSEN.2022.3149337

    Article  Google Scholar 

  37. Gu F, Chung M-H, Chignell M, Valaee S, Zhou B, Liu X (2021) A survey on deep learning for human activity recognition. ACM Comput Surv 54:8. https://doi.org/10.1145/3472290

    Article  Google Scholar 

  38. Glorot X, Bengio Y (2010) Understanding the difficulty of training deep feedforward neural networks. In: Teh YW, Titterington M (eds.) Proceedings of the Thirteenth International Conference on Artificial Intelligence and Statistics. Proceedings of Machine Learning Research, vol. 9, pp. 249–256. PMLR, Chia Laguna Resort, Sardinia, Italy

  39. Chavarriaga R, Sagha H, Calatroni A, Digumarti ST, Tröster G, Millán R, Roggen D (2013) The opportunity challenge: a benchmark database for on-body sensor-based activity recognition. Pattern Recogn Lett 34(15):2033–2042. https://doi.org/10.1016/j.patrec.2012.12.014

    Article  Google Scholar 

  40. Micucci D, Mobilio M, Napoletano P (2017) Unimib shar: A dataset for human activity recognition using acceleration data from smartphones. Appl Sci 7:10. https://doi.org/10.3390/app7101101

    Article  Google Scholar 

  41. Ige AO, Mohd Noor MH (2022) A survey on unsupervised learning for wearable sensor-based activity recognition. Appl Soft Comput 127:109363. https://doi.org/10.1016/j.asoc.2022.109363

    Article  Google Scholar 

  42. Vaswani A, Shazeer N, Parmar N, Uszkoreit J, Jones L, Gomez AN, Kaiser L, Polosukhin I (2023) Attention is all you need

  43. Kingma DP, Ba J (2017) Adam: a method for stochastic optimization

  44. Loshchilov I, Hutter F (2019) Decoupled weight decay regularization

  45. Du KL, Swamy MNS (2019) Fundamentals of machine learning. Springer, London, pp 21–63. https://doi.org/10.1007/978-1-4471-7452-3_2

    Book  Google Scholar 

  46. Raschka S (2018) Model evaluation, model selection, and algorithm selection in machine learning. arXiv preprint arXiv:1811.12808

  47. Ghosal S, Sarkar M, Sarkar R (2022) Nofed-net: Non-linear fuzzy ensemble of deep neural networks for human activity recognition. IEEE Internet Things J. https://doi.org/10.1109/JIOT.2022.3155560

    Article  Google Scholar 

  48. Ma C, Li W, Cao J, Du J, Li Q, Gravina R (2020) Adaptive sliding window based activity recognition for assisted livings. Inf Fusion 53:55–65. https://doi.org/10.1016/j.inffus.2019.06.013

    Article  Google Scholar 

  49. Noor MHM, Salcic Z, Wang KI-K (2017) Adaptive sliding window segmentation for physical activity recognition using a single tri-axial accelerometer. Pervasive Mob Comput 38:41–59. https://doi.org/10.1016/j.pmcj.2016.09.009

    Article  Google Scholar 

  50. Kwapisz JR, Weiss GM, Moore SA (2011) Activity recognition using cell phone accelerometers. ACM SIGKDD Explor Newsl 12(2):74–82

    Article  Google Scholar 

  51. Reiss A, Stricker D (2012) Introducing a new benchmarked dataset for activity monitoring. In: 2012 16th International Symposium on Wearable Computers, pp. 108–109. IEEE

  52. Anguita D, Ghio A, Oneto L, Parra X, Reyes-Ortiz JL et al (2013) A public domain dataset for human activity recognition using smartphones. Esann 3:3

    Google Scholar 

  53. Ferrari A, Micucci D, Mobilio M, Napoletano P (2020) On the personalization of classification models for human activity recognition. IEEE Access 8:32066–32079. https://doi.org/10.1109/ACCESS.2020.2973425

    Article  Google Scholar 

  54. Ferrari Anna NP, Daniela M (2023) Deep learning and model personalization in sensor-based human activity recognition. J Reliable Intell Environ 9:27–39. https://doi.org/10.1007/s40860-021-00167-w

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Mechanical–Electrical Engineering, Universidad de Piura, Av. Ramón Mugica 131, Piura, 20009, Piura, Peru

    Javier Machacuay

  2. Department of Industrial and Systems Engineering, Universidad de Piura, Av. Ramón Mugica 131, Piura, 20009, Piura, Peru

    Mario Quinde

Authors
  1. Javier Machacuay
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Mario Quinde
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

JM developed the idea of the proposed AFs, coded the experiments to validate the AFs and wrote the main manuscript text. MQ designed the methodology to validate the proposed AFs and support the manuscript’s writing. All authors reviewed the manuscript.

Corresponding author

Correspondence to Mario Quinde.

Ethics declarations

Conflict of interest

The authors declare no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Machacuay, J., Quinde, M. Trainable Gaussian-based activation functions for sensor-based human activity recognition. J Reliable Intell Environ (2024). https://doi.org/10.1007/s40860-024-00221-3

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s40860-024-00221-3

Keywords

Search

Navigation