Skip to main content
SpringerLink
Log in

A Gaussian RBM with binary auxiliary units

  • Original Article
  • Published:
International Journal of Machine Learning and Cybernetics Aims and scope Submit manuscript

Abstract

Restricted Boltzmann Machines (RBM) have been widely applied in image processing. For RBM-based models on image recognition and image generation tasks, extracting expressive real-valued features and alleviating the overfitting problem are extremely important. In this paper, we propose a Gaussian Restricted Boltzmann Machine with binary Auxiliary units (GARBM), which designs binary auxiliary units in its visible layer and constructs parameterized real-valued features in its hidden layer. Specifically, based on the designed energy function in GARBM, activated auxiliary units are directly used to control probabilities of visible units and hidden units to extract real-valued features. Moreover, auxiliary units and their resulting feature selection mechanism not only alleviate the “gradient-variance” problem, but also provide certain randomness to other units to alleviate overfitting without introducing more hyperparameters. To build more effective deep models, we propose GARBM-based deep neural networks, and the effectiveness of proposed neural networks is verified in experiments.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

References

  1. Lopez R, Regier J, Jordan M et al (2019) Information constraints on auto-encoding variational Bayes. Adv Neural Inf Process Syst 2019

  2. Yan Q, Wang M, Huang W et al (2019) Automatically synthesizing DoS attack traces using generative adversarial networks. Int J Mach Learn Cybern 10(12):3387–3396

    Article  Google Scholar 

  3. Kuleshov V, Ermon S (2017) Neural variational inference and learning in undirected graphical models. Adv Neural Inf Process Syst

  4. Wang X, Zhao Y, Pourpanah F (2020) Recent advances in deep learning. Int J Mach Learn Cybern 11(11):747–750

    Article  Google Scholar 

  5. Srivastava N, Hinton G, Krizhevsky A et al (2014) Dropout: a simple way to prevent neural networks from overfitting. J Mach Learn Res 15(1):1929–1958

    MathSciNet  MATH  Google Scholar 

  6. Blundell C, Cornebise J, Kavukcuoglu K (2015) Weight uncertainty in neural networks. In: International conference on machine learning

  7. Zhang N, Ding S, Zhang J et al (2017) Research on point-wise gated deep networks. Appl Soft Comput 52:1210–1221

    Article  Google Scholar 

  8. Ranzato M, Krizhevsky A, Hinton GE (2010) Factored 3-Way restricted Boltzmann machines for modeling natural images. J Mach Learn Res 9:621–628

    Google Scholar 

  9. Courville A, Desjardins G, Bergstra J et al (2014) The spike-and-slab RBM and extensions to discrete and sparse data distributions. IEEE Trans Pattern Anal Mach Intell 36(9):1874–1887

    Article  Google Scholar 

  10. Ding S, Zhang N, Zhang J et al (2017) Unsupervised extreme learning machinewith representational features. Int J Mach Learn Cybern 8(2):587–595

    Article  Google Scholar 

  11. Zhai J, Zhou X, Zhang S et al (2019) Ensemble RBM-based classifier using fuzzy integral for big data classification. Int J Mach Learn Cybern 10:3327–3337

    Article  Google Scholar 

  12. Wen Y, Erick D (2019) Deep Boltzmann machine for nonlinear system modelling. Int J Mach Learn Cybern 10:1705–1716

    Article  Google Scholar 

  13. Schmitt J, Roth S (2021) Sampling-free variational inference for neural networks with multiplicative activation noise. arXiv:2103.08497

  14. Mescheder L, Nowozin S, Geiger A (2017) Adversarial variational Bayes: unifying variational autoencoders and generative adversarial networks. In: International conference on machine learning

  15. Tolstikhin I, Bousquet O, Gelly S, et al (2017) Wasserstein Auto-Encoders. arXiv:1711.01558, arXiv, 2017

  16. Felhi G, Leroux J, Seddah D (2020) Controlling the interaction between generation and inference in semi-supervised variational autoencoders using importance weighting. arXiv:2010.06549.

  17. Vahdat A, Macready W G, Bian Z, et al (2018) DVAE++: discrete variational autoencoders with overlapping transformations. In: International conference on machine learning

  18. Nair V, Hinton GE (2010) Rectified linear units improve restricted Boltzmann machines. In: International conference on international conference on machine learning

  19. Su Q, Liao X, Li C et al (2017) Unsupervised learning with truncated gaussian graphical models. In: AAAI conference on artificial intelligence

  20. Cho K, Raiko T, Ilin A (2014) Gaussian-Bernoulli deep Boltzmann machine. In: IEEE international joint conference on neural networks

  21. Simonyan K, Zisserman A (2015) Very deep convolutional networks for large-scale image recognition. In: International conference of learning representation (oral)

Download references

Acknowledgements

This work is supported by the National Natural Science Foundations of China (Nos. 61976216 and 61672522).

Author information

Authors and Affiliations

  1. School of Computer Science and Technology, China University of Mining and Technology, Xuzhou, 221116, China

    Jian Zhang, Shifei Ding, Tongfeng Sun & Lili Guo

  2. Mine Digitization Engineering Research Center of Ministry of Education of the People’s Republic of China, Xuzhou, 221116, China

    Jian Zhang, Shifei Ding, Tongfeng Sun & Lili Guo

Authors
  1. Jian Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Shifei Ding
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Tongfeng Sun
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Lili Guo
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Shifei Ding.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zhang, J., Ding, S., Sun, T. et al. A Gaussian RBM with binary auxiliary units. Int. J. Mach. Learn. & Cyber. 13, 2425–2433 (2022). https://doi.org/10.1007/s13042-022-01534-6

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s13042-022-01534-6

Keywords

Search

Navigation