Skip to main content
SpringerLink
Log in

Towards implementation of residual-feedback GMDH neural network on parallel GPU memory guided by a regression curve

  • Published:
The Journal of Supercomputing Aims and scope Submit manuscript

Abstract

GMDH, which stands for Group Method Data Handling, is an evolutionary type of neural network. It has received much attention in the supercomputing research community because of its ability to optimize its internal structure for maximum prediction accuracy. GMDH works by evolving itself from a basic network, expanding its number of neurons and hidden layer until no further performance gain can be obtained. Earlier on, the authors proposed a novel strategy that extends existing GMDH neural network techniques. The new strategy, called residual-feedback, retains and reuses past prediction errors as part of the multivariate sample data that provides relevant multivariate inputs to the GMDH neural networks. This is important because the strength of GMDH, like any neural network, is in predicting outcomes from multivariate data, and it is very noise-tolerant. GMDH is a well-known ensemble type of prediction method that is capable of modeling highly non-linear relations. Maximum accuracy is often achieved by using only the minimum amount of network neurons and simplest layered structure. This paper contributes to the technical design of implementing GMDH on GPU memory where all the weight computations run on parallel GPU memory blocks. It is a first step towards developing complex neural network architecture on GPU with the capability of evolving and expanding its structure to minimally sufficient for obtaining the maximum prediction accuracy based on the given input data.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18
Fig. 19
Fig. 20
Fig. 21
Fig. 22
Fig. 23
Fig. 24
Fig. 25
Fig. 26

Similar content being viewed by others

Notes

  1. http://srdata.nist.gov/gateway/gateway?keyword=handwriting+recognition.

  2. http://archive.ics.uci.edu/ml/datasets/Smartphone-Based+Recognition+of+Human+Activities+and+Postural+Transitions.

  3. https://archive.ics.uci.edu/ml/datasets/Human+Activity+Recognition+Using+Smartphones.

  4. https://archive.ics.uci.edu/ml/datasets/Gas+Sensor+Array+Drift+Dataset.

References

  1. Ivakhnenko AG (1971) Polynomial theory of complex systems. IEEE Trans Syst Man Cybern SMC–1(4):364–378

    Article  MathSciNet  Google Scholar 

  2. Yefimenko SM (2012) Optimal Paralleling for Solving Combinatorial Modelling Problems using Graphics Processing Units, The 5th International Workshop on Inductive Modelling IWIM 2012, pp 57–59

  3. Sarka D (2009) The ART (and ARIMA) of forecasting, Technical Presentation, Solid Quality Mentors, http://blogs.solidq.com/dsarka/home.aspx, Last accessed on April 23

  4. Fong S, Nannan Z, Wong RK, Yang X-S (2012) Rare events forecasting using a residual-feedback GMDH neural network. 2012 IEEE 12th international conference on data mining workshops (ICDMW), pp 464–473

  5. Ivakhnenko AG (1970) Heuristic self-organization in problems of engineering cybernetics. Automatica 6:207–219

    Article  Google Scholar 

  6. Ivakhnenko AG, Zholnarskiy AA (1992) Estimating the coefficients of polynomials in parametric GMDH algorithms by the improved instrumental variables method. J Autom Inf Sci 25(3):25–32

    MathSciNet  Google Scholar 

  7. Sarychev AP (1984) A multilayer nonlinear harmonic GMDH algorithm for self-organization of predicting models. Sov Autom Control 17(4):90–95

    MATH  Google Scholar 

  8. Duffy JJ, Franklin MA (1975) A learning identification algorithm and its application to an environmental system. IEEE Trans Syst Man Cybern SMC–5(2):226–240

    Article  Google Scholar 

  9. Kozubovskiy SF (1986) Determination of the optimal set of lagging arguments for a difference predicting model by correlation analysis. Sov J Autom Inf Sci 19(2):77–79

    Google Scholar 

  10. Karnazes PA, Bonnell RD (1982) Systems identification techniques using the group method of data handling. In: Proceedings of the 6th symposium on identification and system parameter estimation, vol 1, Washington DC. International Federation of Automatic Control, Pergamon, pp 713–718

  11. Kendall M, Gibbons JD (1990) Rank correlation methods. Edward Arnold, London

    MATH  Google Scholar 

  12. Loquasto F, Seborg DE (2003) Monitoring model predictive control systems using pattern classification and neural networks. J Ind Eng Chem Res 42:4689–4701

    Article  Google Scholar 

  13. Narasimhana S, Vachhanib P, Rengaswamya R (2008) New nonlinear residual feedback observer for fault diagnosis in nonlinear systems. Automatica 44(9):2222–2229

    Article  MathSciNet  Google Scholar 

  14. Lanitis A (2003) Building statistical appearance models using residual information. in Proceedings of 9th Panhellenic Conference in Informatics. Thessaloniki, Nov., pp 547–559

  15. Rossen A (2011) On the predictive content of nonlinear transformations of lagged autoregression residuals and time series observations, HWWI Research Papers, No. 113, Hamburg Institute of International Economics, pp 1–24

  16. Schetinin V (2003) A learning algorithm for evolving cascade neural networks. J Neural Process Lett 17(1):21–31

    Article  Google Scholar 

  17. Varahrami V (2011) Recognition of good prediction of gold price between MLFF and GMDH neural network. J Econ Int Financ 3(4):204–210

    Google Scholar 

  18. LeCun Y, Bottou L, Bengio Y, Haffner P (1998) Gradient-based learning applied to document recognition. Proc IEEE 86(11):2278–2324

    Article  Google Scholar 

  19. Qin Yechena, Langari Rezab, Lianga Gu (2015) A new modeling algorithm based on ANFIS and GMDH. J Intell Fuzzy Syst 29(4):1321–1329

    Article  Google Scholar 

  20. Wang Lipo, Quek Hou Chai, Tee Keng Hoe, Zhou Nina, Wan Chunru (2005) Optimal Size of a Feedforward Neural Network: How Much does it Matter?, Joint International Conference on Autonomic and Autonomous Systems and International Conference on Networking and Services, pp 6–9 Date 23–28 Oct. (2005)

  21. Brito Ricardo, Fong Simon, Cho Kyungeun, Song Wei, Wong Raymond, Mohammed Sabah, Fiaidhi Jinan (2016) GPU-enabled back-propagation artificial neural network for digit recognition in parallel, The Journal of Supercomputing, Springer, February 2016, pp 1–19

Download references

Acknowledgments

The authors are thankful for the financial support from the research grant “Rare Event Forecasting and Monitoring in Spatial Wireless Sensor Network Data,” Grant No. MYRG2014-00065-FST, and Grant No. FDCT/126/2014/A3, offered by the University of Macau, and Macau SAR Government.

Author information

Authors and Affiliations

  1. Department of Computer and Information Science, University of Macau, Taipa, Macau SAR, People’s Republic of China

    Ricardo Brito & Simon Fong

  2. Department of Computer and Multimedia Engineering, Dongguk University, Seoul, Korea

    Kyungeun Cho

  3. College of Information Engineering, North China University of Technology, Beijing, China

    Wei Song

  4. School of Computer Science and Engineering, University of New South Wales, Sydney, Australia

    Raymond Wong

  5. Department of Computer Science, Lakehead University, Thunder Bay, Canada

    Sabah Mohammed & Jinan Fiaidhi

Authors
  1. Ricardo Brito
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Simon Fong
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Kyungeun Cho
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Wei Song
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Raymond Wong
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Sabah Mohammed
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Jinan Fiaidhi
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Simon Fong.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Brito, R., Fong, S., Cho, K. et al. Towards implementation of residual-feedback GMDH neural network on parallel GPU memory guided by a regression curve. J Supercomput 72, 3993–4020 (2016). https://doi.org/10.1007/s11227-016-1740-9

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11227-016-1740-9

Keywords

Search

Navigation