Cognitive Computation

, Volume 7, Issue 6, pp 715–730 | Cite as

Bivariate Nonisotonic Statistical Regression by a Lookup Table Neural System



Linear data regression is a fundamental mathematical tool in engineering and applied sciences. However, for complex nonlinear phenomena, the standard linear least-squares regression may prove ineffective, hence calling for more involved data modeling techniques. The current research work investigates in particular nonlinear statistical regression of bivariate data that do not exhibit a monotonic dependency. The current contribution proposes a neural-network-based data processing method, termed data monotonization, followed by neural isotonic statistical regression. Such data monotonization processing is performed by means of an adaptive neural network that learns its nonlinear transfer function from the training set. The artificial neural system that performs data monotonization is implemented through a lookup table (LUT), which entails few computationally inexpensive algebraic operations to adapt and to compute the output from the input data-set. A number of learning rules to adapt such LUT-based neural system are introduced and compared in order to elucidate their relative merits and drawbacks.


Nonlinear neural modeling Lookup table neural network Statistical regression Isotonic regression Bivariate data monotonization 


  1. 1.
    Abdullah A, Hussain A. A cognitively inspired approach to two-way cluster extraction from one-way clustered data. Cognit Comput. 2014;7:161–82.CrossRefGoogle Scholar
  2. 2.
    Ahuja S, Lakshminarayana A, Shukla SK. Statistical regression based power models. In: Low power design with high-level power estimation and power-aware synthesis. New York: Springer; 2012. p. 59 – 70Google Scholar
  3. 3.
    Angelov S, Harb B, Kannan S, Wang L-S. Weighted isotonic regression under the \(\text{ L }_1\) norm. In: Proceedings of the Seventeenth Annual ACM-SIAM Symposium on Discrete Algorithms (SODA), 2006. p. 783 – 791.Google Scholar
  4. 4.
    Blanco JL, Hernández LA, Fernández R, Ramos D. Improving automatic detection of obstructive sleep apnea through nonlinear analysis of sustained speech. Cognit Comput. 2013;5:458–72.CrossRefGoogle Scholar
  5. 5.
    Breiman L. Statistical modeling: the two cultures. Stat Sci. 2001;16(3):199–231.CrossRefGoogle Scholar
  6. 6.
    Brunk HD. Maximum likelihood estimates of monotone parameters. Ann Math Stat. 1955;26:607–16.CrossRefGoogle Scholar
  7. 7.
    Burnham KP, Anderson DR. Multimodel inference: understanding AIC and BIC in model selection. Sociol Methods Res. 2004;33(2):261–304.CrossRefGoogle Scholar
  8. 8.
    Chakravarti N. Sensitivity analysis in isotonic regression. Discrete Appl Math. 1993;45(3):183–96.CrossRefGoogle Scholar
  9. 9.
    Calzada Defez Á, Socoró Carrie JC. Voice quality modification using a harmonics plus noise model. Cognit Comput. 2013;5:473–82.CrossRefGoogle Scholar
  10. 10.
    Domínguez-Menchero JS, González-Rodríguez G. Analyzing an extension of the isotonic regression problem. Metrika. 2007;66(1):19–30.CrossRefGoogle Scholar
  11. 11.
    Eckerle K. Circular interference transmittance study. Report of the National Institute of Standards and Technology (NIST), US Department of Commerce, USA. Unpublished report, 1979. Data publicly available from the repository
  12. 12.
    Ezeiza A, López de Ipiña K, Hernández C, Barroso N. Enhancing the feature extraction process for automatic speech recognition with fractal dimensions. Cognit Comput. 2013;5:545–50.CrossRefGoogle Scholar
  13. 13.
    Fiori S. Hybrid independent component analysis by adaptive LUT activation function neurons. Neural Netw. 2002;15(1):85–94.CrossRefPubMedGoogle Scholar
  14. 14.
    Fiori S. Statistical nonparametric bivariate isotonic regression by lookup-table-based neural networks. In: Proceedings of the 2011 International Conference on Neural Information Processing (ICONIP 2011, Shanghai (China), November 14–17, 2011), B.-L. Lu, L. Zhang, and J. Kwok (Eds.), Part III, LNCS 7064, Heidelberg: Springer; 2011. p. 365 – 372.Google Scholar
  15. 15.
    Fiori S. Fast statistical regression in presence of a dominant independent variable. Neural Computing and Applications. (Special issue of the 2011 International Conference on Neural Information Processing - ICONIP’2011), 2013; 22(7): 1367 – 1378.Google Scholar
  16. 16.
    Fiori S. An isotonic trivariate statistical regression method. Adv Data Anal Classif. 2013;7(2):209–35.CrossRefGoogle Scholar
  17. 17.
    Fiori S. A two-dimensional Poisson equation formulation of non-parametric statistical non-linear modeling. Comput Math Appl. 2014;67(5):1171–85.CrossRefGoogle Scholar
  18. 18.
    Forrest DR, Hetland RD, DiMarco SF. Multivariable statistical regression models of the areal extent of hypoxia over the Texas-Louisiana continental shelf. Environ Res Lett. 2011;6(4):045002 (10 pp).CrossRefGoogle Scholar
  19. 19.
    Friedman J, Tibshirani R. The monotone smoothing of scatterplots. Technometrics. 1984;26(3):243–50.CrossRefGoogle Scholar
  20. 20.
    Gómez-Vilda P, Rodellar-Biarge V, Nieto-Lluis V, Munöz-Mulas C, Mazaira-Fernández LM, Martínez-Olalla R, Álvarez-Marquina A, Ramírez-Calvo C, Fernández-Fernández M. Characterizing neurological disease from voice quality biomechanical analysis. Cognit Comput. 2013;5:399-42.CrossRefGoogle Scholar
  21. 21.
    Hall P, Huang L-S. Nonparametric kernel regression subject to monotonicity constraints. Ann Stat. 2001;29(3):624–47.CrossRefGoogle Scholar
  22. 22.
    Jaqaman K, Danuser G. Linking data to models: data regression. Nat Rev Mol Cell Biol. 2006;7:813–9.CrossRefPubMedGoogle Scholar
  23. 23.
    Kulkarni MA, Patil S, Rama GV, Sen PN. Wind speed prediction using statistical regression and neural network. J Earth Syst Sci. 2008;117(4):457–63.CrossRefGoogle Scholar
  24. 24.
    Li X, Liu H-Z. Statistical regression for efficient high-dimensional modeling of analog and mixed-signal performance variations. In: Proceedings of the \(45^{\rm th}\) ACM/IEEE Design Automation Conference (DAC 2008, Anaheim Convention Center, California, USA, June 9–13, 2008), p. 38 – 43, June 2008.Google Scholar
  25. 25.
    Liu S, Gao RX, He Q, Staudenmayer J, Freedson P. Development of statistical regression models for ventilation estimation. In: Proceedings of the Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC, September 3–6, 2009), 2009. p. 1266 – 1269.Google Scholar
  26. 26.
    Liu J, Li H. Application research of a statistical regression algorithm in the IVR system. In: Proceedings of the 2010 International Conference on Educational and Network Technology (ICENT, Qinhuangdao (China), June 25–27, 2010), 2010. p. 358 – 360.Google Scholar
  27. 27.
    Naji M, Firoozabadi M, Azadfallah P. Classification of music-induced emotions based on information fusion of forehead biosignals and electrocardiogram. Cognit Comput. 2014;6:241–52.CrossRefGoogle Scholar
  28. 28.
    Papoulis A. Probability and statistics. Upper Saddle River: Prentice Hall; 1996.Google Scholar
  29. 29.
    Punera K, Gosh J. Enhanced hierarchical classification via isotonic smoothing. In: Proceedings of the \(17^{\rm th}\) International Conference on World Wide Web (April 21–25, 2008, Beijing–China), 2008. p. 151–160.Google Scholar
  30. 30.
    Ramsay JO. Monotone regression splines in action. Stat Sci. 1988;3(4):425–41.CrossRefGoogle Scholar
  31. 31.
    Siegert I, Philippou-Hübner D, Hartmann K, Böck R, Wendemuth A. Investigation of speaker group-dependent modelling for recognition of affective states from speech. Cognit Comput. 2014;6:892–913.CrossRefGoogle Scholar
  32. 32.
    Specht DE. A general regression neural network. IEEE Trans Neural Netw. 1991;2(6):568–76.CrossRefPubMedGoogle Scholar
  33. 33.
    Tam DN. Computation in emotional processing: quantitative confirmation of proportionality hypothesis for angry unhappy emotional intensity to perceived loss. Cognit Comput. 2011;3:394–415.CrossRefGoogle Scholar
  34. 34.
    Tran TP, Ong E, Hodges AP, Paternostro G, Piermarocchi C. Prediction of kinase inhibitor response using activity profiling, in vitro screening, and elastic net regression. BMC Syst Biol. 2014;8(74):1–10.Google Scholar
  35. 35.
    Yan X. Dissociated emergent response system and fine-processing system in human neural network and a heuristic neural architecture for autonomous humanoid robots. Cognit Comput. 2011;3:367–73.CrossRefGoogle Scholar
  36. 36.
    Zhu X, Welling M, Jin F, Lowengrub J. Predicting simulation parameters of biological systems using a Gaussian process model. Stat Anal Data Min: ASA Data Sci J (Special Issue: “Best Papers from the SLDM Competition”). 2012;5(6):509 – 522.Google Scholar
  37. 37.
    Zou L, Xu S, Ma Z, Lu J, Su W. Automatic removal of artifacts from attention deficit hyperactivity disorder electroencephalograms based on independent component analysis. Cognit Comput. 2013;5:225–33.CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media New York 2015

Authors and Affiliations

  1. 1.Dipartimento di Ingegneria dell’InformazioneUniversità Politecnica delle MarcheAnconaItaly
  2. 2.School of ComputingNational University of SingaporeSingaporeRepublic of Singapore
  3. 3.Bioinformatics InstituteA*STAR Biomedical Sciences InstitutesSingaporeRepublic of Singapore

Personalised recommendations