# Bivariate Nonisotonic Statistical Regression by a Lookup Table Neural System

- 101 Downloads
- 2 Citations

## Abstract

Linear data regression is a fundamental mathematical tool in engineering and applied sciences. However, for complex nonlinear phenomena, the standard linear least-squares regression may prove ineffective, hence calling for more involved data modeling techniques. The current research work investigates in particular nonlinear statistical regression of bivariate data that do not exhibit a monotonic dependency. The current contribution proposes a neural-network-based data processing method, termed *data monotonization*, followed by neural isotonic statistical regression. Such data monotonization processing is performed by means of an adaptive neural network that learns its nonlinear transfer function from the training set. The artificial neural system that performs data monotonization is implemented through a lookup table (LUT), which entails few computationally inexpensive algebraic operations to adapt and to compute the output from the input data-set. A number of learning rules to adapt such LUT-based neural system are introduced and compared in order to elucidate their relative merits and drawbacks.

## Keywords

Nonlinear neural modeling Lookup table neural network Statistical regression Isotonic regression Bivariate data monotonization## References

- 1.Abdullah A, Hussain A. A cognitively inspired approach to two-way cluster extraction from one-way clustered data. Cognit Comput. 2014;7:161–82.CrossRefGoogle Scholar
- 2.Ahuja S, Lakshminarayana A, Shukla SK. Statistical regression based power models. In: Low power design with high-level power estimation and power-aware synthesis. New York: Springer; 2012. p. 59 – 70Google Scholar
- 3.Angelov S, Harb B, Kannan S, Wang L-S. Weighted isotonic regression under the \(\text{ L }_1\) norm. In: Proceedings of the Seventeenth Annual ACM-SIAM Symposium on Discrete Algorithms (SODA), 2006. p. 783 – 791.Google Scholar
- 4.Blanco JL, Hernández LA, Fernández R, Ramos D. Improving automatic detection of obstructive sleep apnea through nonlinear analysis of sustained speech. Cognit Comput. 2013;5:458–72.CrossRefGoogle Scholar
- 5.Breiman L. Statistical modeling: the two cultures. Stat Sci. 2001;16(3):199–231.CrossRefGoogle Scholar
- 6.Brunk HD. Maximum likelihood estimates of monotone parameters. Ann Math Stat. 1955;26:607–16.CrossRefGoogle Scholar
- 7.Burnham KP, Anderson DR. Multimodel inference: understanding AIC and BIC in model selection. Sociol Methods Res. 2004;33(2):261–304.CrossRefGoogle Scholar
- 8.Chakravarti N. Sensitivity analysis in isotonic regression. Discrete Appl Math. 1993;45(3):183–96.CrossRefGoogle Scholar
- 9.Calzada Defez Á, Socoró Carrie JC. Voice quality modification using a harmonics plus noise model. Cognit Comput. 2013;5:473–82.CrossRefGoogle Scholar
- 10.Domínguez-Menchero JS, González-Rodríguez G. Analyzing an extension of the isotonic regression problem. Metrika. 2007;66(1):19–30.CrossRefGoogle Scholar
- 11.Eckerle K. Circular interference transmittance study. Report of the National Institute of Standards and Technology (NIST), US Department of Commerce, USA. Unpublished report, 1979. Data publicly available from the repository http://www.itl.nist.gov/div898/strd/nls/data/eckerle4.shtml.
- 12.Ezeiza A, López de Ipiña K, Hernández C, Barroso N. Enhancing the feature extraction process for automatic speech recognition with fractal dimensions. Cognit Comput. 2013;5:545–50.CrossRefGoogle Scholar
- 13.Fiori S. Hybrid independent component analysis by adaptive LUT activation function neurons. Neural Netw. 2002;15(1):85–94.CrossRefPubMedGoogle Scholar
- 14.Fiori S. Statistical nonparametric bivariate isotonic regression by lookup-table-based neural networks. In: Proceedings of the 2011 International Conference on Neural Information Processing (ICONIP 2011, Shanghai (China), November 14–17, 2011), B.-L. Lu, L. Zhang, and J. Kwok (Eds.), Part III, LNCS 7064, Heidelberg: Springer; 2011. p. 365 – 372.Google Scholar
- 15.Fiori S. Fast statistical regression in presence of a dominant independent variable. Neural Computing and Applications. (Special issue of the 2011 International Conference on Neural Information Processing - ICONIP’2011), 2013; 22(7): 1367 – 1378.Google Scholar
- 16.Fiori S. An isotonic trivariate statistical regression method. Adv Data Anal Classif. 2013;7(2):209–35.CrossRefGoogle Scholar
- 17.Fiori S. A two-dimensional Poisson equation formulation of non-parametric statistical non-linear modeling. Comput Math Appl. 2014;67(5):1171–85.CrossRefGoogle Scholar
- 18.Forrest DR, Hetland RD, DiMarco SF. Multivariable statistical regression models of the areal extent of hypoxia over the Texas-Louisiana continental shelf. Environ Res Lett. 2011;6(4):045002 (10 pp).CrossRefGoogle Scholar
- 19.Friedman J, Tibshirani R. The monotone smoothing of scatterplots. Technometrics. 1984;26(3):243–50.CrossRefGoogle Scholar
- 20.Gómez-Vilda P, Rodellar-Biarge V, Nieto-Lluis V, Munöz-Mulas C, Mazaira-Fernández LM, Martínez-Olalla R, Álvarez-Marquina A, Ramírez-Calvo C, Fernández-Fernández M. Characterizing neurological disease from voice quality biomechanical analysis. Cognit Comput. 2013;5:399-42.CrossRefGoogle Scholar
- 21.Hall P, Huang L-S. Nonparametric kernel regression subject to monotonicity constraints. Ann Stat. 2001;29(3):624–47.CrossRefGoogle Scholar
- 22.Jaqaman K, Danuser G. Linking data to models: data regression. Nat Rev Mol Cell Biol. 2006;7:813–9.CrossRefPubMedGoogle Scholar
- 23.Kulkarni MA, Patil S, Rama GV, Sen PN. Wind speed prediction using statistical regression and neural network. J Earth Syst Sci. 2008;117(4):457–63.CrossRefGoogle Scholar
- 24.Li X, Liu H-Z. Statistical regression for efficient high-dimensional modeling of analog and mixed-signal performance variations. In: Proceedings of the \(45^{\rm th}\) ACM/IEEE Design Automation Conference (DAC 2008, Anaheim Convention Center, California, USA, June 9–13, 2008), p. 38 – 43, June 2008.Google Scholar
- 25.Liu S, Gao RX, He Q, Staudenmayer J, Freedson P. Development of statistical regression models for ventilation estimation. In: Proceedings of the Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC, September 3–6, 2009), 2009. p. 1266 – 1269.Google Scholar
- 26.Liu J, Li H. Application research of a statistical regression algorithm in the IVR system. In: Proceedings of the 2010 International Conference on Educational and Network Technology (ICENT, Qinhuangdao (China), June 25–27, 2010), 2010. p. 358 – 360.Google Scholar
- 27.Naji M, Firoozabadi M, Azadfallah P. Classification of music-induced emotions based on information fusion of forehead biosignals and electrocardiogram. Cognit Comput. 2014;6:241–52.CrossRefGoogle Scholar
- 28.Papoulis A. Probability and statistics. Upper Saddle River: Prentice Hall; 1996.Google Scholar
- 29.Punera K, Gosh J. Enhanced hierarchical classification via isotonic smoothing. In: Proceedings of the \(17^{\rm th}\) International Conference on World Wide Web (April 21–25, 2008, Beijing–China), 2008. p. 151–160.Google Scholar
- 30.Ramsay JO. Monotone regression splines in action. Stat Sci. 1988;3(4):425–41.CrossRefGoogle Scholar
- 31.Siegert I, Philippou-Hübner D, Hartmann K, Böck R, Wendemuth A. Investigation of speaker group-dependent modelling for recognition of affective states from speech. Cognit Comput. 2014;6:892–913.CrossRefGoogle Scholar
- 32.Specht DE. A general regression neural network. IEEE Trans Neural Netw. 1991;2(6):568–76.CrossRefPubMedGoogle Scholar
- 33.Tam DN. Computation in emotional processing: quantitative confirmation of proportionality hypothesis for angry unhappy emotional intensity to perceived loss. Cognit Comput. 2011;3:394–415.CrossRefGoogle Scholar
- 34.Tran TP, Ong E, Hodges AP, Paternostro G, Piermarocchi C. Prediction of kinase inhibitor response using activity profiling, in vitro screening, and elastic net regression. BMC Syst Biol. 2014;8(74):1–10.Google Scholar
- 35.Yan X. Dissociated emergent response system and fine-processing system in human neural network and a heuristic neural architecture for autonomous humanoid robots. Cognit Comput. 2011;3:367–73.CrossRefGoogle Scholar
- 36.Zhu X, Welling M, Jin F, Lowengrub J. Predicting simulation parameters of biological systems using a Gaussian process model. Stat Anal Data Min: ASA Data Sci J (Special Issue: “Best Papers from the SLDM Competition”). 2012;5(6):509 – 522.Google Scholar
- 37.Zou L, Xu S, Ma Z, Lu J, Su W. Automatic removal of artifacts from attention deficit hyperactivity disorder electroencephalograms based on independent component analysis. Cognit Comput. 2013;5:225–33.CrossRefGoogle Scholar