# A learning strategy for developing neural networks using repetitive observations

- 21 Downloads

## Abstract

Neural networks can model system behaviors by learning past system observations. As system observations are usually collected by human judgments, physical experiments or sensor measures, they can be inherently imprecise and inconsistent over time. System behaviors can be learned more completely from repetitive observations. However, repetitive observations can be very different due to system or measurement uncertainty. If abnormal observations are used for developing neural networks, spurious behaviors can be learnt and the neural networks are likely to generate spurious prediction. If abnormal observations are excluded, important system behaviors can partially be ignored. In this paper, a novel strategy is proposed to develop neural networks by learning repetitive observations. Numerous neural networks are developed individually based on either abnormal or normal observations. The predictions generated based on the individual neural networks are integrated to a single prediction. Analytical proof indicates that the overall observation uncertainty involved on the proposed learning strategy is less than the uncertainty involved on the general ones. As less uncertainty is involved, more effective learning can be performed on the proposed strategy. Two case studies are conducted in order to evaluate the effectiveness of the proposed learning strategy, where the two case studies are involved data collection from either sensor measures or human evaluations. Numerical results indicate that the proposed strategy can generate better neural networks which have higher fitting capability to captured observations and higher generalization capability to uncaptured samples.

## Keywords

Empirical modeling Repetitive observations Imprecise or uncertain measures Subjective evaluations or assessments## Notes

## Compliance with ethical standards

## Conflict of interest

The authors declare that they have no conflict of interest.

## Human and animal rights

This article does not contain any studies with human participants performed by any of the authors.

## References

- Abdi J, Moshiri B, Abdulhai B (2013) Emotional temporal difference Q-learning signals in multi-agent system cooperation: real case studies. IET Intel Transport Syst 7(3):315–326CrossRefGoogle Scholar
- Akkuzu N, Akçay H (2011) The design of a learning environment based on the theory of multiple intelligence and the study its effectiveness on the achievements, attitudes and retention of students. Procedia Comput Sci 3:1003–1008CrossRefGoogle Scholar
- Al-Abdullaha KIA, Abdia H, Lima CP, Yassinb WA (2018) Force and temperature modelling of bone milling using artificial neural networks. Measurement 116:25–37CrossRefGoogle Scholar
- Aw A, Rascle M (2000) Resurrection of "second order" models of traffic flow. Siam J Appl Math 60(3):916–938MathSciNetCrossRefMATHGoogle Scholar
- BT.500-11 I.-R (2002) IUT, Methodology for the subjective assessment of the quality of television pictures (Recommendation BT.500-11), International Telecommunication Union, 2002Google Scholar
- Bagheri A, Pistone E, Rizzo P (2015) Outlier analysis and artificial neural network for the noncontact nondestructive evaluation of immersed plates. Res Nondestruct Eval 26:154–173CrossRefGoogle Scholar
- Barrow D, Kourentze N (2018) The impact of special days in call arrivals forecasting: a neural network approach to modelling special days. Eur J Oper Res 264:967–977MathSciNetCrossRefMATHGoogle Scholar
- Bartlett MS, Kendall DG (1946) The statistical analysis of variance heterogeneity and the logarithmic transformation. J Roy Stat Soc 8(1):128–138MathSciNetMATHGoogle Scholar
- Brazier ER, Beven JK, Freer J, Rowan JS (2000) Equifinality and uncertainty in physically based soil erosion models: application of the glue methodology to WEPP—the water erosion prediction project—for sites in the UK and USA. Earth Surface Process Landf 25(8):825–845CrossRefGoogle Scholar
- Carson ER, Cobelli C, Finkelstein L (1983) The mathematical modelling of metabolic and endocrine systems: model formulation, identification and validation. Wiley, New YorkGoogle Scholar
- Chan KY, Dillon TS, Singh J, Chang E (2012) Neural-network-based models for short-term traffic flow forecasting using a hybrid exponential smoothing and levenberg-marquardt algorithm. IEEE Trans Intell Transp Syst 13(2):644–654CrossRefGoogle Scholar
- Dinga L, Fang W, Luoa H, Lovec PED, Zhonga B, Ouyang X (2018) A deep hybrid learning model to detect unsafe behavior: integrating convolution neural networks and long short-term memory. Autom Constr 86:118–124CrossRefGoogle Scholar
- Dubois D, Foulloy L, Mauris G, Prade H (2004) Probability-possibility transformations, triangular fuzzy sets, and probabilistic inequalities. Reliab Comput 10(4):273–297MathSciNetCrossRefMATHGoogle Scholar
- Engelke U, Maeder A, Zepernick HJ (2012) Human observer confidence in image quality assessment. Sig Process Image Commun 27:935–947CrossRefGoogle Scholar
- Esbensen KH, Wagner C (2016) Sampling quality assessment: the replication experiment. Sampl Column 28(1):20–25Google Scholar
- Finkelstein L, Morawski RZ (2003) Fundamental concepts of measurement. Measurement 34(1):1–2CrossRefGoogle Scholar
- Gunvig A, Hansen F, Borggaard C (2013) A mathematical model for predicting growth/no-growth of psychrotrophic C. botulinum in meat products with five variables. Food Control 29(2):309–317CrossRefGoogle Scholar
- ISO (1993) ISO standards, Uncertainty of measurement–Part 3: Guide to the expression of uncertainty in measurement, ISO/IEC Guide 98-3:2008(en), International Organization for Standardization, 1995Google Scholar
- Kendall M, Stuart A (1977) The advanced theory of statistics. Griffin, LondonMATHGoogle Scholar
- Kharoufeh JP, Chandra MJ (2002) Statistical tolerance analysis for non-normal or correlated normal component characteristics. Int J Prod Res 40(2):337–352CrossRefMATHGoogle Scholar
- Ko CN (2012) Identification of nonlinear systems with outliers using wavelet neural networks based on annealing dynamical learning algorithm. Eng Appl Artif Intell 25:533–543CrossRefGoogle Scholar
- Kung CH, Yang WS, Kung CM (2011) A study on image quality assessment using neural networks and structure similarity. J Comput 6(10):2221–2228CrossRefGoogle Scholar
- Kuo SS, Ko CN (2014) Adaptive annealing learning algorithm-based robust wavelet neural networks for function approximation with outliers. Artif Life Robot 19:186–192CrossRefGoogle Scholar
- Li Y, Po LM, Xu X, Feng L, Yuan F (2015) No-reference image quality assessment with shearlet transform and deep neural networks. Neurocomputing 154:94–109CrossRefGoogle Scholar
- Li J, Zou L, Yan J, Deng D, Qu T, Xie G (2016) No-reference image quality assessment using Prewitt magnitude based on convolutional neural networks. Signal Image Video Process 10(4):609–616CrossRefGoogle Scholar
- Liu J, Gader P (2002) Neural networks with enhanced outlier rejection ability for off-line handwritten word recognition. Pattern Recognit 35:2061–2071CrossRefMATHGoogle Scholar
- Liu Y, Sun W, Yuan Z, Fish J (2016) A nonlocal multiscale discrete-continuum model for predicting mechanical behavior of granular materials. Int J Numer Methods Eng 105(2):129–160MathSciNetCrossRefMATHGoogle Scholar
- Marziliano P, Dufaux F, Winkler S, & Ebrahimi T (2002) A no-reference perceptual blur metric. In: Paper presented at the proceedings of IEEE international conference on image processingGoogle Scholar
- Mauris G (2010) Transformation of bimodal probability distributions into possibility distributions. IEEE Trans Instrum Meas 59(1):39–47CrossRefGoogle Scholar
- Mauris G (2013) A review of relationships between possibility and probability representations of uncertainty in measurement. IEEE Trans Instrum Meas 62(3):622–632CrossRefGoogle Scholar
- Michael AJ (1997) Testing prediction methods: earthquake clustering versus the Poisson model. Geophys Res Lett 24(15):1891–1894CrossRefGoogle Scholar
- Miyahara M, Kotani K, Algazi VR (1998) Objective picture quality scale (PQS) for image coding. IEEE Trans Commun 46(9):1215–1226CrossRefGoogle Scholar
- Morawski RZ (2013) An application-oriented mathematical meta-model of measurement. Measurement 46:3753–3765CrossRefGoogle Scholar
- Myers RH, Montgomery DC (1995) Response surface methodology: process and product optimization using designed experiments. Wiley, New YorkMATHGoogle Scholar
- Omar F, Brousseau E, Elkaseer A, Kolew A, Prokopovich P, Dimov S (2014) Development and experimental validation of an analytical model to predict the demoulding force in hot embossing. J Micromech Microeng 24:1–11CrossRefGoogle Scholar
- Passow BN, Elizondo D, Chiclana F, Witheridge S (2013) Adapting traffic simulation for traffic management: a neural network approach. In: Paper presented at the proceedings of the IEEE conference on intelligent transportation systemsGoogle Scholar
- Pearson RK (2002) Outliers in process modeling and identification. IEEE Trans Control Syst Technol 10(1):55–63CrossRefGoogle Scholar
- Poggio T, Girosi F (1990) Networks for approximation and learning. Proc IEEE 78(9):1481–1497CrossRefMATHGoogle Scholar
- Reznik L, Dabke KP (2004) Measurement models: application of intelligent methods. Measurement 35(1):47–58CrossRefGoogle Scholar
- Saha S, Vemuri R (2000) An analysis on the effect of image features on lossy coding performance. IEEE Signal Process Lett 7:104–107CrossRefGoogle Scholar
- Sakar CO, Kursun O (2017) Discriminative feature extraction by a neural implementation of canonical correlation analysis. IEEE Trans Neural Netw Learn Syst 28(1):164–176CrossRefGoogle Scholar
- Schadschneider A (2002) Traffic flow: a statistical physics point of view. Phys A 313:153–187MathSciNetCrossRefMATHGoogle Scholar
- Scholz F (1995) Tolerance Stack Analysis Methods A Critical Review. Research and Technology Report, Boeing Information & Support Services, 1995Google Scholar
- Tan MC, Wong C, Xu JM, Guan ZR (2009) An aggregation approach to short-term traffic flow prediction. IEEE Trans Intell Transp Syst 10(1):60–69CrossRefGoogle Scholar
- Tomić AS, Antanasijevic D, Ristić M, Grujic AP, Pocajt V (2018) A linear and non-linear polynomial neural network modeling of dissolved oxygen content in surface water: inter- and extrapolation performance with inputs’ significance analysis. Sci Total Environ 610–611:1038–1046Google Scholar
- Wanas N, Auda G, Kamel M S, & Karray F (1998) On the optimal number of hidden nodes in a neural network. In: Paper presented at the proceedings of the IEEE Canadian conference on electrical and computer engineeringGoogle Scholar
- Wang Z, Sheikh H R, & Bovik A C (2002) No-reference perceptual quality assessment of JPEG compressed images. In: Paper presented at the proceedings of IEEE international conference on image processingGoogle Scholar
- Wang Z, Bovik AC, Sheikh HR, Simoncelli EP (2004) Image quality assessment: from error visibility to structural similarity. IEEE Trans Image Process 13(4):600–612CrossRefGoogle Scholar
- Woo YM (2015) Image quality evaluation using deep learning. (BEng BEng Thesis), Curtin UniversityGoogle Scholar
- Yu P, Low MY, Zhou W (2018) Development of a partial least squares-artificial neural network (PLS-ANN) hybrid model for the prediction of consumer liking scores of ready-to-drink green tea beverages. Food Res Int 103:68–75CrossRefGoogle Scholar
- Zadeh LA (1999) Fuzzy sets as a basis for a theory of possibility. Fuzzy Sets Syst 100:9–34CrossRefGoogle Scholar
- Zaric A, Tatalovic N, Brajkovic N, Hlevnjak H, Loncaric M, Dumic E, Grgic S (2012) VCL@FER image quality assessment database. Automatika 53(4):344–354Google Scholar
- Zhang J, Kamel AE (2018) Virtual traffic simulation with neural network learned mobility model. Adv Eng Softw 115:103–111CrossRefGoogle Scholar
- Zhang C, Luo J, Wang B (1999) Statistical tolerance synthesis using distribution function zones. Int J Prod Res 37(17):3995–4006CrossRefMATHGoogle Scholar