Abstract
This paper presents a feedforward network estimation algorithm that addresses two issues, (i) avoiding local inferior minima to the performance criteria, and (ii) imposinga priori constraints to improve generalization and test economic hypotheses. The algorithm combines methods either previously developed or obviously beneficial but not yet combined. These involve combining linear least squares with simulated annealing, along with weight space reducing methods, to considerably improve its speed relative to pure simulated annealing. We present evidence on the algorithm's reliability at finding a global minimum. We also demonstrate how to constrain the estimation process to find networks that satisfy a givena priori condition. We provide examples of imposinga priori information to (i) prevent the trained network from making fundamental errors and (ii) to test economically interesting hypotheses.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Baba, N., 1989, A new approach for finding the global minimum of error function of neural networks.Neural Networks,2, 367–373.
Barron, A.R., 1991, Statistical properties of artificial neural networks. InProceedings NATO ASI on Nonparametric Functional Estimation. Kluwer Academic Publishers.
Barron, A.R., and Barron, R., 1988, Statistical learning networks: A unifying view. In Wegman, E. (ed.),Computing Science and Statistics: Proc. 20th Symp. Interface., pp. 192–203, Washington, D.C. American Statistical Association.
Bishop, C.M., 1990, Curvature-driven smoothing in backpropagation neural networks. InProceedings of the International Conference on Neural Networks, pp. 749–752 New York. IEEE Press.
Christensen, L., Jorgensen, D.W., and Lau, L., 1975, Transcendental logarithmic utility functions.American Economic Review,65, 367–383.
Corana, A., Marchesi, M., Martini, C., & Ridella, S., 1987, minimizing multimodal functions of continuous variables with the ‘simulated annealing’ algorithm.ACM Transactions on Mathematical Software,13(3), 262–280.
Davidson, R., & MacKinnon, J.G. (eds.), 1993,Estimation and Interface in Econometrics. Oxford University Press, New York, NY.
Dorsey, R., Johnson, J., and Mayer, W., 1992, A genetic algorithm for the training of feedforward neural networks. In Whinston, A., and Johnson, J. (eds.),Advances in Artificial Intelligence in Economics, Finance, and Management. JAI Press, Greenwich, CN. (forthcoming).
Gallant, A., 1981, On the bias in flexible functional forms and an essentially unbiased form.Journal of Econometrics,15, 211–245.
Gallant, A., and Golub, G., 1984, Imposing curvature restrictions on flexible functional forms.Journal of Econometrics,26, 295–321.
Gallant, A., & White, H. (1988).A Unified Theory of Estimation and Inference for Nonlinear Dynamic Models. Basil Blackwell, Oxford.
Geman, S., Bienenstock, E., & Doursat, R. (1992). Neural networks and the bias/variance dilemma.Neural Computation, 4, 1–58.
Geman, S., & Hwang, C.-R. (1986). Diffusions for global optimization.Siam Journal of Control and Optimization, 24, 1031–1043.
Gill, P., Murray, W., & Wright, M. (1981).Practical Optimization. Academic Press, New York, NY.
Goldberg, D. (1989).Genetic Algorithms in Search, Optimization and Machine Learning. Addison-Wesley.
Joerding, W., Li, Y., Hu, S., & Meador, J. (1992). Approximating production technologies with feedforward neural networks. In Whinston, A., & Johnson, J. (eds.),Advances in Artificial Intelligence in Economics, Finance, and Management. JAI Press, Greenwich, CN. forthcoming.
Joerding, W.H., & Meador, J.L. (1991). Encoding a priori information in feedforward networks.Neural Networks, 4(6), 847–856.
Joerding, W., & Li, Y. (1993). Approximating production functions with feedforward networks. Tech. rep., Washington State University, Department of Economics, Pullman, WA 99164.
Kirkpatrick, S., Gelatt, C., & Vecchi, M. (1983). Optimization by simulated annealing.Science, 2(220), 671–780.
Kruschke, J. (1989). Improving generalization in back-propagation networks with distributed bottlenecks. InInternational Joint Conference on Neural Networks, Vol. I, pp. 443–447 New York. IEEE Press.
Li, Y. (1993).Semi-Nonparametric Estimation and Testing of Consumer Demand Systems. Ph.D. thesis, Washington State University.
McInerney, J., Haines, K., Biafore, S., & Hecht-Nielsen, R. (1989). Back propagation error surfaces can have local minima. InIEEE Conference on Neural Networks.
Morgan, N., & Bourlard, H. (1990). Generalization and parameter estimation in feedforward nets: Some experiments. In Touretzky, D.S. (Eds.),Neural Information Processing Systems II Morgan Kaufmann, San Mateo, CA.
Psaltis, D., & Neifeld, M. (1988). The emergence of generalization in networks with constrained representations. InInternational Joint Conference on Neural Networks New York. IEEE Press.
Stinchcombe, M., & White, H. (1990). Approximating, and learning unknown mappings using multilayer feedforward networks with bounded weights. InProceedings of the International Joint Conference on Neural Networks, Vol. III, pp. 7–16 New York. IEEE Press.
Styblinski, M., & Tang, T.-S. (1990). Experiments in nonconvex optimization: Stochastic approximation with function smoothing and simulated annealing.Neural Networks, 3, 467–483.
Szu, H. (1986). Fast simulated annealing. InAIP Conference Proceedings 151: Neural Network for Computing.
Szu, H., & Hartley, R. (1987). Fast simulated annealing.Physics Letters A, 122, 157–162.
van Laarhoven, P. (1988).Theoretic and Computational Aspects of Simulated Annealing. Centrum voor Wiskunde en Informatica, Amsterdam, Netherlands.
van Laarhoven, P., & Aarts, E. (1987).Simulated Annealing: Theory and Applications. Kluwer Academic Publishers, Hingham, MA.
Wasserman, P.D. (1988). Combined backpropagation/cauchy machine. InNeural Networks: Abstracts of the First INNS Meeting, p. 556 Elmsford, NY. Pergamon Press.
Webb, A., & Lowe, D. (1988). A hybrid optimization strategy for adaptive feedforward layered networks. Tech. rep. 4193, Royal Signals and Radar Establishment Memorandum, Ministry of Defence, Malvern, UK.
White, H. (1989). Learning in artificial neural networks: A statistical perspective.Neural Computing, 1, 425–464.
White, H. (1990). Connectionist nonparametric regression: Multilayer feedforward networks can learn arbitrary mappings.Neural Networks, 3(5), 535–549.
Young, D. (1982). Economics of fertilizer use. InThe Fertilizer Handbook, pp. 209–221. The Fertilizer Institute, Washington DC.
Author information
Authors and Affiliations
Additional information
This research was partially funded by National Science Foundation Grant No. SES-9022773 and by an equipment grant from Torque Systems, Inc.
Rights and permissions
About this article
Cite this article
Joerding, W., Li, Y. Global estimation of feedforward networks with a priori constraints. Comput Econ 7, 73–87 (1994). https://doi.org/10.1007/BF01299568
Received:
Issue Date:
DOI: https://doi.org/10.1007/BF01299568