A New Propagator for Two-Layer Neural Networks in Empirical Model Learning
This paper proposes a new propagator for a set of Neuron Constraints representing a two-layer network. Neuron Constraints are employed in the context of the Empirical Model Learning technique, that enables optimal decision making over complex systems, beyond the reach of most conventional optimization techniques. The approach is based on embedding a Machine Learning-extracted model into a combinatorial model. Specifically, a Neural Network can be embedded in a Constraint Model by simply encoding each neuron as a Neuron Constraint, which is then propagated individually. The price for such simplicity is the lack of a global view of the network, which may lead to weak bounds. To overcome this issue, we propose a new network-level propagator based on a Lagrangian relaxation, that is solved with a subgradient algorithm. The approach is tested on a thermal-aware dispatching problem on multicore CPUs, and it leads to a massive reduction of the size of the search tree, which is only partially countered by an increased propagation time.
KeywordsHide Layer Search Tree Hide Neuron Lagrangian Relaxation Subgradient Method
Unable to display preview. Download preview PDF.
- 2.Bartolini, A., Lombardi, M., Milano, M., Benini, L.: Optimization and Controlled Systems: A Case Study on Thermal Aware Workload Dispatching. Proc. of AAAI (2012)Google Scholar
- 3.Belew, R.K., McInerney, J., Schraudolph, N.N.: Evolving networks: Using the genetic algorithm with connectionist learning. In: Proc. of Second Conference on Artificial Life, pp. 511–547 (1991)Google Scholar
- 5.Frangioni, A., D’Antonio, G.: Deflected Conditional Approximate Subgradient Methods (Tech. Rep. TR-07-20). Technical report, University of Pisa (2007)Google Scholar
- 6.Ge, S.S., Hang, C.C., Lee, T.H., Zhang, T.: Stable adaptive neural network control. Springer Publishing Company, Incorporated (2010)Google Scholar
- 7.Glover, F., Kelly, J.P., Laguna, M.: New Advances for Wedding optimization and simulation. In: Proc. of WSC, pp. 255–260 (1999)Google Scholar
- 12.Ljung, L.: System identification. Wiley Online Library (1999)Google Scholar
- 13.Montana, D.J., Davis, L.: Training feedforward neural networks using genetic algorithms. In: Proc. of IJCAI, pp. 762–767 (1989)Google Scholar
- 14.Moore, J., Chase, J.S., Ranganathan, P.: Weatherman: Automated, Online and Predictive Thermal Mapping and Management for Data Centers. In: Proc. of IEEE ICAC, pp. 155–164. IEEE (2006)Google Scholar