A New Propagator for Two-Layer Neural Networks in Empirical Model Learning

  • Michele Lombardi
  • Stefano Gualandi
Part of the Lecture Notes in Computer Science book series (LNCS, volume 8124)

Abstract

This paper proposes a new propagator for a set of Neuron Constraints representing a two-layer network. Neuron Constraints are employed in the context of the Empirical Model Learning technique, that enables optimal decision making over complex systems, beyond the reach of most conventional optimization techniques. The approach is based on embedding a Machine Learning-extracted model into a combinatorial model. Specifically, a Neural Network can be embedded in a Constraint Model by simply encoding each neuron as a Neuron Constraint, which is then propagated individually. The price for such simplicity is the lack of a global view of the network, which may lead to weak bounds. To overcome this issue, we propose a new network-level propagator based on a Lagrangian relaxation, that is solved with a subgradient algorithm. The approach is tested on a thermal-aware dispatching problem on multicore CPUs, and it leads to a massive reduction of the size of the search tree, which is only partially countered by an increased propagation time.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Bartolini, A., Lombardi, M., Milano, M., Benini, L.: Neuron Constraints to Model Complex Real-World Problems. In: Lee, J. (ed.) CP 2011. LNCS, vol. 6876, pp. 115–129. Springer, Heidelberg (2011)CrossRefGoogle Scholar
  2. 2.
    Bartolini, A., Lombardi, M., Milano, M., Benini, L.: Optimization and Controlled Systems: A Case Study on Thermal Aware Workload Dispatching. Proc. of AAAI (2012)Google Scholar
  3. 3.
    Belew, R.K., McInerney, J., Schraudolph, N.N.: Evolving networks: Using the genetic algorithm with connectionist learning. In: Proc. of Second Conference on Artificial Life, pp. 511–547 (1991)Google Scholar
  4. 4.
    Chow, T.T., Zhang, G.Q., Lin, Z., Song, C.L.: Global optimization of absorption chiller system by genetic algorithm and neural network. Energy and Buildings 34(1), 103–109 (2002)CrossRefGoogle Scholar
  5. 5.
    Frangioni, A., D’Antonio, G.: Deflected Conditional Approximate Subgradient Methods (Tech. Rep. TR-07-20). Technical report, University of Pisa (2007)Google Scholar
  6. 6.
    Ge, S.S., Hang, C.C., Lee, T.H., Zhang, T.: Stable adaptive neural network control. Springer Publishing Company, Incorporated (2010)Google Scholar
  7. 7.
    Glover, F., Kelly, J.P., Laguna, M.: New Advances for Wedding optimization and simulation. In: Proc. of WSC, pp. 255–260 (1999)Google Scholar
  8. 8.
    Gopalakrishnan, K., Ph, D., Asce, A.M.: Neural Network Swarm Intelligence Hybrid Nonlinear Optimization Algorithm for Pavement Moduli Back-Calculation. Journal of Transportation Engineering 136(6), 528–536 (2009)CrossRefGoogle Scholar
  9. 9.
    Huang, W., Ghosh, S., Velusamy, S.: HotSpot: A compact thermal modeling methodology for early-stage VLSI design. IEEE Trans. on VLSI 14(5), 501–513 (2006)CrossRefGoogle Scholar
  10. 10.
    Jayaseelan, R., Mitra, T.: A hybrid local-global approach for multi-core thermal management. In: Proc. of ICCAD, pp. 314–320. ACM Press, New York (2009)CrossRefGoogle Scholar
  11. 11.
    Kiranyaz, S., Ince, T., Yildirim, A., Gabbouj, M.: Evolutionary artificial neural networks by multi-dimensional particle swarm optimization. Neural Networks 22(10), 1448–1462 (2009)CrossRefGoogle Scholar
  12. 12.
    Ljung, L.: System identification. Wiley Online Library (1999)Google Scholar
  13. 13.
    Montana, D.J., Davis, L.: Training feedforward neural networks using genetic algorithms. In: Proc. of IJCAI, pp. 762–767 (1989)Google Scholar
  14. 14.
    Moore, J., Chase, J.S., Ranganathan, P.: Weatherman: Automated, Online and Predictive Thermal Mapping and Management for Data Centers. In: Proc. of IEEE ICAC, pp. 155–164. IEEE (2006)Google Scholar
  15. 15.
    Zhang, G., Patuwo, B.E., Hu, M.Y.: Forecasting with artificial neural networks: The state of the art. International Journal of Forecasting 14(1), 35–62 (1998)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2013

Authors and Affiliations

  • Michele Lombardi
    • 1
  • Stefano Gualandi
    • 2
  1. 1.DISI, Dipartimento di Informatica: Scienza ed IngegneriaUniversità of BolognaItaly
  2. 2.Dipartimento di MatematicaUniversity of PaviaItaly

Personalised recommendations