Soft Computing

, Volume 21, Issue 21, pp 6369–6379 | Cite as

Optimization of electricity consumption in office buildings based on adaptive dynamic programming

  • Guang Shi
  • Qinglai Wei
  • Derong Liu
Methodologies and Application


In this paper, an optimization method based on adaptive dynamic programming is developed to improve the electricity consumption of rooms in office buildings through optimal battery management. Rooms in office buildings are generally divided into office rooms, computer rooms, storage rooms, meeting rooms, etc., and each category of rooms have different characteristics of electricity consumption, which is divided into electricity consumption from sockets, lights and air-conditioners in this paper. The developed method based on action-dependent heuristic dynamic programming is explained in detail, and different optimization strategies of electricity consumption in different categories of rooms are proposed in accordance with the developed method. Finally, a detailed case study on an office building is given to demonstrate the practical effect of the developed method.


Office buildings Electricity consumption optimization Battery management Optimal control Adaptive dynamic programming Neural networks 



This work was supported in part by the National Natural Science Foundation of China under Grants 61374105, 61233001, and 61273140.

Compliance with ethical standards

Conflict of interest

The authors declare that they have no conflict of interest.


  1. Amjadi Z, Williamson SS (2010) Power-electronics-based solutions for plug-in hybrid electric vehicle energy storage and management systems. IEEE Trans Industr Electron 57(2):608–616CrossRefGoogle Scholar
  2. Anvari-Moghaddam A, Monsef H, Rahimi-Kian A (2015) Cost-effective and comfort-aware residential energy management under different pricing schemes and weather conditions. Energy Build 86:782–793CrossRefGoogle Scholar
  3. Arsuaga-Rios M, Vega-Rodriguez MA (2015) Multi-objective energy optimization in grid systems from a brain storming strategy. Soft Comput 19(11):3159–3172CrossRefGoogle Scholar
  4. Bellman RE (1957) Dynamic programming. Princeton University Press, PrincetonzbMATHGoogle Scholar
  5. Bertsekas DP, Tsitsiklis JN (1996) Neuro-dynamic programming. Athena Scientific, BelmontzbMATHGoogle Scholar
  6. Boaro M, Fuselli D, Angelis FD, Liu D, Wei Q, Piazza F (2013) Adaptive dynamic programming algorithm for renewable energy scheduling and battery management. Cogn Comput 5(2):264–277CrossRefGoogle Scholar
  7. Data of real-time electricity price from ComEd Company, the United States. [Online].
  8. Enns R, Si J (2003) Helicopter trimming and tracking control using direct neural dynamic programming. IEEE Trans Neural Netw 14(8):929–939CrossRefGoogle Scholar
  9. Fuselli D, Angelis FD, Boaro M, Liu D, Wei Q, Squartini S, Piazza F (2013) Action dependent heuristic dynamic programming for home energy resource scheduling. Int J Electr Power Energy Syst 48(6):148–160CrossRefGoogle Scholar
  10. Guerrero JM, Loh PC, Lee TL, Chandorkar M (2013) Advanced control architectures for intelligent microgrids-part II: power quality, energy storage, and AC/DC microgrids. IEEE Trans Industr Electron 60(4):1263–1270CrossRefGoogle Scholar
  11. Huang T, Liu D (2013) A self-learning scheme for residential energy system control and management. Neural Comput Appl 22(2):259–269CrossRefGoogle Scholar
  12. Jaeger H (2001) The ‘echo state’ approach to analysing and training recurrent neural networks. German National Research Center for Information Technology, St. Augustin, Germany, Technical Report 148Google Scholar
  13. Jaeger H, Haas H (2004) Harnessing nonlinearity: predicting chaotic systems and saving energy in wireless communication. Science 304:78–80CrossRefGoogle Scholar
  14. Lewis FL, Vrabie D, Vamvoudakis KG (2012) Reinforcement learning and feedback control: using natural decision methods to design optimal adaptive controllers. IEEE Control Syst 32(6):76–105MathSciNetCrossRefGoogle Scholar
  15. Li C, Zhang G, Wang M, Yi J (2013) Data-driven modeling and optimization of thermal comfort and energy consumption using type-2 fuzzy method. Soft Comput 17(11):2075–2088CrossRefGoogle Scholar
  16. Liu D, Wei Q (2012) An iterative \(\varepsilon \)-optimal control scheme for a class of discrete-time nonlinear systems with unfixed initial state. Neural Netw 32:236–244CrossRefzbMATHGoogle Scholar
  17. Ma K, Hu G, Spanos CJ (2014) Distributed energy consumption control via real-time pricing feedback in smart grid. IEEE Trans Control Syst Technol 22(5):1907–1914CrossRefGoogle Scholar
  18. Molina D, Venayagamoorthy GK, Liang J, Harley RG (2013) Intelligent local area signals based damping of power system oscillations using virtual generators and approximate dynamic programming. IEEE Trans Smart Grid 4(1):498–508CrossRefGoogle Scholar
  19. Na J, Herrmann G (2014) Online adaptive approximate optimal tracking control with simplified dual approximation structure for continuous-time unknown nonlinear systems. IEEE/CAA J Autom Sin 1(4):412–422CrossRefGoogle Scholar
  20. Ni Z, He H (2013) Heuristic dynamic programming with internal goal representation. Soft Comput 17(11):2101–2108CrossRefGoogle Scholar
  21. Ni Z, He H, Wen J (2013) Adaptive learning in tracking control based on the dual critic network design. IEEE Trans Neural Netw Learn Syst 24(6):913–928CrossRefGoogle Scholar
  22. Prokhorov DV, Wunsch DC (1997) Adaptive critic designs. IEEE Trans Neural Netw 8(5):997–1007CrossRefGoogle Scholar
  23. Severini M, Squartini S, Piazza F (2013) Hybrid soft computing algorithmic framework for smart home energy management. Soft Comput 17(11):1983–2005CrossRefGoogle Scholar
  24. Shi G, Wei Q, Liu Y, Guan Q, Liu D (2015) Data-driven room classification for office buildings based on echo state network. The 2015 27th Chinese control and decision conference (CCDC), IEEE, pp 2602–2607Google Scholar
  25. Si J, Wang YT (2001) On-line learning control by association and reinforcement. IEEE Trans Neural Netw 12(2):264–276MathSciNetCrossRefGoogle Scholar
  26. Song R, Xiao W, Wei Q (2013) Multi-objective optimal control for a class of nonlinear time-delay systems via adaptive dynamic programming. Soft Comput 17(11):2109–2115CrossRefzbMATHGoogle Scholar
  27. Wang F, Zhang H, Liu D (2009) Adaptive dynamic programming: an introduction. Comput Intell 4(2):39–47CrossRefGoogle Scholar
  28. Wei Q, Liu D (2014) Adaptive dynamic programming for optimal tracking control of unknown nonlinear systems with application to coal gasification. IEEE Trans Autom Sci Eng 11(4):1020–1036CrossRefGoogle Scholar
  29. Wei Q, Wang F, Liu D, Yang X (2014) Finite-approximation-error based discrete-time iterative adaptive dynamic programming. IEEE Trans Cybern 44(12):2820–2833CrossRefGoogle Scholar
  30. Wei Q, Liu D, Shi G (2015a) A novel dual iterative Q-learning method for optimal battery management in smart residential environments. IEEE Trans Industr Electron 62(4):2509–2518CrossRefGoogle Scholar
  31. Wei Q, Liu D, Shi G, Liu Y (2015b) Multibattery optimal coordination control for home energy management systems via distributed iterative adaptive dynamic programming. IEEE Trans Industr Electron 62(7):4203–4214CrossRefGoogle Scholar
  32. Wei Q, Liu D, Xu Y (2016a) Neuro-optimal tracking control for a class of discrete-time nonlinear systems via generalized value iteration adaptive dynamic programming approach. Soft Comput 20(2):697–706CrossRefzbMATHGoogle Scholar
  33. Wei Q, Liu D, Liu Y, Song R (2016b) Optimal constrained self-learning battery sequential management in microgrids via adaptive dynamic programming. IEEE/CAA J Autom Sin (accepted)Google Scholar
  34. Wei Q, Lewis FL, Sun Q, Yan P, Song R (2016c) Discrete-time deterministic Q-learning: a novel convergence analysis. IEEE Trans Cybern. doi: 10.1109/TCYB.2016.2542923
  35. Werbos PJ (1977) Advanced forecasting methods for global crisis warning and models of intelligence. General Syst Yearb 22:25–38Google Scholar
  36. Werbos PJ (1991) A menu of designs for reinforcement learning over time. In: Miller WT, Sutton RS, Werbos PJ (eds) Neural networks for control. MIT Press, CambridgeGoogle Scholar
  37. Xu H, Jagannathan S (2013) Stochastic optimal controller design for uncertain nonlinear networked control system via neuro dynamic programming. IEEE Trans Neural Netw Learn Syst 24(3):471–484CrossRefGoogle Scholar
  38. Xu B, Yang C, Shi Z (2014a) Reinforcement learning output feedback NN control using deterministic learning technique. IEEE Trans Neural Netw Learn Syst 25(3):635–641Google Scholar
  39. Xu X, Lian C, Zuo L, He H (2014b) Kernel-based approximate dynamic programming for real-time online learning control: an experimental study. IEEE Trans Control Syst Technol 22(1):146–156CrossRefGoogle Scholar
  40. Zhao Q, Xu H, Jagannathan S (2014) Near optimal output feedback control of nonlinear discrete-time systems based on reinforcement neural network learning. IEEE/CAA J Autom Sin 1(4):372–384CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2016

Authors and Affiliations

  1. 1.The State Key Laboratory of Management and Control for Complex SystemsInstitute of Automation, Chinese Academy of SciencesBeijingChina
  2. 2.The School of Automation and Electrical EngineeringUniversity of Science and Technology BeijingBeijingChina

Personalised recommendations