Skip to main content
SpringerLink
Log in

Optimization of electricity consumption in office buildings based on adaptive dynamic programming

  • Methodologies and Application
  • Published:
Soft Computing Aims and scope Submit manuscript

Abstract

In this paper, an optimization method based on adaptive dynamic programming is developed to improve the electricity consumption of rooms in office buildings through optimal battery management. Rooms in office buildings are generally divided into office rooms, computer rooms, storage rooms, meeting rooms, etc., and each category of rooms have different characteristics of electricity consumption, which is divided into electricity consumption from sockets, lights and air-conditioners in this paper. The developed method based on action-dependent heuristic dynamic programming is explained in detail, and different optimization strategies of electricity consumption in different categories of rooms are proposed in accordance with the developed method. Finally, a detailed case study on an office building is given to demonstrate the practical effect of the developed method.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12

Similar content being viewed by others

References

  • Amjadi Z, Williamson SS (2010) Power-electronics-based solutions for plug-in hybrid electric vehicle energy storage and management systems. IEEE Trans Industr Electron 57(2):608–616

    Article  Google Scholar 

  • Anvari-Moghaddam A, Monsef H, Rahimi-Kian A (2015) Cost-effective and comfort-aware residential energy management under different pricing schemes and weather conditions. Energy Build 86:782–793

    Article  Google Scholar 

  • Arsuaga-Rios M, Vega-Rodriguez MA (2015) Multi-objective energy optimization in grid systems from a brain storming strategy. Soft Comput 19(11):3159–3172

    Article  Google Scholar 

  • Bellman RE (1957) Dynamic programming. Princeton University Press, Princeton

    MATH  Google Scholar 

  • Bertsekas DP, Tsitsiklis JN (1996) Neuro-dynamic programming. Athena Scientific, Belmont

    MATH  Google Scholar 

  • Boaro M, Fuselli D, Angelis FD, Liu D, Wei Q, Piazza F (2013) Adaptive dynamic programming algorithm for renewable energy scheduling and battery management. Cogn Comput 5(2):264–277

    Article  Google Scholar 

  • Data of real-time electricity price from ComEd Company, the United States. [Online]. https://rrtp.comed.com/live-prices/?

  • Enns R, Si J (2003) Helicopter trimming and tracking control using direct neural dynamic programming. IEEE Trans Neural Netw 14(8):929–939

    Article  Google Scholar 

  • Fuselli D, Angelis FD, Boaro M, Liu D, Wei Q, Squartini S, Piazza F (2013) Action dependent heuristic dynamic programming for home energy resource scheduling. Int J Electr Power Energy Syst 48(6):148–160

    Article  Google Scholar 

  • Guerrero JM, Loh PC, Lee TL, Chandorkar M (2013) Advanced control architectures for intelligent microgrids-part II: power quality, energy storage, and AC/DC microgrids. IEEE Trans Industr Electron 60(4):1263–1270

    Article  Google Scholar 

  • Huang T, Liu D (2013) A self-learning scheme for residential energy system control and management. Neural Comput Appl 22(2):259–269

    Article  Google Scholar 

  • Jaeger H (2001) The ‘echo state’ approach to analysing and training recurrent neural networks. German National Research Center for Information Technology, St. Augustin, Germany, Technical Report 148

  • Jaeger H, Haas H (2004) Harnessing nonlinearity: predicting chaotic systems and saving energy in wireless communication. Science 304:78–80

    Article  Google Scholar 

  • Lewis FL, Vrabie D, Vamvoudakis KG (2012) Reinforcement learning and feedback control: using natural decision methods to design optimal adaptive controllers. IEEE Control Syst 32(6):76–105

    Article  MathSciNet  Google Scholar 

  • Li C, Zhang G, Wang M, Yi J (2013) Data-driven modeling and optimization of thermal comfort and energy consumption using type-2 fuzzy method. Soft Comput 17(11):2075–2088

    Article  Google Scholar 

  • Liu D, Wei Q (2012) An iterative \(\varepsilon \)-optimal control scheme for a class of discrete-time nonlinear systems with unfixed initial state. Neural Netw 32:236–244

    Article  MATH  Google Scholar 

  • Ma K, Hu G, Spanos CJ (2014) Distributed energy consumption control via real-time pricing feedback in smart grid. IEEE Trans Control Syst Technol 22(5):1907–1914

    Article  Google Scholar 

  • Molina D, Venayagamoorthy GK, Liang J, Harley RG (2013) Intelligent local area signals based damping of power system oscillations using virtual generators and approximate dynamic programming. IEEE Trans Smart Grid 4(1):498–508

    Article  Google Scholar 

  • Na J, Herrmann G (2014) Online adaptive approximate optimal tracking control with simplified dual approximation structure for continuous-time unknown nonlinear systems. IEEE/CAA J Autom Sin 1(4):412–422

    Article  Google Scholar 

  • Ni Z, He H (2013) Heuristic dynamic programming with internal goal representation. Soft Comput 17(11):2101–2108

    Article  Google Scholar 

  • Ni Z, He H, Wen J (2013) Adaptive learning in tracking control based on the dual critic network design. IEEE Trans Neural Netw Learn Syst 24(6):913–928

    Article  Google Scholar 

  • Prokhorov DV, Wunsch DC (1997) Adaptive critic designs. IEEE Trans Neural Netw 8(5):997–1007

    Article  Google Scholar 

  • Severini M, Squartini S, Piazza F (2013) Hybrid soft computing algorithmic framework for smart home energy management. Soft Comput 17(11):1983–2005

    Article  Google Scholar 

  • Shi G, Wei Q, Liu Y, Guan Q, Liu D (2015) Data-driven room classification for office buildings based on echo state network. The 2015 27th Chinese control and decision conference (CCDC), IEEE, pp 2602–2607

  • Si J, Wang YT (2001) On-line learning control by association and reinforcement. IEEE Trans Neural Netw 12(2):264–276

    Article  MathSciNet  Google Scholar 

  • Song R, Xiao W, Wei Q (2013) Multi-objective optimal control for a class of nonlinear time-delay systems via adaptive dynamic programming. Soft Comput 17(11):2109–2115

    Article  MATH  Google Scholar 

  • Wang F, Zhang H, Liu D (2009) Adaptive dynamic programming: an introduction. Comput Intell 4(2):39–47

    Article  Google Scholar 

  • Wei Q, Liu D (2014) Adaptive dynamic programming for optimal tracking control of unknown nonlinear systems with application to coal gasification. IEEE Trans Autom Sci Eng 11(4):1020–1036

    Article  Google Scholar 

  • Wei Q, Wang F, Liu D, Yang X (2014) Finite-approximation-error based discrete-time iterative adaptive dynamic programming. IEEE Trans Cybern 44(12):2820–2833

    Article  Google Scholar 

  • Wei Q, Liu D, Shi G (2015a) A novel dual iterative Q-learning method for optimal battery management in smart residential environments. IEEE Trans Industr Electron 62(4):2509–2518

    Article  Google Scholar 

  • Wei Q, Liu D, Shi G, Liu Y (2015b) Multibattery optimal coordination control for home energy management systems via distributed iterative adaptive dynamic programming. IEEE Trans Industr Electron 62(7):4203–4214

    Article  Google Scholar 

  • Wei Q, Liu D, Xu Y (2016a) Neuro-optimal tracking control for a class of discrete-time nonlinear systems via generalized value iteration adaptive dynamic programming approach. Soft Comput 20(2):697–706

    Article  MATH  Google Scholar 

  • Wei Q, Liu D, Liu Y, Song R (2016b) Optimal constrained self-learning battery sequential management in microgrids via adaptive dynamic programming. IEEE/CAA J Autom Sin (accepted)

  • Wei Q, Lewis FL, Sun Q, Yan P, Song R (2016c) Discrete-time deterministic Q-learning: a novel convergence analysis. IEEE Trans Cybern. doi:10.1109/TCYB.2016.2542923

  • Werbos PJ (1977) Advanced forecasting methods for global crisis warning and models of intelligence. General Syst Yearb 22:25–38

    Google Scholar 

  • Werbos PJ (1991) A menu of designs for reinforcement learning over time. In: Miller WT, Sutton RS, Werbos PJ (eds) Neural networks for control. MIT Press, Cambridge

    Google Scholar 

  • Xu H, Jagannathan S (2013) Stochastic optimal controller design for uncertain nonlinear networked control system via neuro dynamic programming. IEEE Trans Neural Netw Learn Syst 24(3):471–484

    Article  Google Scholar 

  • Xu B, Yang C, Shi Z (2014a) Reinforcement learning output feedback NN control using deterministic learning technique. IEEE Trans Neural Netw Learn Syst 25(3):635–641

  • Xu X, Lian C, Zuo L, He H (2014b) Kernel-based approximate dynamic programming for real-time online learning control: an experimental study. IEEE Trans Control Syst Technol 22(1):146–156

    Article  Google Scholar 

  • Zhao Q, Xu H, Jagannathan S (2014) Near optimal output feedback control of nonlinear discrete-time systems based on reinforcement neural network learning. IEEE/CAA J Autom Sin 1(4):372–384

    Article  Google Scholar 

Download references

Acknowledgments

This work was supported in part by the National Natural Science Foundation of China under Grants 61374105, 61233001, and 61273140.

Author information

Authors and Affiliations

  1. The State Key Laboratory of Management and Control for Complex Systems, Institute of Automation, Chinese Academy of Sciences, Beijing, 100190, China

    Guang Shi & Qinglai Wei

  2. The School of Automation and Electrical Engineering, University of Science and Technology Beijing, Beijing, 100083, China

    Derong Liu

Authors
  1. Guang Shi
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Qinglai Wei
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Derong Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Qinglai Wei.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Additional information

Communicated by V. Loia.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Shi, G., Wei, Q. & Liu, D. Optimization of electricity consumption in office buildings based on adaptive dynamic programming. Soft Comput 21, 6369–6379 (2017). https://doi.org/10.1007/s00500-016-2194-y

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00500-016-2194-y

Keywords

Search

Navigation