1 Introduction

Due to the ongoing advancement of urbanization, there has been an increase in energy consumption during the past few decades (Bhosale and Gadekar 2014). 40% of the world's electrical energy is consumed by the building sector. The generation of electricity is closely related to the emission of carbon dioxide (CO2) due to the predominant use of fossil fuels in many electricity generation processes. Fossil fuel-based power plants, such as those fueled by coal, oil, and natural gas, release significant amounts of CO2 into the atmosphere when these fuels are burned to produce electricity. Electricity generation produces CO2 emissions that lead to the greenhouse effect, which traps heat in the atmosphere and causes a climatic change on a worldwide scale. The more electricity generated from fossil fuel sources, the higher the associated CO2 emissions. Burning fossil fuels for power generation accounts for over 40% of energy-related CO2 emissions. Therefore, predicting building energy consumption has gained the utmost importance in tackling the swift rise of CO2 emissions. Furthermore, this capability empowers power engineers to resolve the supply and demand gap. Also, this helps to render proficient and impactful choices while designing, constructing, and operating building structures (Dong et al. 2018). This can be achieved by improving the overall performance of energy prediction in buildings (Zhang et al. 2020a). A minimal improvement in forecast accuracy can also result in savings of millions of rupees (Zor et al. 2020). Hence, accurate load forecasts are also driven by a significant incentive related to economic motivation (Xu et al. 2019). Numerous methodologies and computational techniques have been developed to increase forecast accuracy (Agyeman et al. 2020).

The energy forecasting horizon is usually classified by time horizon into three categories: long-term (more than a year), mid-term (from a month to a year), and short-term (from one day to a month) forecasting (Kiprijanovska et al. 2020; Liu et al. 2020). Forecasting methods can be classified as supervised learning (neural network models such as support vector machines and classifiers (Das et al. 2020), extreme learning machines (ELM; Fu 2018), random forests (Wang and Srinivasan 2017), and DL algorithms) and unsupervised learning (a variety of clustering algorithms such as k-means clustering (Singh and Dwivedi 2018), fuzzy clustering (Cheng and Li 2008), and other improved clustering methods). The choice of forecasting method depends on the forecasting horizon, which refers to the period for which predictions are made. Short-term forecasting often uses methods like autoregressive integrated moving average (ARIMA; Farzana et al. 2014), exponential smoothing (Liu et al. 2016), or recurrent neural networks (RNNs). Medium-term forecasting may employ seasonal ARIMA, state space models, or machine learning algorithms. Long-term forecasting (years or more) relies on trend extrapolation, econometric models, or causal models. However, the selection should also consider data availability, quality, underlying assumptions, and specific problem characteristics. Combinations and overlaps of methods can occur based on the forecasting task. The literature has extensive research on predicting energy usage for both commercial and residential buildings. Recently, several academics have concentrated on enhancing the effectiveness and accuracy of electricity consumption forecasts using deep learning architectures (Bandic and Kevric 2018; Munkhdalai and Munkhdalai 2019). Most of the research community uses deep learning-based techniques because of their excellent results on unsupervised problems. DL is a complex computational model designed with multiple hidden layers that use features as input to represent data with different abstractions. DL algorithms are used for various learning tasks, especially unsupervised learning. Load forecasting is one application that has benefited from DL algorithms in many kinds of literature. Recently, RNNs and convolution neural networks (CNNs) are two powerful architectures proposed in the literature for the analysis of time series data. For instance, a study presented in Mocanu et al. 2016 proposed a deep CNN network for day-ahead load forecasting and compared the results with an (ELM), ARIMA, CNN, and RNN. Several studies also used RNN models for electricity load prediction, whereas in paper (Ali et al. 2016) used RNN, gated recurrent unit (GRU), and long-short term memory (LSTM) models for electricity load prediction in Turkey and extensively decreased the error. The electricity consumption data is time-series data, which comprises spatial and temporal information. The CNN models perform well for spatial information extraction but are insufficient for temporal information, whereas the RNN models are insufficient for spatial information and can learn temporal information. Therefore, to develop an optimal model for electricity load prediction, hybrid models are introduced in the recent literature.

For instance, another study (Kolehmainen et al. 2015) developed a hybrid model combining CNN with LSTM for short-term load prediction and compared their results with GRU, attention LSTM, LSTM, and bidirectional LSTM. In the paper (Manembu et al. 2018), they also developed a hybrid model with a combination of CNN and multi-layer bidirectional LSTM and compared their results with bidirectional LSTM, LSTM, and CNN-LSTM. Similarly, another study presented in Choksi et al. (2020) integrated a CNN with an LSTM auto-encoder and compared the final results with LSTM, LSTM autoencoder, and CNN-LSTM. Moreover, studies (Jain et al. 2019; Singh and Yassine 2019) presented the performance of a CNN-GRU based model for electricity forecasting. A study (Pirbazari and Sharma 2021) describes solar power forecasting in addition to load forecasting which uses the CNN-LSTM hybrid model with the help of climatic scenarios, and during experimental analysis, they have classified the error based on sunny days, and cloudy days. In recent years, hybrid CNN with CNN-ESN (echo state network) and LSTM-AE (autoencoder) have the potential to enhance the overall efficiency of the existing forecasting models. In a paper (Khan et al. 2020), they proposed a hybrid CNN with the LSTM-AE model. A CNN model extracts features from the input data, which are fed to an LSTM encoder to produce an encoded sequence. The encoded sequence is decoded by another subsequent LSTM decoder and forwarded to the final dense layer for energy prediction. In the paper (Liu et al. 2022), the author proposed the temporal convolutional network model (TCN) architecture and proved with better accuracy than LSTM. Most of the forecasting methods are designed to forecast a single or small group of time series. However, in the paper (Rick and Berton 2020), the author focused on short-term forecasting over many time series of unequal lengths. The author (Rick and Berton 2020) proposed a deep learning approach based on LSTM, CNN, and auto-encoder for training only one model for the many time series. Compared to TCN, they achieved a smaller error rate. The author suggests a hybrid combination technique, called CESN, based on a deep learning model combining Convolutional Neural Networks and Echo State Networks, to produce a high-quality prediction of power consumption (Ghimire et al. 2023). Peak handling is another difficult area for researchers in load predictions. The occurrences of the peaks are irregular, and their time of occurrence cannot be determined apriori because the customer’s consumption behavior is uncertain. This poses a challenge in modeling peaks as there is only a minimal dependency of present consumption on its past data. Thus the peaks have to be modeled precisely. The available state-of-the-art techniques (Gao et al. 2012) are not able to make accurate predictions at peak load conditions as machine learning (ML) algorithms poorly predict peak for hourly mean load prediction. In the paper (Sulaiman et al. 2022), the author proposed a novel hybrid method based on EMD (extreme mode decomposition) with ELM to handle peak residential loads. In the paper (Imani and Ghassemian 2019), wavelet decomposition is applied to remove redundant values. In addition, a collaborative representation is introduced including information on the neighboring points (previous and future time instances) of the considered load point. Hybrid models exploited well in other fields like the finance sector to predict stock market trends are not explored that much in power consumption forecasting. The finance sector benefits from combining LSTM models with Generative Adversarial Networks (GANs; Zhang, et al. 2015; Polamuri et al. 2022) to predict stock market trends. Such hybrid models can be applied to capture the stochastic nature of power consumption data and to generate synthetic datasets for augmentation. The performance of the hybrid model is very promising, reaching the highest level of accuracy. However, the best prediction of the electricity load needs further improvement by choosing an appropriate method.

The main contributions of this paper are:

  • Literature review of the previous research works for energy consumption forecasting in smart buildings, exploring their contribution and inference

  • Detailed framework for power forecasting

  • Analysis of the various methodologies used in the forecasting of energy consumption in buildings from various perspectives, their findings, and limitations

  • A research gap in the existing literature is identified and suggestions are given for the new researchers working in this area

The remainder of the paper is organized in the following way. Section 2 discusses the common framework for power forecasting, In Sect. 3, different DL algorithm is discussed based on energy consumption forecasting Then the paper is followed by Sects. 4 and 5. Section 4 briefly discusses about application of DL using forecasting horizon and Sect. 5 discusses evaluation metrics. Section 6 describes a detailed discussion of energy forecasting methodologies. Finally, the paper concludes with the identification of the research gap in Sect. 7 and concludes in Sect. 8.

2 Inference from literature

The literature survey has provided valuable insights into the field of energy consumption forecasting in smart buildings. The diverse body of research underscores several key trends and perspectives that significantly contribute to this critical domain. Power forecasting needs to analyze large amounts of historical power data, especially for high-resolution forecasting. Different ML techniques have been applied for predicting future electricity consumption for a more ideal solution, which lacks in handling large datasets. This gap is filled by DL based techniques because they can handle and learn from massive datasets more efficiently than traditional ML models First and foremost, data-driven approaches have emerged as a fundamental pillar in energy consumption forecasting. Leveraging advanced machine learning, deep learning, and data analytics techniques, harness historical data, weather information, occupancy patterns, and other relevant variables to create accurate forecasts, offering a promising avenue for improving energy management in smart buildings.

Previous research in energy consumption forecasting for smart buildings has revealed several notable trends and unique perspectives. Consumers are now becoming prosumers, who not only consume electricity but also generate it, often through renewable energy sources like solar panels, wind turbines, or small-scale hydroelectric systems. Prosumers are essentially consumers who become self-sufficient energy producers, and they may feed surplus energy back into the grid. While the literature primarily focuses on load forecasting, recent trends in renewable energy forecasting should be considered. The integration of renewable energy sources, such as solar and wind power, into the energy grid requires accurate forecasting to ensure grid stability and efficient energy management. Accurate forecasting is crucial for optimizing energy storage systems, which play a key role in grid integration of renewables and in managing demand spikes, such as during peak loads. Peak load prediction remains a challenge, and most machine learning algorithms struggle to provide accurate forecasts for peak consumption. Improving peak load prediction is crucial for efficient grid management. The literature suggests the potential of hybrid methods, such as combining EMD with ELM, wavelet decomposition, and collaborative representation, to enhance the accuracy of residential load forecasting.

3 Basic forecasting framework

Predicting power consumption is critical for smart grids to manage and conserve energy, avoid waste, and use it efficiently. Due to the influence of numerous unpredictable situations or the noisy disordering of smart meter data, it is difficult to anticipate power usage accurately, and the methods employed can sometimes yield inaccurate results. Moreover, various techniques based on conventional networks have been developed, but they cannot predict energy demand efficiently (Cai et al. 2019). Conventional networks have problems related to short-term memory and learning from scratch. These problems are easily solved using LSTM, a special type of RNN that has attracted a lot of attention in the field of deep learning. LSTM networks have a unique architecture that includes memory cells and gates. The memory cells allow the network to store and retain information over longer sequences, enabling the model to capture and learn from long-term dependencies in the data. The common schematic diagram used for forecasting power is shown in Fig. 1.

Fig. 1
figure 1

Basic block diagram for forecasting power

Full size image

3.1 Smart meter data

The first step is to collect smart meter data. Smart meters collect a huge range of data related to power consumption and weather data. It typically requires separate weather monitoring equipment or accessing weather data from weather stations (Makonin 2019). Weather data includes various parameters such as temperature, humidity, wind speed, and wind speed precipitation. It records data with a timestamp, allowing for real-time monitoring and historical analysis. Hence, the smart meter data becomes time series data (Le and Vo 2020). The specific data collected can vary based on the utility's requirements and the capabilities of the smart meter. It's important to note that smart meters provide valuable insights for consumers and utilities. Privacy and security issues must be considered to protect sensitive information and ensure responsible data usage.

3.2 Smart meter configuration

Smart meters are advanced digital devices used to measure and record electricity usage in residential, commercial, and industrial environments. Unlike traditional meters, smart meters provide two-way communication between utilities and consumers. They can send data back to the utility company in real time, which enables better monitoring, management, and optimization of energy usage. Designing a smart meter for building energy consumption forecasting involves several key considerations to ensure accurate data collection, communication, and compatibility with forecasting algorithms. The outline of the smart meter design is depicted in Fig. 2. Here's a conceptual outline of how a smart meter can be designed to support energy consumption forecasting (https: www.electronics-notes.com/articles/eco-green-engineering/smart-energy-meters/smart-meter-electronic-circuit-design.php):

Fig. 2
figure 2

Design of Smart meter for building energy consumption forecasting

Full size image

3.2.1 Data collection

  • High-Frequency Data: The smart meter should be capable of collecting energy consumption data at regular intervals (e.g., every 15 min or hourly) to capture detailed consumption patterns.

  • Timestamps: Each data point should be associated with a timestamp, enabling chronological analysis and forecasting.

  • Granular Data: The meter should record various consumption parameters such as active power, reactive power, voltage, and current.

3.2.2 Communication

  • Two-Way Communication: Implement bidirectional communication capabilities to allow data transmission to utility companies and receipt of signals or commands.

  • Data Transmission: Use wired or wireless communication protocols (e.g., cellular, Wi-Fi, Zigbee) to transmit collected data to central servers or cloud platforms.

3.2.3 Data storage and logging

  • Local Storage: Incorporate local storage to buffer and log data in case of communication disruptions, ensuring no data loss.

  • Data Logging: Store consumption data along with timestamps in a structured format for easy retrieval and analysis.

3.2.4 Data quality and accuracy

  • Calibration: Calibrate the smart meter to ensure accurate energy readings and minimize measurement errors.

  • Quality Control: Implement mechanisms to monitor and ensure data accuracy over time. Regular maintenance and calibration checks are essential.

3.2.5 Security and privacy

  • Data Encryption: Encrypt collected data to ensure its security during transmission and storage.

  • Anonymization: Protect consumer privacy by anonymizing data before analysis, and removing personally identifiable information.

3.2.6 Compatibility

  • Open Standards: Design the smart meter to adhere to open communication protocols and standards to ensure interoperability with utility systems.

  • Integration: Ensure compatibility with utility company systems and energy management platforms to facilitate data integration and usage.

3.2.7 Remote management and updates

  • Remote Configuration: Allow for remote configuration and parameter adjustments to adapt to changing requirements.

  • Firmware Updates: Design the smart meter to receive firmware updates remotely to improve functionality and security (Munoz and Ruelas 2022).

3.2.8 Data analysis and forecasting support

  • Data Preprocessing: Include capabilities for data preprocessing, such as filtering out noise and handling missing data.

  • Integration with Forecasting Algorithms: Enable integration with energy consumption forecasting algorithms, whether based on time series analysis, deep learning, or statistical methods.

3.2.9 User interfaces

  • Display and User Interface: Provide an intuitive display that shows real-time consumption data to users for energy awareness and management. For integration with the forecasting algorithm, the is retrieved in Excel or notepad form.

  • Interfaces for Utilities: Implement interfaces that utilities can use to retrieve historical and real-time consumption data (Jaiswal and Thakre 2022).

3.2.10 Scalability and future-proofing

  • Scalability: Design the smart meter to handle large volumes of data as the number of smart meters in deployment increases.

  • Future Upgrades: Plan for potential enhancements in communication technologies and data analytics methods to ensure the meter's long-term relevance.

The design of a smart meter for energy consumption forecasting requires collaboration between meter manufacturers, utility companies, and data analytics experts. It should strike a balance between accurate data collection, data security, interoperability, and usability to provide reliable and valuable insights for energy consumption forecasting.

3.3 Review of smart meter dataset

Smart meters usually collect data every second, minute, or hour. The smart meters record detailed information about electricity consumption, providing valuable insights into usage patterns and trends. Researchers and analysts often utilize publicly available smart meter datasets to develop and test energy consumption forecasting models, anomaly detection algorithms, and other applications related to smart building and energy management. Table 1 provides the various datasets, their availability as public or private, along with the website link, data characteristics, and a number of papers referring to the dataset of smart buildings. Table 2 provides a sample smart meter dataset. Researchers carried out their research using different datasets. Here are a few limitations and challenges associated with smart meter datasets.

Table 1 Dataset availability and their website, smart building characteristics
Full size table
Table 2 Sample smart meter dataset (Makonin 2019)
Full size table

3.3.1 Limitation in dataset selection

  • The availability of certain datasets may change over time. A dataset that was accessible during the initial stages of a project may become unavailable, which could disrupt ongoing research or analysis.

  • Metadata, which provides context and additional information about the data, may be lacking in some datasets. This absence can make it challenging to interpret the data accurately.

  • Working with large datasets can pose limitations in terms of storage, computational resources, and processing time, particularly for researchers with limited infrastructure.

3.3.2 Challenges in datasets selection

Selecting effective power consumption datasets can be challenging, as it involves various considerations and potential pitfalls. Here are some challenges that might be encountered during the selection of datasets.

(A)

Data availability

Availability of quality power consumption data can be a significant challenge. Not all regions or utilities provide access to detailed consumption data in realtime setups, especially for research or non-commercial purposes. There is a need to create a benchmark dataset for power prediction research. Exploring open data initiatives and repositories that may host publicly available power consumption data. The research also suggests that in the absence of a real-time setup, there is a need to develop simulation software or tools to generate.

(B)

Data preprocessing

Due to several number of reasons, including faults in meter sensors, variable weather conditions, and abnormal customer consumption patterns, the raw power consumption data often requires preprocessing to clean, normalize, and transform it for analysis. This can be time-consuming and may introduce errors if not done carefully.

(C)

Data granularity

The level of granularity in the data can be a challenge. Some datasets might provide data at a very high temporal resolution (e.g., every minute), while others might only offer daily or monthly averages. The level of granularity that needs to be depends on the specific research or analysis goals. Therefore, the selection of datasets must match the level of granularity needed for the specific analysis.

(D)

Data heterogeneity

The different utilities and regions may use different data formats and collection methods. The data needs to deal with various sources, and integrating heterogeneous data can be complex. So, the data collection requires standardization in a data format.

3.4 Data pre-processing

Next, the data is split into training and testing sets. Smart meters collect data that may contain missing, redundant, and outlier values. To address these issues, preprocessing techniques are applied to both the training and testing sets. Before training the model, the data needs to be refined. Therefore, before training, the input raw dataset is refined by adding missing values and removing outliers.

Missing values in smart meter data can occur due to communication failures, technical malfunctions, meter installation/replacement, data processing errors, or intentional anonymization/aggregation (Dahunsi and Olawumi 2021). These factors can lead to gaps in the data, where certain time intervals or measurements are not captured or recorded. This may lead to incorrect data collection.

Redundant values refer to duplicate or repetitive entries in the smart meter data. They can arise from issues like data transmission errors, system glitches, or multiple data sources providing overlapping or identical information (Ali et al. 2016). Redundant values can skew the analysis and inflate the importance of certain data points if not properly identified and handled.

Outlier values in smart meter data are extreme or unusual measurements that deviate significantly from the expected or normal range. Outliers can occur due to meter malfunctions, data transmission errors, incorrect sensor readings, or anomalies in the underlying processes being measured (Wang et al. 2018). Outliers can negatively impact the accuracy and reliability of data analysis and forecasting models if not detected and appropriately treated.

Similarly, power consumption patterns vary widely and networks are sensitive to them. Hence, the data normalization techniques are applied to keep the dataset within the normal range. (Pascanu et al. 2013; Khan et al. 2020a; Rick and Berton 2020).

3.5 Training deep learning model and error metrics

Then, the normalized data is fed to the deep learning model to create a final model. Then, the final model is used for testing to provide forecasted power as output at every instant of time.

For analyzing and verifying the effectiveness of the model With different kinds of experiments, its performance is evaluated and analyzed using error metrics and cross-validation (Ghimire et al. 2023). The quality of the prediction model was evaluated using the error metrics used.

This is the common framework used by the researchers. But still, the researchers can incorporate some advanced techniques to make the framework give more accurate results. The framework excels in integrating a wide array of data sources, including real-time data, and making effective use of this information for forecasting. This results in more accurate and dynamic predictions.

There exist several strategies to deal with multistep forecasting problems (Gao et al. 2012): the recursive strategy, which performs one-step predictions and feeds the result as the last input for the next prediction; the direct strategy, which builds one model for each time step; and the multi-output approach, which outputs the complete forecasting horizon vector using just one model. The SISO strategy belongs to the time series problem. As suggested in recent forecasting studies (Sulaiman et al. 2022; Imani and Ghassemian 2019), the single input and single output (SISO) strategy is not performed in most of the studies. The recursive approach is particularly useful for capturing short-term patterns and trends in the data. By using the most recent prediction as input, it can quickly adjust to changes in the time series, making it responsive to short-term fluctuations.

The Key Component and Innovation in the framework is integrating hybrid deep learning models. TCNs are employed to capture long-range dependencies in time series data. This innovative architecture provides an alternative to traditional RNNs and LSTMs, making it suitable for complex temporal patterns in power consumption.

4 Application of deep learning for forecasting energy consumption

Deep learning methods are promising for time series forecasting, such as automatically learning temporal dependencies and automatically handling temporal structures such as trends and seasonality (Bandic and Kevric 2018). In the paper (Pirbazari and Sharma 2021), they have used several common algorithms in time series forecasting, e.g., support vector regression (SVR), ARIMA, LSTM, etc. for training. RNNs can be extended to deep recurrent neural networks (DRNNs) in various ways (Cai et al. 2019). According to the RNN framework, we can deepen the hidden function, the transition function, and the hidden output function to create a DRNN (Cai et al. 2019). There are different variants of DRNN, but the focus is on the widely known LSTM network and GRU network used in the context of electricity predictions. Because the LSTM and GRU techniques have long term dependencies. A multilayer perceptron neural network (MLP) is a unique form of feedforward network called a universal approximation. Due to its simplicity, it is one of the most widely used neural network frameworks. (Pascanu, et al. 2013).

In the paper (Dehalwar et al. 2017), LSTM neural networks are used to perform prediction operations. They proved that stationarization of the wavelet transform could improve the LSTM prediction results. Finally, the prediction results are synthesized using the inverse stationary wavelet transform. In a paper (Lara-benitez 2020), the researchers suggested a TCN model with (DenseNet) densely connected convolution networks. In a paper (Mocanu et al. 2016), they present a hybrid intelligent technique that combines a CNN with a multi-layer bidirectional long-short term memory (M-BDLSTM) method. In paper (Rahman et al. 2018), for very short term forecasting, proposed a hybrid electricity demand forecasting model that combines LSTM and CNN. The input sequence consists of multiple pairs and the key value is the power demand value. Context values contain contextual information such as temperature, humidity, and time of year. In a paper (Spiliotis et al. 2020), for a small industry profile, they performed a 30-min energy consumption forecast. Instead of the conventional MSE, the pinball loss was used as a guideline for adjusting the LSTM neural network's parameters. Pinball loss is indeed a term used in the context of training and tuning parameters for LSTM models, particularly in time series forecasting. Pinball loss, also known as quantile loss or quantile regression loss, is a loss function used to train models to estimate conditional quantiles of a target variable. In the case of time series forecasting with LSTM models, it is often used to estimate and optimize quantiles of the predicted distribution for different forecasting horizons.

Table 3 depicts the summary of previous work related to deep learning. In the table, research findings and their corresponding accuracy were mentioned. The limitations of each method were tabulated. From the table, inferred that the evaluation metrics play a vital role in prediction accuracy. And also input data contributes more to DL performance and forecasting accuracy, especially weather variables.

Table 3 The summary of deep learning related work, limitations, and findings
Full size table

5 Various applications of forecasting using forecasting horizon

Electricity load forecasting plays an important role in the planning and operation of energy systems. It is the process of predicting future power consumption using historical load profiles and weather information. Electricity prediction is categorized into three groups. These include short-term forecasts, typically ranging from days to weeks, medium-term forecasts, typically ranging from weeks to a year, and long-term forecasts, typically ranging from a year or more (Sajjad et al. 2020). The decision-making process, such as the scheduling of generation amount, maintenance and investment plan, and so on, is supported by the expected amount of energy demand (Le et al. 2020). Based on different applications of forecasting in the electrical field, it is classified into two categories.

5.1 Based on HVAC

The energy consumption of a building is impacted by several factors, including the building's structure, HVAC system, occupant behavior, and lighting (Liu et al. 2020). Predicting building energy usage can be categorized into five main types: heating energy, cooling energy, combined heating and cooling energy, total building energy consumption, and other factors (Mocanu et al. 2016). Anticipating power consumption has become essential for enhancing power management and fostering collaboration between a building's energy consumption and the electrical grid (Xu et al. 2019). Accurate predictions play a crucial role in enabling efficient power management, such as helping suppliers generate the right amount of electricity to meet demand. Energy/cost investment funds and general control execution are legitimately impacted by how accurately energy use is estimated (Cai et al. 2019).

5.2 Based on power grid planning

It is crucial to categorize and forecast the energy consumption of residential buildings using historical data based on the impact of dynamic real-time changes on both the supply and demand sides to provide adequate decision-making for planning power transmission configuration patterns that maintain regional characteristics. Short and very short term forecasting techniques for energy consumption are useful for domestic energy demand management, electricity price market design, energy efficiency, and maintenance planning for large and complex smart grids (Abuella and Chowdhury 2017; Khan et al. 2019; Tokgoz and Unal 2018). Forecasting aims to provide as accurate predictions as possible, utilizing the wealth of available data (Kim and Cho 2019b; Ullah et al. 2019). In recent years, numerous approaches have been suggested to harness this information for making predictions (Afrasiabi et al. 2020). These forecasting techniques effectively anticipate energy consumption by establishing connections between energy and its consuming systems, thereby minimizing inefficiencies arising from over or undersupply and undersupply (Mocanu et al. 2016).

Figure 3 shows the forecasting horizon in the works of literature. It depicts the percentage of literature that contributes to Short term, Medium term, and Long term forecasting. Most of the literature concentrates more on short-term forecasting. Short-term energy prediction is preferred over medium and long-term prediction due to its higher accuracy, immediate applicability for real-time decision making, optimal resource allocation, contribution to grid stability, and reliance on up-to-date data. The more predictable patterns and behaviors of energy consumption in the short term, along with the availability of timely data, make short-term energy prediction more reliable and actionable for efficient energy management and operational adjustments in response to changing demand and supply dynamics.

Fig. 3
figure 3

Related works by forecasting horizon

Full size image

6 Evaluation metrics

To validate the predictive models, a variety of accuracy criteria are employed. The most commonly used are mean squared error (MSE), root mean squared error (RMSE), coefficient of variation of root mean square error (CV-RMSE), mean absolute error (MAE), mean absolute percentage error (MAPE), and determination coefficient (R2). The evaluation metrics used to validate forecasting models are shown in Fig. 3. Each of these metrics has a specific function. MSE is a commonly used metric in the context of forecasting and predictive modeling to evaluate the accuracy of a forecasting model's predictions. It measures the average squared difference between the predicted values and the actual (observed) values. It is a way to quantify how well the forecasting model's predictions match the actual outcomes. RMSE is widely utilized to examine the accuracy of various anticipating criteria because it is generally offered as an expectation quality estimation (Kim et al. 2019). CV-RMSE can standardize the expected error and provide a useful unitless metric. MAE relies on absolute error and can indicate the normal distance between expected and actual values. Large errors are eliminated because the measurement mathematically enhances the error (Wang et al. 2019). MAPE exhibits rate accuracy and mitigates the impact of absolute error caused by a single exception (Kim et al. 2019). R2 evaluates how well the model fits the real information and provides a measure of model consistency (Amasyali and El-Gohary 2018).

The five evaluation metrics are listed in Table 4. Let \(n\) be the number of test data, \({x}_{pred}\) be the value predicted by the proposed algorithm, and \({x}_{act}\) be the actual value in the quantitative, \({\overline{x} }_{act}\) be the average of the actual value.

Table 4 Evaluation metrics equation
Full size table

7 Discussion

Based on the review of previous research work, a detailed summary of building energy consumption forecasting methods is presented in Table 5. The summary includes the contribution of each paper and inference from that work was discussed briefly. And also, explains the lists of buildings used, the network used, the input parameters used, and the types of forecasting used.

Table 5 Summary of previous research papers, their contribution, and inference
Full size table

All prediction methods discussed in the literature have weaknesses, which depend on the approach chosen. A table comparing different forecasting methods is provided to help you easily identify which forecasting model type to use. Due to the relevance of information for planning, a lot of effort is required when it comes to power forecasting. As stated in the literature, numerous strategies have been used. Based on the survey, it is obvious that DL has been proven to outperform other prediction approaches in terms of accuracy. It has been discovered that linear models, which were previously relegated because of their inability to solve nonlinear problems, are nevertheless applicable in the context of energy projections. RNNs are effective in solving nonlinear issues with great prediction accuracy. Despite the power of RNN’s as discussed in the literature, it has a significant drawback, called the problem of vanishing gradient For this reason, few researchers have applied traditional RNNs. DRNN improved the boundary vanishing gradient by introducing memory (Kaur and Ahuja 2017). Therefore, LSTM with GRU is currently used in the context of energy prediction because it can model complex functions with high accuracy.

Accurate customer-level energy forecasting has a direct impact on overall system efficiency. However, it is difficult to predict building energy consumption, especially in the medium and long term changes in climatic conditions, thermal system performance, and patterns in occupancy. Therefore, current state-of-the-art technology cannot contain building-level uncertainty due to many influencing factors. These can include changes in weather, occupant behavior, changes in building structure and operations, missing data in datasets, computation time impacting forecast accuracy, and more. As reported in the literature, few researchers have used occupancy profiles in predictive models (Nepal 2020; Zhao and Magoules 2020; Ma et al. 2019), and only one study (Das et al. 2020) considered building design.

From the statistical report of this review paper, about 72% of the research reviewed focused on developing methods to predict building energy use in non-residential buildings, 18% on residential buildings, and just 10% on both. It is depicted in Fig. 4.

Fig. 4
figure 4

Comparison chart between building type

Full size image

Regarding the forecast horizon, about 68% of the studies focused on short-term forecasts, 27% on long-term forecasts, and 5% on medium-term forecasts. A comparison of building types and prediction horizon is shown in Fig. 5. This explains the increasing use of demand-side management strategies such as shifting of load, which allows the loads to shift from peak hours to off-peak hours and knowing the energy consumption of building for 24 h. It should be noted that the short-term horizon has expanded in recent studies since it has proven to be quite useful in predicting the supply of energy resources in buildings.

Fig. 5
figure 5

Comparison chart between type of building and Forecast horizon

Full size image

About 54% of the research focused on the use of calendar and historical data in prediction methods, 41% focused on the use of weather, calendar data, and historical and 3% on historical, occupancy rates and focusing on the use of calendar data. Only 2% of the studies used calendar, weather, occupancy, and historical data. These phenomena may arise from the reliance on diverse sensors to capture building energy consumption, while data concerning occupancy often remains unaccounted for due to the intricate data acquisition process, which can vary based on the building's characteristics. In Fig. 6. comparison chart between the type of building and input parameters was given. On the other hand, the availability of this data will further improve the accuracy of the predictions.

Fig. 6
figure 6

Comparison chart between type of building and input parameters. His indicates Historical, cal indicates calendar, wea represents weather, and occ indicates Occupancy

Full size image

In summary, the analysis of methodologies for energy consumption forecasting has revealed the prominence of deep learning, the importance of feature selection, and the influence of weather and occupancy patterns. There is a need for real-time data integration, the adoption of ensemble and hybrid models, the emphasis on explainability and behavioral insights, and the consideration of energy efficiency measures. These findings and observations collectively contribute to the advancement of accurate and actionable energy consumption forecasts for smart buildings. The novel deep learning architectures, including CNN-LSTM, CNN-LSTM autoencoders, and TCN, have been proposed to address the complexities of energy consumption time series data and to achieve better forecasting results. While machine learning algorithms have advanced the field of energy forecasting, challenges remain, particularly in peak load prediction. Ongoing research focuses on improving the accuracy of these models and addressing their limitations.

These findings and insights reflect the dynamic nature of the field of power forecasting, with researchers continually developing and refining methodologies to address the complexities of energy consumption and integrate renewable energy sources for a more sustainable energy system. These critical observations reflect the ongoing evolution of energy forecasting methodologies, with a focus on enhancing accuracy, addressing challenges, and adapting to the changing energy landscape.

8 Research gap

Energy demand forecasting is a useful tool for identifying, measuring, and managing demand flexibility. Nowadays, renewable resources play a vital role in energy generation. The review of such vast existing research work suggested the necessity to address the obstacles in the future. Further research work needs to focus on the following:

  • Research gap-1 A key component that reduces the prediction accuracy is the weather (temperature, humidity, etc.). Therefore, future studies should concentrate on the impact of weather variables on power forecasting and incorporate weather components by analyzing the contribution and impact of each variable on the consumption profile. (Mocanu et al. 2016).

  • Implication-1 A deeper understanding of how weather influences power consumption can lead to more energy-efficient practices. For instance, businesses and households can optimize their energy use by adjusting their consumption patterns in response to weather forecasts, potentially reducing costs and environmental impact.

  • Avenues -1 A detailed analysis is needed about, how weather-informed forecasts can optimize energy storage systems. To determine how renewable energy generation can be better aligned with weather patterns to match demand. A Study needs how consumers respond to weather-informed energy consumption predictions. To investigate the effectiveness of behavioral interventions based on weather forecasts in reducing energy consumption.

  • Research gap-2 Based on the knowledge from the works of literature, no study takes into account the functionality of the buildings in terms of space and share percentages, nor is there one that uses ML approaches and future weather scenarios to evaluate the effects of global climatic change on the energy performance of urban buildings. This will be a worthwhile area for research.

  • Implication-2 Research in this area can contribute to more sustainable urban development by optimizing the energy performance of buildings, which is crucial for reducing carbon emissions and mitigating the effects of climate change in urban areas.

  • Avenues-2 To develop and refine energy simulation models that incorporate detailed building functionality data, real-world weather scenarios, and ML techniques for accurate predictions of energy consumption. The creation of databases that consolidate building functionality data, historical weather data, and building energy usage data.

  • Research gap-3 Future lines of research should encourage considering various time scales, different environmental conditions, and various horizons like hours, months, and years. To enable the efficient use of electrical energy across various industries and smart grids, these horizons can be used (Mocanu et al. 2016).

  • Implication-3 This research approach promotes multi-temporal energy planning that spans various time scales. It enables utilities, industries, and smart grid operators to develop comprehensive strategies for optimizing energy use, production, and distribution. For long-term forecasts, the model needs a huge number of data to make efficient predictions. Therefore, research in this area can drive the development of data collection technologies and data analytics tools that are tailored to various time scales and environmental conditions. Addressing different time horizons allows for more accurate energy demand forecasting. Short-term forecasts (hours) help manage grid stability, while long-term forecasts (months and years) inform infrastructure investments.

  • Avenues-3 Invest in advanced data collection and data analytics tools to process and analyze diverse data sources, enabling precise multi-temporal forecasting and decision-making.

  • Research gap4- Challenges such as compensation for forecasting errors, problems with dynamic model selection, the creation of adaptive predictive models, and data integrity must be addressed by current techniques. (Singh and Dwivedi 2018). Improving energy management requires achieving high accuracy in the use of energy forecasts. In any instance, this necessitates the selection of appropriate estimating models that are prepared to capture each of the predicted array's features, which is a task fraught with uncertainty (Wen et al. 2020).

  • Implication 4 Improving the accuracy of energy forecasts can enhance grid stability and resilience. Grid operators can better anticipate demand fluctuations, reducing the risk of blackouts and ensuring the reliability of energy supply, even in the face of unexpected events. Energy markets rely on accurate energy forecasts. Addressing these challenges can lead to more efficient market operations, better matching supply and demand, and potentially reducing price volatility.

  • Avenues-4 Developing advanced forecasting models that integrate dynamic model selection and adaptive predictive capabilities to improve forecasting accuracy and robustness in the face of changing conditions.

  • Research gap-5 Most research has been done on predicting power usage in a single building, but there hasn't been much done to aggregate the power usage across a wider area using a large number of data samples to assure the model's accuracy.

  • Implication-5 Accurate and aggregated energy data can facilitate the design of targeted energy efficiency programs that address the specific needs of a region or community, resulting in reduced energy waste. By aggregating data, it becomes easier to manage the integration of renewable energy sources into the grid. This research can lead to better strategies for balancing energy supply and demand.

  • Avenues-5 Develop real-time monitoring and control systems that provide actionable insights and allow for dynamic adjustments in energy use and distribution based on aggregated data. Most of the real-time datasets are not publicly available on the website. So there is a need for a benchmark dataset for this domain.

The identified research gaps in the area of energy consumption forecasting have significant implications for improving the accuracy and sustainability of energy management. Focusing on the impact of weather variables, integrating building functionality with DL, addressing long-term forecasting challenges, and considering various time scales can lead to more precise predictions,, better energy planning, and enhanced grid efficiency. The identified gaps impact the field by reducing the accuracy and efficiency of energy consumption forecasts in smart buildings. Inaccurate forecasts can result in suboptimal energy management, increased costs, and a failure to meet peak demand. Research in this area can support smart city initiatives by providing accurate and comprehensive data on energy consumption patterns across various sectors, enabling better urban planning and resource allocation.

9 Conclusion

The 40% rise in the amount of electricity used in residential and commercial is because of the recent increase in urbanization. Accurately predicting electricity demand has become crucial. Forecasting can be used to meet the supply and demand gap for electrical energy. Since it helps decision-makers and planners in government, this forecast is important on a global scale. To increase the accuracy of energy consumption predictions, new automated paradigms are required. This paper reviews the recent advanced forecasting methods. While analyzing other methods, DL gives better results. DNN has recently been effectively used in this context. Furthermore, the study provides taxonomies for these methodologies based on various forecasting horizons and data sources utilized to predict future energy usage. As a result, this assessment can help researchers identify research gaps that require addressing in the future and come up with novel approaches to enhance power forecasting in commercial and residential buildings. This review paper presented the significance of smart meter data for energy forecasting. Researchers can take the initiative to extract useful information from the smart meter for the benefit of society. The review of prior works will provide useful guidance to future researchers. Based on the continuation of this review paper, a novel hybrid deep learning approach will be proposed for improving forecasting accuracy in residential buildings from short to long term horizon.