Keywords

1 Introduction

Automotive manufacturing industries are faced with new challenges in technology adoption, environmental degradation from a significant proportion of carbon emission, supply change for the digital economy, and multi-faceted sustainability drives [1]. Energy efficient system within complex automotive manufacturing has great potential for energy consumption reduction, relative to size, season, and types of manufactured components. Decision makers require fourth industrial revolution (4IR) application techniques to ensure that the industry’s operational energy use is efficiently managed and decarbonized. Understanding the meaning of ‘decarbonization’ is important as it is a term that is used for ‘reduction or total removal of carbon dioxide (CO2) emissions [2].

Accurate forecasting is a challenging process and requires statistical or machine learning techniques with a strong capability for predicting energy consumption and carbon emission, which is part of decarbonization process planning. Time series forecasting is known for a collection of past observation data of the same variables, which can be analyzed to develop a model for future prediction [3]. The widely applied prediction model for time series (TS) data stream is a statistical technique; autoregressive integrated moving average (ARIMA), Box Jenkins methodology proposed in 1970 [4, 5]. ARIMA has been applied to any form of process challenges within the different industries to assist decision makers to plan future-based predictions on trusted applications [6,7,8,9]. ARIMA ais quite flexible in that they can models several different types of time series data, i.e. pure autoregressive (AR), pure moving average (MA) and combined AR and MA (ARMA) series, with major limitation as pre-assumed linear form of the model [3].

Machine Learning (ML) as a subset of artificial intelligence (AI) is considered for the non-linear application data pattern. ML algorithms are used for data-driven fault prediction technology [10] and consist of the process of building an inductive model that learns from a limited amount of data without specialist intervention [11]. There are various ML techniques such as convolution neural network (CNN), gated recurrent network (GCN), artificial neural network (ANN), recurrent neural network (RNN), and long short-term memory (LSTM). Recently, LSTM has transformed from a modified RNN architecture introduced in 1997 [12] and has attracted attention for its capability to capture non-linear trends and dependencies [5]. LSTM-based deep learning methods have achieved great success in artificial intelligence fields involving large datasets [13, 14].

In this paper, we propose a hybrid time series forecasting approach using ARIMA and LSTM models for an automotive component manufacturing company data using the 4IR concept, based on the following motivation: in practice, it is challenging to determine whether the study is generated from a linear or non-linear process and time series are rarely linear or nonlinear in the data patterns. Comparison of the result of the ARIMA model and ARIMA-LSTM model using predictive evaluation indicators (PMI); mean absolute error (MAE), root mean square error (RMSE), and mean percentage error (MPE) are evaluated to calculate prediction accuracy.

2 Related Work on ARIMA-LSTM

The hybrid models are becoming popular in decision-making. This is due to the combination of linear and nonlinear aspects of the data pattern, which further increases the accuracy of the predictions in an application. In recent years, it has become evident that hybrid methods yielded better results compared to a single method. A summary of studies that applied hybrid model for accurate prediction: Soy Temür et al. [15] proposed a hybrid model which consists of a combination of the linear model (ARIMA), nonlinear model (LSTM), and hybrid (LSTM and ARIMA) model to improve system performance compared to a single model. A prediction method (GA‐CNN‐LSTM) combines a convolutional neural network (CNN) and a long‐short‐term memory network (LSTM) and is optimized by a genetic algorithm (GA) [16]. Authors [17] proposed a new hybrid model using long short-term memory (LSTM), a recurrent neural network (RNN) technique, and autoregressive integrated moving average (ARIMA), as time series forecasting technique to capture live stock market data, the method performed very well compared to clairvoyant forecasting library. The work of [18] provided Indonesian governments with an accurate prediction of future exports, a hybrid model that integrates ARIMA and LSTM models based on their specialties, where LSTM was applied to the non-linear component of the data, and ARIMA was applied to the linear component of the data. The results showed that the hybrid (LSTM-ARIMA) model achieved the lowest error metrics among all the tested models. ARIMA and LSTM techniques were used to establish rolling forecast models, which greatly improve the accuracy and efficiency of demand, and inventory forecasting, while the authors proposed ARIMA and LSTM as superior to the manufacturer’s empirical model prediction results [19].

The proposed hybrid-based model on deep learning methods integrates ARIMA and LSTM model to improve the accuracy of short-term drought prediction [20] and the results state that the ARIMA-LSTM model has the highest prediction accuracy [21]. In the paper titled, we present a novel hybrid ARIMA-LSTM model for automotive component manufacturing company production data forecasting considering manual operations to establish the advantages of linearity and nonlinearity, which exhibited better results than the individual models.

3 Methodology

3.1 Data Set and Processing

The study uses tier 2 automotive company-generated electricity data retrieved from National Cleaner Production Centre (NCPC) as secondary data obtained through its energy management training program. The carbon emission equivalent was derived using compared country-specific energy guidelines according to Intergovernmental Panel on Climate Change (IPCC) 2019 emission factors for coal mining [22, 23].

3.2 Auto-regressive Integrated Moving Average (ARIMA)

ARIMA model are a transformational statistical method that supports seasonality in data prediction [24], which have gained popularity among researchers due to their vast applications in manufacturing [25]. The models are known for their notable forecasting OF time series data accuracy and flexibility in different applications [26]. ARIMA models use regression equation to determine how variables respond to stochastic dissimilarity [24]. The independent variables are dependent on the lagged value of the previous values of the forecast as proposed by Box and Jenkins in the 1970s [4]. The equation is given as:

$$y^{\prime}\left( t \right)\, = \,c\, + \,\varphi_i \, \ast \,y^{\prime}\left( {t - 1} \right)\, + \, \ldots \, + \,\varphi_i \,p \ast \,y^{\prime}\,\left( {t - p} \right)\, + \,\theta_i \, \ast \,\varepsilon \left( {t - 1} \right) + \, \ldots \, + \,\theta_{p\,} \, \ast \,\varepsilon \left( {t - q} \right)\, + \,\varepsilon t$$
(1)

\({y}^{^{\prime}}\left(t\right)\) = differenced series, (p = order of lag depicted as autoregression, q is the order of error lag (moving average) and ε \(\left(t-1\right)\) is residuals of past observation, \({\varphi }_{i}\) is the coefficient of the first AR term, \({\theta }_{i}\) is the coefficient of the first MA term.

3.3 Long Short-Term Memory (LSTM)

Recurrent neural networks are difficult to train, as they often suffer from the exploding/vanishing gradient problem [27]. To overcome this shortcoming when learning long-term dependencies, the LSTM architecture [12] was introduced. The LSTM architecture consists of a set of recurrently connected sub-networks, known as memory blocks, the idea behind the memory block is to maintain its state over time and regulate the information flow through non-linear gating units [28]. Figure 1 is the N architecture of LSTM adopted from the work of [15]. Figure 2 depicted LSTM Structure.

3.4 Hybrid ARIMA-LSTM

ARIMA filters linear trends in the TS data and the residual values are passed to the LSTM model for training and residuals predict for the upcoming year. The LSTM has longer memory and works well for the non-stationary section of the data. LSTM works well for the non-stationary portion of the data with a relatively longer memory. LSTM’s capability of capturing nonlinear patterns in TS data is one of the method’s main advantages, as an attempt to overcome the challenges of obtaining an accurate forecasting model considering the intrinsic characteristics of the demand time series (being nonlinear and non-stationary).

Fig. 1.
figure 1

N architecture of LSTM

Full size image
Fig. 2.
figure 2

LSTM structure

Full size image

Xt represents the input data at t time step and the output of the previous unit, ht is the hidden units’ output, while ht-1 is their previous output. The new memory in Eq. 3 is the LSTM unit calculated from Eq. 2

$$f_t^j = \tanh \left( {W_{xc} x_t + W_{ht} h_{h - 1} + b_c } \right)^j$$
(2)
$$h_t^j = \sigma_t^j tanh\left( {c_t^j } \right)$$
(3)

3.5 Predictive Evaluation Indicators Validation

The time series prediction performance assessment was validated to evaluate the accuracy of trained models and identified efficient models between ARIMA and ARIMA – LSTM hybrid models. The root mean square error (RMSE) is a means of measuring the error in predicting quantitative data, it is a normalized distance between the observed values and the predicted values. The predictive evaluation indicators are engaged heuristically to decrease the non-absolute size of the error iteration from one step to the next.

$${\text{RMSE}} = \sqrt {\frac{1}{n}\mathop \sum \nolimits_{i = 1}^n \left( {x_{1 } \left( t \right.} \right) - } \left. {y_t \left( t \right)} \right)$$
(4)

where yt is the actual value, ft is the predicted value; k is the sample size square error. In this paper, the proposed hybrid formulation is the residuals from the rolling LSTM model analysis for further optimization, given its non-linear form.

4 Results

ARIMA and ARIMA-LSTM models were used to forecast the electricity generated from coal and carbon dioxide emissions. Figure 3 is the initial visual observation of tier 2 automotive component manufacturing electricity generation. RMSE is used as an indicator for analysis comparing ARIMA as a single model and the performance of the hybrid model ARIMA-LSTM. The RMSE for ARIMA is 448.89 from the initial visual observed data in Fig. 4 and the ARIMA-LSTM actual and predicted are 59.52 and 58.41 as presented in Fig. 5. The results of ARIMA and ARIMA-LSTM predictions are depicted in Fig. 6. The RMSE values of ARIMA-LSTM are smaller than ARIMA, which proves that the hybrid ARIMA-LSTM is more suitable for prediction than the single model that is ARIMA.

Fig. 3.
figure 3

Initial visual observation of tier 2 automotive component manufacturing electricity generation

Full size image
Fig. 4.
figure 4

ARIMA RMSE results

Full size image
Fig. 5.
figure 5

RMSE for actual train data and predicted test data

Full size image
Fig. 6.
figure 6

ARIMA and ARIMA-LSTM predictions

Full size image

5 Conclusion

The long short-term memory network (LSTM) model was used to forecast energy consumption and carbon emission within the manufacturing facility using the 4IR concept. The method captures linear and nonlinear patterns in time series data, ARIMA captures the linear features and LSTM captures the long dependencies in the data. The obtained result with hybrid models were individually compared, it was observed that they could reduce the general variance or error, even if they are unrelated. Due to this reason, hybrid models are recognized as the most successful models for forecasting tasks [15]. This information will support the decision makers for energy management and decarbonization planning. The objective to construct two models was to test which model will best fit for prediction, using the configuration that gives the lowest root mean square error.