Active Power Load Data Dimensionality Reduction Using Autoencoder

Abstract

Dimensionality reduction is a machine learning based technique used to convert the data from higher dimensionality space to lower dimensionality space. This technique helps to build lighter version machine learning based predictive models. In this paper, a deep learning model i.e.. autoencoders is used to reduce the dimensionality of active power load data from higher dimensionality space consists 14 input features to lower dimensionality space consists 7 input features. Original active power load data is prepared based on data collected from 33/11KV substation located in Godishala village, Telangana State, India, from 01.01.2021 to 31.12.2021. Autoencoder model is trained and tested with python program using visual studio. From the simulation results, observed that autoencoder model reduces the dimensionality space of load data with almost same variance that original data exist.

Appendix

1.1 Autoencoder

This section presents step by step procedure for training of autoencoder and also explains how trained encoder is used to reconstruct the data from original dataset. The sample architecture of autoencoder that used in this section to explain the training  and data reconstruction procedure is shown in Fig. 5.

Fig. 5

Sample autoencoder architecture

The initial weight and bias matrices are shown in equations (12–15). The data samples that are used to train the autoencoder is shown in Table 9.

$$\begin{aligned} W_{ih}=\begin{bmatrix} 0.1 &{} 0.2\\ -0.2 &{} -0.1\\ 0.2 &{} 0.1 \end{bmatrix} \end{aligned}$$

(12)

$$\begin{aligned} W_{hk}=\begin{bmatrix} 0.1 &{} -0.2 &{} 0.1\\ -0.1 &{} 0.2 &{} -0.1\\ \end{bmatrix} \end{aligned}$$

(13)

$$\begin{aligned} b_{h}=\begin{bmatrix} 0.1\\ -0.1\\ \end{bmatrix} \end{aligned}$$

(14)

$$\begin{aligned} b_{k}=\begin{bmatrix} 0.1\\ -0.2\\ 0.1\\ \end{bmatrix} \end{aligned}$$

(15)

Table 9 Sample training data

1.2 Auto Encoder Training

In this section, update of weights and bias parameters of autoencoder based on given training data for one iteration  is explained.

• Iteration 01

  • Sample 01 = [ 0.1 0.2 0.3]

Calculate net input to hidden layer/latent space using Eq. (4) and the result for the first sample is shown below

$$\begin{aligned} net_{h}= \begin{bmatrix} 0.1 &{} -0.2 &{} 0.2\\ 0.2 &{} -0.1 &{} 0.1\\ \end{bmatrix} * \begin{bmatrix} 0.1\\ 0.2\\ 0.3\\ \end{bmatrix} + \begin{bmatrix} 0.1\\ -0.1\\ \end{bmatrix} = \begin{bmatrix} 0.13\\ -0.07\\ \end{bmatrix} \end{aligned}$$

(16)

Calculate the output of hidden layer/latent space using Eq. (5) and the result for the first sample is shown below

$$\begin{aligned} O_{h} = \max \left( 0,\begin{bmatrix} 0.13 \\ -0.07 \\ \end{bmatrix}\right) = \begin{bmatrix} 0.13 \\ 0 \\ \end{bmatrix} \end{aligned}$$

(17)

Calculate the input of output layer using Eq. (6) and the result for the first sample is shown below

$$\begin{aligned} net_{k}= \begin{bmatrix} 0.1 &{} -0.1 \\ -0.2 &{} 0.2 \\ 0.1 &{} -0.1 \\ \end{bmatrix} * \begin{bmatrix} 0.13 \\ 0 \\ \end{bmatrix} + \begin{bmatrix} 0.1 \\ -0.2 \\ 0.1 \\ \end{bmatrix} = \begin{bmatrix} 0.11 \\ -0.23 \\ 0.11\\ \end{bmatrix} \end{aligned}$$

(18)

As linear activation function is used in the output layer, the output and input for output layer remains same and that is

$$\begin{aligned} net_{k}=L'_{k}=\begin{bmatrix} A' \\ B' \\ C' \\ \end{bmatrix}=\begin{bmatrix} 0.11 \ 0.23 \\ 0.11 \\ \end{bmatrix} \end{aligned}$$

(19)

Calculate the change in weights connected to each neuron in the output layer using Eq. (20). Where n is a neuron in output layer.

$$\begin{aligned} \Delta W_{hk}^{n} = \eta * (L(n)-L'(n)) * o_{j} \end{aligned}$$

(20)

$$\begin{aligned} \Delta W_{hk}^{1} = 0.1 * (0.1-0.11) * \begin{bmatrix} 0.13\\ 0\\ \end{bmatrix}=\begin{bmatrix} -0.0001\\ 0\\ \end{bmatrix} \end{aligned}$$

(21)

$$\begin{aligned} \Delta W_{hk}^{2} = 0.1 * (0.2+0.23) * \begin{bmatrix} 0.13\\ 0\\ \end{bmatrix}=\begin{bmatrix} 0.0056\\ 0\\ \end{bmatrix} \end{aligned}$$

(22)

$$\begin{aligned} \Delta W_{hk}^{3} = 0.1 * (0.3-0.11) * \begin{bmatrix} 0.13\\ 0\\ \end{bmatrix}=\begin{bmatrix} 0.0025\\ 0\\ \end{bmatrix} \end{aligned}$$

(23)

Finally, change in weights between latent space and output layer is given by

$$\begin{aligned} \Delta W_{hk} =\begin{bmatrix} \Delta W_{hk}^{1} &{} \Delta W_{hk}^{2} &{} \Delta W_{hk}^{3}\\ \end{bmatrix}=\begin{bmatrix} -0.0001 &{} 0.0056 &{} 0.0025\\ 0 &{} 0 &{} 0\\ \end{bmatrix} \end{aligned}$$

(24)

Now update the weights between latent space and output layer using equation (25).

$$\begin{aligned} W_{hk}=W_{hk}+\Delta W_{hk} \end{aligned}$$

(25)

$$\begin{aligned} W_{hk}=\begin{bmatrix} 0.1 &{} -0.2 &{} 0.1\\ -0.1 &{} 0.2 &{} -0.1\\ \end{bmatrix}+\begin{bmatrix} -0.0001 &{} 0.0056 &{} 0.0025\\ 0 &{} 0 &{} 0\\ \end{bmatrix}= \begin{bmatrix} 0.0999 &{} -0.1944 &{} 0.1025\\ -0.1 &{} 0.2 &{} -0.1\\ \end{bmatrix} \end{aligned}$$

(26)

Calculate the change in bias connected to each neuron in the output layer using Eq. (27) and update bias parameters using Eq. (29)

$$\begin{aligned} \Delta b_{k}=\eta * \left( \begin{bmatrix} A\\ B\\ C\\ \end{bmatrix}-\begin{bmatrix} A'\\ B'\\ C'\\ \end{bmatrix}\right) \end{aligned}$$

(27)

$$\begin{aligned} \Delta b_{k}=0.1 * \left( \begin{bmatrix} 0.1\\ 0.2\\ 0.3\\ \end{bmatrix}-\begin{bmatrix} 0.11\\ -0.23\\ 0.11\\ \end{bmatrix}\right) = \begin{bmatrix} -0.001\\ 0.043\\ 0.019\\ \end{bmatrix} \end{aligned}$$

(28)

$$\begin{aligned} b_{k}=b_{k}+\Delta b_{k} \end{aligned}$$

(29)

$$\begin{aligned} b_{k}=\begin{bmatrix} 0.1\\ -0.2\\ 0.1\\ \end{bmatrix}+\begin{bmatrix} -0.001\\ 0.043\\ 0.019\\ \end{bmatrix} = \begin{bmatrix} 0.099\\ -0.157\\ 0.119\\ \end{bmatrix} \end{aligned}$$

(30)

Calculate the change in weights connected to each neuron in the latent space using Eq. (31). Where n is a neuron in output layer and j is neuron in latent space.

$$\begin{aligned} \Delta W_{ih}^{j} = \eta * L * \sum (L(n)-L'(n)) * W_{hk}^{n} \end{aligned}$$

(31)

$$\begin{aligned} \Delta W_{ih}^{1} = 0.1 * \begin{bmatrix} 0.1\\ 0.2\\ 0.3\\ \end{bmatrix} * [(0.1-0.11)*0.0999+(0.2+0.23)*-0.1944+(0.3-0.11)*0.1025] \end{aligned}$$

(32)

$$\begin{aligned} \Delta W_{ih}^{1} = \begin{bmatrix} -0.0007\\ -0.0013\\ -0.002\\ \end{bmatrix} \end{aligned}$$

(33)

$$\begin{aligned} \Delta W_{ih}^{2} = 0.1 * \begin{bmatrix} 0.1\\ 0.2\\ 0.3\\ \end{bmatrix} * [(0.1-0.11)*-0.1+(0.2+0.23)*0.2+(0.3-0.11)*-0.1] \end{aligned}$$

(34)

$$\begin{aligned} \Delta W_{ih}^{2} = \begin{bmatrix} 0.0007\\ 0.0014\\ 0.002\\ \end{bmatrix} \end{aligned}$$

(35)

Finally, change in weights between latent space and input layer is given by

$$\begin{aligned} \Delta W_{ih} =\begin{bmatrix} \Delta W_{ih}^{1} &{} \Delta W_{ih}^{2} \\ \end{bmatrix}=\begin{bmatrix} -0.0007 &{} 0.0007\\ -0.0013 &{} 0.0014\\ -0.002 &{} 0.002\\ \end{bmatrix} \end{aligned}$$

(36)

Now update the weights between latent space and input layer using Eq. (37).

$$\begin{aligned} W_{ih}=W_{ih}+\Delta W_{ih} \end{aligned}$$

(37)

$$\begin{aligned} W_{ih}=\begin{bmatrix} 0.1 &{} 0.2\\ -0.2 &{} -0.1\\ 0.2 &{} 0.1 \end{bmatrix} +\begin{bmatrix} -0.0007 &{} 0.0007\\ -0.0013 &{} 0.0014\\ -0.002 &{} 0.002\\ \end{bmatrix}= \begin{bmatrix} 0.0993 &{} 0.2007\\ -0.2013 &{} 0.1014\\ 0.198/ &{} 0.102\\ \end{bmatrix} \end{aligned}$$

(38)

Calculate the change in bias connected to each neuron “j” in the latent space using Eq. (39) and update bias parameters using equation (??).

$$\begin{aligned} \Delta b_{h}^{j}= \eta * \sum (L(n)-L'(n)) * W_{hk}^{n} \end{aligned}$$

(39)

$$\begin{aligned} \Delta b_{h}^{1} = 0.1 * [(0.1-0.11)*0.0999+(0.2+0.23)*-0.1944+(0.3-0.11)*0.1025]=-0.0065 \end{aligned}$$

(40)

$$\begin{aligned} \Delta b_{h}^{2} = 0.1 * [(0.1-0.11)*-0.1+(0.2+0.23)*0.2+(0.3-0.11)*-0.1] = 0.0068 \end{aligned}$$

(41)

Finally, change in bias parameter for the neurons in latent space are given by

$$\begin{aligned} \Delta b_{h} =\begin{bmatrix} \Delta b_{h}^{1} &{} \Delta b_{h}^{2} \\ \end{bmatrix}=\begin{bmatrix} -0.0065\\ 0.0068\\ \end{bmatrix} \end{aligned}$$

(42)

$$\begin{aligned} b_{h}=b_{h}+\Delta b_{h} \end{aligned}$$

(43)

$$\begin{aligned} b_{h}=\begin{bmatrix} 0.1\\ -0.1\\ \end{bmatrix}+\begin{bmatrix} -0.0065\\ 0.0068\\ \end{bmatrix} = \begin{bmatrix} 0.0935\\ -0.0932\\ \end{bmatrix} \end{aligned}$$

(44)

• Iteration 01

  • Sample 02 = [ 0.3 0.2 0.1]

Calculate net  input to hidden layer/latent space using Eq. (4) and the result for the second sample is shown below

$$\begin{aligned} net_{h}= \begin{bmatrix} 0.0993 &{} -0.2013 &{} 0.198 \\ 0.2007 &{} 0.1014 &{} 0.102 \\ \end{bmatrix} * \begin{bmatrix} 0.3 \\ 0.2 \\ 0.1 \\ \end{bmatrix}+\begin{bmatrix} 0.0935 \\ -0.0932 \\ \end{bmatrix} = \begin{bmatrix} 0.1028 \\ -0.0025 \\ \end{bmatrix} \end{aligned}$$

(45)

Calculate the output of hidden layer/latent space using Eq. (5) and the result for the first sample is shown below

$$\begin{aligned} O_{h}=max(0,\begin{bmatrix} 0.1028\\ -0.0025\\ \end{bmatrix})=\begin{bmatrix} 0.1028\\ 0\\ \end{bmatrix} \end{aligned}$$

(46)

Calculate the input of output layer using Eq. (6) and the result for the first sample is shown below

$$\begin{aligned} net_{k} = \begin{bmatrix} 0.0999 &{} -0.1 \\ -0.1944 &{} 0.2 \\ 0.1025 &{} -0.1 \\ \end{bmatrix} * \begin{bmatrix} 0.1028 \\ 0 \\ \end{bmatrix} + \begin{bmatrix} 0.099 \\ -0.157 \\ 0.119 \\ \end{bmatrix}=\begin{bmatrix} 0.1093 \\ -0.177 \\ 0.1295 \\ \end{bmatrix} \end{aligned}$$

(47)

As linear activation function is used in the output layer, the output and input for output layer remains same and that is

$$\begin{aligned} net_{k}=L'_{k}=\begin{bmatrix} A' \\ B' \\ C' \\ \end{bmatrix}= \begin{bmatrix} 0.1093 \\ -0.177 \\ 0.1295\\ \end{bmatrix} \end{aligned}$$

(48)

Calculate the change in weights connected to each neuron in the output layer using Eq. (20).

$$\begin{aligned} \Delta W_{hk}^{1} = 0.1 * (0.3-0.1093) * \begin{bmatrix} 0.1028\\ 0\\ \end{bmatrix}=\begin{bmatrix} -0.002\\ 0\\ \end{bmatrix} \end{aligned}$$

(49)

$$\begin{aligned} \Delta W_{hk}^{2} = 0.1 * (0.2+0.177) * \begin{bmatrix} 0.1028\\ 0\\ \end{bmatrix}=\begin{bmatrix} 0.0039\\ 0\\ \end{bmatrix} \end{aligned}$$

(50)

$$\begin{aligned} \Delta W_{hk}^{3} = 0.1 * (0.1-0.1295) * \begin{bmatrix} 0.1028\\ 0\\ \end{bmatrix}=\begin{bmatrix} -0.0003\\ 0\\ \end{bmatrix} \end{aligned}$$

(51)

Finally, change in weights between latent space and output layer is given by

$$\begin{aligned} \Delta W_{hk} =\begin{bmatrix} \Delta W_{hk}^{1} &{} \Delta W_{hk}^{2} &{} \Delta W_{hk}^{3}\\ \end{bmatrix}=\begin{bmatrix} 0.00196 &{} 0.00388 &{} -0.0003\\ 0 &{} 0 &{} 0\\ \end{bmatrix} \end{aligned}$$

(52)

Now update the weights between latent space and output layer using Eq. (25).

$$\begin{aligned} W_{hk}=\begin{bmatrix} 0.0999 &{} -0.1944 &{} 0.1025\\ -0.1 &{} 0.2 &{} -0.1\\ \end{bmatrix}+\begin{bmatrix} 0.00196 &{} 0.00388 &{} -0.0003\\ 0 &{} 0 &{} 0\\ \end{bmatrix}= \begin{bmatrix} 0.1019 &{} -0.1905 &{} 0.1022\\ -0.1 &{} 0.2 &{} -0.1\\ \end{bmatrix} \end{aligned}$$

(53)

Calculate the change in bias connected to each neuron in the output layer using Eq. (27) and update bias parameters using Eq. (29)

$$\begin{aligned} \Delta b_{k}=0.1 * \left( \begin{bmatrix} 0.3\\ 0.2\\ 0.1\\ \end{bmatrix}-\begin{bmatrix} 0.1093\\ -0.177\\ 0.1295\\ \end{bmatrix}\right) = \begin{bmatrix} 0.019\\ 0.038\\ 0.0103\\ \end{bmatrix} \end{aligned}$$

(54)

$$\begin{aligned} b_{k}=\begin{bmatrix} 0.099\\ -0.157\\ 0.119\\ \end{bmatrix}+\begin{bmatrix} 0.019\\ 0.038\\ 0.0103\\ \end{bmatrix} = \begin{bmatrix} 0.118\\ -0.119\\ 0.1293\\ \end{bmatrix} \end{aligned}$$

(55)

Calculate the change in weights connected to each neuron in the latent space using Eq. (31). Where n is a neuron in output layer and j is neuron in latent space.

$$\begin{aligned} \Delta W_{ih}^{1} = 0.1 * \begin{bmatrix} 0.3\\ 0.2\\ 0.1\\ \end{bmatrix} * [(0.3-0.1093)*0.1019+(0.2+0.177)*-0.1905+(0.1-0.1295)*0.1022] \end{aligned}$$

(56)

$$\begin{aligned} \Delta W_{ih}^{1} = \begin{bmatrix} -0.0017\\ -0.0011\\ -0.0006\\ \end{bmatrix} \end{aligned}$$

(57)

$$\begin{aligned} \Delta W_{ih}^{2} = 0.1 * \begin{bmatrix} 0.3\\ 0.2\\ 0.1\\ \end{bmatrix} * [(0.3-0.1093)*-0.1+(0.2+0.177)*0.2+(0.1-0.1295)*-0.1] \end{aligned}$$

(58)

$$\begin{aligned} \Delta W_{ih}^{2} = \begin{bmatrix} 0.0018\\ 0.0012\\ 0.0006\\ \end{bmatrix} \end{aligned}$$

(59)

Finally, change in weights between latent space and input layer is given by

$$\begin{aligned} \Delta W_{ih} =\begin{bmatrix} \Delta W_{ih}^{1} &{} \Delta W_{ih}^{2} \\ \end{bmatrix}=\begin{bmatrix} -0.0017 &{} 0.0018\\ -0.0011 &{} 0.0012\\ -0.0006 &{} 0.0006\\ \end{bmatrix} \end{aligned}$$

(60)

Now update the weights between latent space and input layer using Eq. (37).

$$\begin{aligned} W_{ih}=\begin{bmatrix} 0.0993 &{} 0.2007\\ -0.2013 &{} 0.1014\\ 0.198/ &{} 0.102\\ \end{bmatrix} + \begin{bmatrix} -0.0017 &{} 0.0018\\ -0.0011 &{} 0.0012\\ -0.0006 &{} 0.0006\\ \end{bmatrix}= \begin{bmatrix} 0.0976 &{} 0.2025\\ -0.2024 &{} 0.1026\\ 0.1974/ &{} 0.1026\\ \end{bmatrix} \end{aligned}$$

(61)

Calculate the change in bias connected to each neuron “j” in the latent space using Eq. (39) and update bias parameters using Eq. (29).

$$\begin{aligned} \Delta b_{h}^{1} = 0.1 * [(0.3-0.1093)*0.1019+(0.2+0.177)*-0.1905+(0.1-0.1295)*0.1022]=-0.0055 \end{aligned}$$

(62)

$$\begin{aligned} \Delta b_{h}^{2} = 0.1 * [(0.3-0.1093)*-0.1+(0.2+0.177)*0.2+(0.1-0.1295)*-0.1] = 0.0059 \end{aligned}$$

(63)

Finally, change in bias parameter for the neurons in latent space are given by

$$\begin{aligned} \Delta b_{h} =\begin{bmatrix} \Delta b_{h}^{1} &{} \Delta b_{h}^{2} \\ \end{bmatrix}=\begin{bmatrix} -0.0055\\ 0.0059\\ \end{bmatrix} \end{aligned}$$

(64)

$$\begin{aligned} b_{h}=\begin{bmatrix} 0.0935\\ -0.0932\\ \end{bmatrix}+\begin{bmatrix} -0.0055\\ 0.0059\\ \end{bmatrix} = \begin{bmatrix} 0.088\\ -0.0873\\ \end{bmatrix} \end{aligned}$$

(65)

$$\begin{aligned} net_{h}= \begin{bmatrix} 0.0993 &{} -0.2013 &{} 0.198\\ 0.2007 &{} 0.1014 &{} 0.102\\ \end{bmatrix} * \begin{bmatrix} 0.3\\ 0.2\\ 0.1\\ \end{bmatrix}+\begin{bmatrix} 0.0935\\ -0.0932\\ \end{bmatrix} = \begin{bmatrix} 0.1028\\ -0.0025\\ \end{bmatrix} \end{aligned}$$

(66)

Calculate the output of hidden layerlatent space using Eq. (5) and the result for the first sample is shown below

$$\begin{aligned} O_{h} = \max \left( 0, \begin{bmatrix} 0.1028\\ -0.0025\\ \end{bmatrix}\right) = \begin{bmatrix} 0.1028\\ 0\\ \end{bmatrix} \end{aligned}$$

(67)

1.3 Reconstruction of Data Using Trained Autoencoder

Encoder part of trained autoencoder which is shown in Fig. 6 is used to reconstruct the data. Weight and bias matrices of encoder part are shown  below.

Fig. 6

Encoder part of sample autoencoder architecture

$$\begin{aligned} W_{ih}=\begin{bmatrix} 0.0976 &{} 0.2025\\ -0.2024 &{} 0.1026\\ 0.1974/ &{} 0.1026\\ \end{bmatrix} \end{aligned}$$

(68)

$$\begin{aligned} b_{h}=\begin{bmatrix} 0.088\\ -0.0873\\ \end{bmatrix} \end{aligned}$$

(69)

A sample data [0.2,0.3,0.4] is reconstructed into a data sample consists only two features as shown below

$$\begin{aligned} net_{h}= \begin{bmatrix} 0.0976 &{} -0.2024 &{} 0.1974\\ 0.2025 &{} 0.1026 &{} 0.1026\\ \end{bmatrix} * \begin{bmatrix} 0.2\\ 0.3\\ 0.4\\ \end{bmatrix}+\begin{bmatrix} 0.088\\ -0.0873\\ \end{bmatrix} = \begin{bmatrix} 0.13\\ 0.01\\ \end{bmatrix} \end{aligned}$$

(70)

The output of hidden layer/latent space is reconstructed data.

$$\begin{aligned} O_{h}=\begin{bmatrix} F_{1}\\ F_{2}\\ \end{bmatrix}=\max \left( 0,\begin{bmatrix} 0.13\\ 0.01\\ \end{bmatrix}\right) =\begin{bmatrix} 0.13\\ 0.01\\ \end{bmatrix} \end{aligned}$$

(71)