Mining Load Profile Patterns for Australian Electricity Consumers

  • Vanh Khuyen NguyenEmail author
  • Wei Emma Zhang
  • Quan Z. Sheng
  • Jason Merefield
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10604)


The transformation from centralized and fossil-based electricity generation to distributed and renewable energy sources is an inevitable trend in the energy industry. One of the prime challenges in this transformation is the task of load/battery management, especially at the residential level. In solving this task, it is critical that a good strategy for analyzing and grouping residential electricity consumption patterns is in place so that further optimization strategies can be devised for different groups of consumers. Based on the real data from an Australian electricity retailer, we propose a clustering process to determine typical customer load profiles. It can be served as a standard framework for dealing with real-world unsupervised problems. In addition, some statistical techniques, including cumulative sum and calculation of the most frequent value in dataset by using mode, are integrated into our data preprocessing and analysis. CUSUM chart is a graphical method to clearly visualize as well as detect changes in time-series data and then using mode values is to replace missing values in the dataset. Furthermore, in our framework, more practical Elbow method is conducted to determine appropriated number of clusters for k-centers algorithm. We then apply multiple state-of-the-art clustering methods for time series data and benchmark their respective performance. We found that k-centers clustering techniques produces better results compared to exemplar-based methods. Additionally, choosing appropriated number of clusters for k-means can improve performance of clustering model. For example, k-means++ with \(k=2\) has significantly outperformed other methods in our experiment.


Time series clustering Residential electricity consumption Data mining 



This study was funded by Capital Markets Cooperative Research Centre (CMCRC) ( and supported for data collection by Mojo Power, Australia.


  1. 1.
    AEMO. Emerging Technologies Information Paper (2015)Google Scholar
  2. 2.
    Albanese, D., Visintainer, R., Merler, S., Riccadonna, S., Jurman, G., Furlanello, C.: mlpy: Machine Learning Python. CoRR (2012)Google Scholar
  3. 3.
    Anuar, N., Zakaria, Z.: Electricity load profile determination by using fuzzy C-means and probability neural network. Energ. Procedia 14, 1861–1869 (2012)CrossRefGoogle Scholar
  4. 4.
    Arthur, D., Vassilvitskii, S.: k-means++: the advantages of careful seeding (2007)Google Scholar
  5. 5.
    Bholowalia, P., Kumar, A.: EBK-means: a clustering technique based on elbow method and K-means in WSN. IJCA 105(9), 17–24 (2014)Google Scholar
  6. 6.
    Chicco, G., Napoli, R., Piglione, F.: Comparisons among clustering techniques for electricity customer classification. IEEE Trans. Power Syst. 21, 933–940 (2006)CrossRefGoogle Scholar
  7. 7.
    Frey, B., Dueck, D.: Clustering by passing messages between data points. Science 315(5814), 972–976 (2007). (Washington)MathSciNetCrossRefzbMATHGoogle Scholar
  8. 8.
    Fu, T.C.: A review on time series data mining. Eng. Appl. Artif. Intell. 24(1), 164–181 (2011)CrossRefGoogle Scholar
  9. 9.
    Gerbec, D., Gasperic, S., Smon, I., Gubina, F.: Allocation of the load profiles to consumers using probabilistic neural networks. IEEE Trans. Power Syst. 20(2), 548–555 (2005)CrossRefGoogle Scholar
  10. 10.
    Hino, H., Shen, H., Murata, N., Wakao, S., Hayashi, Y.: A versatile clustering method for electricity consumption pattern analysis in households. IEEE Trans. Smart Grid 4(2), 1048–1057 (2013)CrossRefGoogle Scholar
  11. 11.
    Hou, L., Kwok, J.T., Zurada, J.M.: Efficient learning of timeseries shapelets. In: Proceedings - 30th AAAI on Artificial Intelligence, pp. 1209–1215 (2016)Google Scholar
  12. 12.
    Mcloughlin, F., Duffy, A., Conlon, M.: A clustering approach to domestic electricity load profile characterisation using smart metering data. Appl. Energ. 141, 190–199 (2015)CrossRefGoogle Scholar
  13. 13.
    Mesnil, B., Petitgas, P.: Detection of changes in time-series of indicators using CUSUM control charts. Aquat. Living Res. 22(2), 187–192 (2009)CrossRefGoogle Scholar
  14. 14.
    Mesquita, D., Gomes, J., Rodrigues, L.: K-means for datasets with missing attributes: building soft constraints with observed and imputed values. In: 24th ESANN, pp. 27–29 (2016)Google Scholar
  15. 15.
    Paparrizos, J., Gravano, L.: k-shape. ACM SIGMOD Rec. 45(1), 69–76 (2016)CrossRefGoogle Scholar
  16. 16.
    Pedregosa, F., Varoquaux, G., Gramfort, A., Michel, V., Thirion, B., Grisel, O., Blondel, M., Prettenhofer, P., Weiss, R., Dubourg, V., Vanderplas, J., Passos, A., Cournapeau, D., Brucher, M., Perrot, M., Duchesnay, E.: Scikit-learn: machine learning in python. J. Mach. Learn. Res. 12, 2825–2830 (2011)Google Scholar
  17. 17.
    Rani, S., Sikka, G., Liao, T.W.: Recent techniques of clustering of time series data: a survey. Pattern Recognit. 52(15), 1–9 (2005)Google Scholar
  18. 18.
    Rousseeuw, P.J.: Silhouettes: a graphical aid to the interpretation and validation of cluster analysis. J. Comput. Appl. Math. 20(C), 53–65 (1987)Google Scholar
  19. 19.
    Schaul, T., Bayer, J., Wierstra, D., Sun, Y., Felder, M., Sehnke, F., Ruckstiess, T., Schmidhuber, J.: PyBrain. J. Mach. Learn. Res. 11, 743–746 (2010)Google Scholar
  20. 20.
    Sculley, D.: Web-scale k-means clustering. In: Proceedings - 19th WWW, p. 1177 (2010)Google Scholar
  21. 21.
    Wagstaff, K.: Clustering with missing values: no imputation required. In: Banks, D., McMorris, F.R., Arabie, P., Gaul, W. (eds.) Classification, Clustering, and Data Mining Applications. Studies in Classification, Data Analysis, and Knowledge Organisation, pp. 649–658. Springer, Heidelberg (2004)Google Scholar
  22. 22.
    Wang, C.D., Lai, J.H., Suen, C.Y., Zhu, J.Y.: Multi-exemplar affinity propagation. IEEE Trans. Pattern Anal. Mach. Intell. 35(9), 2223–2237 (2013)CrossRefGoogle Scholar
  23. 23.
    Zakaria, J., Mueen, A., Keogh, E.: Clustering time series using unsupervised-shapelets. In: Proceedings - IEEE ICDM, pp. 785–794 (2012)Google Scholar

Copyright information

© Springer International Publishing AG 2017

Authors and Affiliations

  • Vanh Khuyen Nguyen
    • 1
    Email author
  • Wei Emma Zhang
    • 1
  • Quan Z. Sheng
    • 1
  • Jason Merefield
    • 2
  1. 1.Department of ComputingMacquarie UniversitySydneyAustralia
  2. 2.Mojo Power CompanySydneyAustralia

Personalised recommendations