Skip to main content


Log in

Predicting unusual energy consumption events from smart home sensor network by data stream mining with misclassified recall

  • Original Research
  • Published:
Journal of Ambient Intelligence and Humanized Computing Aims and scope Submit manuscript


With the popularity and affordability of ZigBee wireless sensor technology, IoT-based smart controlling system for home appliances becomes prevalent for smart home applications. From the data analytics point of view, one important objective from analyzing such IoT data is to gain insights from the energy consumption patterns, thereby trying to fine-tune the energy efficiency of the appliance usage. The data analytics usually functions at the back-end crunching over a large archive of big data accumulated over time for learning the overall pattern from the sensor data feeds. The other objective of the analytics, which may often be more crucial, is to predict and identify whether an abnormal consumption event is about to happen. For example, a sudden draw of energy that leads to hot spot in the power grid in a city, or black-out at home. This dynamic prediction is usually done at the operational level, with moving data stream, by data stream mining methods . In this paper, an improved version of very fast decision tree (VFDT) is proposed, which learns from misclassified results for the sake of filtering the noisy data from learning and maintaining sharp classification accuracy of the induced prediction model. Specifically, a new technique called misclassified recall (MR), which is a pre-processing step for self-rectifying misclassified instances, is formulated. In energy data prediction, most misclassified instances are due to data transmission errors or faulty devices. The former case happens intermittently, and the errors from the latter cause may persist for a long time. By caching up the data at the MR pre-processor, the one-pass online model learning can be effectively shielded in case of intermitting problems at the wireless sensor network; likewise the stored data could be investigated afterwards should the problem persist for long. Simulation experiments over a dataset about predicting exceptional appliances energy use in a low energy building are conducted. The reported results validate the efficacy of the new methodology VFDT + MR, in comparison to a collection of popular data stream mining algorithms from the literature.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Subscribe and save

Springer+ Basic
EUR 32.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or Ebook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Fig. 1
Fig. 2

Image courtesy by

Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13

Similar content being viewed by others


  • Bottou L, Bousquet O (2008) The tradeoffs of large scale learning. Adv Neural Inf Process Syst 20:161–168

    Google Scholar 

  • Candanedo LM, Feldheim V, Deramaix D (2017) Data driven prediction models of energy use of appliances in a low-energy house, energy and buildings, vol 140. Springer, New York, pp 81–97

  • Ceci M, Cassavia N, Corizzo R, Dicosta P, Malerba D, Maria G, Masciari E, Pastura C (2015) Big data techniques for renewable energy market,, 2015.

  • Ceci M, Corizzo R, Fumarola F, Ianni M, Malerba D, Maria G, Masciari E, Oliverio M, Rashkovska A (2015) Big data techniques for supporting accurate predictions of energy production from renewable sources. In: IDEAS ‘15 proceedings of the 19th international database engineering & applications symposium, pp. 62–71

  • Cerquitelli T (2017) Predicting large scale fine grain energy consumption. In: 8th international conference on sustainability in energy and buildings, SEB-16, energy procedia 111 (2017), pp. 1079–1088

  • Choi JH, Shin HS, Nasridinov A (2016) A comparative study on data mining classification techniques for military applications. J Converg 7.

  • Darrell S, Indyk (eds). (2005) Nearest-neighbor methods in learning and vision. MIT Press, Cambridge, 0-262-19547-X

  • Domingos P, Hulten G (2000) Mining high-speed data streams. In: Proceedings KDD 2000, ACM Press, New York, NY, USA, pp. 71–80

  • Egarter D, Monacchi A, Khatib T, Elmenreich W (2016) Integration of legacy appliances into home energy management systems. J Ambient Intell Hum Comput 7(2):171–185

    Article  Google Scholar 

  • Finogeev AG, Parygin DS, Finogeev AA (2017) The convergence computing model for big sensor data mining and knowledge discovery. Human Centri Comput Inf Sci 7(11):1–16

    Google Scholar 

  • Fong S, Hu S, Wei S, Cho K, Wong RK, Mohammed S (2017) On recognizing abnormal human behaviors by data stream mining with misclassified recalls. In: WWW ‘17 proceedings of the 26th international conference on World Wide Web companion, pp. 1129–1135

  • Ikonomovska E, Gama J, Džeroski S (2010) Learning model trees from evolving data streams. Data Min Knowl Discov 23(1):128–168

    Article  MathSciNet  MATH  Google Scholar 

  • Ikonomovska E, Gama J, Zenko B Džeroski S (2011) Speeding-up hoeffding-based regression trees with options. ICML 2011:537–544

    Google Scholar 

  • Kang WM, Moon SY, Park JH (2017) An enhanced security framework for home appliances in smart home. Hum Centric Comput Inf Sci 7(6):1–12

    Google Scholar 

  • Lachut D, Banerjee N, Rollins S, (2014) Predictability of energy use in homes. In: 2014 international green computing conference (IGCC), 3–5 Nov. 2014, pp. 1–10

  • Maity S, Park JH (2016) Powering IoT devices: a novel design and analysis technique. J Converg 7:1–18 (Article ID 16071001)

    Google Scholar 

  • Martin EG, Lavesson N, Grahn H (2015) Energy efficiency in data stream mining. In: 2015 IEEE/ACM international conference on advances in social networks analysis and mining (ASONAM), 25–28 Aug. 2015, Paris, France, pp. 1125–1132

  • Mocanu E, Nguyen PH, Gibescu M, Kling WL (2014) Comparison of machine learning methods for estimating energy consumption in buildings. In: 2014 international conference on probabilistic methods applied to power systems (PMAPS), 7–10 July 2014, pp. 1–6

  • Pughat A, Sharma V (2015) A review on stochastic approach for dynamic power management in wireless sensor networks. Human Centric Comput Inf Sci 4:1–14

    Google Scholar 

  • Parra J Jr, Kiekintveld C (2013) Initial exploration of machine learning to predict customer demand in an energy market simulation, trading agent design and analysis: papers from the AAAI 2013 workshop, pp 29–32

  • Perera KS, Aung Z, Woon WL (2014) Machine learning techniques for supporting renewable energy generation and integration: a survey. In: DARE'14 Proceedings of the second international conference on data analytics for renewable energy integration, Nancy, France. Springer, Switzerland, pp 81–96.

  • Quinlan R (1993) C4.5: programs for machine learning. Morgan Kaufmann Publishers, San Mateo

  • Russell S, Norvig P (2003) [1995] Artificial intelligence: a modern approach (2nd edn). Prentice Hall, New Jersey, 978-0137903955

  • Settles B (2010) Active learning literature survey, computer sciences technical report 1648. University of Wisconsin–Madison, retrieved 2014-11-18

  • Shcherbakov MV, Brebels A, Shcherbakova NL, Kamaev VA, Gerget OM, Devyatykh D (2016) Outlier detection and classification in sensor data streams for proactive decision support systems. J Phys Conf Ser 803(Number 1):1–10

    Google Scholar 

  • Song T-W, Yang C-S (2008) A connectivity improving mechanism for zigbee wireless sensor networks. In: IEEE/IFIP international conference on embedded and ubiquitous computing, 2008. EUC ‘08. 17–20 Dec. 2008, pp. 495–500

  • Verma P, Sood SK, Kalra S (2017) Cloud-centric IoT based student healthcare monitoring framework, J Ambient Intell Hum Comput. (Springer)

  • Wang D, Fong S, Wong RK, Mohammed S, Fiaidhi F, Wong KKL (2017) Robust high-dimensional bioinformatics data streams mining by ODR-ioVFDT. Sci Rep 7:43167.

  • Yang H, Fong S (2013) Countering the concept-drift problem in big data using iOVFDT. In: Proceedings of the 2013 IEEE international congress BIGDATACONGRESS’13, June 27–July 02, 2013, pp. 126–132

Download references

Author information

Authors and Affiliations


Corresponding author

Correspondence to Wei Song.



See Table 4

Table 4 Notation of the symbols defined for the model formulation

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Fong, S., Li, J., Song, W. et al. Predicting unusual energy consumption events from smart home sensor network by data stream mining with misclassified recall. J Ambient Intell Human Comput 9, 1197–1221 (2018).

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: