Data Mining and Knowledge Discovery

, Volume 29, Issue 6, pp 1622–1664 | Cite as

A general framework for never-ending learning from time series streams

  • Yanping Chen
  • Yuan Hao
  • Thanawin Rakthanmanon
  • Jesin Zakaria
  • Bing Hu
  • Eamonn Keogh


Time series classification has been an active area of research in the data mining community for over a decade, and significant progress has been made in the tractability and accuracy of learning. However, virtually all work assumes a one-time training session in which labeled examples of all the concepts to be learned are provided. This assumption may be valid in a handful of situations, but it does not hold in most medical and scientific applications where we initially may have only the vaguest understanding of what concepts can be learned. Based on this observation, we propose a never-ending learning framework for time series in which an agent examines an unbounded stream of data and occasionally asks a teacher (which may be a human or an algorithm) for a label. We demonstrate the utility of our ideas with experiments that consider real-world problems in domains as diverse as medicine, entomology, wildlife monitoring, and human behavior analyses.


Never-ending learning Classification Data streams Time series 


  1. Achtert E, Bohm C, Kriegel H-P, Kröger P (2005) Online hierarchical clustering in a data warehouse environment data mining. ICDM, pp 10–17Google Scholar
  2. Ambert JD, Hodgman TP, Laurent EJ, Brewer GL, Iliff MJ, Dettmers R (2009) The northeast bird monitoring handbook. American Bird Conservancy, The Plains, VAGoogle Scholar
  3. Bache K, Lichman M (2013) UCI machine learning repository. University of California, School of Information and Computer Science, Irvine, CA.
  4. Bardeli R, Wolff D, Kurth F, Koch M, Frommolt KH (2010) Detecting bird sounds in a complex acoustic environment and application to bioacoustic monitoring. Pattern Recognit Lett 31:1524–1534CrossRefGoogle Scholar
  5. Barrenetxea G et al (2008) Sensorscope: out-of-the-box environmental monitoring. In: IPSN, San FranciscoGoogle Scholar
  6. Batista G, Keogh E, Mafra-Neto A, Rowton E (2011) Sensors and software to allow computational entomology, an emerging application of data mining. In: KDD, pp 761–764Google Scholar
  7. Berges M, Goldman E, Matthews HS, Soibelman L (2010) Enhancing electricity audits in residential buildings with non-intrusive load monitoring. J Ind Ecol 14(5):844–858CrossRefGoogle Scholar
  8. Berlin E, Laerhoven K (2012) Detecting leisure activities with dense motif discovery. In: Proceedings of the 2012 intl conference on uniquitous computing, pp 250–259Google Scholar
  9. Borazio M, Laerhoven K (2012) Combining wearable and environmental sensing into an unobtrusive tool for long-term sleep studies. In: 2nd ACM SIGHITGoogle Scholar
  10. Campanharo ASLO, Sirer MI, Malgren RD, Ramos FM, Nunes LAN (2011) Duality between time series and networks. Plos One 6:e23378CrossRefGoogle Scholar
  11. Carlson A, Betteridge J, Kisiel B, Settles B, Hruschka Jr ER, Mitchell TM (2010) Toward an architecture for never-ending language learning. In: Proc’ AAAIGoogle Scholar
  12. Charikar M, Chen K, Farach-Colton M (2002) Finding frequent items in data streams. InL Proceedings of the 29th ICALP international colloquium on automata, languages and programming, pp 693–703Google Scholar
  13. Chen Y (2014) Project webpage. Accessed 03 April 2014
  14. Chen Y, Why A, Batista G, Mafra-Neto A, Keogh E (2014) Flying insect classification with inexpensive sensors. J Insect Behav 27(5):657–677CrossRefGoogle Scholar
  15. Chiu B, Keogh E, Lonardi S (2003) Probabilistic discovery of time series motifs. In: Proceedings of the ninth ACM SIGKDD international conference on knowledge discovery and data mining, pp 493–498Google Scholar
  16. Cormode G, Hadjieleftheriou M (2010) Methods for finding frequent items in data streams. VLDB J 19(1):3–20CrossRefGoogle Scholar
  17. Dagan I, Engelson SP (1995) Committee-based sampling for training probabilistic classifiers. In: ICML, vol 95, pp 150–157Google Scholar
  18. Dawson DK, Efford MG (2009) Bird Population Density Estimated from Acoustic Signals. Journal of Applied Ecology. 46(6):1201–1209CrossRefGoogle Scholar
  19. Ding H, Trajcevski G, Scheuermann P, Wang X, Keogh EJ (2008) Querying and mining of time series data. Experimental comparison of representations and distance measures. PVLDB 1(2):1542–1552Google Scholar
  20. Dodge Y (2003) Oxford dictionary of statistical terms. OUP, Oxford. ISBN 0-19-850994-4MATHGoogle Scholar
  21. Elkan C, Noto K (2008) Learning classifiers from only positive and unlabeled data. KDD, pp 213–220Google Scholar
  22. Estan C, Varghese G (2003) New directions in traffic measurement and accounting: focusing on the elephants, ignoring the mice. ACM Trans Comput Syst 21(3):270–313CrossRefGoogle Scholar
  23. Ferreira F, Bota D, Bross A, Mélot C, Vincent J (2001) Serial evaluation of the SOFA score to predict outcome in critically ill patients. JAMA 286(14):1754–1758CrossRefGoogle Scholar
  24. Fujii A, Tokunaga T, Inui K, Tanaka H (1998) Selective sampling for example-based word sense disambiguation. Comput Linguist 24(4):573–597Google Scholar
  25. Goldberger A et al (2000) PhysioBank, PhysioToolkit, and PhysioNet: components of a new research resource for complex physiologic signals. Circulation 101(23):215–220CrossRefGoogle Scholar
  26. Goldberger A et al (2013) Physionet. Accessed 04 Feb 2013
  27. Google Prediction API. Accessed 31 Jul 2013
  28. Gupta S, Reynolds S, Patel SN (2010) ElectriSense: single-point sensing using EMI for electrical event detection and classification in the home. In: Proceedings of the conference on ubiquitous computingGoogle Scholar
  29. Hinman J, Hickey E (2009) Modeling and forecasting short-term electricity load using regression analysis. Working paper, Illinois State University, Normal (US), FallGoogle Scholar
  30. Holyoak DT (2001) Nightjars and their allies: the caprimulgiformes. Oxford University Press, Oxford, New York. ISBN 0-19-854987-3Google Scholar
  31. Hu B, Chen Y, Keogh EJ (2013) Time series classification under more realistic assumptions. In: SDMGoogle Scholar
  32. Jin C, Qian W, Sha C, Yu J, Zhou A (2003) Dynamically maintaining frequent items over a data stream. In: Proceedings of the 12th ACM CIKM international conference on information and knowledge management, pp 287–294Google Scholar
  33. Jin S, Chen Z, Backus E, Sun X, Xiao B (2012) Characterization of EPG waveforms for the tea green leafhopper on tea plants and their correlation with stylet activities. J Insect Physiol 58:1235–1244CrossRefGoogle Scholar
  34. Karp R, Papadimitriou C, Shenker S (2003) A simple algorithm for finding frequent elements in sets and bags. ACM Trans Database Syst 28:51–55CrossRefGoogle Scholar
  35. Keogh E, Zhu Q, Hu B, Hao Y, Xi X, Wei L, Ratanamahatana CA (2011) The UCR time series classification/clustering homepage.
  36. Kolter J, Jaakkola T (2012) Approximate inference in additive factorial HMMs with application to energy disaggregation. J Mach Learn Res 22:1472–1482Google Scholar
  37. Krishnamurthy A, Balakrishnan S, Xu M, Singh A (2012) Efficient active algorithms for hierarchical clustering. arXiv:1206.4672
  38. Lines J, Bagnall A, Caiger-Smith P, Anderson S (2011) Classification of household devices by electricity usage profiles. In: IDEAL, pp 403–412Google Scholar
  39. MacLeod J, Greene T, MacKenzie DI, Allen RB (2012) Monitoring widespread and common bird species on New Zealand’s conservation lands: a pilot study. N Z J Ecol 36(3):300–311Google Scholar
  40. Manku G, Motwani R (2002) Approximate frequency counts over data streams. In: International conference on very large databases, pp 346–357Google Scholar
  41. Mermelstein P (1976) Distance measures for speech recognition, psychological and instrumental. In: Chen CH (ed) Pattern recognition and artificial intelligence. Academic Press, New YorkGoogle Scholar
  42. Metwally A, Agrawal D, Abbadi AE (2005) Efficient computation of frequent and top-k elements in data streams. In: International conference on database theoryGoogle Scholar
  43. Mitchell L (1981) Time segregated mosquito collections with a CDC miniature light trap. Mosquito News 42:12Google Scholar
  44. Morales L, Arbetman MP, Cameron SA, Aizen MA (2013) Rapid ecological replacement of a native bumble bee by invasive species. Frontiers Ecol Environ 11:529–534CrossRefGoogle Scholar
  45. Mueen A, Keogh EJ, Young N (2011) Logical-shapelets: an expressive primitive for time series classification. In: KDD, pp 1154–1162Google Scholar
  46. Mueen A, Keogh EJ (2010) Online discovery and maintenance of time series motifs. In: KDD, pp 1089–1098Google Scholar
  47. Nassar S, Sander J, Cheng C (2004) Incremental and effective data summarization for dynamic hierarchical clustering. In: SIGMOD Conference, pp 467–478Google Scholar
  48. Norris JR (1998) Markov chains. Cambridge university press, CambridgeMATHGoogle Scholar
  49. PAMAP (2012) Physical activity monitoring for aging people. Retrieved 2012
  50. Robbins CS (1981) Effect of time of day on bird activity. Stud Avian Biol 6:275–286Google Scholar
  51. Roggen D et al (2012) Collecting complex activity data sets in highly rich networked sensor environments. In: Proc’ 7th IEEE INSS, pp 233–240Google Scholar
  52. Rakthanmanon T, Campana B, Mueen A, Batista G, Westover B, Zhu Q et al (2012) Searching and mining trillions of time series subsequences under dynamic time warping. Proceedings of the 18th ACM SIGKDD intersnational conference on knowledge discovery and data mining. ACM, New York, pp 262–270Google Scholar
  53. Rowling JK (1997) Harry Potter and the chamber of secrets. Levine Books (scholastics), New York, Read by Stephan Fry, Arthur AGoogle Scholar
  54. Salton G, McGill MJ (1986) Introduction to modern information retrieval. McGraw-Hill, New York. ISBN 0-07-054484-0Google Scholar
  55. Settles B (2012) Active learning. Morgan & Claypool, San RafaelMATHGoogle Scholar
  56. Settles B, Craven M, Friedland L (2008) Active learning with real annotation costs. In: Proceedings of the NIPS workshop on cost-sensitive learningGoogle Scholar
  57. Shrivastava N, Buragohain C, Agrawal D, Suri S (2004) Medians and beyond: new aggregation techniques for sensor networks. In: ACM SenSysGoogle Scholar
  58. Stikic M, Huynh T, Laerhoven KV, Schiele B (2008) ADL recognition based on the combination of RFID and accelerometer sensing. In: PervasiveHealth, pp 258–263Google Scholar
  59. Tur G, Hakkani-Tür D, Schapire RE (2005) Combining active and semi-supervised learning for spoken language understanding. Speech Commun 45(2):171–186CrossRefGoogle Scholar
  60. Van Rijsbergen J (1979) Information retrieval, 2nd edn. Butterworths, LondonGoogle Scholar
  61. Veeraraghavan A, Chellappa R, Roy-Chowdhury AK (2006) The function space of an activity. In: Computer vision and pattern recognitionGoogle Scholar
  62. Wu Y, Zhou C, Xiao J, Kurths J, Schellnhuber HJ (2010) Evidence for a bimodal distribution in human communication. Proc Natl Acad Sci USA 107:18803–18808CrossRefGoogle Scholar
  63. Yu H (2005) SVM selective sampling for ranking with application to data retrieval. In: Proceedings of the eleventh ACM SIGKDD international conference on knowledge discovery in data mining. ACM, New YorkGoogle Scholar
  64. Zhai S, Kristensson PO, Appert C, Anderson TH, Cao X (2012) Foundational issues in touch-surface stroke gesture design-an integrative review. Found Trends Hum Comput Interact 5(2):97–205CrossRefGoogle Scholar

Copyright information

© The Author(s) 2014

Authors and Affiliations

  • Yanping Chen
    • 1
  • Yuan Hao
    • 1
  • Thanawin Rakthanmanon
    • 2
  • Jesin Zakaria
    • 1
  • Bing Hu
    • 1
  • Eamonn Keogh
    • 1
  1. 1.Department of Computer Science & EngineeringUniversity of California, RiversideRiversideUSA
  2. 2.Department of Computer Science & EngineeringKasetsart UniversityBangkokThailand

Personalised recommendations