Concept Tracking and Adaptation for Drifting Data Streams under Extreme Verification Latency
Abstract
When analyzing large-scale streaming data towards resolving classification problems, it is often assumed that true labels of the incoming data are available right after being predicted. This assumption allows online learning models to efficiently detect and accommodate non-stationarities in the distribution of the arriving data (concept drift). However, this assumption does not hold in many practical scenarios where a delay exists between predicted and class labels, to the point of lacking this supervision for an infinite period of time (extreme verification latency). In this case, the development of learning algorithms capable of adapting to drifting environments without any external supervision remains a challenging research area to date. In this context, this work proposes a simple yet effective learning technique to classify non-stationary data streams under extreme verification latency. The intuition motivating the design of our technique is to predict the trajectory of concepts in the feature space. The estimation of the region where concepts may reside in the future can be then exploited for producing more updated predictions for newly arriving examples, ultimately enhancing its accuracy during this unsupervised drifting period. Our approach is compared to a benchmark of incremental and static learning methods over a set of public non-stationary synthetic datasets. Results obtained by our passive learning method are promising and encourage further research aimed at generalizing its applicability to other types of drifts.
Keywords
Classification Extreme verification latency Concept drift Non-stationary environmentsNotes
Acknowledgements
This work was supported in part by the Basque Government under the EMAITEK funding program. Jesus L. Lobo also thanks the funding support from the EU project Pacific Atlantic Network for Technical Higher Education and Research - PANTHER (grant number 2013-5659/004-001 EMA2).
References
- 1.Bose, R.J.C., van der Aalst, WM., Žliobaite, I., Pechenizkiy, M.: Handling concept drift in process mining. In: International Conference on Advanced Information Systems Engineering, pp. 391–405. Springer (2011)Google Scholar
- 2.Dehghan, M., Beigy, H., ZareMoodi, P.: A novel concept drift detection method in data streams using ensemble classifiers. Intell. Data Anal. 20(6), 1329–1350 (2016)CrossRefGoogle Scholar
- 3.Delany, S.J., Cunningham, P., Tsymbal, A., Coyle, L.: A case-based technique for tracking concept drift in spam filtering. Knowl.-Based Syst. 18(4–5), 187–195 (2005)CrossRefGoogle Scholar
- 4.Ditzler, G., Roveri, M., Alippi, C., Polikar, R.: Learning in nonstationary environments: a survey. IEEE Comput. Intell. Mag. 10(4), 12–25 (2015)CrossRefGoogle Scholar
- 5.Dyer, K.B., Capo, R., Polikar, R.: Compose: a semisupervised learning framework for initially labeled nonstationary streaming data. IEEE Trans. Neural Netw. Learn. Syst. 25(1), 12–26 (2014)CrossRefGoogle Scholar
- 6.Elwell, R., Polikar, R.: Incremental learning of concept drift in nonstationary environments. IEEE Trans. Neural Netw. 22(10), 1517–1531 (2011)CrossRefGoogle Scholar
- 7.Escovedo, T., Koshiyama, A., da Cruz, A.A., Vellasco, M.: DetectA: abrupt concept drift detection in non-stationary environments. Appl. Soft Comput. 62, 119–133 (2018)CrossRefGoogle Scholar
- 8.Fan, W., Bifet, A.: Mining big data: current status, and forecast to the future. ACM SIGKDD Explor. Newsl. 14(2), 1–5 (2013)CrossRefGoogle Scholar
- 9.Frederickson, C., Gracie, T., Portley, S., Moore, M., Cahall, D., Polikar, R.: Adding adaptive intelligence to sensor systems with mass. In: IEEE Sensors Applications Symposium (SAS), pp. 1–6 (2017)Google Scholar
- 10.Fritzke, B.: A growing neural gas network learns topologies. In: Advances in Neural Information Processing Systems, pp. 625–632 (1995)Google Scholar
- 11.Gaber, M.M., Zaslavsky, A., Krishnaswamy, S.: Mining data streams: a review. ACM Sigmod Rec. 34(2), 18–26 (2005)CrossRefGoogle Scholar
- 12.Gama, J., Žliobaite, I., Bifet, A., Pechenizkiy, M., Bouchachia, A.: A survey on concept drift adaptation. ACM Comput. Surv. 46(4), 44 (2014)CrossRefGoogle Scholar
- 13.Gonçalves Jr., P.M., De Barros, R.S.M.: RCD: a recurring concept drift framework. Pattern Recogn. Lett. 34(9), 1018–1025 (2013)CrossRefGoogle Scholar
- 14.Hofer, V., Krempl, G.: Drift mining in data: a framework for addressing drift in classification. Comput. Stat. Data Anal. 57(1), 377–391 (2013)MathSciNetCrossRefGoogle Scholar
- 15.Kambatla, K., Kollias, G., Kumar, V., Grama, A.: Trends in big data analytics. J. Parallel Distrib. Comput. 74(7), 2561–2573 (2014)CrossRefGoogle Scholar
- 16.Khan, L., Fan, W.: Tutorial: data stream mining and its applications. In: International Conference on Database Systems for Advanced Applications, pp. 328–329 (2012)CrossRefGoogle Scholar
- 17.Krempl, G.: The algorithm apt to classify in concurrence of latency and drift. In: International Symposium on Intelligent Data Analysis, pp. 222–233 (2011)CrossRefGoogle Scholar
- 18.Krempl, G., Žliobaite, I., Brzeziński, D., Hüllermeier, E., Last, M., Lemaire, V., Noack, T., Shaker, A., Sievi, S., Spiliopoulou, M.: Open challenges for data stream mining research. ACM SIGKDD Explor. News. 16(1), 1–10 (2014)CrossRefGoogle Scholar
- 19.Lobo, J.L., Del Ser, J., Bilbao, M.N., Perfecto, C., Salcedo-Sanz, S.: DRED: an evolutionary diversity generation method for concept drift adaptation in online learning environments. Appl. Soft Comput. 68, 693–709 (2018)CrossRefGoogle Scholar
- 20.Losing, V., Hammer, B., Wersing, H.: Tackling heterogeneous concept drift with the self-adjusting memory (sam). Knowl. Inf. Syst. 1–31 (2018)Google Scholar
- 21.Marrs, G.R., Hickey, R.J., Black, M.M.: The impact of latency on online classification learning with concept drift. International Conference on Knowledge Science, Engineering and Management, pp. 459–469. Springer (2010)Google Scholar
- 22.Minku, L.L., White, A.P., Yao, X.: The impact of diversity on online ensemble learning in the presence of concept drift. IEEE Trans. Knowl. Data Eng. 22(5), 730–742 (2010)CrossRefGoogle Scholar
- 23.Pang, S., Ozawa, S., Kasabov, N.: Incremental linear discriminant analysis for classification of data streams. IEEE Trans. Syst. Man Cybern. Part B (Cybern.) 35(5), 905–914 (2005)CrossRefGoogle Scholar
- 24.Souza, V.M., Silva, D.F., Gama, J., Batista, G.E.: Data stream classification guided by clustering on nonstationary environments and extreme verification latency. In: SIAM International Conference on Data Mining, pp. 873–881 (2015)CrossRefGoogle Scholar
- 25.Stanley, K.O.: Learning concept drift with a committee of decision trees. Report UT-AI-TR-03-302, Department of Computer Sciences, University of Texas at Austin, USA (2003)Google Scholar
- 26.Street, W.N., Kim, Y.: A streaming ensemble algorithm (SEA) for large-scale classification. In: ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 377–382 (2001)Google Scholar
- 27.Tsai, C.W., Lai, C.F., Chao, H.C., Vasilakos, A.V.: Big data analytics: a survey. J. Big Data 2(1), 21 (2015)CrossRefGoogle Scholar
- 28.Tsymbal, A., Pechenizkiy, M., Cunningham, P., Puuronen, S.: Dynamic integration of classifiers for handling concept drift. Inf. Fusion 9(1), 56–68 (2008)CrossRefGoogle Scholar
- 29.Umer, M., Frederickson, C., Polikar, R.: Learning under extreme verification latency quickly: fast compose. In: IEEE Symposium Series on Computational Intelligence (SSCI), pp. 1–8. IEEE (2016)Google Scholar
- 30.Widmer, G., Kubat, M.: Learning in the presence of concept drift and hidden contexts. Mach. Learn. 23(1), 69–101 (1996)Google Scholar
- 31.Xioufis, E.S., Spiliopoulou, M., Tsoumakas, G., Vlahavas, IP.: Dealing with concept drift and class imbalance in multi-label stream classification. In: International Joint Conferences on Artificial Intelligence, pp. 1583–1588 (2011)Google Scholar