Advertisement

Conformal decision-tree approach to instance transfer

  • S. ZhouEmail author
  • E. N. Smirnov
  • G. Schoenmakers
  • R. Peeters
Open Access
Article

Abstract

Instance transfer for classification aims at boosting generalization performance of classification models for a target domain by exploiting data from a relevant source domain. Most of the instance-transfer approaches assume that the source data is relevant to the target data for the complete set of features used to represent the data. This assumption fails if the target data and source data are relevant only for strict subsets of the input features which we call “partially input-feature relevant”. In this case these approaches may result in sub-optimal classification models or even in a negative transfer. This paper proposes a new decision-tree approach to instance transfer when the source data are partially input-feature relevant to the target data. The approach selects input features for tree nodes using univariate transfer of source instances. The instance transfer is guided by a conformal test for source relevance estimation. Experimental results on real-world data sets demonstrate that the new decision-tree approach is capable of outperforming existing instance-transfer approaches, especially, when the source data are partially input-feature relevant to the target data.

Keywords

Instance transfer Classification Decision trees Conformal prediction framework 

Mathematics Subject Classification (2010)

97R40 

References

  1. 1.
  2. 2.
    Al-Stouhi, S., Reddy, C.K.: Adaptive boosting for transfer learning using dynamic updates. In: Machine Learning and Knowledge Discovery in Databases, pp. 60–75. Springer (2011)Google Scholar
  3. 3.
    Aldous, D.J.: Exchangeability and related topics. In: École D’été De ProbabilitéS De Saint-Flour, XIII—1983, vol. 1117, pp. 1–198. Springer, Berlin (1985)Google Scholar
  4. 4.
    Borgwardt, K.M., Gretton, A., Rasch, M.J., Kriegel, H.P., Schölkopf, B., Smola, A.J.: Integrating structured biological data by kernel maximum mean discrepancy. Bioinformatics 22(14), e49–e57 (2006)CrossRefGoogle Scholar
  5. 5.
    Cortez, P., Cerdeira, A., Almeida, F., Matos, T., Reis, J.: Modeling wine preferences by data mining from physicochemical properties. Decis. Support. Syst. 47(4), 547–553 (2009)CrossRefGoogle Scholar
  6. 6.
    Cortez, P., Silva, A.M.G.: Using data mining to predict secondary school student performance. EUROSIS (2008)Google Scholar
  7. 7.
    Dai, W., Yang, Q., Xue, G.R., Yu, Y.: Boosting for transfer learning. In: Proceedings of the 24th International Conference on Machine Learning, pp. 193–200. ACM (2007)Google Scholar
  8. 8.
    Jolliffe, I.: Principal component analysis. Wiley Online Library (2002)Google Scholar
  9. 9.
    Kamishima, T., Hamasaki, M., Akaho, S.: Trbagg: a Simple Transfer Learning Method and Application to Personalization in Collaborative Tagging. In: Proceedings of the 9th IEEE International Conference on Data Mining, pp. 219–228 (2009)Google Scholar
  10. 10.
    Lehmann, E.L., D’Abrera, H.J.: Nonparametrics: Statistical Methods based on Ranks. Springer, New York (2006)zbMATHGoogle Scholar
  11. 11.
    Lichman, M.: UCI Machine learning repository. http://archive.ics.uci.edu/ml (2013)
  12. 12.
    Lin, D., An, X., Zhang, J.: Double-bootstrapping source data selection for instance-based transfer learning. Pattern Recogn. Lett. 34(11), 1279–1285 (2013)CrossRefGoogle Scholar
  13. 13.
    Pan, S.J., Kwok, J.T., Yang, Q.: Transfer learning via dimensionality reduction AAAI, vol. 8, pp. 677–682 (2008)Google Scholar
  14. 14.
    Pan, S.J., Tsang, I.W., Kwok, J.T., Yang, Q.: Domain adaptation via transfer component analysis. IEEE Trans. Neural Netw. 22(2), 199–210 (2011)CrossRefGoogle Scholar
  15. 15.
    Pan, S.J., Yang, Q.: A survey on transfer learning. IEEE Trans. Knowl. Data Eng. 22(10), 1345–1359 (2010)CrossRefGoogle Scholar
  16. 16.
    Quinlan, J.R.: C4.5: Programs for machine learning. Morgan Kaufmann Publishers Inc, San Mateo (1993)Google Scholar
  17. 17.
    Schapire, R.E.: The strength of weak learnability. Mach. Learn. 5(2), 197–227 (1990)Google Scholar
  18. 18.
    Shafer, G., Vovk, V.: A tutorial on conformal prediction. J. Mach. Learn. Res. 9, 371–421 (2008)MathSciNetzbMATHGoogle Scholar
  19. 19.
    Torrey, L., Shavlik, J.: Transfer learning. Handbook of Research on Machine Learning Applications and Trends: Algorithms, Methods, and Techniques 1, 242 (2009)Google Scholar
  20. 20.
    Uguroglu, S., Carbonell, J.: Feature selection for transfer learning Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 430–442. Springer (2011)Google Scholar
  21. 21.
    Vovk, V.: The basic conformal prediction framework Conformal Prediction for Reliable Machine Learning Theory, Adaptations and Applications, pp. 1–20 (2014)Google Scholar
  22. 22.
    Zhou, S., Smirnov, E., Schoenmakers, G., Driessens, K., Peeters, R.: Testing exchangeability for transfer decision. Pattern Recognit. Lett. 88(C), 64–71 (2017)CrossRefGoogle Scholar
  23. 23.
    Zhou, S., Smirnov, E., Schoenmakers, G., Peeters, R.: Conformity-based source subset selection for instance transfer. Neurocomputing. doi: 10.1016/j.neucom.2016.11.071 (2017)
  24. 24.
    Zhou, S., Smirnov, E.N., Peeters, R.: Conformal region classification with instance-transfer boosting. Int. J. Artif. Intell. Tools 24(6), 1560,002 (2015)CrossRefGoogle Scholar

Copyright information

© The Author(s) 2017

Open AccessThis article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.

Authors and Affiliations

  • S. Zhou
    • 1
    Email author
  • E. N. Smirnov
    • 1
  • G. Schoenmakers
    • 1
  • R. Peeters
    • 1
  1. 1.Department of Data Science and Knowledge EngineeringMaastricht UniversityMaastrichtThe Netherlands

Personalised recommendations