Combining Task Predictors via Enhancing Joint Predictability

Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 12361)


Predictor combination aims to improve a (target) predictor of a learning task based on the (reference) predictors of potentially relevant tasks, without having access to the internals of individual predictors. We present a new predictor combination algorithm that improves the target by i) measuring the relevance of references based on their capabilities in predicting the target, and ii) strengthening such estimated relevance. Unlike existing predictor combination approaches that only exploit pairwise relationships between the target and each reference, and thereby ignore potentially useful dependence among references, our algorithm jointly assesses the relevance of all references by adopting a Bayesian framework. This also offers a rigorous way to automatically select only relevant references. Based on experiments on seven real-world datasets from visual attribute ranking and multi-class classification scenarios, we demonstrate that our algorithm offers a significant performance gain and broadens the application range of existing predictor combination approaches.



This work was supported by UNIST’s 2020 Research Fund (1.200033.01), National Research Foundation of Korea (NRF) grant NRF-2019R1F1A1061603, and Institute of Information & Communications Technology Planning & Evaluation (IITP) grant (No. 20200013360011001, Artificial Intelligence Graduate School support (UNIST)) funded by the Korean government (MSIT).

Supplementary material

504471_1_En_26_MOESM1_ESM.pdf (495 kb)
Supplementary material 1 (pdf 494 KB)


  1. 1.
    Argyriou, A., Evgeniou, T., Pontil, M.: Convex multi-task feature learning. Mach. Learn. 73(3), 243–272 (2008). Scholar
  2. 2.
    Chen, L., Zhang, Q., Li, B.: Predicting multiple attributes via relative multi-task learning. In: CVPR, pp. 1027–1034 (2014)Google Scholar
  3. 3.
    Everingham, M., Eslami, S.M.A., Van Gool, L., Williams, C.K.I., Winn, J., Zisserman, A.: The Pascal visual object classes challenge: a retrospective. Int. J. Comput. Vis. 111(1), 98–136 (2015). Scholar
  4. 4.
    Evgeniou, T., Micchelli, C.A., Pontil, M.: Learning multiple tasks with kernel methods. JMLR 6, 615–637 (2005)MathSciNetzbMATHGoogle Scholar
  5. 5.
    Farhadi, A., Endres, I., Hoiem, D., Forsyth, D.: Describing objects by their attributes. In: CVPR, pp. 1778–1785 (2009)Google Scholar
  6. 6.
    Gong, P., Ye, J., Zhang, C.: Robust multi-task feature learning. In: KDD, pp. 895–903 (2012)Google Scholar
  7. 7.
    He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: CVPR, pp. 770–778 (2016)Google Scholar
  8. 8.
    Hein, M., Maier, M.: Manifold denoising. In: NIPS, pp. 561–568 (2007)Google Scholar
  9. 9.
    Joachims, T.: Optimizing search engines using clickthrough data. In: KDD, pp. 133–142 (2002)Google Scholar
  10. 10.
    Kim, K.I., Chang, H.J.: Joint manifold diffusion for combining predictions on decoupled observations. In: CVPR, pp. 7549–7557 (2019)Google Scholar
  11. 11.
    Kim, K.I., Tompkin, J., Richardt, C.: Predictor combination at test time. In: ICCV, pp. 3553–3561 (2017)Google Scholar
  12. 12.
    Kovashka, A., Parikh, D., Grauman, K.: WhittleSearch: image search with relative attribute feedback. In: CVPR, pp. 2973–2980 (2012)Google Scholar
  13. 13.
    Mejjati, Y.A., Cosker, D., Kim, K.I.: Multi-task learning by maximizing statistical dependence. In: CVPR, pp. 3465–3473 (2018)Google Scholar
  14. 14.
    Meng, Z., Adluru, N., Kim, H.J., Fung, G., Singh, V.: Efficient relative attribute learning using graph neural networks. In: Ferrari, V., Hebert, M., Sminchisescu, C., Weiss, Y. (eds.) Computer Vision – ECCV 2018. LNCS, vol. 11218, pp. 575–590. Springer, Cham (2018). Scholar
  15. 15.
    Oliva, A., Torralba, A.: Modeling the shape of the scene: a holistic representation of the spatial envelope. Int. J. Comput. Vis. 42(3), 145–175 (2001). Scholar
  16. 16.
    Pan, S.J., Yang, Q.: A survey on transfer learning. IEEE Trans. Knowl. Data Eng. 22(10), 1345–1359 (2010)CrossRefGoogle Scholar
  17. 17.
    Parikh, D., Grauman, K.: Relative attributes. In: ICCV, pp. 503–510 (2011)Google Scholar
  18. 18.
    Passos, A., Rai, P., Wainer, J., Daumé III, H.: Flexible modeling of latent task structures in multitask learning. In: ICML, pp. 1103–1110 (2012)Google Scholar
  19. 19.
    Rasmussen, C.E., Williams, C.K.I.: Gaussian Processes for Machine Learning. MIT Press, Cambridge (2006)zbMATHGoogle Scholar
  20. 20.
    Seeger, M., Williams, C.K.I., Lawrence, N.D.: Fast forward selection to speed up sparse Gaussian process regression. In: International Workshop on Artificial Intelligence and Statistics (2003)Google Scholar
  21. 21.
    Snelson, E., Ghahramani, Z.: Sparse Gaussian processes using pseudo-inputs. In: NIPS (2006)Google Scholar
  22. 22.
    Wah, C., Branson, S., Welinder, P., Perona, P., Belongie, S.: The Caltech-UCSD Birds-200-2011 Dataset. Technical report CNS-TR-2011-001, California Institute of Technology (2011)Google Scholar
  23. 23.
    Xian, Y., Lampert, C.H., Schiele, B., Akata, Z.: Zero-shot learning - a comprehensive evaluation of the good, the bad and the ugly. IEEE TPAMI 41(9), 2251–2265 (2019)CrossRefGoogle Scholar
  24. 24.
    Yang, X., Zhang, T., Xu, C., Yan, S., Hossain, M.S., Ghoneim, A.: Deep relative attributes. IEEE T-MM 18(9), 1832–1842 (2016)Google Scholar
  25. 25.
    Yu, A., Grauman, K.: Fine-grained visual comparisons with local learning. In: CVPR, pp. 192–199 (2014)Google Scholar
  26. 26.
    Zamir, A.R., Sax, A., Shen, W., Guibas, L., Malik, J., Savarese, S.: Taskonomy: disentangling task transfer learning. In: CVPR, pp. 3712–3722 (2018)Google Scholar
  27. 27.
    Zhou, X., Belkin, M., Srebro, N.: An iterated graph Laplacian approach for ranking on manifolds. In: KDD, pp. 877–885 (2011)Google Scholar

Copyright information

© Springer Nature Switzerland AG 2020

Authors and Affiliations

  1. 1.UNISTUlsanKorea
  2. 2.University of BathBathUK
  3. 3.University of BirminghamBirminghamUK

Personalised recommendations