Expectation Propagation for Bayesian Multi-task Feature Selection

  • Daniel Hernández-Lobato
  • José Miguel Hernández-Lobato
  • Thibault Helleputte
  • Pierre Dupont
Conference paper

DOI: 10.1007/978-3-642-15880-3_39

Volume 6321 of the book series Lecture Notes in Computer Science (LNCS)
Cite this paper as:
Hernández-Lobato D., Hernández-Lobato J.M., Helleputte T., Dupont P. (2010) Expectation Propagation for Bayesian Multi-task Feature Selection. In: Balcázar J.L., Bonchi F., Gionis A., Sebag M. (eds) Machine Learning and Knowledge Discovery in Databases. ECML PKDD 2010. Lecture Notes in Computer Science, vol 6321. Springer, Berlin, Heidelberg

Abstract

In this paper we propose a Bayesian model for multi-task feature selection. This model is based on a generalized spike and slab sparse prior distribution that enforces the selection of a common subset of features across several tasks. Since exact Bayesian inference in this model is intractable, approximate inference is performed through expectation propagation (EP). EP approximates the posterior distribution of the model using a parametric probability distribution. This posterior approximation is particularly useful to identify relevant features for prediction. We focus on problems for which the number of features d is significantly larger than the number of instances for each task. We propose an efficient parametrization of the EP algorithm that offers a computational complexity linear in d. Experiments on several multi-task datasets show that the proposed model outperforms baseline approaches for single-task learning or data pooling across all tasks, as well as two state-of-the-art multi-task learning approaches. Additional experiments confirm the stability of the proposed feature selection with respect to various sub-samplings of the training data.

Keywords

Multi-task learning feature selection expectation propagation approximate Bayesian inference 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

Copyright information

© Springer-Verlag Berlin Heidelberg 2010

Authors and Affiliations

  • Daniel Hernández-Lobato
    • 1
  • José Miguel Hernández-Lobato
    • 2
  • Thibault Helleputte
    • 1
  • Pierre Dupont
    • 1
  1. 1.Machine Learning Group, ICTEAM instituteUniversité catholique de LouvainLouvain-la-NeuveBelgium
  2. 2.Computer Science DepartmentUniversidad Autónoma de MadridMadridSpain