TOP

, 16:345

Optimizing logistic regression coefficients for discrimination and calibration using estimation of distribution algorithms

  • V. Robles
  • C. Bielza
  • P. Larrañaga
  • S. González
  • L. Ohno-Machado
Original Paper

DOI: 10.1007/s11750-008-0054-3

Cite this article as:
Robles, V., Bielza, C., Larrañaga, P. et al. TOP (2008) 16: 345. doi:10.1007/s11750-008-0054-3

Abstract

Logistic regression is a simple and efficient supervised learning algorithm for estimating the probability of an outcome or class variable. In spite of its simplicity, logistic regression has shown very good performance in a range of fields. It is widely accepted in a range of fields because its results are easy to interpret. Fitting the logistic regression model usually involves using the principle of maximum likelihood. The Newton–Raphson algorithm is the most common numerical approach for obtaining the coefficients maximizing the likelihood of the data.

This work presents a novel approach for fitting the logistic regression model based on estimation of distribution algorithms (EDAs), a tool for evolutionary computation. EDAs are suitable not only for maximizing the likelihood, but also for maximizing the area under the receiver operating characteristic curve (AUC).

Thus, we tackle the logistic regression problem from a double perspective: likelihood-based to calibrate the model and AUC-based to discriminate between the different classes. Under these two objectives of calibration and discrimination, the Pareto front can be obtained in our EDA framework. These fronts are compared with those yielded by a multiobjective EDA recently introduced in the literature.

Keywords

Logistic regression Evolutionary algorithms Estimation of distribution algorithms Calibration and discrimination 

Mathematics Subject Classification (2000)

62J12 90C59 90C29 

Copyright information

© Sociedad de Estadística e Investigación Operativa 2008

Authors and Affiliations

  • V. Robles
    • 1
  • C. Bielza
    • 2
  • P. Larrañaga
    • 2
  • S. González
    • 1
  • L. Ohno-Machado
    • 3
  1. 1.Departamento de Arquitectura y Tecnologia de Sistemas InformáticosUniversidad Politecnica de MadridMadridSpain
  2. 2.Departamento de Inteligencia ArtificialUniversidad Politecnica de MadridMadridSpain
  3. 3.Division of Health Science and TechnologyHarvard University and MITBostonUSA

Personalised recommendations