Rule-Based Prediction of Rare Extreme Values

  • Rita Ribeiro
  • Luís Torgo
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 4265)


This paper describes a rule learning method that obtains models biased towards a particular class of regression tasks. These tasks have as main distinguishing feature the fact that the main goal is to be accurate at predicting rare extreme values of the continuous target variable. Many real-world applications from scientific areas like ecology, meteorology, finance,etc., share this objective. Most existing approaches to regression problems search for the model parameters that optimize a given average error estimator (e.g. mean squared error). This means that they are biased towards achieving a good performance on the most common cases. The motivation for our work is the claim that being accurate at a small set of rare cases requires different error metrics. Moreover, given the nature and relevance of this type of applications an interpretable model is usually of key importance to domain experts, as predicting these rare events is normally associated with costly decisions. Our proposed system (R-PREV) obtains a set of interpretable regression rules derived from a set of bagged regression trees using evaluation metrics that bias the resulting models to predict accurately rare extreme values. We provide an experimental evaluation of our method confirming the advantages of our proposal in terms of accuracy in predicting rare extreme values.


Regression Tree Domain Expert Target Variable Minority Class Interpretable Model 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Breiman, L., Friedman, J., Olshen, R., Stone, C.: Classification and Regression Trees. Statistics/Probability Series. Wadsworth & Brooks/Cole Advanced Books & Software (1984)Google Scholar
  2. 2.
    Cleveland, W.: Visualizing data. Hobart Press (1993)Google Scholar
  3. 3.
    Demšar, J.: Statistical comparisons of classifiers over multiple data sets. Journal of Machine Learning Research 7, 1–30 (2006)Google Scholar
  4. 4.
    Indurkhya, N., Weiss, S.M.: Solving regression problems with rule-based ensemble classifiers. In: KDD 2001: Proceedings of the seventh ACM SIGKDD international conference on Knowledge discovery and data mining, pp. 287–292. ACM Press, New York (2001)CrossRefGoogle Scholar
  5. 5.
    Joshi, M.V., Agarwal, R.C., Kumar, V.: Predicting rare classes: Comparing two-phase rule induction to cost-sensitive boosting. In: Elomaa, T., Mannila, H., Toivonen, H. (eds.) PKDD 2002. LNCS, vol. 2431, pp. 237–249. Springer, Heidelberg (2002)CrossRefGoogle Scholar
  6. 6.
    Ribeiro, R.: Prediction models for rare phenomena. Master’s thesis, Faculty of Economics, University of Porto, Portugal (February 2004)Google Scholar
  7. 7.
    Van Rijsbergen, C.: Information Retrieval, 2nd edn. Dept. of Computer Science, University of Glasgow (1979)Google Scholar
  8. 8.
    Torgo, L., Gama, J.: Regression using classification algorithms. Intelligent Data Analysis 1(4) (1997)Google Scholar
  9. 9.
    Torgo, L., Ribeiro, R.: Predicting outliers. In: Lavrač, N., Gamberger, D., Todorovski, L., Blockeel, H. (eds.) PKDD 2003. LNCS (LNAI), vol. 2838, pp. 447–458. Springer, Heidelberg (2003)CrossRefGoogle Scholar
  10. 10.
    Turney, P.: Types of cost in inductive learning. In: Proceedings of the Workshop on cost-sensitive learning at the 17th ICML, pp. 15–21 (2000)Google Scholar
  11. 11.
    UCI Machine Learning Repository,
  12. 12.
    Weiss, G., Hirsh, H.: Learning to predict rare events in categorical time-series data. In: AAAI Workshop on Predicting the Future: AI Approaches to Time-Series Problems, vol. WS-98-07, pp. 83–90. AAAI Press, Menlo Park (1998)Google Scholar
  13. 13.
    Weiss, G., Hirsh, H.: Learning to predict extremely rare events. In: AAAI Workshop on Learning from Imbalanced Data Sets, vol. WS-00-05, pp. 64–68. AAAI Press, Menlo Park (2000)Google Scholar
  14. 14.
    Weiss, G., Provost, F.: The effect of class distribution on classifier learning: an empirical study. Technical Report ML-TR-44, Department of Computer Science, Rutgers University (2001)Google Scholar
  15. 15.
    Weiss, S.M., Indurkhya, N.: Rule-based machine learning methods for functional prediction. Journal of Artificial Intelligence Research 3, 383–403 (1995)MATHGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2006

Authors and Affiliations

  • Rita Ribeiro
    • 1
  • Luís Torgo
    • 2
  1. 1.LIACCUniversity of PortoPortoPortugal
  2. 2.FEP/LIACCUniversity of PortoPortoPortugal

Personalised recommendations