Journal of Classification

, Volume 34, Issue 3, pp 399–426 | Cite as

Modeling Threshold Interaction Effects Through the Logistic Classification Trunk



We introduce a model dealing with the identification of interaction effects in binary response data, which integrates recursive partitioning and generalized linear models. It derives from an ad-hoc specification and consequent implementation of the Simultaneous Threshold Interaction Modeling Algorithm (STIMA). The model, called Logistic Classification Trunk, allows us to obtain regression parameters by maximum likelihood through the simultaneous estimation of both main effects and threshold interaction effects. The main feature of this model is that it allows the user to evaluate a unique model and simultaneously the importance of both effects obtained by first growing a classification trunk and then by pruning it back to avoid overfitting. We investigate the choice of a suitable pruning parameter through a simulation study and compare the classification accuracy of the Logistic Classification Trunk with that of 13 alternative models/classifiers on 25 binary response datasets.


STIMA Generalized linear modeling Logistic Regression Recursive partitioning Interaction effects Regression trunk 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. ALLEN, J., and LE, H. (2008), “An Additional Measure of Overall Effect Size for Logistic Regression Models”, Journal of Educational and Behavioral Statistics, 33(4), 416–441.CrossRefGoogle Scholar
  2. ASSMANN, S.F., HOSMER, D.W., LEMESHOW, S., and MUNDT, K.A. (1996), “Confidence Intervals for Measures of Interaction”, Epidemiology, 7(3), 286–290.CrossRefGoogle Scholar
  3. BACHE, K., and LICHMAN, M. (2013), “UCI Machine Learning Repository”, University of California, Irvine, School of Information and Computer Sciences,
  4. BALLI, H.O., and SORENSON, B.E. ( 2013), “Interaction Effects in Econometrics”, Empirical Economics, 45, 583–603.CrossRefGoogle Scholar
  5. BERRINGTON DE GONZÁLEZ, A., and COX, D.R. (2007), “Interpretation of Interaction: A Review”, The Annals of Applied Statistics, 1(2), 371–385.MathSciNetCrossRefMATHGoogle Scholar
  6. BREIMAN, L. (1996), “Bagging Predictors”, Machine Learning, 24(2), 123–140.MATHGoogle Scholar
  7. BREIMAN, L. (2001), “Random Forests”, Machine Learning, 45(1), 5–32.CrossRefMATHGoogle Scholar
  8. BREIMAN, L., FRIEDMAN, J., OLSHEN, R., and STONE, C. (1984), Classification and Regression Trees, Monterey CA: Wadsworth and Brooks.MATHGoogle Scholar
  9. CHIPMAN, H.A., GEORGE, E.I., and MCCULLOCH, R.E. (2010), “Bart: Bayesian Additive Regression Trees”, The Annals of Applied Statistics, 4(1), 266–298.MathSciNetCrossRefMATHGoogle Scholar
  10. COHEN, J., COHEN, P., WEST, S., and AIKEN, L. (2003), Applied Multiple Regression/Correlation Analysis for the Behavioral Sciences, Mahwah NJ: Lawrence Erlbaum.Google Scholar
  11. CULP, M., JOHNSON, K., and MICHAILIDIS, G. (2012), ada: An R Package for Stochastic Boosting, R package version 2.0-3,
  12. DAWSON, J. (2014), “Moderation in Management Research: What, Why, When, and How”, Journal of Business and Psychology, 29(1), 1–19.MathSciNetCrossRefGoogle Scholar
  13. DEMŠAR, J. (2006), “Statistical Comparisons of Classifiers Over Multiple Data Sets”, Journal of Machine Learning Research, 7, 1–30.MathSciNetMATHGoogle Scholar
  14. DUSSELDORP, E., CONVERSANO, C., and VAN OS, B.J. (2010), “Combining an Additive and Tree-Based Regression Model Simultaneously: Stima”, Journal of Computational and Graphical Statistics, 19(3), 514–530.MathSciNetCrossRefGoogle Scholar
  15. FLEMING, T., and HARRINGTON, D. (1991), Counting Processes and Survival Analysis, Hoboken NJ: John Wiley and Sons, Inc.MATHGoogle Scholar
  16. FRIEDMAN, J.H. (1991), “Multivariate Adaptive Regression Splines”, The Annals of Statistics, 19(1), 1–67.MathSciNetCrossRefMATHGoogle Scholar
  17. FRIEDMAN, J.H., HASTIE, T., and TIBSHIRANI, R. (2000), “Additive Logistic Regression: A Statistical View of Boosting”, The Annals of Statistics, 28(2), 337–374.MathSciNetCrossRefMATHGoogle Scholar
  18. GRUBINGER, T., ZEILEIS, A., and PFEIFFER, K.P. (2011), “Evolutionary Learning of Globally Optimal Classification and Regression Trees in R”,Working Paper 2011-20. Working Papers in Economics and Statistics, Research Platform Empirical and Experimental Economics, Universitaet Innsbruck,
  19. HALVORSEN, T. (2012), “ElemStatLearn: Data Sets, Functions and Examples”, in The Elements of Statistical Learning, Data Mining, Inference, and Prediction, T. Hastie, R. Tibshirani, and J. Friedman, R package version 2012.04-0,
  20. HAND, D.J. (1997), Construction and Assessment of Classification Rules, Chichester: Wiley.MATHGoogle Scholar
  21. HASTIE, T.J., FRIEDMAN, J.H., and TIBSHIRANI, R.J. (2009), Elements of Statistical Learning, New York: Springer.CrossRefMATHGoogle Scholar
  22. HASTIE, T.J., and TIBSHIRANI, R.J. (1990), Generalized AdditiveModels, London: Chapman and Hall.MATHGoogle Scholar
  23. HASTIE, T. (2013), gam: Generalized Additive Models, R package version 1.09,
  24. HASTIE, T., and TIBSHIRANI, R. (2013), mda: Mixture and Flexible Discriminant Analysis, R package version 0.4-4,S original by Hastie and Tishirani, original R port by Leisch, Hornik, and Ripley,
  25. HOTHORN, T., HORNIK, K., and ZEILEIS, A. (2006) , “Unbiased Recursive Partitioning: A Conditional Inference Framework”, Journal of Computational and Graphical Statistics, 15(3), 651–674.MathSciNetCrossRefGoogle Scholar
  26. HOSMER, D.W., LEMESHOW, S., and STURDIVANT, R.X. (2013), Applied Logistic Regression (3rd. ed.), Hoboken, NJ: John Wiley and Sons, Inc.Google Scholar
  27. HOSMER, D.W., and LEMESHOW, S. (2000), Applied Logistic Regression (2nd ed.), Hoboken NJ: John Wiley and Sons, Inc.Google Scholar
  28. HOSMER, D.W., and LEMESHOW, S. (1989), Applied Logistic Regression (1st ed.), Hoboken NJ: John Wiley and Sons, Inc.Google Scholar
  29. HOTHORN, T. (2014), “TH.Data: TH’s Data Archive”, R package version 1.0-3,
  30. KALBFLEISCH, J.D., and PRENTICE, R.L.(1980), The Statistical Analysis of Failure Time Data, New York: John Wiley and Sons.MATHGoogle Scholar
  31. KAPELNER, A., and BLEICH, J. (2013), “bartMachine: A Powerful Tool for Machine Learning”, ArXiv e-prints,
  32. KARATZOGLOU, A., MEYER, D., and HORNIK, K. (2006), “Support Vector Machines in R”, Journal of Statistical Software, 15(9), 1–28.CrossRefGoogle Scholar
  33. KARATZOGLOU, A., SMOLA, A., HORNIK, K., and ZEILEIS, A. (2004), “kernlab - An S4 Package for Kernel Methods in R”, Journal of Statistical Software, 11(9), 1–20.CrossRefGoogle Scholar
  34. KIM, H., and LOH, W. (2001), ‘Classification TreesWith Unbiased Multiway Splits’, Journal of the American Statistical Association, 96, 589–604.MathSciNetCrossRefGoogle Scholar
  35. KUHN, A. [Contributions from J. Wing, S. Weston, A. Williams, C. Keefer, A. Engelhardt, T. Cooper, Z. Mayer and the R Core Team] (2014), “caret: Classification and Regression Training. R package version 6.0-30”,
  36. KUHN, M., WESTON, S., and COULTER, N. (2014), “C50: C5.0 Decision Trees and Rule-Based Models”, R package version 0.1.0-19, (C code for C5.0 by R. Quinlan),
  37. LANDWEHR, N., HALL, M. and FRANK, E. (2005), “Logistic Model Trees”, Machine Learning, 59(1-2), 161–205.CrossRefMATHGoogle Scholar
  38. LEISCH, F., and DIMITRIADU, E. (2010), “mlbench: Machine Learning Benchmark Problems”, R package version 2.1-1,
  39. LIAW, A., and WIENER, M. (2002), “Classification and Regression by randomForest”, R News, 2(3), 18–22.Google Scholar
  40. LOH, W.-Y. (2009), “Improving the Precision of Classification Trees”, The Annals of Applied Statistics, 3(4), 1710–1737.MathSciNetCrossRefMATHGoogle Scholar
  41. LOH, W.-Y., and SHIH, Y.-S. (1997), “Split Selection Methods for Classification Trees”, Statistica Sinica, 7(4), 815–840.MathSciNetMATHGoogle Scholar
  42. MCCULLAGH, P., and NELDER, J.A. (1989), Generalized Linear Models, London: Chapman and Hall.CrossRefMATHGoogle Scholar
  43. MCFADDEN, D. (1974), “The Measurement of Urban Travel Demand”, Journal of Public Economics, 3(4), 303–328.CrossRefGoogle Scholar
  44. MENARD, S. (2000), “Coefficients of Determination for Multiple Logistic Regression Analysis”, The American Statistician, 54(1), 17–24.Google Scholar
  45. MORGAN, J.N., and SONQUIST, J.A. (1963), “Problems in the Analysis of Survey Data, and a Proposal”, Journal of the American Statistical Association, 58(302), 415–434.CrossRefMATHGoogle Scholar
  46. NAGLER, J. (1993), “Scobit: An Alternative Estimator to Logit and Probit”, American Journal of Political Science, 38(1), 230–255.CrossRefGoogle Scholar
  47. NELDER, J.A., and WEDDERBURN, R.W.M. (1972), “Generalized Linear Models”, Journal of the Royal Statistical Society, Series A, 135, 370–384.CrossRefGoogle Scholar
  48. PETERS, A., and HOTHORN, T. (2013), “ipred: Improved Predictors”, R package version 0.9-3,
  49. QUINLAN, J.R. (1993), C4.5: Programs for Machine Learning, San Franciso CA: Morgan Kaufmann Publishers Inc.Google Scholar
  50. R CORE TEAM (2016), R: A Language and Environment for Statistical Computing, R Foundation for Statistical Computing, Vienna, Austria.Google Scholar
  51. RIPLEY, B.D. (1996), Pattern Recognition and Neural Networks, New York NY: Cambridge University Press.Google Scholar
  52. RUSCH, T., LEE, I., HORNIK, K., JANK, W., and ZEILEIS, A. (2013), “Influencing Elections with Statistics: Targeting Voters with Logistic Regression Trees”, The Annals of Applied Statistics, 7(3), 1612–1639.MathSciNetCrossRefMATHGoogle Scholar
  53. THERNEAU, T., ATKINSON, B., and RIPLEY, B.D. (2014). rpart: Recursive Partitioning and Regression Trees. R package version 4.1-5,
  54. VENABLES, W.N., and RIPLEY, B.D. (2002) Modern Applied Statistics with S, NewYork: Springer.CrossRefMATHGoogle Scholar
  55. WITTEN, I.H., and FRANK, E. (2005), Data Mining: Practical Machine Learning Tools and Techniques, San Francisco CA: Morgan Kaufmann.MATHGoogle Scholar
  56. ZEILEIS, A., HOTHORN, T., and HORNIK, K.(2008), “Model-Based Recursive Partitioning”, Journal of Computational and Graphical Statistics, 17(2), 492–514.MathSciNetCrossRefGoogle Scholar

Copyright information

© Classification Society of North America 2017

Authors and Affiliations

  1. 1.Dipartimento di Scienze Economiche e AziendaliUniversità di CagliariCagliariItaly
  2. 2.Leiden UniversityLeidenThe Netherlands
  3. 3.Netherlands Organization for Applied Scientific Research (TNO)The HagueThe Netherlands

Personalised recommendations