A Practical Variable Selection for Linear Models

Abstract

In the analysis of experiments, there are many variable selection algorithms for linear models. Most of these approaches select the best model based on some criteria such as AIC. These criteria do not allow for any relationship between predictors. However, in practice, the analysis is driven by following three principles: Effect Hierarchy, Effect Sparsity, and Effect Heredity Principle. The approach depending solely on those criteria ignore these principles, so it would often select a hard to interpretable models, for instance, which are consisted with only interaction terms. In this article, we extend the LASSO method to identify significant interaction terms mainly focusing on the heredity principle. And we compare the proposed method with ordinary LASSO and traditional variable selection approach. In the example, we analyze the data obtained from designed experiments such as Placket-Burman design and supersaturated design.