A Non-Parametric Approach to Model Evaluation

Abstract

All existing tests and criteria for model evaluation are based on the sum of squared residuals. These measures do not evaluate the performance of a model in predicting turning points in the variable of interest. Here, three tests are proposed to evaluate models on the basis of their ability to predict turning points and to give an unbiased prediction of the relative order of magnitude of changes. Five models are evaluated using the proposed tests. Results indicate that the tests expose facets of models that R² and sum of squared residuals are unable to highlight. Hence, the tests are suggested to complement existing model evaluation techniques.