In several places in this and other chapters we referred to forecasting accuracy of a model and used the Mean Absolute Percentage Error (MAPE) as a measure of estimation accuracy. MAPE is one of many measures of forecasting error. Two other widely used are Mean Absolute Forecasting Error (MAFE) and Root Mean Square Error (RMSE). These measures are valuable in evaluating the strength of a model in terms of generating more accurate estimations or forecasts of values of a variable compared to other competing mathematical/statistical models. In the text we introduced two macroeconomic models of national income (example 1 and 2) leading to two different first order difference equations. Solutions to these difference equations were consequently used to predict values of the US GDP from 1999 to 2005. It was then judged, based on MAPE, that the first model was more accurate compared to the second model. What follows is a short description of three most widely used measure of estimation accuracy.

Assume a model estimate or forecast

\(n\) values of a variable. Let

\(E_i\,\,i = 1,2, \ldots , n \;\) denote the estimated values and

\( A_i\,\,i = 1,2, \ldots , n\) denote the actual or observed values of the variable. The difference between actual and estimated values

\(A_i\;-E_i\,\,i = 1,2, \ldots , n\) are

\(n\) estimation or forecasting errors. A model that generates quality forecast should minimize these errors. One measure of forecasting accuracy could be the average of these errors,

*Mean Forecasting Error* (MFE)

$$\begin{aligned} MAF = \frac{\sum _{i=1}^n (A_i-E_i)}{n} \end{aligned}$$

The main problem with MFE is that a model could some times over-estimate and some other times under-estimate the actual values [that is

\((A_i-E_i < 0 )\) or

\((A_i-E_i > 0 )\)], and by summing these values positive and negative numbers may canceled each other out, leading to a small and misleading number for MFE. To avoid this problem we can compute

*Mean Absolute Forecasting Error* (MAFE), by calculating the average of absolute values of forecasting errors

$$\begin{aligned} MAFE = \frac{\sum _{i=1}^n |A_i-E_i|}{n} \end{aligned}$$

An alternative way of measuring estimation error is to express it in percentage terms. A large estimation error in absolute terms might be very small in percentage terms. For example, a model may overestimate the United States GDP by $200 billion. In absolute term $200 billion is a huge number, but not in relative terms. $200 billion overestimation of a $14 trillion economy amounts to only about 1.4 % error. To measure the relative size of the estimation error the

*Mean Absolute Percentage Error* (MAPE) is computed as

$$\begin{aligned} MAPE = \frac{\sum _{i=1}^n |A_i-E_i|/A_i}{n}\;*100 \end{aligned}$$

The third measure of estimation error is the

*Root Mean Square Error* (RMSE). RMSE is defined as

$$\begin{aligned} RMSE = \sqrt{\frac{\sum _{i=1}^n (A_i-E_i)^2}{n}} \end{aligned}$$

This measure penalize larger errors more heavily, make it more appropriate for use in a situation where the costs of making errors proportionately larger than the size of error.

Regardless of which measure of estimation error is used, the model generating the smallest MAFE, MAPE, or RMSE is the most accurate.

**Example**

The following table for the US GDP (in Billion) is constructed from Example 1 discussed in the chapter.

Using the figures from the last three columns of the table we compute MAFE, MAPE, and RMSE for the model in Example 1 as

$$\begin{aligned} {\textit{MAFE}}&= \frac{548.3}{6}=91.38\\ {\textit{MAPE}}&= \frac{0.0538}{6}*100 = 0.897\\ {\textit{RMSE}}&= \sqrt{\frac{65680.5}{6}} = 104.63 \end{aligned}$$

The following table is constructed from Example 2 discussed in the chapter.

Again using the figures from the last three columns of the above table we compute MAFE, MAPE, and RMSE for the model in Example 2 as

$$\begin{aligned} {\textit{MAFE}}&= \frac{4972.9}{6}=828.8\\ {\textit{MAPE}}&= \frac{0.471}{6}*100 = 7.8\\ {\textit{RMSE}}&= \sqrt{\frac{5780102.0}{6}} = 981.5 \end{aligned}$$

In the text the two models were compared based on MAPE. But comparing MAFE and RMSE for the two models also indicates that the model 1 is superior to the model 2 base on these measures.