Forecast error

Forecast error is the difference between the actual value of a variable and its predicted or forecasted value. It's a measure of how well a forecast aligns with reality. In essence, it quantifies the "off-by" amount in a prediction.

Different types of errors

• MAD
  
• MAE
• MASE
• MAPE

MAD (Mean Absolute Deviation):

• Measures the average absolute deviation from the mean of the data, not necessarily from predictions.

• Formula (population MAD):

  $$MAD = \frac{1}{n} \sum_{i=1}^{n} |x_i - \bar{x}|$$
  

  where $\bar{x}$ is the mean of the data $x_i$.

Example:

Let’s say you have actual sales: [100, 120, 130]

• Predicted:[110, 115, 125]

• Mean of actuals: 116.67

MAD:

$$\frac{|100 - 116.67| + |120 - 116.67| + |130 - 116.67|}{3} = \frac{16.67 + 3.33 + 13.33}{3} = 11.11$$

MAE (Mean Absolute Error):

• Measures the average absolute difference between actual and predicted values.

• Formula:

  $$MAE = \frac{1}{n} \sum_{i=1}^{n} |y_i - \hat{y}_i|$$
  

  where $y_i$ is the actual value and $\hat{y}_i$ is the predicted value.

MAE:

$$\frac{|100 - 110| + |120 - 115| + |130 - 125|}{3} = \frac{10 + 5 + 5}{3} = 6.67$$

MAPE (Mean Absolute Percentage Error):

MAPE measures the average of the absolute percentage errors between actual and predicted values.

$$MAPE = \frac{1}{n} \sum_{i=1}^{n} \left| \frac{y_i - \hat{y}_i}{y_i} \right| \times 100$$

• $y_i$= actual value

• $\hat{y}_i$ = predicted value

• $n$ = number of observations

Example

Actual values:[100, 200, 300]
Predicted:[90, 210, 310]

$$MAPE = \frac{1}{3} \left( \frac{10}{100} + \frac{10}{200} + \frac{10}{300} \right) \times 100 \\ = \frac{1}{3} (0.1 + 0.05 + 0.0333) \times 100 = 6.1\%$$

MASE (Mean Absolute Scaled Error)

MASE compares the model's prediction error to the error of a naïve forecast (usually the one-step lagged actuals). It scales the mean absolute error (MAE) of your forecast by the mean absolute error of the naïve forecast.

$$\text{MASE} = \frac{\frac{1}{n} \sum_{t=1}^{n} |y_t - \hat{y}_t|}{\frac{1}{n-1} \sum_{t=2}^{n} |y_t - y_{t-1}|}$$

• $y_t$= Actual value at time $t$

• $\hat{y}_t$ = Predicted value at time $t$

Example (Small Forecast Series):

Actual:[100, 110, 120, 130]

Predicted:[98, 111, 118, 129]




Model MAE = average of |100-98|, |110-111|, |120-118|, |130-129| = (2+1+2+1)/4 = 1.5

Naïve MAE = average of |110-100|, |120-110|, |130-120| = (10+10+10)/3 = 10

$$MASE=\frac{1.5}{10}=0.15\mathrm{<br>}\mathrm{<br>}\mathrm{<br>}\mathrm{<br>}\mathrm{<br>}\mathrm{<br>}\mathrm{<br>}\mathrm{<br>}$$

• MASE < 1: Your model is better than the naïve forecast.

• MASE = 1: Your model performs the same as the naïve forecast.

• MASE > 1: Your model is worse than the naïve forecast.