Hypothetical Mean

Commentary from an Actuarial and Economic Perspective

Archive for the ‘Forecasting’ Category

We Must First Forecast the Past

leave a comment »

All forecasts are properly viewed as historical studies. Forecasts are, by definition, historical. The art is to discern which portions of the past are most relevant to understand the future. We want to choose objects in the mirror that are not just closer than they appear, but that are also predictive of where the objects will be. To be a good forecaster, it is first good to be a great historian.

A great example of a historical forecasting methodology is PECOTA. PECOTA forecasts future performance for a player by first going back in time and finding historical players that are “similar”. It then uses information on how those historical players evolved over their careers to project how the current player will evolve. So far, this approach is as good as it can get for forecasting player performance.

Not all forecasts are this obviously historical, but all forecasts really are about intelligent selection of historical comparators.

This key relationship indicates why forecasts will always need both quantitative and qualitative components. Quantitative components — from numerical data that describes the past — are key to anchoring estimates of magnitude in an objective way.[1]

Qualitative components are necessary to adjust for limitations in the data and to accommodate the possibility that this time “really is different”. Frequently, historical data exists because of convenience or some other business purpose; rarely is the historical data directly applicable to the current problem. A significant degree of wisdom is required to judge when “this time really is different”, as is perhaps obvious.

This post may appear to be a truism, but I've found that model interpretation and forecasting errors frequently stem from a lack of appreciation regarding the relationship between history and prediction. Curiosity, energy, and time are all required to investigate the past in a comprehensive way. It is difficult — even in retrospect — to identify key causes for historical events. It is exponentially more difficult to select and measure which of those causal relationships will be the key drivers in the future.

Companies would do well to keep in mind that forecasts are as much about the past as they are about the future. The better you know where you've been, and why, the better you will be able to navigate where you will be.

[1] Even quantitative measures are susceptible to subjective interpretation and biases that influence the selection of the data. Nevertheless, quantitative evaluation helps provide a degree of dispassion, if used wisely.

 

Advertisements

Written by Victor

September 1, 2014 at 7:01 pm

Posted in Forecasting, Modelling

Tagged with