Forecast Value Added: The Key to Managing the Business Forecasting Process
Business executives want their processes to be effective, efficient, and void of waste. They don't want to squander company time and resources on activities that have no benefit to customers, or to their own bottom line. So when it comes to the business forecasting process, how can they distinguish good performance from bad? How do they know that efforts are "adding value" by making the forecasts more accurate, thereby enabling better service to customers, and making more money?
Business forecasting can be a significant consumer of company resources. There may be forecasting software to license, install, and maintain, and forecast analysts to hire and train to generate the forecasts. There is also, quite commonly, an elaborate consensus or collaborative processes where forecasts are reviewed and adjusted by stakeholders in sales, marketing, finance, operations, or elsewhere. And there can also be a final executive review and signoff, where a general manager or CEO can make final adjustments before "approving" the forecast.
All of this is high-cost management time. We tend to assume these extra reviews and inputs and adjustments are making the forecast better. But the reality is that each human touch points is just one more place that biases and politics and personal agendas can negatively impact forecast accuracy. The unfortunate thing is that our traditional forecasting metrics, by themselves, cannot tell us this.
Traditional forecasting performance metrics, such as Mean Absolute Percent Error (MAPE), tell you the size of your forecast error. But MAPE tells you nothing about what the error should be - what is the best you can expect to do? And MAPE tells you nothing about how efficient you were at achieving the level of forecast accuracy you did attain. Traditional metrics, by themselves, are not enough to properly evaluate and manage forecasting process performance.
Forecast Value Added (FVA) is a forecasting performance metric that has gained wide industry adoption. FVA is defined as "The change in a performance metric that can be attributed to a particular step or participant in the forecasting process." FVA works with whatever traditional metric you use (commonly MAPE, Mean Absolute Deviation, Bias, etc.). FVA is concerned about the change in the metric due to some activity in the forecasting process. Consider an example of a simple forecasting process:
Sales History→Forecasting Model→Statistical Forecast→Analyst Override→Final Forecast
In this process, historical sales information is read into forecasting software, which models the history and generates what we call the "statistical forecast" (i.e., the forecast generated by the software). At that point, the forecast analyst can review and adjust the statistical forecast, resulting in the final forecast.
FVA analysis is the application of basic scientific method to the business forecasting process. Just like the evaluation of a new drug, it involves comparing a treatment (e.g., the new drug, or the statistical forecast) to a placebo. If those patients who take the new drug do better than those who take the placebo, we may conclude that the drug is "adding value" by helping cure their affliction. Similar, if the statistical forecast is more accurate than a "naïve forecast" (described below), then we may conclude that our software and modeling efforts are "adding value" by making the forecast better.
A naïve forecast is something simple to compute, requiring the minimum of effort, and serves as the "placebo" in FVA analysis. For example, using last month's actual sales as the forecast for this month's sales. Such a forecast can be generated at virtually no cost to the organization. So, if our resource consuming forecasting process is not performing any better than the naïve forecast, why bother? Simply use the naïve forecast and free those resources to do more productive activities (or just eliminate those resources that had been used in forecasting).
In conducting FVA analysis, we make this kind of comparison for each sequential step in the forecasting process. In our process example above, we would compare the statistical forecast to a naïve forecast, and also compare the analyst overridden final forecast to the statistical forecast. We might find, for example, that the statistical forecast is better than the naïve forecast (we should certainly hope to find this, given how much we spend on forecasting software!), but that the analyst override just made it worse.
FVA is a tool in the "lean" approach to business management. FVA allows the organization to identify waste - those process steps that are failing to improve the forecast, or may even be making it worse. By eliminating the non-value adding steps or participants from the forecasting process, those resources can be redirected to more productive activities. And by eliminating those steps that are actually making the forecast worse, you can achieve better forecasts with no additional investment.
[For more information on FVA analysis, contact the author at mike.gilliland@sas.com, or download the whitepaper "Forecast Value Added Analysis: Step-by-Step."]