Don’t Worry (too much) about Forecast Accuracy!

Demand Planners spend an inordinate amount of time, frequently using very sophisticated software, trying to generate accurate sku level forecasts. They are often disappointed when their efforts fail to achieve a ‘world class’ mix accuracy performance of above 75% and can often feel aggrieved when their perceived failings are blamed as the cause of service misses or inventory problems.

Even if world class performance is achieved, however, it will inevitably include some individual sku’s with accuracies in the upper 90s but also many well below 50% and maybe even in negative territory  (ie. the error is greater than the forecast , as is often the case with supply chains in which integrated DRP isn’t feasible due to lack of appropriate systems so the supplier is having to service a country distributor, maybe an affiliate, on an ex-stock forecast driven basis). The range of sku level accuracies within a  ‘world  class’ forecast  is related to the level of sku  demand volatilities. In general, the greater the demand volatility (as measured by the coefficient of variation) the more inaccurate the forecasts will be. This is because large random errors cannot be predicted,  forecasting  algorithms  tend towards stability (as designed for driving supply) and such demand patterns often  attract additional, well intentioned, forecasting intelligence which, ironically, tends to generate even greater volatility and inaccuracy.

To get round the problem of forecast error, forecast consumption rules can be selected (although choice of the most appropriate is not always obvious and they can generate quite different outputs) and there is always available the calculation and use of ‘backcast’ error based safety stock to buffer the supply schedules. Forecasting systems have also become ever more sophisticated through using Bayesian techniques and,  recently, ‘big data’ demand sensing and shaping technologies between trading partners have begun to appear using the premise of ‘Why forecast when you can calculate?’

In general however, most sku level forecasts, even those which are ‘world class; are so inaccurate that they shouldn’t be used to directly drive replenishment execution. 75% accuracy is, of course, 25% wrong and most products in a portfolio are below this – it  is only the relatively few stable demand sku’s which can be forecasted with accuracies above 95% and it is these which positively skew average performances towards ‘world class’ due to their high volumes. Even if the forecasts were all above 90% accuracy, use of the forecast to drive replenishment would still be wrong because there is a replenishment signal which is always 100% accurate………….that signal, of course, is real demand itself – so why use a forecast?

To understand why ‘demand pull’ is vastly superior to ‘forecast push’ it is worth considering the impact of forecast inaccuracy upon  DRP/MRP calculations. These generate replenishment recommendations aimed at  achieving safety stock levels but, because the forecasts are incorrect,  and are always being updated,  exception messages  are ceaselessly generated  and suppliers  and factories are frequently asked to amend and change their schedules at short notice.

Not only do these exception messages indicate that the wrong quantities of stock have been sent to the wrong places, and the wrong product mix is being produced, the schedule changes also lead to unplanned machine changeovers and  lost capacity. Inevitable knock on effects up and down  the supply chain cause lead times (and  WIP) to fluctuate and,  unfortunately, as ‘Factory Physics’ (1) proves, the greater the level of variability that a supply chain experiences, and the higher are desired levels of capacity utilisation, the longer and more volatile lead times become which is absolutely antithetical to the core MRP tenet that they be stable. Service issues and supply schedule instability are the result with their consequent costs, and average stock levels tend to exceed the theoretical ‘safety stock plus half the average batch size’ as planners tend towards ‘safe scheduling’ rather than actually using the safety stock.

As if  forecast induced  volatilities aren’t bad enough by themselves, they are a whole lot worse when the end to end supply chain is affected by ‘bullwhip’, which it always is when “forecast push DRP/MRP” techniques are used. ‘Bullwhip’ occurs when small changes in consumer demand get amplified by the forecast  as they are passed up the supply chain causing factories and suppliers to respond to very much higher levels of sku demand variability (and its attendant costs) than they otherwise would.

Bullwhip is caused by batching and the impact of the well intentioned behaviours of the many supply chain players all attempting, in their own way, to make sensible forecasting and replenishment decisions. Unfortunately, due to poor supply chain visibility and information delays (ie. ‘latency’)  the outcome is a cycle of increasing over and under error correction resulting in end to end demand amplification and volatility as explained through, and can be modelled by, engineering control theory, systems theory and, even, chaos theory (2). Readers may have experienced these problems when playing the ‘Beer Game’.

So, if your factory and your suppliers are working hard to eradicate their sources of variability then your ‘forecast push DRP/MRP’ driven replenishment and ordering process is simply adding another source of that same variability which is increasing  costs and/or causing you/them to have to work with unnecessarily high levels of stock buffer or extended response lead times. Any  instability, of course, can also disrupt and slow down the CI process as everyone focuses on meeting the latest service issue instead!

The impact upon product costs of bullwhip volatility has been demonstrated to reduce product margins by up to 30% (3). This is due to the stock holding costs and because all operations (ie. those of factories, warehouses and freight as well as those of suppliers) perform most efficiently when under predictable  stable conditions as demonstrated by the well known Toyota House schematic in which ‘Stability’ is the foundation;  and as any Operations or Supply  Manager will tell you.

Despite all the efforts of Demand Planners and investment in technologically sophisticated software, the use of forecasts to drive replenishment through MRP/DRP logic  will always generate the volatility creating latency and inaccuracy issues to a greater or lesser extent. The only way to avoid generating forecast induced variability and bullwhip is to stop using ‘forecast push’ replenishment.

Is there an alternative? Fortunately there is and it’s called ‘Demand Driven Planning & Replenishment’ (DDPR) in which, at each stock location for each item, an appropriate non-forecast based replenishment technique is selected and, across the full end to end supply chain, the techniques, replenishment triggers and buffers are aligned to ensure they successfully support each other and the rates of consumption at each and every echelon. These techniques don’t just apply to a company’s internal supply chain of course, they can also be the basis for collaboration with suppliers and customers.…… clearly the benefits from DDPR increase in line with the share of the supply and demand network that is managed using its principles.

From a stability perspective, these techniques protect Operations and Suppliers from demand volatility by preventing it being amplified into ”bullwhip” and ensuring that any remaining is correctly buffered and that the safety stock is actually used..

The key ‘make to stock’ demand driven replenishment techniques available are, broadly, the following

  • Consumption-based pull – in which inventory buffers are located, as appropriate, in the supply chain to decouple processes and minimize lead times. Supply activities (eg. inventory movements, production and purchase orders) are scheduled according to a time phased cycle and the quantities triggered, rounded as necessary, replace what has been taken from the location immediately downstream. This affords protection against demand uncertainty and minimises amplification by only building to a replenishment signal, not to a forecast. This technique can be used even if demand has trend or is seasonal so long as the replenishment parameters reflect future demand patterns appropriately.
  • Rate-based or Level Schedule – where demand is high and relatively stable (as it often is for mature products and for upstream items before sku specific customisation takes place), supply can be levelled at a suitable fixed rate, subject to periodic review . In Lean language this is called ‘heijunka’ or ‘mixed model scheduling’ and it works best when it involves high frequency/small batch production.  Ironically, stable products which are suitable for level schedule are also those which are easiest to forecast – but why use a forecast driven replenishment technique when a more effective and simple alternative is available? Level schedule is also very useful for product launches, with regular review, as it generates the stability that operations need to enable them to focus upon  improving the, possibly, new manufacturing processes.

Whereas ‘forecast push’ requires the forecast to be accurate on a time phased basis, which it can never be, these ‘demand pull’ processes allow the supply chain to autonomously respond to demand variations and enables Planners to concentrate properly upon Inventory Planning. Inventory is now effectively managed as ‘capacity’ to respond to demand and meet service requirements. Through replenishment parameter management, inventory planning therefore has as significant a role to play as conventional capacity planning. The parameters up and down the supply chain need to be aligned and should include an appropriately calculated element of safety stock to reflect demand uncertainty. Unlike ‘forecast push’, however, these safety stocks are actually planned to be used and are designed to protect operations against any residual variability. Despite the criticality of effective inventory planning however, the replenishment parameters should not be changed every month (if they were you may as well use the forecast!); they should certainly be reviewed regularly but only  c5% will actually need amending at any one time.

In general, the more volatile is demand the more important it  is that replenishment is not driven with a forecast due to the inevitably high levels of error and ‘bullwhip’. When volatility is very significant, and ex-stock service is not an option, other demand driven replenishment technique options can be selected. These are those in which time is used as the buffer and, depending on the required service strategy, the technique chosen might be ‘assemble to order’ (ATO) in association with postponement strategies, or ‘make to order’ (MTO). These techniques can be used when customer demand is genuinely volatile (eg. response to tenders and price promotions) as well as forming part of an ‘abnormal demand’ management process. In this way postponement with ATO, and supported as necessary by appropriate ‘design for manufacture’ and asset configuration, combined with upstream ‘demand pull’ or level schedule, can deliver cost effective and responsive ‘Agility with Stability’.

S&OP, of course, is an essential support process for demand driven replenishment as it is  forecast based and forward looking. In addition to aligning commercial and SC operations with the financial plan, one of S&OP’s aims is to also align material and capacity availability with the demand plan. To achieve this, however, high levels of time phased sku level forecast accuracy aren’t required. As work centre capacities and materials are used across a range of products, aggregated forecast accuracies of 95% are generally quite easy to achieve.

Companies that use DDPR not only benefit from significantly better operational performance but also experience vastly improved S&OP collaboration between the Commercial, Supply Chain, Operations and Procurement functions. Not only is this due to the generally better relations that might be expected from a more successful process, but also because there is less ‘blaming’ of the forecast for service misses and less short term schedule changes which reduces stress, pressure and the need to ‘achieve the impossible!’

If DDPR is so effective, it’s reasonable to ask why it isn’t practiced more widely? ‘Bullwhip’ was first written about by Forrester and Burbidge in 1961 which was the same year Kingman mathematically formalised the relationship between variability and capacity utilisation with average queue times. DDPR has been part of the Lean toolbox since at least the early 1980s and, in fact, ‘pull’ is increasingly common within the factory walls and, in some industries, with suppliers (eg. grocery multiples and automotive). ‘Pull’ is rarely seen, however, all the way across companies’ distribution channels and linked into their factories through to suppliers. This might explain why many companies often find that all their Lean Factory and Supplier Engineering efforts haven’t translated into the significant performance benefits they expected. By now the reader will know why.

The reasons for the relative scarcity of “end to end” DDPR can only be speculative but might include the fact that it is counter intuitive, neither widely or fully understood and difficult to implement successfully without appropriate software support.

Clearly Factory Managers, and Procurement have an interest in how their company’s supply chain is managed if it has a significant impact upon their performance and their ability to contribute to the bottom line, let alone if it could prevent material shortages and service issues. In some companies the CEO might also be interested?

Fortunately, the outlook for DDPR is positive with the  impact of a Lean education upon the new generation of supply chain leaders and the emergence of specialist “software as a service” (SaaS) DDPR vendors such as Orchestr8(UK), Ultriva and Demand Driven Technologies (both US)

It is important however, not to  assume that DDPR is an easy fix. Successful implementation requires a clear understanding of its rationale and benefits (a successful pilot is often an important element here), adequately robust, user friendly  and functionality rich planning systems, senior cross functional leadership support and a capable and interested supply chain team that includes Order Management through to  Supply Management.

How would successful implementation of DDPR affect Planning and Supply Management activities?  The good news is that it will remove a lot of tedious and non -value added work such as continuously cutting and re-cutting of supply plans, expediting and then having to explain why there are service/inventory issues, inbound late deliveries and lost production hours. Planners should be able to allow Operations to “respond to what they can” while they plan what is necessary. These planning activities will include S&OP, demand profile analysis, replenishment technique selection and management of  replenishment cycles and inventory parameters…………all of which might be termed supply chain “conditioning” or “tuning”. Other key activities which can now receive adequate focus will be collaboration initiatives with customers and suppliers, new product launch planning, promotions and tenders management and, by exception, the abnormal demand (and maybe supply) management process. Supplier Management teams will also be able  to spend more time on their value add activities such as supplier S&OP, price/cost analysis, supplier development etc.

Additional significant benefits are that warehouses, factories and suppliers will all notice that their daily ‘work to’ lists become far more predictable and stable. In consequence adverse cost variances will diminish, unplanned overtime will become a thing of the past and more quality time can be spent on CI initiatives that minimise supply variability and cost  instead of chasing the latest backorder.  Supplier Management should be able to share their now relatively stable requirements with suppliers and allow them to reliably schedule their operations accordingly and, perhaps, even manage their own replenishment schedules if working with a collaboration platform. Perhaps, very importantly, suppliers will be able to operate far more efficiently and some of the consequent cost savings should accrue to their customers!

Should we be concerned about forecast accuracy? If using DDPR, forecast accuracy needs only reach that required for supporting ‘family’ level S&OP and this is relatively easy to achieve. SKU level accuracy on a time phased basis is not at all important and little effort should be put into achieving a ’world class’ mix performance. Instead, Planners should focus upon the far more feasible and value add task of Inventory Planning  in order to ensure that the replenishment parameters are aligned and capable of meeting average demand levels and trends.



1 – Hopp & Spearman (1996) “Factory Physics”

2 – See for instance

-Disney, Dejonckheere,  Lambrecht, Towill (2003) “Measuring and avoiding the bullwhip effect: a control heretic approach” European Journal of Operational Research  147(3), p567-590

– Sterman (2006) “Operational and Behavioural Causes of Supply Chain Instability” p17-56The Bullwhip Effect in Supply Chains: A Review of Methods, Components and Cases

-Wilding (1998) “Chaos Theory; Implications for Supply Chain Management”, International Journal of Logistics Management, 9(1), p43-56

3 – Metters (1997)  “Quantifying the bullwhip effect in supply chains”.  Journal of Operations Management  15(2)  p89-100