Those forecasting toy demand on an e-commerce site can benefit from several forecasting advances, including the integration of more contextual, operational real-time data, along with demand insights, pricing indicators, and promotions. But even with these advances, seasonality for instance, in retail, remains an issue.
Forecasting becomes increasingly difficult and less accurate, the further out it reaches in time.
The biggest issue? No data exists from the future. I don’t state that factiously, it is a fact. No matter how much data you feed into a model, all the data will be historical, unless you create data to simulate a future state. That simulated data must be based on something, and that something is usually too basic to be meaningful.
Data-driven forecasting needs scenarios because businesses looking out beyond a year require a consistent set of documented assumptions to create a context for forecasts. Each scenario in a scenario set can include parameters for major variables that can feed into a forecasting model. Rather than a forecast that looks at minor variability or at some less-than-robust assumptions, scenarios offer a way to consistently model demand under diverse circumstances.
Note: this post focuses on business forecasting, not engineering or scientific forecasting. When forecasting the performance of a design, for instance, the designers are likely to include all the physics required to test their design. In science, many areas of uncertainty remain, but scenario planning is not suited to solving problems like missing particles in the standard model. Long term projections, however, such as climate models, will benefit from scenario planning because they rely not just on the physics of the planet, but on enacting and enforcing government policy, individual human choices that can be seen as societal-level behaviors, and the priorities placed on the economic impact of climate change.
The limits of data-driven forecasting
Even the most sophisticated data-driven approaches to forecasting rely on historical data. Even if that data is only a nano-second old, it remains historical. The combination of historical data and a model that relies on that data to forecast future demand, or performance, will set up a narrow set of expectations. Important factors, usually those beyond the control of the model owner, or where data is hard to come by, are left out. Even complex models that combine data from supply chains, or operations in an attempt to narrow the forecasting gap will miss the impact of more external factors.
Scenario planning begins with a study of the influential components that cannot be controlled by the individual. the industry or the organization. Even In areas like regulatory frameworks, where industry works to influence the language of the regulation, its ultimate impact remains uncertain until it is enacted—and the influence still remains uncertain as to the enforcement, and continued willingness to enforce, remains uncertain because of politics and social acceptance.
Just because a law exists, does not mean it will be enforced. That is the antithesis of data-driven. The law is data and it should be incorporated into the model. The experience captured in scenarios can produce futures that offer various takes on the enforcement of one law, or many—or even a variety of frameworks with differing characteristics.
Forecasts rarely emerge with a range of models. We see this regularly in Hurricane forecasting, but while informative it is a short-term modeling issue. Within a few days, the Hurricane does what it does and the uncertainty drops to zero. The same is true over a slightly longer scale with infection rates from flu or COVID. These forecasts are instructive because they employ different models, not just different data—the models make different assumptions about the underlying cause of Hurricane movement, or the behavior of infected people, and those who want to remain uninfected. Scenarios offer businesses a framework for building alternative models that may help anticipate growth and downturns more effectively.
Forecasting the right thing
If you assert, as Majd Kharfan et al do in A data-driven forecasting approach for newly launched seasonal products by leveraging machine-learning approaches (at Semantic Scholar here) that:
Consumers have become more demanding and less predictable in their purchasing behavior that expects high quality, guaranteed availability and fast delivery. Besides, the imported items from far countries are needed to be predicted up to almost 1 year before the raw materials are ordered.1
you must join the assertion with a general assumption forecasting demand is the right thing to do. That statement also assumes the need for a match between raw materials and finished goods that requires orders to be placed one year ahead of demand.
Scenarios can also help in the planning of forecasts and their associated models. Although a business will likely always want to forecast revenue, forecasting the supply chain may prove as important as was discovered during COVID. Yes, much of the data may not be as readily available but the growth curve usually looks at demand, not the inability of a manufacturer or retailer to meet demand. With issues of raw material, labor, and transportation disrupting the supply chain, an optimistic growth curve will likely prove unattainable in light of underlying issues.
Scenario planning can help identify the areas where complementary and supplementary models may offer value in relaying equally credible forecasts. They may well not illustrate reaching currently stated goals, but ideally, they will help better model viable alternative realities, which can be addressed should they happen rather than reacted to afterward.
Challenging assumptions
Regardless of the temporal stretch associated with a model, scenarios can play a pivotal role in the definition of which factors of influence exist for the model. Operational or marketing professionals may view their domains too narrowly when building models. Even as they expand the number of factors in a model, the model may still prove biased toward the discipline. Models that cross disciplines, for instance, combining operational, supply chain, marketing, and financial measures will likely remain narrowcast to include only trusted, controlled data sources, and do so within an industry framework.
Unfortunately, an evidence-based bias reinforces existing assumptions. Data rarely emerges that undermines models within the forecast period. It is only through hindsight, when a model performs poorly, that the forecast’s assumptions get examined.
Scenarios that deliver radically different narratives about the future can help identify the narrowness and bias in the organization, and generate strategic dialog about it, so those challenged to forecast the right things, for instance, become more accepting of an open and inclusive set of model components.
The consistent application of scenario-based variables
Scenarios coalesce around variables, known in scenario planning circles as uncertainties. Crafters of scenarios assign a value to uncertainties within the content of a scenario. A set of scenarios offers at least four values for the same uncertainty in the end state. Model builders can leverage scenarios to define a consistent set of future states with shared assumptions to drive forecast variability. The benefit is not only a set of models founded on clearly stated assumptions about the future, but that forecasts applying the foundations for different models will be comparable—they will live in the same universe or future state.
Complexity beyond the data
The other problem with data-driven forecasts comes from the complexity of influence relationships across what scenario planners call STEEP, or Social-Technological-Economic-Environmental-Political categories of uncertainty.
Forecast models, for instance, rarely account for legislative or regulatory action that may affect supply chains, prices, or perceptions. Social movements may influence what people prioritize or care about. Since there is no data about the future, not only must the data associated with narrative assertions be generated, the relationships and influences between scenario components must also be documented.
Generally applied forecasting methods do not take into account enough variables, nor relationships, and therefore deliver mathematically accurate results that may not be accurate in practice. Of course, a forecast is just that, a forecast—but the goal of applying statistics, and more recently machine learning (ML) is to create more complex, more accurate models. Scenarios may well suggest that even ML models are not complex enough—that their training sets don’t account for everything they should. And again, there may be a data access or even data existence issue, but if that is true, the model builders need to share explicitly the shortcomings of their methods and not pass off completed mathematics that results in a forecast as a credible or good forecast.
At a minimum, scenarios offer a way to tie a narrative to a model’s shortcomings. Model builders can share the context of the scenario and state what they could and could not model.
On a secondarily level, scenarios can also inform model hypotheses. Some of the narrative aspects of the scenarios may not prove fundamental to the model. Modelers can test the suggested relationships and data components to see which ones actually matter, which will help the scenarios planners refine their narratives, and in a practical way, help identify if a key performance indicator matters or not.
Future Key Performance Indicators
Key performance indicators (KPIs) need to matter. Because each future in a set of scenarios offers its own version of what good and bad looks like in the context of the focal question, the type of KPIs, and their values, will also vary from scenario to scenario—what proves important in one future, may prove less important in another. We see this, for instance, with the emerging need to capture and quantify ESG factors. A few years ago, organizations seeking to define themselves as environmentally or socially progressive may have reported figures to establish their credibility as environmental or social leaders or both. Those figures may or may not have accurately reflected the organization’s actual performance on pollution or CO2 generation.
The rise of activist investors and related regulatory frameworks, primarily in the EU, have crafted more stringent definitions for ESG components, and investors and analysts are more actively asking organizations to adhere to their commitments or risk disinvestment.
ESG matters now. The housing crisis created stress tests on banking liquidity. Scenarios provide prompts for models by helping people ask, “What would be important to measure in this future?”
I recommend that organizations compare their current KPIs and those from the various futures to identify core measurements that reflect the business regardless of circumstance, and contingent KPIs that may become important under certain circumstances As with all scenario-based work,
The future of forecasting
Very few organizations share their forecast failures. Most published forecasts act as replacements for a previous forecast. Forecasts, for almost anything, published in late 2019, for instance, show no anticipation of the COVID-19 virus and its impact on the economy. The forecasts now show recovery from a downturn and a growth line coming out of 2020. Some may go back far enough to also capture the economic downturn heralded by the United States Housing Crisis. While forecasts that include actuals will acknowledge disruption, they rarely show that the forecast completely missed the disruption.
Using scenarios, forecasts models can be built that anticipate disruption and discontinuity. The timing may not be right, but if the forecast suggests a significant downturn based on the logic of the scenarios, then a dialog can be convened about three important things,
- How do we anticipate such a downturn? What are the indicators to watch for?
- How do we avoid such a downturn affecting our business negatively, or at least, minimize our exposure?
- How do we recover should such a downturn happen, and we can’t get out of its way?
Economic downturns are not the only disruptions models in scenarios. New technology, new competition, new regulatory frameworks, and new customer attitudes can also invalidate a current business model, leaving forecasts based just on current data dreadfully inaccurate.
What scenario planners need to do to help data-driven forecasting
It is important to point out that most scenarios do not reach the level of detail I lean upon in the advice above. It is not that they can’t, but they often do not. Developing good scenarios is a costly and time-consuming endeavor. Unless an organization adopts scenarios as part of its strategic operating model, then they are usually developed for a purpose with little hope of additional investment once they meet their purpose. Adding another layer, a much more detailed layer, would appear untenable.
For as much as they cost to build, many organizations accept their initial outcome as enough proof enough of value—maintaining them, and drilling into them requires continued staffing or consulting investment. What I illustrate above, however, offers the foundations of justification for a deeper commitment to, and engagement with, scenarios.
The acknowledgment that current data cannot accurately forecast the future is an evidence-based assertion that makes the case for scenarios to support a data-driven organization. If you don’t have data, then you need a robust, repeatable, transparent process for crafting the underlying assumptions about the world at the point the data no longer makes sense. Scenarios are the only way to identify the uncertain variables, assign them meaningful values, and create a context where multiple models can draw upon shared perspectives.
If scenarios are about reperceiving strategy, they are also about reperceiving the assumptions that underly the models built to investigate and support strategy. Scenario planners, and those who fund the work, need to encourage and support deeper, more innovative use of scenarios, enhancing the value derived from investment, and helping the organization anticipate the future at more operational levels than is normally the case.
- Kharfan, M., Chan, V.W., & Efendigil, T. (2021). A data-driven forecasting approach for newly launched seasonal products by leveraging machine-learning approaches. Annals of Operations Research, 1-16.
For more serious insights on strategy click here.
Leave a Reply