Big company forecasting power for the mid market
Forecasting is criticalForecasting product demand drives supply chain management. Poor forecasts means products are unavailable or wasted. For retailers and product businesses, improving forecast accuracy by just 1% could mean a 2-4% improvement in profit.
|
Midsize is opportunityThe largest businesses already employ armies of Ph.D's to create customized tools. Mid-size companies ($100 Million -$1 Billion) represent 1/3 of US GDP and are starting to hire small data science teams.
|
Legacy tools = problemHowever, the legacy forecasting tools date from another era, when forecasting was done by Finance. These tools are often inflexible, with black box forecast models, making it difficult for data scientists to actually apply their skills.
|
We help data scientists make great forecasts
Get started fastUpload some data on our web app and immediately see fits in multiple models. Visualize results online, change parameters and and see the impact. Download all results for further analysis any time you want.
|
Cutting edge toolsWe support common time series techniques like ETS and SARIMAX. But also bring you the latest tools like Facebook's Prophet and deep learning forecasts from Amazon DeepAR. We're built to expand.
|
Customize all you wantBuilt on common platforms like Python and R, giving you full access to data pipeline and algorithms, so you can extend for your own use case and incorporate domain-specific knowledge.
|
Workflow
1. Upload and Prep Data
|
2. Select Model and Fit
|
3. View Forecasts
|
4. Review Results and Download
|
Results
We built Demanding Solutions using real data from a mid-sized product company in the $100Million-$1Billion revenue range. This data covered over 1000 SKUs for a a 4 year period and included orders from dozens of different retail customers, for well over 1 Million records. We used both order data and retail Point-of-Sale data where available.
The company currently creates its forecasts with a combination of legacy systems, targeted modelling and human intervention. We benchmarked forecast performance for models vs the existing forecasts.
Our objective was to to get the automatic fit out of the box for our product to be in a similar range of accuracy as the company's existing production forecasts, which already incoporate all their best knowledge. So far, we find the results quite encouraging, with a range of success across the techniques and our best result beating the company's own current best forecasts by several percentage points -- a very practically significant improvement.
The company currently creates its forecasts with a combination of legacy systems, targeted modelling and human intervention. We benchmarked forecast performance for models vs the existing forecasts.
Our objective was to to get the automatic fit out of the box for our product to be in a similar range of accuracy as the company's existing production forecasts, which already incoporate all their best knowledge. So far, we find the results quite encouraging, with a range of success across the techniques and our best result beating the company's own current best forecasts by several percentage points -- a very practically significant improvement.
Facebook Prophet |
79.5% |
Benchmark: System Forecast plus user knowledge |
75.2% |
ETS with adjustments |
74.4% |
(S)ARIMA(X) |
71.9% |
Amazon DeepAR |
Poor |
The variation across models is what we’d expect as we think that certain techniques will have more applicability to some company’s data than others. In this case, for example, DeepAR appears to perform poorly, which we hypothesize is because many of the SKUs are highly promotion-sensitive, and since that technique can't incorporate additional information about the timing of promotions, it suffers greatly as a result. We also believe that ARIMA could be significantly improved with more model tuning as it is a technique that often requires careful adjustment for best performance. Our ETS model is the technique that is most similar to the company's existing approach, and seems to perform very similarly, even before adjustment, although it is not as manipulable as some of the other techniques, so its ceiling on performance may be lower.
Product Roadmap
The current product provides the basic functionality to quickly explore different models, experiment with hyperparameters and retrieve results. However, a number of directions for future development exist.
User Interface
The web app user interface could continue to be made more refined, with additional visualization and seamless integration of SageMaker workflow.
Additional Models
There are many other potentially useful forecasting models and packages that could also be integrated as options, including TBATS, a recent model that combines some of the benefits of ARIMA and ETS, and GARCH, a technique very popular in modeling financial time series with varying periods of volatility. Adding these would give users more ability to find a technique most suitable to their particular data and needs.
Modelling speedup
Fitting modelled forecasts across hundreds of SKUs can be time consuming, especially when generating simulated historical forecasts to validate models, or doing grid searches to identify the best model parameters by sku. Running our current product across a few hundred skus takes a few hours, so possible to do overnight, but not ideal for the user experience. Fortunately this is a highly parallelizable task for most of our models, with each sku/parameter set/time slice able to be run independently, so with some more development and using multiple machines in the backend, the run across hundreds or thousands of skus could be reduced to minutes or less.
Forecasting Use Cases outside Retail
We have built our initial product to solve a specific problem for retail demand forecasting. However, there are many other important forecasting problems, including forecasting labor schedules, projected attendance or managing scarce inventory like hotel rooms or airline seats.
We believe that our core insight applies in those areas, too - mid-sized organizations are big enough to afford generalist data scientists and could get tremendous value from them, but those generalists need products designed to help them specifically.
They need a simple way to get started, they need the core technologies to be cutting edge, and they need to be able to customize to use their domain knowledge and skills. So it would certainly also be possible to expand our product to serve other forecasting niches. But beyond that, it may be possible to extend beyond forecasting to include support for other high value and common areas of data science, too.
We believe that our core insight applies in those areas, too - mid-sized organizations are big enough to afford generalist data scientists and could get tremendous value from them, but those generalists need products designed to help them specifically.
They need a simple way to get started, they need the core technologies to be cutting edge, and they need to be able to customize to use their domain knowledge and skills. So it would certainly also be possible to expand our product to serve other forecasting niches. But beyond that, it may be possible to extend beyond forecasting to include support for other high value and common areas of data science, too.
Amazon is eating retail.
Use their tools to bite back!
Use their tools to bite back!