What is Backtesting?
Backtesting is the process of testing a model against historical data. By simulating past data conditions, you can ensure that your model is accurate. Think of it as cross-validation for time-series models.
Why is the backtesting of your models important?
Validating your models against historical data is important for two reasons:
✅It allows you to ensure that your forecasts are accurate and consistent over a long period of time
✅It can help you to find and correct errors in your models.
✅It can help you to identify missing external factors in your data that would help the model make better forecasts
Backtesting allows you to validate your models against historical data to ensure that they are accurate over a longer period of time. This can help you to find and correct errors in your models. By doing this, you can ensure that your predictions are as accurate as possible and that the model is not just relying on recent trends to make predictions.
Backtesting can also be used to improve the accuracy of your model by identifying any areas where it may be over-or under- estimating future events. By carefully examining the results of backtests, you can identify and correct any errors in your model before applying it to new data.
How is backtesting different from model testing?
Model testing encompasses a broader and deeper set of checks of a model that not only verifies performance but also boundary conditions and general sanity checks.
By performing these tests, you can ensure that your model is capable of forecasting future data conditions and adheres to the expected outputs (e.g. strictly positive forecast values or that values fit inside a range that makes sense from a business domain perspective). Think of these as the basic integrity checks.
Conclusion
By backtesting your models against historical data, you can ensure that your forecasts are accurate and reliable and avoid that your model “just lucked out”. Armed with this information, you can put your time series forecasting models to the best possible use.
Want to learn more?
Sign up for our newsletter (free) and we’ll occasionally send you interesting bits and pieces around #AI, #ML, and #forecasting
