High-frequency forecasting for grocery point-of-sales: intervention in practice and theoretical implications for operational design

Abstract

Food waste in grocery supply chains may exceed one third of the total volume, depending on the category. To address this problem effectively, grocery retailers are introducing automated systems for more efficient store replenishment and dynamic pricing. The stock keeping unit (SKU) and store level forecast is pivotal in these operations management solutions, but operationally challenging. Large grocery retailers have millions of SKU-store combinations that depending on the operational application would need to be forecasted on a weekly, daily, hourly, or even 15-min frequency. However, in grocery it is challenging to account for demand variation at high frequencies without introducing manual decisions into the process of forecast model configuration. To investigate the limits of current practice and explore opportunities of technology-enabled change, we explore how an advanced forecasting method for electricity demand, called TBATS, can automate daily forecasting for grocery store replenishment. Adopting an interventionist approach, we explore the implications for the design of the operational process in the operational setting provided by the case company. We find that TBATS can produce high frequency base forecasts for the SKU-store level accurately for a period exceeding 3 months. This finding points to the opportunity of shifting operational focus from recalculating forecasts to monitoring forecast errors. Introducing variable, even indefinite re-training frequencies for forecasting models is a significant change of the forecasting process for situations where monitoring requires less computation than retraining, potentially reducing the time and cost associated with increasing the forecast frequency.

This is a preview of subscription content, access via your institution.

Fig. 1
Fig. 2

Notes

  1. 1.

    On the retail supply chain level, van Donselaar et al. (2006) investigated daily replenishment, but not forecasting, of perishables in six supermarkets. Daily forecasting of aggregate replenishment orders placed to the distribution center using different types of demand signals (Narayanan et al. 2019) and different types of demand representations (Sillanpää and Liesiö 2018) was recently investigated. Note that daily forecasting of aggregate replenishment orders is different by an order of magnitude from daily forecasting in individual supermarkets.

  2. 2.

    See Pulkka (2020) for a further description of the components and functionality of the Bayesian Regression Model applied in the context of grocery retailing.

References

  1. Alftan A, Kaipia R, Loikkanen L, Spens K (2015) Centralised grocery supply chain planning: Improved exception management. Int J Phys Distrib Logist Manag 45(3):237–259

    Article  Google Scholar 

  2. Ali ÖG, Sayın S, Van Woensel T, Fransoo J (2009) SKU demand forecasting in the presence of promotions. Expert Syst Appl 36(10):12340–12348

    Article  Google Scholar 

  3. Amine A, Cadenat S (2003) Efficient retailer assortment: A consumer choice evaluation perspective. Int J Retail Distrib Manag 31(10):486–497

    Article  Google Scholar 

  4. Anderson Consulting (1996) Where to look for incremental sales gains: The retail problem of out-of-stock merchandise. http://www.ccrrc.org/wp-content/uploads/sites/24/2014/02/Where_to_Look_for_Incremental_Sales_Gains_The_Retail_Problem_of_Out-of-Stock_Merchandise_1996.pdf. Accessed 20 October 2019

  5. Armstrong JS (2001) Evaluating forecasting methods. In: Armstrong, J.S. (ed) Principles of forecasting: a handbook for researchers and practitioners. Springer Science & Business Media, pp 441–472

  6. Bishop CM, Tipping ME (2003) Bayesian regression and classification. Nato Science Series sub Series III Computer and Systems Sciences 190:267–288

    Google Scholar 

  7. Box GE, Jenkins GM, Reinsel GC, Ljung GM (2015) Time series analysis: forecasting and control. John Wiley & Sons

  8. Brozyna J, Mentel G, Szetela B, Strielkowski W (2018) Multi-seasonality in the TBATS model using demand for electric energy as a case study. Econ Comput Econ Cybern Stud Res 52(1)

  9. Chai T, Draxler RR (2014) Root mean square error (RMSE) or mean absolute error (MAE)?– Arguments against avoiding RMSE in the literature. Geosci Model Dev 7(3):1247–1250

    Article  Google Scholar 

  10. Chand S, Hsu VN, Sethi S (2002) Forecast, solution, and rolling horizons in operations management problems: A classified bibliography. Manuf Serv Oper Manag 4(1):25–43

    Article  Google Scholar 

  11. Choi TM, Wallace SW, Wang Y (2018) Big data analytics in operations management. Prod Oper Manag 27(10):1868–1883

    Article  Google Scholar 

  12. Christodoulos C, Michalakelis C, Varoutas D (2010) Forecasting with limited data: Combining ARIMA and diffusion models. Technol Forecast Soc Chang 77(4):558–565

    Article  Google Scholar 

  13. Corsten D, Gruen T (2003) Desperately seeking shelf availability: an examination of the extent, the causes, and the efforts to address retail out-of-stocks. Int J Retail Distrib Manag 31(12):605–617

    Article  Google Scholar 

  14. De Livera AM, Hyndman RJ, Snyder RD (2011) Forecasting time series with complex seasonal patterns using exponential smoothing. J Am Stat Assoc 106(496):1513–1527

    Article  Google Scholar 

  15. Denyer D, Tranfield D, van Aken JE (2008) Developing design propositions through research synthesis. Organ Stud 29:393–413

    Article  Google Scholar 

  16. De Toni AF, Zamolo E (2005) From a traditional replenishment system to vendor-managed inventory: A case study from the household electrical appliances sector. Int J Prod Econ 96(1):63–79

    Article  Google Scholar 

  17. Ehrenthal JCF, Honhon D, Van Woensel T (2014) Demand seasonality in retail inventory management. Eur J Oper Res 238(2):527–539

    Article  Google Scholar 

  18. Fernie J, Grant DB (2008) On-shelf availability: The case of a UK grocery retailer. Int J Logist Manag 19(3):293–308

    Article  Google Scholar 

  19. Fildes R, Hibon M, Makridakis S, Meade N (1998) Generalising about univariate forecasting methods: Further empirical evidence. Int J Forecast 14(3):339–358

    Article  Google Scholar 

  20. Fildes R, Ma S, Kolassa S (2019) Retail forecasting: Research and practice. Int J Logist Manag

  21. Gardner ES (2006) Exponential smoothing: The state of the art, Part II. Int J Forecast 22(4):637–666

    Article  Google Scholar 

  22. Gardner ES Jr, McKenzie E (1985) Forecasting Trends in Time Series. Manag Sci 31(10):1237–1246

    Article  Google Scholar 

  23. Gelman A, Carlin JB, Stern HS, Dunson DB, Vehtari A, Rubin DB (2013) Bayesian data analysis. Chapman and Hall/CRC

  24. Godfrey GA, Powell WB (2000) Adaptive estimation of daily demands with complex calendar effects for freight transportation. Transport Res B-Meth 34(6):451–469

    Article  Google Scholar 

  25. Goodwin P (2010) The holt-winters approach to exponential smoothing: 50 years old and going strong. Foresight 19(19):30–33

    Google Scholar 

  26. Gould PG, Koehler AB, Ord JK, Snyder RD, Hyndman RJ, Vahid-Araghi F (2008) Forecasting time series with multiple seasonal patterns. Eur J Oper Res 191(1):207–222

    Article  Google Scholar 

  27. Grmanová G, Laurinec P, Rozinajová V, Ezzeddine AB, Lucká M, Lacko P, Návrat P (2016) Incremental ensemble learning for electricity load forecasting. Acta Polytech Hung 13(2):97–117

    Google Scholar 

  28. Gruber V, Holweg C, Teller C (2016) What a waste! Exploring the human reality of food waste from the store manager’s perspective. J Public Policy Mark 35(1):3–25

    Article  Google Scholar 

  29. Holmström J, Holweg M, Lawson B, Pil FK, Wagner SM (2019) The digitalization of operations and supply chain management: Theoretical and methodological implications. J Oper 65(8):728–734

    Article  Google Scholar 

  30. Holmström J, Ketokivi M, Hameri AP (2009) Bridging practice and theory: A design science approach. Decis Sci 40(1):65–87

    Article  Google Scholar 

  31. Hyndman RJ, Athanasopoulos G (n.d.) Complex seasonality. https://otexts.com/fpp2/complexseasonality.html. Accessed 1 October 2019

  32. Hyndman RJ, Athanasopoulos G, Bergmeir C, Caceres G, Chhay L, O’Hara-Wild M, Petropoulos F, Razbash S, Wang E, Yasmeen F (2018) forecast: Forecasting functions for time series and linear models. Software, R package

    Google Scholar 

  33. Hyndman RJ, Koehler AB (2006) Another look at measures of forecast accuracy. Int J Forecast 22(4):679–688

    Article  Google Scholar 

  34. Hyndman RJ, Koehler AB, Ord JK, Snyder RD (2008) Forecasting with exponential smoothing: the state space approach. Springer Science & BusinessMedia, pp 14-27

  35. Karabiber OA, Xydis G (2019) Electricity Price Forecasting in the Danish Day-Ahead Market Using the TBATS. ANN and ARIMA Methods Energies 12(5):928

    Google Scholar 

  36. Kiil K, Dreyer HC, Hvolby HH, Chabada L (2018) Sustainable food supply chains: the impact of automatic replenishment in grocery stores. Prod Plan Control 29(2):106–116

    Article  Google Scholar 

  37. Koehler AB, Murphree ES (1988) A comparison of results from state space forecasting with forecasts from the Makridakis competition. Int J Forecast 4(1):45–55

    Article  Google Scholar 

  38. Kotzab H, Teller C (2003) Value-adding partnerships and co-opetition models in the grocery industry. Int J Phys Distrib Logist Manag 33(3):268–281

    Article  Google Scholar 

  39. Koupriouchina L, van der Rest JP, Schwartz Z (2014) On revenue management and the use of occupancy forecasting error measures. Int J Hosp Manag 41:104–114

    Article  Google Scholar 

  40. Kourentzes N, Crone SF (2008) Forecasting high-frequency time series with neural networks - an analysis of modelling challenges from increasing data frequency. The 4th International Conference on Data Mining

  41. Kourentzes N, Petropoulos F, Trapero JR (2014) Improving forecasting by estimating time series structural components across multiple frequencies. Int J Forecast 30(2):291–302

    Article  Google Scholar 

  42. Kourentzes N, Trapero JR, Svetunkov I (2014) Measuring the behaviour of experts on demand forecasting: a complex task http://kourentzes.com/forecasting/wp-content/uploads/2014/12/Kourentzes_Complex-bias.pdf. Accessed 1 October 2019

  43. Lago J, De Ridder F, De Schutter B (2018) Forecasting spot electricity prices: Deep learning approaches and empirical comparison of traditional algorithms. Appl Energy 221:386–405

    Article  Google Scholar 

  44. Ma S, Fildes R, Huang T (2016) Demand forecasting with high dimensional data: The case of SKU retail sales forecasting with intra-and inter-category promotional information. Eur J Oper Res 249(1):245–257

    Article  Google Scholar 

  45. Narayanan A, Sahin F, Robinson EP (2019) Demand and order-fulfillment planning: The impact of point-of-sale data, retailer orders and distribution center orders on forecast accuracy. J Oper Manag 65(5):468–486

    Article  Google Scholar 

  46. Oliva R (2019) Intervention as a research strategy. J Oper Manag 65(7):710–724

    Article  Google Scholar 

  47. Pereira LN (2016) An introduction to helpful forecasting methods for hotel revenue management. Int J Hosp Manag 58:13–23

    Article  Google Scholar 

  48. Peterson H (2014) 4 Ways American Grocery Shopping Is Changing Forever https://www.businessinsider.com/trends-that-are-changing-grocery-stores-2014-4?r=US&IR=T&IR=T. Accessed 4 October 2019

  49. Poler R, Mula J (2011) Forecasting model selection through out-of-sample rolling horizon weighted errors. Expert Syst Appl 38(12):14778–14785

    Article  Google Scholar 

  50. Pulkka A (2020) Applying Bayesian regression to forecast retail demand in the Christmas season. M.Sc. thesis, Aalto University, Finland

  51. Ramos P, Fildes R (2017) Characterizing retail demand with promotional effects. In International Symposium on Forecasting, International Institute of Forecasters Cairns, Australia

    Google Scholar 

  52. Schmenner RW, Swink ML (1998) On theory in operations management. J Oper Manag 17(1):97–113

    Article  Google Scholar 

  53. Seaman B (2018) Considerations of a retail forecasting practitioner. Int J Forecast 34(4):822–829

    Article  Google Scholar 

  54. Segal D (2019) The world wastes tons of food. A grocery ‘happy hour’ in one answer. The New York Times. https://www.nytimes.com/2019/09/08/business/food-waste-climate-change.html. Accessed 3 February 2020

  55. Sillanpää V, Liesiö J (2018) Forecasting replenishment orders in retail: value of modelling low and intermittent consumer demand with distributions. Int J Prod Res 56(12):4168–4185

    Article  Google Scholar 

  56. Taylor JW (2003) Short-term electricity demand forecasting using double seasonal exponential smoothing. J Oper Res Soc 54(8):799–805

    Article  Google Scholar 

  57. Taylor JW, De Menezes LM, McSharry PE (2006) A comparison of univariate methods for forecasting electricity demand up to a day ahead. Int J Forecast 22(1):1–16

    Article  Google Scholar 

  58. Taylor JW (2007) Forecasting daily supermarket sales using exponentially weighted quantile regression. Eur J Oper Res 178(1):154–167

    Article  Google Scholar 

  59. Taylor JW (2010) Triple seasonal methods for short-term electricity demand forecasting. Eur J Oper Res 204(1):139–152

    Article  Google Scholar 

  60. Taylor JW (2011) Multi-item sales forecasting with total and split exponential smoothing. J Oper Res Soc 62(3):555–563

    Article  Google Scholar 

  61. Van Donselaar K, van Woensel T, Broekmeulen RACM, Fransoo J (2006) Inventory control of perishables in supermarkets. Int J Prod Econ 104(2):462–472

    Article  Google Scholar 

  62. Veit A, Goebel C, Tidke R, Doblander C, Jacobsen HA (2014) Household electricity demand forecasting: benchmarking state-of-the-art methods. In Proceedings of the 5th international conference on Future energy systems pp 233–234

  63. Whiteoak P (2004) Rethinking efficient replenishment in the grocery sector. In: Fernie, J. & Sparks, L. (ed) Logistics and Retail Management, 2nd edn. London & Sterling, pp 138–163

  64. Winters PR (1960) Forecasting sales by exponentially weighted moving averages. Manage Sci 6(3):324–342

    Article  Google Scholar 

  65. Yokuma JT, Armstrong JS (1995) Beyond accuracy: Comparison of criteria used to select forecasting methods. Int J Forecast 11(4):591–597

    Article  Google Scholar 

Download references

Author information

Affiliations

Authors

Corresponding author

Correspondence to Jan Holmström.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Appendices

Appendix 1

BATS and TBATS models

De Livera et al. (2011) introduced both BATS and TBATS models. BATS cannot handle non-integer seasonality, which is why TBATS was created. TBATS uses Fourier terms to represent seasonal components in a trigonometric manner, thus allowing for non-integer seasonality. The components of both models are explained in Table 8.

Table 8 Description BATS and TBATS model components

Comparison of approaches for high frequency forecasting

Table 9 Summary of the comparison between selected high frequency forecasting models

Appendix 2

Evaluation criteria

The case company noted the importance of highlighting measures that show that a forecasting model(s) has reduced large forecast errors/ biases significantly when compared to another model, as this is crucial in practice. Due to the occurrence of zero sales, we disregard scale independent measures as they become undefined in the presence of zeros. We also calculate the forecast bias separately because even though some amount of bias is permissible, it is necessary to escalate extreme biases to management to be dealt with manually. We use four measures overall (explained in detail in Table 10):

  • Root Mean Square Error (RMSE) and Mean Absolute Error (MAE): for forecast accuracy

  • Mean Error (ME) and bias coefficient: for forecast bias

RMSE, MAE and ME are well established in literature while the bias coefficient was proposed recently by Kourentzes, Trapero, Svetunkov (2014). We use the bias coefficient to calculate the number of SKU-stores showing extremely high biases (the value of ‘extreme bias’ can be decided based on the industry and products in question). Since they most highlight/ penalize large errors, we pay special consideration to the RMSE as the primary measure of forecast accuracy and extremely high bias as the primary measure of forecast bias.

Table 10 Measures of forecast accuracy and bias

Aggregation and presentation of results

It is necessary to aggregate forecast errors for ease of evaluation and comparison between solutions. We chose the entire forecast period (126 days) and product category as aggregation levels for all four measures.

Some additional steps are also needed for the RMSE, MAE and ME. Since different products can have vastly different sales volumes even within the same product category, these forecast errors should be weighted based on product volume. We also present these results relative to the current solution for increased readability (since the original numbers are quite small). This process is summarized in Table 11.

Table 11 Methods of aggregation for forecast measures

Level of automation

Automation eliminates user involvement and thus reduces effort. However, computational efficiency is likely to decrease when complex methods are fully automated. Therefore, the feasibility of automating such methods and whether other aspects such as increased accuracy can be used as tradeoffs have to be discussed before they are used practically. When measuring the performance of the new solution, we calculated the time taken to create the forecasting model(s) and forecast as a measure of computational efficiency. These numbers were discussed with case company representatives to understand their practical relevance. We also calculated the time to forecast using an assumed model (in place of the case company’s current solution) in order to perform a more direct comparison.

Transparency and ease of usage

Forecasting models are not used by researchers or statisticians in business contexts, but usually by nonprofessionals in the fields of statistics and numerical modeling. Therefore, the new solution must ideally be intuitive and easy to use. We discussed and compared the new solution’s complexity with the current solution, with feedback from the case company. It was also necessary to consider if there are justifiable tradeoffs for decreased transparency, which we reasonably expect with a more intricate forecasting model such as TBATS.

Appendix 3

Monthly box plots

Fig. 3
figure3

Category 1 product sales per month

Fig. 4
figure4

Category 2 product sales per month

Fig. 5
figure5

Category 3 product sales per month

Fig. 6
figure6

Category 4 product sales per month

Daily box plots

Fig. 7
figure7

Category 1 product sales per day

Fig. 8
figure8

Category 2 product sales per day

Fig. 9
figure9

Category 3 product sales per day

Fig. 10
figure10

Category 4 product sales per day

Appendix 4

Practicalities in using the intervention

Since the new solution is fully automatic, a user simply has to have the sales data in a format that can be read by RStudio. Data can be imported into RStudio in a variety of formats including csv, xls and xlsx. This solution currently cannot be used for new product forecasting; therefore, it is recommended to have at least a few weeks of sales data to train the TBATS model.

The user can change certain parameters easily, such as the frequencies considered when training the TBATS model(s), training period for the TBATS model(s), the number of forecasts produced, whether a single or multiple SKUs are forecasted etc. Currently, the unified and segmented approaches are run separately (the user can choose which approach to use). However, this can easily be modified so that the approach is chosen automatically based on some criteria (for e.g. the product category). Rolling forecasts and long forecasts are also separately calculated based on the user’s choice, but this too can be modified in a similar manner.

It is also possible to modify the technical parameters of the TBATS model, such as:

  • Whether or not to use the Box-Cox transform, trend, damping parameter for the trend and ARMA errors

  • The lower and upper limits for the Box-Cox transform

  • The values (p and q) for ARMA errors

However, this is generally unnecessary since all such possible scenarios are automatically tried out and the best options are chosen to train the TBATS model. Modifying these parameters could be useful, for e.g. if it is clear that the time series in question does not need any transform because it is linear (Box-Cox transform is then set to NULL). Setting unnecessary parameters to NULL makes the forecasting process faster, however this requires the user to have expertise in this area.

The produced forecasts are automatically exported to an xlsx file (this can be changed if the user prefers some other format). This xlsx file shows both the dates and corresponding forecasts for the chosen forecast horizon. If multiple SKUs are forecasted at the same time, these results will be displayed in the same file (in separate columns or sheets, as can be chosen by the user).

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Dharmawardane, C., Sillanpää, V. & Holmström, J. High-frequency forecasting for grocery point-of-sales: intervention in practice and theoretical implications for operational design. Oper Manag Res (2021). https://doi.org/10.1007/s12063-020-00176-7

Download citation

Keywords

  • Grocery retail operations
  • Automatic store replenishment
  • High frequency forecasting
  • Retraining frequency
  • Error monitoring