RETAIL FORECASTING AND STOCKOUT DETECTION USING MACHINE LEARNING: A UNIFIED COMPARATIVE FRAMEWORK

Abstract

The retail industry faces a persistent and costly challenge in the form of stockouts, situations where product inventory is depleted before customer demand is met. These events not only result in direct revenue losses but also damage customer loyalty and disrupt supply chain efficiency. The advancement of machine learning technologies, there is a growing opportunity to develop intelligent systems capable of predicting demand patterns and detecting stockout risks before they occur. This study addresses that opportunity through a structured two phase comparative framework applied to sales data of retail. The primary objective of this study is to compare the performance of four machine learning models, Linear Regression, Random Forest, XGBoost, and Long Short Term Memory networks, across two distinct but interconnected phases. Phase 1 focuses on demand forecasting where models predict future sales values and are evaluated using Root Mean Squared Error and Mean Absolute Error. Phase 2 introduces a novel stockout classification layer where the same models are retrained as classifiers to predict whether a stockout event will occur, evaluated using F1 Score and AUC-ROC metrics. The study utilizes the Rossmann Store Sales dataset which contains historical daily sales records from 1115 retail stores across Germany spanning several years. The dataset includes key retail features such as promotional activity, state and school holidays, store type, assortment level, and competitor proximity. After merging and preprocessing the data, a binary stockout label was engineered by identifying days where a store's sales dropped below 50 percent of its historical average while remaining open, a threshold determined through systematic experimentation across multiple levels. To address the severe class imbalance of only 1.65 percent stockout cases in the dataset, the Synthetic Minority Oversampling Technique known as SMOTE was applied to balance the training data before classification model training. Experimental results from Phase 1 reveal that Random Forest achieved the best demand forecasting performance with an RMSE of 922.26 and MAE of 593.05, outperforming XGBoost which recorded an RMSE of 1200.56 and MAE of 859.80. Linear Regression performed the worst with an RMSE of 2805.82 confirming that retail sales data exhibits strong non linear patterns that simple models cannot capture. LSTM recorded an RMSE of 2520.57 suggesting that without proper sequential time series structuring per store, deep learning models do not automatically outperform traditional ensemble methods. In Phase 2, Random Forest again demonstrated the strongest classification performance with an F1 Score of 0.6338 and AUC-ROC of 0.9722, making it the most reliable model for detecting actual stockout events. XGBoost achieved the highest AUC-ROC of 0.9757 indicating superior ability to rank stockout risk across probability thresholds, however its lower F1 Score of 0.4897 suggests it is less precise in directly classifying individual stockout days. LSTM recorded an F1 Score of 0.2829 and Logistic Regression performed the weakest with an F1 Score of 0.1134.