Event-driven supply chain architecture based on Q-Learning under uncertainty

Abstract

The paper considers building an Event-Driven Architecture (EDA) for Supply Chain Management (SCM) systems using Q-learning under uncertainty. It is substantiated that in the presence of high demand volatility, delivery delays, and input data distortions, deterministic and classical statistical forecasting models commit significant errors, leading to increased operational costs due to stockouts or excess inventory. The authors emphasize the expediency of transitioning from scheduled cyclical forecast calculations to real-time response to events in the logistics network. The objective of the study is to develop an event-driven decision-making architecture that combines an event bus, a Reinforcement Learning-based controller, and an asymmetric reward function. The main focus is on abandoning the direct recovery of the demand function in favor of assessing the optimality of actions in specific system states and balancing holding costs and shortage risks. The paper systemizes key solutions: the formalization of a Markov Decision Process (MDP) without constructing a mathematical model of the environment (a model-free approach); asynchronous event processing in network nodes; and the use of Q-learning as an interpretable and computationally lightweight method, serving as an alternative to Deep Reinforcement Learning (Deep RL). Practical analysis proves that applying the proposed approach ensures system resilience to noisy data and minimizes time lags during managerial decision-making. The conclusions outline perspectives for further research: extending the architecture to multi-agent systems (Multi-Agent RL) and developing node-interaction protocols for distributed SCM networks.