Quantile Regression DQN for Battery Energy Arbitrage in the Belgian Imbalance Settlement Mechanism
Abstract: This paper investigates the application of Distributional Reinforcement Learning, specifically Quantile Regression DQN (QR-DQN), for battery energy arbitrage within the Belgian Imbalance Settlement Mechanism. By learning the full return distribution rather than its expected value, the QR-DQN agent exhibits a significantly higher degree of action selectivity compared to the standard DQN baseline. Evaluating the agents on out-of-sample Elia imbalance prices (May 2024?December 2025), the QR-DQN agent demonstrates conservative operational behaviour, executing a sparse strategy that reduces degradation damage. Under a hard daily cycle constraint (2.0 Equivalent Full Cycles), the distributional approach achieves an exceptional proportional efficiency of ? 160.6 per EFC, reducing daily degradation costs by 67.9% against the baseline and preserving a final state of health of 97.67%. These findings indicate that a discrete-action distributional model can effectively internalise hard constraints and match the operational efficiency of more complex continuous-action architectures.
Keywords: Quantile Regression DQN (QR-DQN), Imbalance Settlement Mechanism (ISM), Distributional Reinforcement Learning, Battery Degradation, Energy Arbitrage
How to Cite?: Sylvester Tinashe Mukarakate, Wang Wei, "Quantile Regression DQN for Battery Energy Arbitrage in the Belgian Imbalance Settlement Mechanism", Volume 15 Issue 8, August 2026, International Journal of Science and Research (IJSR), Pages: 1545-1553, https://www.ijsr.net/getabstract.php?paperid=SR26816175559, DOI: https://dx.doi.org/10.21275/SR26816175559