Reinforcement Learning-Based Energy Storage Management for Microgrid Power Exchanges

Intelligent energy management systems are increasingly necessary for integrating renewable energy sources within microgrids. This paper investigates the application of a reinforcement learning (RL) neural network to optimize the operation of an electrochemical storage system in an environment composed of residential loads, commercial loads, and a photovoltaic plant, all connected to the grid. A dataset combining market purchase prices, photovoltaic generation, and residential and commercial load profiles was generated and used to train a Twin Delayed Deep Deterministic Policy Gradient (TD3) agent with the primary goal of deriving a reliable and adaptive post-training policy capable of maximizing photovoltaic self-consumption, minimizing operational costs through intelligent price arbitrage, and ensuring strict compliance with battery physical constraints. The system state includes battery state of charge, load demand, PV generation, and normalized market purchase prices, whereas the action represents the battery’s charge/discharge power, which is restricted from exporting energy to the grid. Results show that the agent learns to effectively store surplus PV energy and minimize grid dependency through dynamic charge management. The proposed approach outperforms strategies based solely on storing surplus self-generated energy and maintains the battery within safe operational limits. Tests with previously unseen data demonstrate robust, adaptive, and economically efficient energy management, highlighting the potential of reinforcement learning in intelligent energy systems.