DEEP REINFORCEMENT LEARNING-BASED ADAPTIVE ENERGY MANAGEMENT FOR LITHIUM-ION BATTERY ENERGY STORAGE SYSTEMS

Abstract

Effective energy management is important for the reliable operation of lithium-ion battery energy storage systems, especially when solar generation and electricity demand vary. This study develops a Deep Reinforcement Learning-based energy management approach using a Deep Deterministic Policy Gradient (DDPG) controller for a photovoltaic–battery energy storage system. The controller uses battery State of Charge (SOC), battery temperature, photovoltaic power, and load demand to determine appropriate charging and discharging actions. To create a realistic operating environment, solar resource data for Ekwulobia, Anambra State, Nigeria, obtained from NASA POWER, were combined with the Ausgrid 2012–2013 residential electricity demand dataset. The combined dataset contained 52,560 observations at 10-minute intervals and was divided chronologically into training, validation, and unseen testing periods. The DDPG controller was tested using the unseen May–June 2013 data and compared with a conventional photovoltaic-priority rule-based energy management system (RB-EMS). During testing, the DDPG controller achieved 575.34 kWh of battery charging, 521.35 kWh of discharging, and 1,096.69 kWh of total battery energy throughput. Grid import was 1,096.86 kWh, representing a 3.39% reduction compared with the RB-EMS, while battery energy throughput was 6.88% lower. The controller maintained SOC, temperature, and voltage within the defined limits without constraint violations. Overall, the results show that DDPG can provide adaptive battery control while reducing grid dependence and battery energy throughput under changing solar and load conditions.