Real-time Operation Optimization in Active Distribution Networks Based on Multi-agent Deep Reinforcement Learning

The increasing integration of intermittent renewable energy sources(RESs)poses great challenges to active distribution networks(ADNs),such as frequent voltage fluctuations.This paper proposes a novel ADN strategy based on multiagent deep reinforcement learning(MADRL),which harnesses the regulating function of switch state transitions for the realtime voltage regulation and loss minimization.After deploying the calculated optimal switch topologies,the distribution network operator will dynamically adjust the distributed energy resources(DERs)to enhance the operation performance of ADNs based on the policies trained by the MADRL algorithm.Owing to the model-free characteristics and the generalization of deep reinforcement learning,the proposed strategy can still achieve optimization objectives even when applied to similar but unseen environments.Additionally,integrating parameter sharing(PS)and prioritized experience replay(PER)mechanisms substantially improves the strategic performance and scalability.This framework has been tested on modified IEEE 33-bus,IEEE 118-bus,and three-phase unbalanced 123-bus systems.The results demonstrate the significant real-time regulation capabilities of the proposed strategy.

Solving Inventory Management Problems through Deep Reinforcement Learning

Inventory management(e.g.lost sales)is a central problem in supply chain management.Lost sales inventory systems with lead times and complex cost function are notoriously hard to optimize.Deep reinforcement learning(DRL)methods can learn optimal decisions based on trails and errors from the environment due to its powerful complex function representation capability and has recently shown remarkable successes in solving challenging sequential decision-making problems.This paper studies typical lost sales and multi-echelon inventory systems.We first formulate inventory management problem as a Markov Decision Process by taking into account ordering cost,holding cost,fixed cost and lost-sales cost and then develop a solution framework DDLS based on Double deep Q-networks(DQN).In the lost-sales scenario,numerical experiments demonstrate that increasing fixed ordering cost distorts the ordering behavior,while our DQN solutions with improved state space are flexible in the face of different cost parameter settings,which traditional heuristics find challenging to handle.We then study the effectiveness of our approach in multi-echelon scenarios.Empirical results demonstrate that parameter sharing can significantly improve the performance of DRL.As a form of information sharing,parameter sharing among multi-echelon suppliers promotes the collaboration of agents and improves the decisionmaking efficiency.Our research further demonstrates the potential of DRL in solving complex inventory management problems.