Simulation is an important area in Operations Research,Management Science and many disciplines,and it has many advantages in analyzing the performance of complex stochastic models.Over the last 15 years,AI has been ga...Simulation is an important area in Operations Research,Management Science and many disciplines,and it has many advantages in analyzing the performance of complex stochastic models.Over the last 15 years,AI has been gaining steam and fundamentally reshaped a number of current and emerging areas.Simulation plays a central role in deep learning and reinforcement learning,which are the foundation of AI,and it can continue improving the AI techniques,particularly in addressing some of its bottlenecks.展开更多
Inventory management(e.g.lost sales)is a central problem in supply chain management.Lost sales inventory systems with lead times and complex cost function are notoriously hard to optimize.Deep reinforcement learning(D...Inventory management(e.g.lost sales)is a central problem in supply chain management.Lost sales inventory systems with lead times and complex cost function are notoriously hard to optimize.Deep reinforcement learning(DRL)methods can learn optimal decisions based on trails and errors from the environment due to its powerful complex function representation capability and has recently shown remarkable successes in solving challenging sequential decision-making problems.This paper studies typical lost sales and multi-echelon inventory systems.We first formulate inventory management problem as a Markov Decision Process by taking into account ordering cost,holding cost,fixed cost and lost-sales cost and then develop a solution framework DDLS based on Double deep Q-networks(DQN).In the lost-sales scenario,numerical experiments demonstrate that increasing fixed ordering cost distorts the ordering behavior,while our DQN solutions with improved state space are flexible in the face of different cost parameter settings,which traditional heuristics find challenging to handle.We then study the effectiveness of our approach in multi-echelon scenarios.Empirical results demonstrate that parameter sharing can significantly improve the performance of DRL.As a form of information sharing,parameter sharing among multi-echelon suppliers promotes the collaboration of agents and improves the decisionmaking efficiency.Our research further demonstrates the potential of DRL in solving complex inventory management problems.展开更多
文摘Simulation is an important area in Operations Research,Management Science and many disciplines,and it has many advantages in analyzing the performance of complex stochastic models.Over the last 15 years,AI has been gaining steam and fundamentally reshaped a number of current and emerging areas.Simulation plays a central role in deep learning and reinforcement learning,which are the foundation of AI,and it can continue improving the AI techniques,particularly in addressing some of its bottlenecks.
基金supported in part by the National Natural Science Foundation of China,under grant Nos.72022001,92146003,,71901003CAAI-Huawei MindSpore,CCF-Tencent Open Research Fund。
文摘Inventory management(e.g.lost sales)is a central problem in supply chain management.Lost sales inventory systems with lead times and complex cost function are notoriously hard to optimize.Deep reinforcement learning(DRL)methods can learn optimal decisions based on trails and errors from the environment due to its powerful complex function representation capability and has recently shown remarkable successes in solving challenging sequential decision-making problems.This paper studies typical lost sales and multi-echelon inventory systems.We first formulate inventory management problem as a Markov Decision Process by taking into account ordering cost,holding cost,fixed cost and lost-sales cost and then develop a solution framework DDLS based on Double deep Q-networks(DQN).In the lost-sales scenario,numerical experiments demonstrate that increasing fixed ordering cost distorts the ordering behavior,while our DQN solutions with improved state space are flexible in the face of different cost parameter settings,which traditional heuristics find challenging to handle.We then study the effectiveness of our approach in multi-echelon scenarios.Empirical results demonstrate that parameter sharing can significantly improve the performance of DRL.As a form of information sharing,parameter sharing among multi-echelon suppliers promotes the collaboration of agents and improves the decisionmaking efficiency.Our research further demonstrates the potential of DRL in solving complex inventory management problems.