Simulation is an important area in Operations Research,Management Science and many disciplines,and it has many advantages in analyzing the performance of complex stochastic models.Over the last 15 years,AI has been ga...Simulation is an important area in Operations Research,Management Science and many disciplines,and it has many advantages in analyzing the performance of complex stochastic models.Over the last 15 years,AI has been gaining steam and fundamentally reshaped a number of current and emerging areas.Simulation plays a central role in deep learning and reinforcement learning,which are the foundation of AI,and it can continue improving the AI techniques,particularly in addressing some of its bottlenecks.展开更多
Inventory management(e.g.lost sales)is a central problem in supply chain management.Lost sales inventory systems with lead times and complex cost function are notoriously hard to optimize.Deep reinforcement learning(D...Inventory management(e.g.lost sales)is a central problem in supply chain management.Lost sales inventory systems with lead times and complex cost function are notoriously hard to optimize.Deep reinforcement learning(DRL)methods can learn optimal decisions based on trails and errors from the environment due to its powerful complex function representation capability and has recently shown remarkable successes in solving challenging sequential decision-making problems.This paper studies typical lost sales and multi-echelon inventory systems.We first formulate inventory management problem as a Markov Decision Process by taking into account ordering cost,holding cost,fixed cost and lost-sales cost and then develop a solution framework DDLS based on Double deep Q-networks(DQN).In the lost-sales scenario,numerical experiments demonstrate that increasing fixed ordering cost distorts the ordering behavior,while our DQN solutions with improved state space are flexible in the face of different cost parameter settings,which traditional heuristics find challenging to handle.We then study the effectiveness of our approach in multi-echelon scenarios.Empirical results demonstrate that parameter sharing can significantly improve the performance of DRL.As a form of information sharing,parameter sharing among multi-echelon suppliers promotes the collaboration of agents and improves the decisionmaking efficiency.Our research further demonstrates the potential of DRL in solving complex inventory management problems.展开更多
In this study,we consider the problem of node ranking in a random network.A Markov chain is defined for the network,and its transition probability matrix is unknown but can be learned by sampling random interactions a...In this study,we consider the problem of node ranking in a random network.A Markov chain is defined for the network,and its transition probability matrix is unknown but can be learned by sampling random interactions among nodes.Our objective is to decompose the Markov chain into several ergodic classes and select the best node in each ergodic class.We propose a dynamic sampling procedure,which gives a probability guarantee on correct decomposition and maximizes a weighted probability of correct selection of the best node in each ergodic class.Numerical experiment results demonstrate the efficiency of the proposed sampling procedure.展开更多
The generalized likelihood ratio(GLR)method is a recently introduced gradient estimation method for handling discontinuities in a wide range of sample performances.We put the GLR methods from previous work into a sing...The generalized likelihood ratio(GLR)method is a recently introduced gradient estimation method for handling discontinuities in a wide range of sample performances.We put the GLR methods from previous work into a single framework,simplify regularity conditions to justify the unbiasedness of GLR,and relax some of those conditions that are difficult to verify in practice.Moreover,we combine GLR with conditional Monte Carlo methods and randomized quasi-Monte Carlo methods to reduce the variance.Numerical experiments show that variance reduction could be significant in various applications.展开更多
文摘Simulation is an important area in Operations Research,Management Science and many disciplines,and it has many advantages in analyzing the performance of complex stochastic models.Over the last 15 years,AI has been gaining steam and fundamentally reshaped a number of current and emerging areas.Simulation plays a central role in deep learning and reinforcement learning,which are the foundation of AI,and it can continue improving the AI techniques,particularly in addressing some of its bottlenecks.
基金supported in part by the National Natural Science Foundation of China,under grant Nos.72022001,92146003,,71901003CAAI-Huawei MindSpore,CCF-Tencent Open Research Fund。
文摘Inventory management(e.g.lost sales)is a central problem in supply chain management.Lost sales inventory systems with lead times and complex cost function are notoriously hard to optimize.Deep reinforcement learning(DRL)methods can learn optimal decisions based on trails and errors from the environment due to its powerful complex function representation capability and has recently shown remarkable successes in solving challenging sequential decision-making problems.This paper studies typical lost sales and multi-echelon inventory systems.We first formulate inventory management problem as a Markov Decision Process by taking into account ordering cost,holding cost,fixed cost and lost-sales cost and then develop a solution framework DDLS based on Double deep Q-networks(DQN).In the lost-sales scenario,numerical experiments demonstrate that increasing fixed ordering cost distorts the ordering behavior,while our DQN solutions with improved state space are flexible in the face of different cost parameter settings,which traditional heuristics find challenging to handle.We then study the effectiveness of our approach in multi-echelon scenarios.Empirical results demonstrate that parameter sharing can significantly improve the performance of DRL.As a form of information sharing,parameter sharing among multi-echelon suppliers promotes the collaboration of agents and improves the decisionmaking efficiency.Our research further demonstrates the potential of DRL in solving complex inventory management problems.
基金This work was supported in part by the National Natural Science Foundation of China(Grants No.72022001,92146003,71901003).
文摘In this study,we consider the problem of node ranking in a random network.A Markov chain is defined for the network,and its transition probability matrix is unknown but can be learned by sampling random interactions among nodes.Our objective is to decompose the Markov chain into several ergodic classes and select the best node in each ergodic class.We propose a dynamic sampling procedure,which gives a probability guarantee on correct decomposition and maximizes a weighted probability of correct selection of the best node in each ergodic class.Numerical experiment results demonstrate the efficiency of the proposed sampling procedure.
基金the National Natural Science Foundation of China(NSFC)under Grant 72022001,92146003,71901003the Air Force Office of Scientific Research under Grant FA95502010211by Discover GrantRGPIN-2018-05795fromNSERCCanada.
文摘The generalized likelihood ratio(GLR)method is a recently introduced gradient estimation method for handling discontinuities in a wide range of sample performances.We put the GLR methods from previous work into a single framework,simplify regularity conditions to justify the unbiasedness of GLR,and relax some of those conditions that are difficult to verify in practice.Moreover,we combine GLR with conditional Monte Carlo methods and randomized quasi-Monte Carlo methods to reduce the variance.Numerical experiments show that variance reduction could be significant in various applications.