基于深度强化学习的多人协同混流装配线平衡优化研究

Multi-manned collaborative mixed-model assembly line balancingoptimization based on deep reinforcement learning

针对大型设备混流装配过程中的多人协同、多工种等特点,提出基于双深度Q网络(double deep Q network, DDQN)的多人协同混流装配线平衡优化算法.首先以工作站和工人数量、工人与工作站间的负载为优化目标,建立多人协同混流装配线平衡问题的多目标优化数学模型.其次,根据装配过程中生产对象的特征设计状态空间,并根据启发式规则设计动作空间,结合优化目标设计奖励函数,从而将数学模型转化为马尔科夫决策模型.在此基础上,对传统DDQN算法进行改进,采用自适应探索概率完成动作决策,并设计基于工人利用率的解码方法.最后,通过混流装配线标准测试实例以及多人协同混流装配线测试实例,将DDQN算法与改进离散水波优化算法和模拟退火算法进行对比,验证算法的寻优精度以及模型的有效性.同时,在车身混流装配实际案例中采用DDQN算法进行平衡优化,验证算法的有效性和实用性.

Considering the characteristics of assembly process such as multiple workers collaborating,the demand for workers with different skills,and mixed-model assembly,this paper proposes a double deep Q network(DDQN)based algorithm to address a multi-manned cooperation mixed-model assembly line balancing problem.Firstly,a mathematical model for the multi-manned cooperation mixed-model assembly line balancing problem is established with the objectives of optimising the number of workstations and workers,the workload between workers and workstations.Secondly,the state space is designed based on the features of production objects.Meanwhile,the action space is designed using heuristic rules.Besides,the reward function is constructed based on the objectives of the model.As a result,the mathematical model is converted into a Markov decision process model.On this basis,an improved DDQN algorithm with an adaptive exploration probability for action decision-making and a decoding method based on worker utilization rate is developed.Finally,the improved DDQN algorithm is compared with the improved discrete water wave optimization algorithm and the simulated annealing algorithm on standard mixed-model assembly line test cases and multi-manned collaborative mixed-model assembly line test cases to verify the accuracy of the algorithm and the effectiveness of the model.The effectiveness and practicality of the algorithm are also verified by applying it to balance optimization in a practical car body mixed-flow assembly process.