基于强化学习的整体式AMHS防堵塞路径规划方法

An Anti-congestion Path Planning Method Based on Reinforcement Learning for Unified AMHS

整体式自动化物料运输系统(automated material handling system,AMHS)广泛应用于12英寸晶圆制造系统,AMHS中天车(overhead hoist transport,OHT)频繁堵塞严重制约AMHS的生产运输效率。为消减OHT堵塞,提出一种实时数据驱动强化学习(real-time data-driven rein⁃forcement learning,RTDRL)的防堵塞路径规划方法。该方法根据导轨实时工况信息建立实时堵塞指标,基于实时堵塞指标与Q值对应的历史通行经验,构建动作选择策略进行路径优化决策,并设计奖励函数,提高Q值反映AMHS通行情况的真实性,从而提高防堵塞效果。实验研究表明,与现有OHT防堵塞路径规划方法相比,该方法在晶圆卡平均运输时间和晶圆卡平均等待时间指标方面具有更好的性能,有效减少堵塞发生,即RTDRL方法是有效的。

A unified automated material handling system(AMHS)is widely used in 12-inch wafer manufacturing systems.The frequent congestion of overhead hoist transport(OHT)seriously restricts the production and transportation efficiency of AMHS.To reduce congestion of OHT,an anticongestion path planning method based on real-time data-driven reinforcement learning(RTDRL)was proposed.In this method,the real-time congestion index was established according to the real-time working condition information of the guide,and the action selection strategy was constructed based on the historical traffic experience corresponding to the real-time clogging index and the Q value to make the path optimization decision.A reward function was designed to realize the real reflection of the Q value to the AMHS traffic situation,so as to improve the anti-congestion effect.Experimental results show that,compared with the existing OHT anti-congestion path planning methods,this method has better performance in the average transportation time of wafer and the average waiting time of wafer,and effectively reduces the occurrence of congestion.The RTDRL method is effective.