结合多模态知识图谱与大语言模型的风机装配工艺问答系统

Assembly process question answering system of wind turbines combining multi-modal knowledge graphs with LLMs

风力发电机是风能发电的基础,但其装配工艺信息通常分散于以三维模型、自然文本和图像等多模态信息构成的工艺文件中,导致历史工艺难以高效复用,装配工艺设计效率低下。针对这一问题,文中通过构建多模态知识图谱的方式整合风力发电机历史装配工艺信息,并且为了能对工艺信息进行有效利用,提出了结合多模态知识图谱与大语言模型的风力发电机装配工艺问答系统,利用大语言模型进行自然语言问题的粗粒度解析,再结合模式层关系细粒度解析问题,最后获得答案。在风力发电机装配工艺数据集上与其他最新KBQA方法和最新大语言模型对比取得优异结果,并在装配工艺可视化问答系统上进行验证,研究结果表明:风机装配工艺设计效率有显著提升。

Wind turbines are the foundation of wind energy generation,but the assembly process information is usually scattered across process documents composed of multimodal information such as 3D models,natural language texts,and images.This leads to difficulties in efficiently reusing historical processes and low efficiency in assembly process design.To address this issue,this paper integrates historical wind turbine assembly process information by constructing a multimodal knowledge graph.To effectively utilize the process information,the paper proposes a wind turbine assembly process question-and-answer system that combines multimodal knowledge graph with a large language model.The system utilizes the large language model for coarse-grained parsing of natural language questions and combines it with fine-grained parsing of pattern-level relationships to obtain answers.The paper achieves outstanding results by comparing with other state-of-the-art KBQA methods and the latest large language models on the wind turbine assembly process dataset.The research results are validated on the assembly process visualization questionand-answer system,demonstrating a significant improvement in assembly process design efficiency.