面向管制语音识别系统的性能评价方法

Performance Evaluation Methods for ATC Speech Recognition Systems

目前,随着空中交通管理领域的发展,越来越多的人工智能技术运用到管制领域。其中自动语音识别技术被用于管制指令纠错、复诵一致性检验等方面,用于提升飞行安全和效率。为了解决自动语音识别系统的性能参差不齐的问题,提出了一种面向管制语音识别系统的性能评价方法,对3个待测系统进行了评价与分析。首先,按照一定的管制场景比例收集管制语音并进行数据标注,建立管制语音识别系统测试语料库;其次,设计管制语音识别系统评价指标体系,并通过层次分析法计算指标权重;最后,提出并训练3个待测管制语音识别系统用于评价分析。结果表明,通过该评价方法可以对管制语音识别系统进行全面的评价以及分析不同管制场景下系统的表现情况,并能够根据不同的管制场景提出性能改进建议。该方法提供了一种直观的评价管制语音识别系统的途径,有望为未来的研究提供有力的指导。

Currently,with the development of air traffic management,an increasing number of artificial intelligence technologies are being applied to the field of control.Among them,automatic speech recognition technology was used for tasks such as correcting control instructions and conducting consistency checks for improving flight safety and efficiency.In order to address the issue of inconsistent performance in automatic speech recognition(ASR)systems,a performance evaluation method tailored to air traffic control(ATC)speech recognition systems was proposed.Three ASR systems were evaluated and analyzed using this method.Firstly,ATC speech data were collected and annotated according to certain proportions of control scenarios to establish a testing corpus for ATC speech recognition systems.Secondly,an evaluation index system for ATC speech recognition systems was designed,and the weights of the indices were calculated using the analytic hierarchy process(AHP).Lastly,three ASR systems tailored to ATC were proposed and trained for evaluation and analysis.The results show that this evaluation method enables comprehensive assessment of ATC speech recognition systems and analysis of their performance across different control scenarios.Moreover,it facilitates the provision of performance improvement suggestions tailored to specific control scenarios.It is concluded that this method offers an intuitive approach to evaluating ATC speech recognition systems and holds promise for guiding future research endeavors.