Audio-Text Multimodal Speech Recognition via Dual-Tower Architecture for Mandarin Air Traffic Control Communications

In air traffic control communications (ATCC), misunderstandings between pilots and controllers could result in fatal aviation accidents. Fortunately, advanced automatic speech recognition technology has emerged as a promising means of preventing miscommunications and enhancing aviation safety. However, most existing speech recognition methods merely incorporate external language models on the decoder side, leading to insufficient semantic alignment between speech and text modalities during the encoding phase. Furthermore, it is challenging to model acoustic context dependencies over long distances due to the longer speech sequences than text, especially for the extended ATCC data. To address these issues, we propose a speech-text multimodal dual-tower architecture for speech recognition. It employs cross-modal interactions to achieve close semantic alignment during the encoding stage and strengthen its capabilities in modeling auditory long-distance context dependencies. In addition, a two-stage training strategy is elaborately devised to derive semantics-aware acoustic representations effectively. The first stage focuses on pre-training the speech-text multimodal encoding module to enhance inter-modal semantic alignment and aural long-distance context dependencies. The second stage fine-tunes the entire network to bridge the input modality variation gap between the training and inference phases and boost generalization performance. Extensive experiments demonstrate the effectiveness of the proposed speech-text multimodal speech recognition method on the ATCC and AISHELL-1 datasets. It reduces the character error rate to 6.54% and 8.73%, respectively, and exhibits substantial performance gains of 28.76% and 23.82% compared with the best baseline model. The case studies indicate that the obtained semantics-aware acoustic representations aid in accurately recognizing terms with similar pronunciations but distinctive semantics. The research provides a novel modeling paradigm for semantics-aware speech recognition in air traffic control communications, which could contribute to the advancement of intelligent and efficient aviation safety management.

Cluster DetectionMethod of Endogenous Security Abnormal Attack Behavior in Air Traffic Control Network

In order to enhance the accuracy of Air Traffic Control(ATC)cybersecurity attack detection,in this paper,a new clustering detection method is designed for air traffic control network security attacks.The feature set for ATC cybersecurity attacks is constructed by setting the feature states,adding recursive features,and determining the feature criticality.The expected information gain and entropy of the feature data are computed to determine the information gain of the feature data and reduce the interference of similar feature data.An autoencoder is introduced into the AI(artificial intelligence)algorithm to encode and decode the characteristics of ATC network security attack behavior to reduce the dimensionality of the ATC network security attack behavior data.Based on the above processing,an unsupervised learning algorithm for clustering detection of ATC network security attacks is designed.First,determine the distance between the clustering clusters of ATC network security attack behavior characteristics,calculate the clustering threshold,and construct the initial clustering center.Then,the new average value of all feature objects in each cluster is recalculated as the new cluster center.Second,it traverses all objects in a cluster of ATC network security attack behavior feature data.Finally,the cluster detection of ATC network security attack behavior is completed by the computation of objective functions.The experiment took three groups of experimental attack behavior data sets as the test object,and took the detection rate,false detection rate and recall rate as the test indicators,and selected three similar methods for comparative test.The experimental results show that the detection rate of this method is about 98%,the false positive rate is below 1%,and the recall rate is above 97%.Research shows that this method can improve the detection performance of security attacks in air traffic control network.

A Robust Conformer-Based Speech Recognition Model for Mandarin Air Traffic Control

This study aims to address the deviation in downstream tasks caused by inaccurate recognition results when applying Automatic Speech Recognition(ASR)technology in the Air Traffic Control(ATC)field.This paper presents a novel cascaded model architecture,namely Conformer-CTC/Attention-T5(CCAT),to build a highly accurate and robust ATC speech recognition model.To tackle the challenges posed by noise and fast speech rate in ATC,the Conformer model is employed to extract robust and discriminative speech representations from raw waveforms.On the decoding side,the Attention mechanism is integrated to facilitate precise alignment between input features and output characters.The Text-To-Text Transfer Transformer(T5)language model is also introduced to handle particular pronunciations and code-mixing issues,providing more accurate and concise textual output for downstream tasks.To enhance the model’s robustness,transfer learning and data augmentation techniques are utilized in the training strategy.The model’s performance is optimized by performing hyperparameter tunings,such as adjusting the number of attention heads,encoder layers,and the weights of the loss function.The experimental results demonstrate the significant contributions of data augmentation,hyperparameter tuning,and error correction models to the overall model performance.On the Our ATC Corpus dataset,the proposed model achieves a Character Error Rate(CER)of 3.44%,representing a 3.64%improvement compared to the baseline model.Moreover,the effectiveness of the proposed model is validated on two publicly available datasets.On the AISHELL-1 dataset,the CCAT model achieves a CER of 3.42%,showcasing a 1.23%improvement over the baseline model.Similarly,on the LibriSpeech dataset,the CCAT model achieves a Word Error Rate(WER)of 5.27%,demonstrating a performance improvement of 7.67%compared to the baseline model.Additionally,this paper proposes an evaluation criterion for assessing the robustness of ATC speech recognition systems.In robustness evaluation experiments based on this criterion,the proposed model demonstrates a performance improvement of 22%compared to the baseline model.

Air Traffic Volume and Air Traffic Control Human Errors

Navigable airspaces are becoming more crowded with increasing air traffic, and the number of accidents caused by human errors is increasing. The main objective of this paper is to evaluate the relationship between air traffic volume and human error in air traffic control (ATC). First, the paper identifies categories and elements of ATC human error through a review of existing literature, and a study through interviews and surveys of ATC safety experts. And then the paper presents the results of an experiment conducted on 52 air traffic controllers sampled from the Korean ATC organization to find out if there is any relationship between traffic volume and air traffic controller human errors. An analysis of the experiment clearly showed that several types of ATC human error are influenced by traffic volume. We hope that the paper will make its contribution to aviation safety by providing a realistic basis for securing proper manpower and facility in accordance with the level of air traffic volume.

Integrated Evaluation of Air Traffic Controller Workload Based on Matter-Element Analysis

A model for evaluating the controller workload was presented based on matter-element analysis, particularly from a mansystem engineering perspective. On the basis of a questionnaire survey, 18 kinds of indexes which influence the controller workload were determined. By establishing the classical field and node field of the controller workload, the correlation function of the controller workload grade was obtained; then the correlation degree and estimated grade of controller workload were given. A case study verifies the feasibility of the proposed evaluation method.

Identification of Information-Seeking Behaviors from Air Traffic Controllers′ Eye Movements

Air traffic controllers are the important parts of air traffic management system who are responsible for the safety and efficiency of the system.They make traffic management decisions based on information acquired from various sources.The understanding of their information seeking behaviors is still limited.We aim to identify controllers′ behavior through the examination of the correlations between controllers′eye movements and air traffic.Sixteen air traffic controllers were invited to participate real-time simulation experiments,during which the data of their eye ball movements and air traffic were recorded.Tweny-three air traffic complexity metrics and six eye movements metrics were calculated to examine their relationships.Two correlational methods,Pearson′s correlation and Spearman′s correlation,were tested between every eye-traffic pair of metrics.The results indicate that controllers′two kinds of information-seeking behaviors can be identified from their eye movements:Targets tracking,and confliction recognition.The study on controllers′ eye movements may contribute to the understanding of information-seeking mechanisms leading to the development of more intelligent automations in the future.

Effect of Working Experience on Air Traffic Controller Eye Movement

Eye movement is an important indicator of information-seeking behavior and provides insight into cognitive strategies which are vital for decision-making.Various measures based on eye movements have been proposed to capture humans’ability to process information in a complex environment.The effectiveness of these measures has not yet been fully explored in the field of air traffic management.This paper presents a comparative study on eye-movement measures in air traffic controllers with different levels of working experience.Two commonly investigated oculomotor behaviors,fixation and saccades,together with gaze entropy,are examined.By comparing the statistical properties of the relevant metrics,it is shown that working experience has a notable effect on eye-movement patterns.Both fixation and saccades differ between qualified and novice controllers,with the former type of controller employing more efficient searching strategies.These findings are useful in enhancing the quality of controller training and contributing to an understanding of the information-seeking mechanisms humans use when executing complex tasks.

A New Approach to an Automated Air Traffic Control

This paper identifies areas of improvements of the air traffic control system and proposes modification of the concept of automation by using available technologies. With the proposed modification, the current Europe wide en route network structure can be modified in order to make routes more optimal. For this new route network structure, a new concept of automation will be used to manage with the air traffic. The first identified area of improvement is implementation of automation process that will enable decentralization of the air traffic control functionality to each individual aircraft and this will be achieved through automated routing of the aircrafts and CD＆R （conflict detection and resolution）. The FMS （flight management system） at the aircraft will make decisions for the optimal flight route based on the sensor inputs, information on selection of the routes, next hope points and flight levels, all received by ADS-B （automatic dependant surveillance-broadcast）. The second area is processing the information about the deviation from the optimal route as in flight plan due to a traffic management （vectoring, level change） and taking it into consideration when further actions are undertaken. For each action, a cost factor will be calculated from the fuel burned for that action. This factor will be used to select conflict resolution protocol. The proposed concept shall increase the capacity of the network, and enable the air traff^c more efficient and more environmentally friendly while maintaining safe separation.

Traffic Control Under Complex Weather Conditions in Suining Airport

Complex weather conditions is meaning thunderstorm freezing turbulence wind-shear low visibility weather affect the flight safety.When confronted with complex weather conditions,the controllers should know the weather condition and trend weather,and notify the aircraft under your control zone.The controllers provide the required services to the pilots,help the pilots to avoid the complex weather.In this paper,through different complex weathers under different control command,get the different methods of control.

OPTIMALFLOWCONTROLPROBLEMINAIRTRAFFICCONTROL：PARTⅠ

研究了多架飞机着陆于单个拥挤机场的最优调度问题。通过对空中交通系统模型的建立，提出了一种新的、有效的地面等待策略算法。

Intelligent Virtual Assistant System(IVAS) for Air Traffic Controllers

Air traffic control is an essential obligation in the aviation industry to have safe and efficient air transportation.Year by year,the workload and on-job-stress of the air traffic controllers are rapidly increasing due to the rapid growth of air traveling.Controllers are usually dealing with multiple aircrafts at a time and must make quick and accurate decisions to ensure the safety of aircrafts.Heavy workload and high responsibilities create air traffic control a stressful job that sometimes could be error-prone and time-consuming,since controlling and decision-making are solely dependent on human intelligence.To provide effective solutions for the mentioned on the job challenges of the controllers,this study proposed an intelligent virtual assistant system(IVAS)to assist the controllers thereby to reduce the controllers’workload.Consisting of four main parts,which are voice recognition,display conversation on screen,task execution,and text to speech,the proposed system is developed with the aid of artificial intelligence(AI)techniques to make speedy decisions and be free of human interventions.IVAS is a computer-based system that can be activated by the voice of the air traffic controller and then appropriately assist to control the flight.IVAS identifies the words spoken by the controller and then a virtual assistant navigates to collect the data requested from the controllers,which allows additional or free time to the controllers to contemplate more on the work or could assist to another aircraft.The Google speech application programming interface(API)converts audio to text to recognize keywords.AI agent is trained using the Hidden marko model(HMM)algorithm such that it could learn the characteristics of the distinct voices of the controllers.At this stage,the proposed IVAS can be used to provide training for novice air traffic controllers effectively.The system is to be developed as a real-time system which could be used at the air traffic controlling base for actual traffic controlling purposes and the system is to be further upgraded to perform the task by recognizing keywords directly from the pilot voice command.

Human Resource Management:A Case Study of the Air Traffic Controller Strike in 1981

The article discusses several hemispheres of human resource management based on a typical case review. The study presents the main problems of the case from both employees and organizations; the issued problems involve compensations and benefits,restructuring,job design,and training. Based on the analysis of the case,two alternative solutions state that the potential routes for the organizations to avoid serious negative results. The study introduces a number of recommendations that can be used as a reference for other organizations to avoid similar risks.

民航管制安全风险主题时空分布规律研究

为了探究民航管制安全风险的时空分布规律,基于潜在迪利克雷分布(Latent Dirichlet Allocation,LDA)主题模型识别出民航管制安全风险主题,定义民航管制安全风险主题强度的定量测度指标,运用全局空间自相关分析和冷热点分析对民航管制安全风险主题的时空分布规律进行研究。结果表明:利用LDA主题模型识别出“管制员指令错误风险”等10个管制安全风险主题;“管制员指令错误风险”主题存在较弱的全局空间自相关性,在2018—2021年,全局Moran’s I总体呈现波动增长的趋势;在2018—2021年,“管制员指令错误风险”主题强度高值聚集的区域由西南向东南转移,高值聚集区域数量变少,且不稳定,低值聚集区域发生转移并在2020年后保持稳定。通过全局空间自相关分析和冷热点分析确定了2018—2021年中国民航不同管制区域的管制安全风险的时空分布格局,为局方进行差异化的安全监管提供决策支持。

考虑空域时空特性的动态多跑道使用策略优化方法

多跑道机场飞行区运行效率低下会导致空域-跑道系统容流供需失衡,进而造成终端区空域交通拥堵、航班延误现象频发。为提升多跑道机场终端区运行效率,借助全空域与机场模型软件(total airspace and airport modeler,TAAM)建立空域仿真模型,针对不同运行模式动态转换下对终端区交通流走向、扇区开合等空域时空特性的影响进行分析,提出1种考虑不同运行时段内终端区机场走廊口流量配比和进离港流量分布的动态多跑道使用策略优化方法。首先,使用TAAM综合考虑不同跑道运行模式下各扇区内航班流量、高度变更、移交协调及冲突解脱对管制负荷的影响,拟合得出不同跑道运行模式下基于当量航空器架次的各扇区管制负荷函数。以终端区内航班平均飞行时间、平均延误时间及管制员工作负荷为优化目标,建立了跑道使用策略优化模型。设计了1种基于航空器基本性能数据库(the base aircraft data,BADA)的多目标非支配排序遗传算法(NSGA-Ⅱ),并结合机场实际运行条件在无运行限制、运行方向限制、运行模式限制等5种场景下进行仿真计算。对各场景Pareto最优解集进行评价得出不同场景下最优跑道使用策略,并使用TAAM进行仿真对比验证。结果表明:无运行限制和运行方向限制相较于单一跑道运行模式的航班服务效率提升10.15%,5.01%;管制员工作负荷减少3.91%,3.4%;延误时间减少28.86%,19.46%。

基于OBE理念的机场管制模拟训练改革与实践

基于OBE理念,从课程目标、实践内容、实践过程和考核方法以及校企联合授课教学等方面对机场管制模拟训练进行改革。实践表明:课程目标反向设计、实践内容与训练难度分层设计、指导教师及学生动态分组、基于教考分离和自动评价的面向过程考核、校企联合教学等改革手段,能较好地解决教学存在的问题。与改革前相比较,不仅课程平均分有所提升,优秀比例也进一步提高,60~69分段比例下降明显,课程达成度提升效果显著。基于OBE理念的改革方案和实践成果为交通运输类专业的实践教学提供了应用案例。

自动飞行员复诵指令生成方法研究

为了提高效率,降低培训成本并推广使用计算机来取代管制模拟机中的飞行员席位,采用集成学习的策略来生成飞行员复诵指令。选用5个大规模预训练语言模型进行微调,并使用K折交叉验证来筛选出性能较好的4个模型作为基础模型来构建集成学习模型。所构建的集成学习模型在管制指令数据集上取得在本领域中的最优效果。在通用的ROUGE(recall-oriented understudy for gisting evaluation)评价标准中,取得R_(OUGE-1)=0.998,R_(OUGE-2)=0.995,R_(OUGE-L)=0.998的最新效果。其中,R_(OUGE-1)关注参考文本与生成文本之间单个单词的匹配度,R_(OUGE-2)则关注两个连续单词的匹配度,R_(OUGE-L)则关注最长公共子序列的匹配度。为了克服通用指标在本领域的局限性,更准确地评估模型性能,针对生成的复诵指令提出一套基于关键词的评价标准。该评价指标准基于管制文本分词后的结果计算各个关键词指标来评估模型的效果。在基于关键词的评价标准下,所构建模型取得整体准确率为0.987的最优效果,对航空器呼号的复诵准确率达到0.998。

中国省会航空节点城市复杂时空系统协调度变化及空间特征分析

航空运输业提升了城市系统的吸引力并激发新的交通需求,引导城市经济和空间结构快速演进。本文构建了包含经济、规模、外贸、科技、社会、环境、航空等要素在内的城市复杂时空系统指标体系,对中国省会航空节点城市及其构成的7个空管区的复杂时空系统发展的协调程度变化情况及时空聚类特征、空间溢出效应进行了分析。研究发现:2001—2019年,中国31个省会(首府、直辖市)航空节点城市在复杂时空系统发展过程中呈现了或分散、或集中的趋势特征,南方城市在研究末期的协调程度表现更好。7个空管区中,华北区协调度最高,新疆区、西北区、华东区、中南区的协调度水平发展增速较快,航空系统的平均协调度最高。各航空节点城市复杂时空系统协调度的时空聚类模式存在空间分异特征与聚类特征,航空系统在空管区的聚类优势最为突出,经济系统在空管区聚类辐射范围最大。航空、规模、科技与环境系统均表现了正向直接空间溢出效应,经济与科技系统表现了正向间接空间溢出效应。通过全面分析航空节点城市及空管区的复杂时空系统协调度变化,精准判定了航空节点城市多指标系统在区域发展过程中的影响程度及时空分布异质性,对拓展航空地理研究及促进航空节点城市高质量综合发展提供理论依据。

基于扇区开合的管制员排班优化

为提高管制员的利用效率和工作满意度,制定出更切合实际运行的管制员排班表,从管制工作的实际需要出发,以人员需求数量最少为目标,在符合民航局各种规章要求的基础上,建立管制员排班模型。在初始模型的基础上,又考虑了人员对工作及休息时间的个性化需求,对模型进行改进。最后以某空中交通管理局区域管制室管制员数据为例进行分析,使用LINGO求解器对模型进行求解。结果表明:与手工编制的排班时间表相比,通过排班优化模型制定出的排班方案减少了执勤人数,避免了人员浪费,并能较为充分地满足管制员的个性化需求。

因天气影响的航空器绕飞管制安全评估及策略分析

为降低航空器在绕飞情况下管制指挥事故的风险,采用事故树分析法(fault tree analysis, FTA)和层次分析法(analytic hierarchy process, AHP)相结合的方式对管制员在进行绕飞管制时发生指挥事故建立综合风险评估模型并制定风险管控措施,以及验证所提管控措施的有效性。首先对各风险因素进行逐层分析,找出其关联性,建立绕飞管制指挥事故的事故树模型。以此为基础,结合层次分析法,确定各层风险指标权重。同时依据各指标总权重的排序结果,提出了在绕飞情况下管制指挥事故的管控措施,最后以长沙区域管制空域为实例对所提方案进行验证。结果表明:在本文方案下虽然雷雨、风切变、协调程序过于复杂和航空器偏离计划航路仍然是导致绕飞管制事故的重要因素,但其重要程度下降了6.2%,且成熟管制员人数不足、扇区流量大和预案频繁发生变化的状况得到有效解决。证明了该方案能够有效抑制导致绕飞管制指挥事故发生的事件的重要程度,减少因极端天气和扇区负荷过大而出现的安全风险事故,整体提高管制系统的稳定性和可靠性。

面向智能视频监控的空中交通管制员图像分割

为解决复杂场景下空中交通管制员检测与分割精度低、鲁棒性差的问题,提出一种基于掩码区域卷积神经网络(Mask Region-based Convolutional Neural Networks, Mask R-CNN)的管制员图像分割模型ATC Mask R-CNN(ATC Mask Region-based Convolutional Neural Networks)。首先,构建管制员监控图像数据集(ATC Monitor Image Dataset, AMID)并用于模型训练、测试;其次,在主干网络中引入瓶颈注意力模块(Bottleneck Attention Module, BAM)以增强管制员特征提取,采取改进的柔性非极大值抑制算法(Soft Non-maximum Suppression, Soft-NMS)替代NMS算法进行候选框选取,提高对遮挡目标的检测分割;最后,基于AMID进行管制员图像分割试验。结果显示:ATC Mask R-CNN的精确率、召回率和平均精度分别为96.49%、95.62%和88.84%,表明了该方法的有效性。与Mask R-CNN相比,ATC Mask R-CNN有效降低了复杂场景的不利影响,更适用于管制员工作场景,可以为管制大厅安全管理自动化应用提供技术支撑。