Traffic Control Based on Integrated Kalman Filtering and Adaptive Quantized Q-Learning Framework for Internet of Vehicles

Intelligent traffic control requires accurate estimation of the road states and incorporation of adaptive or dynamically adjusted intelligent algorithms for making the decision.In this article,these issues are handled by proposing a novel framework for traffic control using vehicular communications and Internet of Things data.The framework integrates Kalman filtering and Q-learning.Unlike smoothing Kalman filtering,our data fusion Kalman filter incorporates a process-aware model which makes it superior in terms of the prediction error.Unlike traditional Q-learning,our Q-learning algorithm enables adaptive state quantization by changing the threshold of separating low traffic from high traffic on the road according to the maximum number of vehicles in the junction roads.For evaluation,the model has been simulated on a single intersection consisting of four roads:east,west,north,and south.A comparison of the developed adaptive quantized Q-learning(AQQL)framework with state-of-the-art and greedy approaches shows the superiority of AQQL with an improvement percentage in terms of the released number of vehicles of AQQL is 5%over the greedy approach and 340%over the state-of-the-art approach.Hence,AQQL provides an effective traffic control that can be applied in today’s intelligent traffic system.

Audio-Text Multimodal Speech Recognition via Dual-Tower Architecture for Mandarin Air Traffic Control Communications

In air traffic control communications (ATCC), misunderstandings between pilots and controllers could result in fatal aviation accidents. Fortunately, advanced automatic speech recognition technology has emerged as a promising means of preventing miscommunications and enhancing aviation safety. However, most existing speech recognition methods merely incorporate external language models on the decoder side, leading to insufficient semantic alignment between speech and text modalities during the encoding phase. Furthermore, it is challenging to model acoustic context dependencies over long distances due to the longer speech sequences than text, especially for the extended ATCC data. To address these issues, we propose a speech-text multimodal dual-tower architecture for speech recognition. It employs cross-modal interactions to achieve close semantic alignment during the encoding stage and strengthen its capabilities in modeling auditory long-distance context dependencies. In addition, a two-stage training strategy is elaborately devised to derive semantics-aware acoustic representations effectively. The first stage focuses on pre-training the speech-text multimodal encoding module to enhance inter-modal semantic alignment and aural long-distance context dependencies. The second stage fine-tunes the entire network to bridge the input modality variation gap between the training and inference phases and boost generalization performance. Extensive experiments demonstrate the effectiveness of the proposed speech-text multimodal speech recognition method on the ATCC and AISHELL-1 datasets. It reduces the character error rate to 6.54% and 8.73%, respectively, and exhibits substantial performance gains of 28.76% and 23.82% compared with the best baseline model. The case studies indicate that the obtained semantics-aware acoustic representations aid in accurately recognizing terms with similar pronunciations but distinctive semantics. The research provides a novel modeling paradigm for semantics-aware speech recognition in air traffic control communications, which could contribute to the advancement of intelligent and efficient aviation safety management.

Cluster DetectionMethod of Endogenous Security Abnormal Attack Behavior in Air Traffic Control Network

In order to enhance the accuracy of Air Traffic Control(ATC)cybersecurity attack detection,in this paper,a new clustering detection method is designed for air traffic control network security attacks.The feature set for ATC cybersecurity attacks is constructed by setting the feature states,adding recursive features,and determining the feature criticality.The expected information gain and entropy of the feature data are computed to determine the information gain of the feature data and reduce the interference of similar feature data.An autoencoder is introduced into the AI(artificial intelligence)algorithm to encode and decode the characteristics of ATC network security attack behavior to reduce the dimensionality of the ATC network security attack behavior data.Based on the above processing,an unsupervised learning algorithm for clustering detection of ATC network security attacks is designed.First,determine the distance between the clustering clusters of ATC network security attack behavior characteristics,calculate the clustering threshold,and construct the initial clustering center.Then,the new average value of all feature objects in each cluster is recalculated as the new cluster center.Second,it traverses all objects in a cluster of ATC network security attack behavior feature data.Finally,the cluster detection of ATC network security attack behavior is completed by the computation of objective functions.The experiment took three groups of experimental attack behavior data sets as the test object,and took the detection rate,false detection rate and recall rate as the test indicators,and selected three similar methods for comparative test.The experimental results show that the detection rate of this method is about 98%,the false positive rate is below 1%,and the recall rate is above 97%.Research shows that this method can improve the detection performance of security attacks in air traffic control network.

Development of a Bias Compensating Q-Learning Controller for a Multi-Zone HVAC Facility

We present the development of a bias compensating reinforcement learning(RL)algorithm that optimizes thermal comfort(by minimizing tracking error)and control utilization(by penalizing setpoint deviations)in a multi-zone heating,ventilation,and air-conditioning(HVAC)lab facility subject to unmeasurable disturbances and unknown dynamics.It is shown that the presence of unmeasurable disturbance results in an inconsistent learning equation in traditional RL controllers leading to parameter estimation bias(even with integral action support),and in the extreme case,the divergence of the learning algorithm.We demonstrate this issue by applying the popular Q-learning algorithm to linear quadratic regulation(LQR)of a multi-zone HVAC environment and showing that,even with integral support,the algorithm exhibits bias issue during the learning phase when the HVAC disturbance is unmeasurable due to unknown heat gains,occupancy variations,light sources,and outside weather changes.To address this difficulty,we present a bias compensating learning equation that learns a lumped bias term as a result of disturbances(and possibly other sources)in conjunction with the optimal control parameters.Experimental results show that the proposed scheme not only recovers the bias-free optimal control parameters but it does so without explicitly learning the dynamic model or estimating the disturbances,demonstrating the effectiveness of the algorithm in addressing the above challenges.

Urban Traffic Control Meets Decision Recommendation System:A Survey and Perspective

Urban traffic control is a multifaceted and demanding task that necessitates extensive decision-making to ensure the safety and efficiency of urban transportation systems.Traditional approaches require traffic signal professionals to manually intervene on traffic control devices at the intersection level,utilizing their knowledge and expertise.However,this process is cumbersome,labor-intensive,and cannot be applied on a large network scale.Recent studies have begun to explore the applicability of recommendation system for urban traffic control,which offer increased control efficiency and scalability.Such a decision recommendation system is complex,with various interdependent components,but a systematic literature review has not yet been conducted.In this work,we present an up-to-date survey that elucidates all the detailed components of a recommendation system for urban traffic control,demonstrates the utility and efficacy of such a system in the real world using data and knowledgedriven approaches,and discusses the current challenges and potential future directions of this field.

Novel Distance Measures on Hesitant Fuzzy Sets Based on Equal-Probability Transformation and Their Application in Decision Making on Intersection Traffic Control

The purpose of this study is to reduce the uncertainty in the calculation process on hesitant fuzzy sets(HFSs).The innovation of this study is to unify the cardinal numbers of hesitant fuzzy elements(HFEs)in a special way.Firstly,a probability density function is assigned for any given HFE.Thereafter,equal-probability transformation is introduced to transform HFEs with different cardinal numbers on the condition into the same probability density function.The characteristic of this transformation is that the higher the consistency of the membership degrees in HFEs,the higher the credibility of the mentioned membership degrees is,then,the bigger the probability density values for them are.According to this transformation technique,a set of novel distance measures on HFSs is provided.Finally,an illustrative example of intersection traffic control is introduced to show the usefulness of the given distance measures.The example also shows that this study is a good complement to operation theories on HFSs.

A Robust Conformer-Based Speech Recognition Model for Mandarin Air Traffic Control

This study aims to address the deviation in downstream tasks caused by inaccurate recognition results when applying Automatic Speech Recognition(ASR)technology in the Air Traffic Control(ATC)field.This paper presents a novel cascaded model architecture,namely Conformer-CTC/Attention-T5(CCAT),to build a highly accurate and robust ATC speech recognition model.To tackle the challenges posed by noise and fast speech rate in ATC,the Conformer model is employed to extract robust and discriminative speech representations from raw waveforms.On the decoding side,the Attention mechanism is integrated to facilitate precise alignment between input features and output characters.The Text-To-Text Transfer Transformer(T5)language model is also introduced to handle particular pronunciations and code-mixing issues,providing more accurate and concise textual output for downstream tasks.To enhance the model’s robustness,transfer learning and data augmentation techniques are utilized in the training strategy.The model’s performance is optimized by performing hyperparameter tunings,such as adjusting the number of attention heads,encoder layers,and the weights of the loss function.The experimental results demonstrate the significant contributions of data augmentation,hyperparameter tuning,and error correction models to the overall model performance.On the Our ATC Corpus dataset,the proposed model achieves a Character Error Rate(CER)of 3.44%,representing a 3.64%improvement compared to the baseline model.Moreover,the effectiveness of the proposed model is validated on two publicly available datasets.On the AISHELL-1 dataset,the CCAT model achieves a CER of 3.42%,showcasing a 1.23%improvement over the baseline model.Similarly,on the LibriSpeech dataset,the CCAT model achieves a Word Error Rate(WER)of 5.27%,demonstrating a performance improvement of 7.67%compared to the baseline model.Additionally,this paper proposes an evaluation criterion for assessing the robustness of ATC speech recognition systems.In robustness evaluation experiments based on this criterion,the proposed model demonstrates a performance improvement of 22%compared to the baseline model.

A Genetic Based Fuzzy Q-Learning Flow Controller for High-Speed Networks

For the congestion problems in high-speed networks, a genetic based fuzzy Q-learning flow controller is proposed. Because of the uncertainties and highly time-varying, it is not easy to accurately obtain the complete information for high-speed networks. In this case, the Q-learning, which is independent of mathematic model, and prior-knowledge, has good performance. The fuzzy inference is introduced in order to facilitate generalization in large state space, and the genetic operators are used to obtain the consequent parts of fuzzy rules. Simulation results show that the proposed controller can learn to take the best action to regulate source flow with the features of high throughput and low packet loss ratio, and can avoid the occurrence of congestion effectively.

Hybrid Q-learning for data-based optimal control of non-linear switching system

In this paper,the optimal control of non-linear switching system is investigated without knowing the system dynamics.First,the Hamilton-Jacobi-Bellman(HJB)equation is derived with the consideration of hybrid action space.Then,a novel data-based hybrid Q-learning(HQL)algorithm is proposed to find the optimal solution in an iterative manner.In addition,the theoretical analysis is provided to illustrate the convergence and optimality of the proposed algorithm.Finally,the algorithm is implemented with the actor-critic(AC)structure,and two linear-in-parameter neural networks are utilized to approximate the functions.Simulation results validate the effectiveness of the data-driven method.

Q-Learning-Based Inter-Networking Mobile Number Portability Congestion Control Mechanism

Inter-networking Mobile Number Portability(MNP) is an important way to promote the development of new telecom services.For the congestion that always occurs on the MNP query nodes,this paper proposes a Q-learning-based inter-networking mobile number portability congestion control mechanism,taking the node queue delay and query service stability as the control targets,based on the optimal decision which is made over mobile user number portability query through self-adaptive connection admission control.The simulation and analysis show that,compared with tail-drop,random early detection and other mechanisms,this mechanism realizes a smoother and more regular queue size when the load is relatively light;and maintains a relatively low and stable queue size with relatively small loss when the load is relatively heavy,thus improving QoS and increasing network throughput at lower cost.

Supervisory control of the hybrid off-highway vehicle for fuel economy improvement using predictive double Q-learning with backup models

This paper studied a supervisory control system for a hybrid off-highway electric vehicle under the chargesustaining(CS)condition.A new predictive double Q-learning with backup models(PDQL)scheme is proposed to optimize the engine fuel in real-world driving and improve energy efficiency with a faster and more robust learning process.Unlike the existing“model-free”methods,which solely follow on-policy and off-policy to update knowledge bases(Q-tables),the PDQL is developed with the capability to merge both on-policy and off-policy learning by introducing a backup model(Q-table).Experimental evaluations are conducted based on software-in-the-loop(SiL)and hardware-in-the-loop(HiL)test platforms based on real-time modelling of the studied vehicle.Compared to the standard double Q-learning(SDQL),the PDQL only needs half of the learning iterations to achieve better energy efficiency than the SDQL at the end learning process.In the SiL under 35 rounds of learning,the results show that the PDQL can improve the vehicle energy efficiency by 1.75%higher than SDQL.By implementing the PDQL in HiL under four predefined real-world conditions,the PDQL can robustly save more than 5.03%energy than the SDQL scheme.

Adaptive signal control and coordination for urban traffic control in a connected vehicle environment: A review

Existing signal control systems for urban traffic are usually based on traffic flow data from fixed location detectors.Because of rapid advances in emerging vehicular communication,connected vehicle(CV)-based signal control demonstrates significant improvements over existing conventional signal control systems.Though various CV-based signal control systems have been investigated in the past decades,these approaches still have many issues and drawbacks to overcome.We summarize typical components and structures of these existing CV-based urban traffic signal control systems and digest several important issues from the summarized vital concepts.Last,future research directions are discussed with some suggestions.We hope this survey can facilitate the connected and automated vehicle and transportation research community to efficiently approach next-generation urban traffic signal control methods and systems.

Empirical Exploration of Air Traffic Control Behaviour at Terminal Maneuvering Area:From an Air Traffic Flow Aspect

In a large-volume,high-density traffic background,air traffic manifests fluid-like microscopical characteristics.The characteristics are formed by the micro tailing actions between individual aircraft.Aircraft headway refers to the time interval between successive flying aircraft in air traffic flow,which is one of the most important characteristics of air traffic flow.The variation in aircraft headway reveals the air traffic control behaviour.In this paper,we study the characteristics of air traffic control behaviours by analyzing radar tracks in a terminal maneuvering area.The headway in arrival traffic flow is measured after the determination of aircraft trailing relationships.The headway evolutionary characteristics for different control decisions and the headway evolutionary characteristics in different phase-states are discussed,and some interesting findings are gotten.This work may be helpful for scholars and managers in understanding the intrinsic nature of air traffic flow and in the development of intelligent assistant decision systems for air traffic management.

Forecast of Air Traffic Controller Demand Based on SVR and Parameter Optimization

As the main body of air traffic control safety,the air traffic controller is an important part of the whole air traffic control system. According to the relevant data of civil aviation over the years,a mapping model between flight support sorties and air traffic controller demand is constructed by using the prediction algorithm of support vector regression(SVR) based on grid search and cross-validation. Then the model predicts the demand for air traffic controllers in seven regions. Additionally,according to the employment data of civil aviation universities,the future training scale of air traffic controller is predicted. The forecast results show that the average relative error of the number of controllers predicted by the algorithm is 1.73%,and the prediction accuracy is higher than traditional regression algorithms. Under the influence of the epidemic,the demand for air traffic controllers will decrease in the short term,but with the control of the epidemic,the demand of air traffic controllers will return to the pre-epidemic level and gradually increase. It is expected that the controller increment will be about 816 by 2028. The forecast results of the demand for air traffic controllers provide a theoretical basis for the introduction and training of medium and long-term air traffic controllers,and also provide method guidance and decision support for the establishment of professional reserve and dynamic control mechanism in the air traffic control system.

Integrated Evaluation of Air Traffic Controller Workload Based on Matter-Element Analysis

A model for evaluating the controller workload was presented based on matter-element analysis, particularly from a mansystem engineering perspective. On the basis of a questionnaire survey, 18 kinds of indexes which influence the controller workload were determined. By establishing the classical field and node field of the controller workload, the correlation function of the controller workload grade was obtained; then the correlation degree and estimated grade of controller workload were given. A case study verifies the feasibility of the proposed evaluation method.

基于Q-learning和BP神经元网络的交叉口信号灯控制

解决单交叉口信号灯最优控制问题。提出了基于强化学习的信号灯控制系统结构,应用强化学习中Q学习,将信号灯最优控制问题转变成是否切换运行相位的决策问题,提出了采用BP神经元网络实现Q学习的信号灯控制系统。应用微观交通仿真软件PARAMICS进行仿真分析,结果表明该系统能够感知交通流变化,并能够自适应地调整信号灯切换策略,以达到最优的控制效果,该方法是可行的,与定时控制相比具有明显的优势。

Effect of Working Experience on Air Traffic Controller Eye Movement

Eye movement is an important indicator of information-seeking behavior and provides insight into cognitive strategies which are vital for decision-making.Various measures based on eye movements have been proposed to capture humans’ability to process information in a complex environment.The effectiveness of these measures has not yet been fully explored in the field of air traffic management.This paper presents a comparative study on eye-movement measures in air traffic controllers with different levels of working experience.Two commonly investigated oculomotor behaviors,fixation and saccades,together with gaze entropy,are examined.By comparing the statistical properties of the relevant metrics,it is shown that working experience has a notable effect on eye-movement patterns.Both fixation and saccades differ between qualified and novice controllers,with the former type of controller employing more efficient searching strategies.These findings are useful in enhancing the quality of controller training and contributing to an understanding of the information-seeking mechanisms humans use when executing complex tasks.

Identification of Information-Seeking Behaviors from Air Traffic Controllers′ Eye Movements

Air traffic controllers are the important parts of air traffic management system who are responsible for the safety and efficiency of the system.They make traffic management decisions based on information acquired from various sources.The understanding of their information seeking behaviors is still limited.We aim to identify controllers′ behavior through the examination of the correlations between controllers′eye movements and air traffic.Sixteen air traffic controllers were invited to participate real-time simulation experiments,during which the data of their eye ball movements and air traffic were recorded.Tweny-three air traffic complexity metrics and six eye movements metrics were calculated to examine their relationships.Two correlational methods,Pearson′s correlation and Spearman′s correlation,were tested between every eye-traffic pair of metrics.The results indicate that controllers′two kinds of information-seeking behaviors can be identified from their eye movements:Targets tracking,and confliction recognition.The study on controllers′ eye movements may contribute to the understanding of information-seeking mechanisms leading to the development of more intelligent automations in the future.

OPTIMAL FLOW CONTROL PROBLEM IN AIR TRAFFIC CONTROL:PARTⅠ

The fundamental case is considered in which flights from many destinations must be scheduled for arrival at a single congested airport having limited capacities.An air traffic control(ATC)model is developed in this case.A new and efficient algorithm for the optimal solution of ground holding strategy problem(GHSP)is put forward and verified by a numerical example.

Intelligent Virtual Assistant System(IVAS) for Air Traffic Controllers

Air traffic control is an essential obligation in the aviation industry to have safe and efficient air transportation.Year by year,the workload and on-job-stress of the air traffic controllers are rapidly increasing due to the rapid growth of air traveling.Controllers are usually dealing with multiple aircrafts at a time and must make quick and accurate decisions to ensure the safety of aircrafts.Heavy workload and high responsibilities create air traffic control a stressful job that sometimes could be error-prone and time-consuming,since controlling and decision-making are solely dependent on human intelligence.To provide effective solutions for the mentioned on the job challenges of the controllers,this study proposed an intelligent virtual assistant system(IVAS)to assist the controllers thereby to reduce the controllers’workload.Consisting of four main parts,which are voice recognition,display conversation on screen,task execution,and text to speech,the proposed system is developed with the aid of artificial intelligence(AI)techniques to make speedy decisions and be free of human interventions.IVAS is a computer-based system that can be activated by the voice of the air traffic controller and then appropriately assist to control the flight.IVAS identifies the words spoken by the controller and then a virtual assistant navigates to collect the data requested from the controllers,which allows additional or free time to the controllers to contemplate more on the work or could assist to another aircraft.The Google speech application programming interface(API)converts audio to text to recognize keywords.AI agent is trained using the Hidden marko model(HMM)algorithm such that it could learn the characteristics of the distinct voices of the controllers.At this stage,the proposed IVAS can be used to provide training for novice air traffic controllers effectively.The system is to be developed as a real-time system which could be used at the air traffic controlling base for actual traffic controlling purposes and the system is to be further upgraded to perform the task by recognizing keywords directly from the pilot voice command.