BCCLR:A Skeleton-Based Action Recognition with Graph Convolutional Network Combining Behavior Dependence and Context Clues

In recent years,skeleton-based action recognition has made great achievements in Computer Vision.A graph convolutional network(GCN)is effective for action recognition,modelling the human skeleton as a spatio-temporal graph.Most GCNs define the graph topology by physical relations of the human joints.However,this predefined graph ignores the spatial relationship between non-adjacent joint pairs in special actions and the behavior dependence between joint pairs,resulting in a low recognition rate for specific actions with implicit correlation between joint pairs.In addition,existing methods ignore the trend correlation between adjacent frames within an action and context clues,leading to erroneous action recognition with similar poses.Therefore,this study proposes a learnable GCN based on behavior dependence,which considers implicit joint correlation by constructing a dynamic learnable graph with extraction of specific behavior dependence of joint pairs.By using the weight relationship between the joint pairs,an adaptive model is constructed.It also designs a self-attention module to obtain their inter-frame topological relationship for exploring the context of actions.Combining the shared topology and the multi-head self-attention map,the module obtains the context-based clue topology to update the dynamic graph convolution,achieving accurate recognition of different actions with similar poses.Detailed experiments on public datasets demonstrate that the proposed method achieves better results and realizes higher quality representation of actions under various evaluation protocols compared to state-of-the-art methods.

Automatic modulation recognition of radio fuzes using a DR2D-based adaptive denoising method and textural feature extraction

The identification of intercepted radio fuze modulation types is a prerequisite for decision-making in interference systems.However,the electromagnetic environment of modern battlefields is complex,and the signal-to-noise ratio(SNR)of such environments is usually low,which makes it difficult to implement accurate recognition of radio fuzes.To solve the above problem,a radio fuze automatic modulation recognition(AMR)method for low-SNR environments is proposed.First,an adaptive denoising algorithm based on data rearrangement and the two-dimensional(2D)fast Fourier transform(FFT)(DR2D)is used to reduce the noise of the intercepted radio fuze intermediate frequency(IF)signal.Then,the textural features of the denoised IF signal rearranged data matrix are extracted from the statistical indicator vectors of gray-level cooccurrence matrices(GLCMs),and support vector machines(SVMs)are used for classification.The DR2D-based adaptive denoising algorithm achieves an average correlation coefficient of more than 0.76 for ten fuze types under SNRs of-10 d B and above,which is higher than that of other typical algorithms.The trained SVM classification model achieves an average recognition accuracy of more than 96%on seven modulation types and recognition accuracies of more than 94%on each modulation type under SNRs of-12 d B and above,which represents a good AMR performance of radio fuzes under low SNRs.

KurdSet: A Kurdish Handwritten Characters Recognition Dataset Using Convolutional Neural Network

Handwritten character recognition(HCR)involves identifying characters in images,documents,and various sources such as forms surveys,questionnaires,and signatures,and transforming them into a machine-readable format for subsequent processing.Successfully recognizing complex and intricately shaped handwritten characters remains a significant obstacle.The use of convolutional neural network(CNN)in recent developments has notably advanced HCR,leveraging the ability to extract discriminative features from extensive sets of raw data.Because of the absence of pre-existing datasets in the Kurdish language,we created a Kurdish handwritten dataset called(KurdSet).The dataset consists of Kurdish characters,digits,texts,and symbols.The dataset consists of 1560 participants and contains 45,240 characters.In this study,we chose characters only from our dataset.We utilized a Kurdish dataset for handwritten character recognition.The study also utilizes various models,including InceptionV3,Xception,DenseNet121,and a customCNNmodel.To show the performance of the KurdSet dataset,we compared it to Arabic handwritten character recognition dataset(AHCD).We applied the models to both datasets to show the performance of our dataset.Additionally,the performance of the models is evaluated using test accuracy,which measures the percentage of correctly classified characters in the evaluation phase.All models performed well in the training phase,DenseNet121 exhibited the highest accuracy among the models,achieving a high accuracy of 99.80%on the Kurdish dataset.And Xception model achieved 98.66%using the Arabic dataset.

Working condition recognition of sucker rod pumping system based on 4-segment time-frequency signature matrix and deep learning

High-precision and real-time diagnosis of sucker rod pumping system(SRPS)is important for quickly mastering oil well operations.Deep learning-based method for classifying the dynamometer card(DC)of oil wells is an efficient diagnosis method.However,the input of the DC as a two-dimensional image into the deep learning framework suffers from low feature utilization and high computational effort.Additionally,different SRPSs in an oil field have various system parameters,and the same SRPS generates different DCs at different moments.Thus,there is heterogeneity in field data,which can dramatically impair the diagnostic accuracy.To solve the above problems,a working condition recognition method based on 4-segment time-frequency signature matrix(4S-TFSM)and deep learning is presented in this paper.First,the 4-segment time-frequency signature(4S-TFS)method that can reduce the computing power requirements is proposed for feature extraction of DC data.Subsequently,the 4S-TFSM is constructed by relative normalization and matrix calculation to synthesize the features of multiple data and solve the problem of data heterogeneity.Finally,a convolutional neural network(CNN),one of the deep learning frameworks,is used to determine the functioning conditions based on the 4S-TFSM.Experiments on field data verify that the proposed diagnostic method based on 4S-TFSM and CNN(4S-TFSM-CNN)can significantly improve the accuracy of working condition recognition with lower computational cost.To the best of our knowledge,this is the first work to discuss the effect of data heterogeneity on the working condition recognition performance of SRPS.

Analysis of RNA Recognition and Binding Characteristics of OsCPPR1 Protein in Rice

Pentatricopeptide repeat(PPR)proteins represent one of the largest protein families in plants and typically localize to organelles like mitochondria and chloroplasts.By contrast,CYTOPLASMLOCALIZED PPR1(OsCPPR1)is a cytoplasm-localized PPR protein that can degrade OsGOLDENLIKE1(OsGLK1)mRNA in the tapetum of rice anther.However,the mechanism,by which OsCPPR1 recognizes and binds to OsGLK1 transcripts,remains unknown.Through protein structure prediction and macromolecular docking experiments,we observed that distinct PPR motif structures of OsCPPR1 exhibited varying binding efficiencies to OsGLK1 RNA.Moreover,RNA-electrophoretic mobility shift assay experiment demonstrated that the recombinant OsCPPR1 can directly recognize and bind to OsGLK1 mRNA in vitro.This further confirmed that the mutations in the conserved amino acids in each PPR motif resulted in loss of activity,while truncation of OsCPPR1 decreased its binding efficiency.These findings collectively suggest that it may require some co-factors to assist in cleavage,a facet that warrants further exploration in subsequent studies.

GeoNER:Geological Named Entity Recognition with Enriched Domain Pre-Training Model and Adversarial Training

As important geological data,a geological report contains rich expert and geological knowledge,but the challenge facing current research into geological knowledge extraction and mining is how to render accurate understanding of geological reports guided by domain knowledge.While generic named entity recognition models/tools can be utilized for the processing of geoscience reports/documents,their effectiveness is hampered by a dearth of domain-specific knowledge,which in turn leads to a pronounced decline in recognition accuracy.This study summarizes six types of typical geological entities,with reference to the ontological system of geological domains and builds a high quality corpus for the task of geological named entity recognition(GNER).In addition,Geo Wo BERT-adv BGP(Geological Word-base BERTadversarial training Bi-directional Long Short-Term Memory Global Pointer)is proposed to address the issues of ambiguity,diversity and nested entities for the geological entities.The model first uses the fine-tuned word granularitybased pre-training model Geo Wo BERT(Geological Word-base BERT)and combines the text features that are extracted using the Bi LSTM(Bi-directional Long Short-Term Memory),followed by an adversarial training algorithm to improve the robustness of the model and enhance its resistance to interference,the decoding finally being performed using a global association pointer algorithm.The experimental results show that the proposed model for the constructed dataset achieves high performance and is capable of mining the rich geological information.

Deep Learning Approach for Hand Gesture Recognition:Applications in Deaf Communication and Healthcare

Hand gestures have been used as a significant mode of communication since the advent of human civilization.By facilitating human-computer interaction(HCI),hand gesture recognition(HGRoc)technology is crucial for seamless and error-free HCI.HGRoc technology is pivotal in healthcare and communication for the deaf community.Despite significant advancements in computer vision-based gesture recognition for language understanding,two considerable challenges persist in this field:(a)limited and common gestures are considered,(b)processing multiple channels of information across a network takes huge computational time during discriminative feature extraction.Therefore,a novel hand vision-based convolutional neural network(CNN)model named(HVCNNM)offers several benefits,notably enhanced accuracy,robustness to variations,real-time performance,reduced channels,and scalability.Additionally,these models can be optimized for real-time performance,learn from large amounts of data,and are scalable to handle complex recognition tasks for efficient human-computer interaction.The proposed model was evaluated on two challenging datasets,namely the Massey University Dataset(MUD)and the American Sign Language(ASL)Alphabet Dataset(ASLAD).On the MUD and ASLAD datasets,HVCNNM achieved a score of 99.23% and 99.00%,respectively.These results demonstrate the effectiveness of CNN as a promising HGRoc approach.The findings suggest that the proposed model have potential roles in applications such as sign language recognition,human-computer interaction,and robotics.

Spatial pattern recognition for near-surface high temperature increases in mountain areas using MODIS and SRTM DEM

Abrupt near-surface temperature changes in mountainous areas are a special component of the mountain climate system.Fast and accurate measurements of the locations,intensity,and width of the near-surface changes are necessary but highly difficult due to the complicated environmental conditions and instrumental issues.This paper develops a spatial pattern recognition method to measure the near-surface high temperature increase(NSHTI),one of the lesser-attended changes.First,raster window measurement was proposed to calculate the temperature lapse rate using MODIS land surface temperature and SRTM DEM data.It fully considers the terrain heights of two neighboring cells on opposite or adjacent slopes with a moving window of 3×3 cell size.Second,a threshold selection was performed to identify the NSHTI cells using a threshold of-0.65℃/100 m.Then,the NSHTI strips were parameterized through raster vectorization and spatial analysis.Taking Yunnan,a mountainous province in southwestern China,as the study area,the results indicate that the NSHTI cells concentrate in a strip-like pattern along the mountains and valleys,and the strips are almost parallel to the altitude contours with a slight northward uplift.Also,they are located mostly at a 3/5 height of high mountains or within 400 m from the valley floors,where the controlling topographic index is the altitude of the terrain trend surface but not the absolute elevation and the topographic uplift height and cutting depth.Additionally,the NSHTI intensity varies with the geographic locations and the proportions increase with an exponential trend,and the horizontal width has a mean of about 1000 m and a maximum of over 5000 m.The result demonstrates that the proposed method can effectively recognize NSHTI boundaries over mountains,providing support for the modeling of weather and climate systems and the development of mountain resources.

Fine-Grained Ship Recognition Based on Visible and Near-Infrared Multimodal Remote Sensing Images: Dataset,Methodology and Evaluation

Fine-grained recognition of ships based on remote sensing images is crucial to safeguarding maritime rights and interests and maintaining national security.Currently,with the emergence of massive high-resolution multi-modality images,the use of multi-modality images for fine-grained recognition has become a promising technology.Fine-grained recognition of multi-modality images imposes higher requirements on the dataset samples.The key to the problem is how to extract and fuse the complementary features of multi-modality images to obtain more discriminative fusion features.The attention mechanism helps the model to pinpoint the key information in the image,resulting in a significant improvement in the model’s performance.In this paper,a dataset for fine-grained recognition of ships based on visible and near-infrared multi-modality remote sensing images has been proposed first,named Dataset for Multimodal Fine-grained Recognition of Ships(DMFGRS).It includes 1,635 pairs of visible and near-infrared remote sensing images divided into 20 categories,collated from digital orthophotos model provided by commercial remote sensing satellites.DMFGRS provides two types of annotation format files,as well as segmentation mask images corresponding to the ship targets.Then,a Multimodal Information Cross-Enhancement Network(MICE-Net)fusing features of visible and near-infrared remote sensing images,has been proposed.In the network,a dual-branch feature extraction and fusion module has been designed to obtain more expressive features.The Feature Cross Enhancement Module(FCEM)achieves the fusion enhancement of the two modal features by making the channel attention and spatial attention work cross-functionally on the feature map.A benchmark is established by evaluating state-of-the-art object recognition algorithms on DMFGRS.MICE-Net conducted experiments on DMFGRS,and the precision,recall,mAP0.5 and mAP0.5:0.95 reached 87%,77.1%,83.8%and 63.9%,respectively.Extensive experiments demonstrate that the proposed MICE-Net has more excellent performance on DMFGRS.Built on lightweight network YOLO,the model has excellent generalizability,and thus has good potential for application in real-life scenarios.

Recognition and rejection of foreign eggs of different colors in Barn Swallows

Brood parasitic birds lay eggs in the nests of other birds,and the parasitized hosts can reduce the cost of raising unrelated offspring through the recognition of parasitic eggs.Hosts can adopt vision-based cognitive mechanisms to recognize foreign eggs by comparing the colors of foreign and host eggs.However,there is currently no uniform conclusion as to whether this comparison involves the single or multiple threshold decision rules.In this study,we tested both hypotheses by adding model eggs of different colors to the nests of Barn Swallows(Hirundo rustica)of two geographical populations breeding in Hainan and Heilongjiang Provinces in China.Results showed that Barn Swallows rejected more white model eggs(moderate mimetic to their own eggs)and blue model eggs(highly non-mimetic eggs with shorter reflectance spectrum)than red model eggs(highly nonmimetic eggs with longer reflectance spectrum).There was no difference in the rejection rate of model eggs between the two populations of Barn Swallows,and clutch size was not a factor affecting egg recognition.Our results are consistent with the single rejection threshold model.This study provides strong experimental evidence that the color of model eggs can has an important effect on egg recognition in Barn Swallows,opening up new avenues to uncover the evolution of cuckoo egg mimicry and explore the cognitive mechanisms underlying the visual recognition of foreign eggs by hosts.

Enhancing Security and Privacy in Distributed Face Recognition Systems through Blockchain and GAN Technologies

The use of privacy-enhanced facial recognition has increased in response to growing concerns about data securityand privacy in the digital age. This trend is spurred by rising demand for face recognition technology in a varietyof industries, including access control, law enforcement, surveillance, and internet communication. However,the growing usage of face recognition technology has created serious concerns about data monitoring and userprivacy preferences, especially in context-aware systems. In response to these problems, this study provides a novelframework that integrates sophisticated approaches such as Generative Adversarial Networks (GANs), Blockchain,and distributed computing to solve privacy concerns while maintaining exact face recognition. The framework’spainstaking design and execution strive to strike a compromise between precise face recognition and protectingpersonal data integrity in an increasingly interconnected environment. Using cutting-edge tools like Dlib for faceanalysis,Ray Cluster for distributed computing, and Blockchain for decentralized identity verification, the proposedsystem provides scalable and secure facial analysis while protecting user privacy. The study’s contributions includethe creation of a sustainable and scalable solution for privacy-aware face recognition, the implementation of flexibleprivacy computing approaches based on Blockchain networks, and the demonstration of higher performanceover previous methods. Specifically, the proposed StyleGAN model has an outstanding accuracy rate of 93.84%while processing high-resolution images from the CelebA-HQ dataset, beating other evaluated models such asProgressive GAN 90.27%, CycleGAN 89.80%, and MGAN 80.80%. With improvements in accuracy, speed, andprivacy protection, the framework has great promise for practical use in a variety of fields that need face recognitiontechnology. This study paves the way for future research in privacy-enhanced face recognition systems, emphasizingthe significance of using cutting-edge technology to meet rising privacy issues in digital identity.

Efficient Object Segmentation and Recognition Using Multi-Layer Perceptron Networks

Object segmentation and recognition is an imperative area of computer vision andmachine learning that identifies and separates individual objects within an image or video and determines classes or categories based on their features.The proposed system presents a distinctive approach to object segmentation and recognition using Artificial Neural Networks(ANNs).The system takes RGB images as input and uses a k-means clustering-based segmentation technique to fragment the intended parts of the images into different regions and label thembased on their characteristics.Then,two distinct kinds of features are obtained from the segmented images to help identify the objects of interest.An Artificial Neural Network(ANN)is then used to recognize the objects based on their features.Experiments were carried out with three standard datasets,MSRC,MS COCO,and Caltech 101 which are extensively used in object recognition research,to measure the productivity of the suggested approach.The findings from the experiment support the suggested system’s validity,as it achieved class recognition accuracies of 89%,83%,and 90.30% on the MSRC,MS COCO,and Caltech 101 datasets,respectively.

Cybernet Model:A New Deep Learning Model for Cyber DDoS Attacks Detection and Recognition

Cyberspace is extremely dynamic,with new attacks arising daily.Protecting cybersecurity controls is vital for network security.Deep Learning(DL)models find widespread use across various fields,with cybersecurity being one of the most crucial due to their rapid cyberattack detection capabilities on networks and hosts.The capabilities of DL in feature learning and analyzing extensive data volumes lead to the recognition of network traffic patterns.This study presents novel lightweight DL models,known as Cybernet models,for the detection and recognition of various cyber Distributed Denial of Service(DDoS)attacks.These models were constructed to have a reasonable number of learnable parameters,i.e.,less than 225,000,hence the name“lightweight.”This not only helps reduce the number of computations required but also results in faster training and inference times.Additionally,these models were designed to extract features in parallel from 1D Convolutional Neural Networks(CNN)and Long Short-Term Memory(LSTM),which makes them unique compared to earlier existing architectures and results in better performance measures.To validate their robustness and effectiveness,they were tested on the CIC-DDoS2019 dataset,which is an imbalanced and large dataset that contains different types of DDoS attacks.Experimental results revealed that bothmodels yielded promising results,with 99.99% for the detectionmodel and 99.76% for the recognition model in terms of accuracy,precision,recall,and F1 score.Furthermore,they outperformed the existing state-of-the-art models proposed for the same task.Thus,the proposed models can be used in cyber security research domains to successfully identify different types of attacks with a high detection and recognition rate.

Japanese Sign Language Recognition by Combining Joint Skeleton-Based Handcrafted and Pixel-Based Deep Learning Features with Machine Learning Classification

Sign language recognition is vital for enhancing communication accessibility among the Deaf and hard-of-hearing communities.In Japan,approximately 360,000 individualswith hearing and speech disabilities rely on Japanese Sign Language(JSL)for communication.However,existing JSL recognition systems have faced significant performance limitations due to inherent complexities.In response to these challenges,we present a novel JSL recognition system that employs a strategic fusion approach,combining joint skeleton-based handcrafted features and pixel-based deep learning features.Our system incorporates two distinct streams:the first stream extracts crucial handcrafted features,emphasizing the capture of hand and body movements within JSL gestures.Simultaneously,a deep learning-based transfer learning stream captures hierarchical representations of JSL gestures in the second stream.Then,we concatenated the critical information of the first stream and the hierarchy of the second stream features to produce the multiple levels of the fusion features,aiming to create a comprehensive representation of the JSL gestures.After reducing the dimensionality of the feature,a feature selection approach and a kernel-based support vector machine(SVM)were used for the classification.To assess the effectiveness of our approach,we conducted extensive experiments on our Lab JSL dataset and a publicly available Arabic sign language(ArSL)dataset.Our results unequivocally demonstrate that our fusion approach significantly enhances JSL recognition accuracy and robustness compared to individual feature sets or traditional recognition methods.

Automatic modulation recognition of radiation source signals based on two-dimensional data matrix and improved residual neural network

Automatic modulation recognition(AMR)of radiation source signals is a research focus in the field of cognitive radio.However,the AMR of radiation source signals at low SNRs still faces a great challenge.Therefore,the AMR method of radiation source signals based on two-dimensional data matrix and improved residual neural network is proposed in this paper.First,the time series of the radiation source signals are reconstructed into two-dimensional data matrix,which greatly simplifies the signal preprocessing process.Second,the depthwise convolution and large-size convolutional kernels based residual neural network(DLRNet)is proposed to improve the feature extraction capability of the AMR model.Finally,the model performs feature extraction and classification on the two-dimensional data matrix to obtain the recognition vector that represents the signal modulation type.Theoretical analysis and simulation results show that the AMR method based on two-dimensional data matrix and improved residual network can significantly improve the accuracy of the AMR method.The recognition accuracy of the proposed method maintains a high level greater than 90% even at -14 dB SNR.

RoBGP:A Chinese Nested Biomedical Named Entity Recognition Model Based on RoBERTa and Global Pointer

Named Entity Recognition(NER)stands as a fundamental task within the field of biomedical text mining,aiming to extract specific types of entities such as genes,proteins,and diseases from complex biomedical texts and categorize them into predefined entity types.This process can provide basic support for the automatic construction of knowledge bases.In contrast to general texts,biomedical texts frequently contain numerous nested entities and local dependencies among these entities,presenting significant challenges to prevailing NER models.To address these issues,we propose a novel Chinese nested biomedical NER model based on RoBERTa and Global Pointer(RoBGP).Our model initially utilizes the RoBERTa-wwm-ext-large pretrained language model to dynamically generate word-level initial vectors.It then incorporates a Bidirectional Long Short-Term Memory network for capturing bidirectional semantic information,effectively addressing the issue of long-distance dependencies.Furthermore,the Global Pointer model is employed to comprehensively recognize all nested entities in the text.We conduct extensive experiments on the Chinese medical dataset CMeEE and the results demonstrate the superior performance of RoBGP over several baseline models.This research confirms the effectiveness of RoBGP in Chinese biomedical NER,providing reliable technical support for biomedical information extraction and knowledge base construction.

3D Road Network Modeling and Road Structure Recognition in Internet of Vehicles

Internet of Vehicles (IoV) is a new system that enables individual vehicles to connect with nearby vehicles,people, transportation infrastructure, and networks, thereby realizing amore intelligent and efficient transportationsystem. The movement of vehicles and the three-dimensional (3D) nature of the road network cause the topologicalstructure of IoV to have the high space and time complexity.Network modeling and structure recognition for 3Droads can benefit the description of topological changes for IoV. This paper proposes a 3Dgeneral roadmodel basedon discrete points of roads obtained from GIS. First, the constraints imposed by 3D roads on moving vehicles areanalyzed. Then the effects of road curvature radius (Ra), longitudinal slope (Slo), and length (Len) on speed andacceleration are studied. Finally, a general 3D road network model based on road section features is established.This paper also presents intersection and road section recognition methods based on the structural features ofthe 3D road network model and the road features. Real GIS data from a specific region of Beijing is adopted tocreate the simulation scenario, and the simulation results validate the general 3D road network model and therecognitionmethod. Therefore, thiswork makes contributions to the field of intelligent transportation by providinga comprehensive approach tomodeling the 3Droad network and its topological changes in achieving efficient trafficflowand improved road safety.

Promotion of structural plasticity in area V2 of visual cortex prevents against object recognition memory deficits in aging and Alzheimer's disease rodents

Memory deficit,which is often associated with aging and many psychiatric,neurological,and neurodegenerative diseases,has been a challenging issue for treatment.Up till now,all potential drug candidates have failed to produce satisfa ctory effects.Therefore,in the search for a solution,we found that a treatment with the gene corresponding to the RGS14414protein in visual area V2,a brain area connected with brain circuits of the ventral stream and the medial temporal lobe,which is crucial for object recognition memory(ORM),can induce enhancement of ORM.In this study,we demonstrated that the same treatment with RGS14414in visual area V2,which is relatively unaffected in neurodegenerative diseases such as Alzheimer s disease,produced longlasting enhancement of ORM in young animals and prevent ORM deficits in rodent models of aging and Alzheimer’s disease.Furthermore,we found that the prevention of memory deficits was mediated through the upregulation of neuronal arbo rization and spine density,as well as an increase in brain-derived neurotrophic factor(BDNF).A knockdown of BDNF gene in RGS14414-treated aging rats and Alzheimer s disease model mice caused complete loss in the upregulation of neuronal structural plasticity and in the prevention of ORM deficits.These findings suggest that BDNF-mediated neuronal structural plasticity in area V2 is crucial in the prevention of memory deficits in RGS14414-treated rodent models of aging and Alzheimer’s disease.Therefore,our findings of RGS14414gene-mediated activation of neuronal circuits in visual area V2 have therapeutic relevance in the treatment of memory deficits.

KGTLIR:An Air Target Intention Recognition Model Based on Knowledge Graph and Deep Learning

As a core part of battlefield situational awareness,air target intention recognition plays an important role in modern air operations.Aiming at the problems of insufficient feature extraction and misclassification in intention recognition,this paper designs an air target intention recognition method(KGTLIR)based on Knowledge Graph and Deep Learning.Firstly,the intention recognition model based on Deep Learning is constructed to mine the temporal relationship of intention features using dilated causal convolution and the spatial relationship of intention features using a graph attention mechanism.Meanwhile,the accuracy,recall,and F1-score after iteration are introduced to dynamically adjust the sample weights to reduce the probability of misclassification.After that,an intention recognition model based on Knowledge Graph is constructed to predict the probability of the occurrence of different intentions of the target.Finally,the results of the two models are fused by evidence theory to obtain the target’s operational intention.Experiments show that the intention recognition accuracy of the KGTLIRmodel can reach 98.48%,which is not only better than most of the air target intention recognition methods,but also demonstrates better interpretability and trustworthiness.

Audio-Text Multimodal Speech Recognition via Dual-Tower Architecture for Mandarin Air Traffic Control Communications

In air traffic control communications (ATCC), misunderstandings between pilots and controllers could result in fatal aviation accidents. Fortunately, advanced automatic speech recognition technology has emerged as a promising means of preventing miscommunications and enhancing aviation safety. However, most existing speech recognition methods merely incorporate external language models on the decoder side, leading to insufficient semantic alignment between speech and text modalities during the encoding phase. Furthermore, it is challenging to model acoustic context dependencies over long distances due to the longer speech sequences than text, especially for the extended ATCC data. To address these issues, we propose a speech-text multimodal dual-tower architecture for speech recognition. It employs cross-modal interactions to achieve close semantic alignment during the encoding stage and strengthen its capabilities in modeling auditory long-distance context dependencies. In addition, a two-stage training strategy is elaborately devised to derive semantics-aware acoustic representations effectively. The first stage focuses on pre-training the speech-text multimodal encoding module to enhance inter-modal semantic alignment and aural long-distance context dependencies. The second stage fine-tunes the entire network to bridge the input modality variation gap between the training and inference phases and boost generalization performance. Extensive experiments demonstrate the effectiveness of the proposed speech-text multimodal speech recognition method on the ATCC and AISHELL-1 datasets. It reduces the character error rate to 6.54% and 8.73%, respectively, and exhibits substantial performance gains of 28.76% and 23.82% compared with the best baseline model. The case studies indicate that the obtained semantics-aware acoustic representations aid in accurately recognizing terms with similar pronunciations but distinctive semantics. The research provides a novel modeling paradigm for semantics-aware speech recognition in air traffic control communications, which could contribute to the advancement of intelligent and efficient aviation safety management.