In air traffic control communications (ATCC), misunderstandings between pilots and controllers could result in fatal aviation accidents. Fortunately, advanced automatic speech recognition technology has emerged as a p...In air traffic control communications (ATCC), misunderstandings between pilots and controllers could result in fatal aviation accidents. Fortunately, advanced automatic speech recognition technology has emerged as a promising means of preventing miscommunications and enhancing aviation safety. However, most existing speech recognition methods merely incorporate external language models on the decoder side, leading to insufficient semantic alignment between speech and text modalities during the encoding phase. Furthermore, it is challenging to model acoustic context dependencies over long distances due to the longer speech sequences than text, especially for the extended ATCC data. To address these issues, we propose a speech-text multimodal dual-tower architecture for speech recognition. It employs cross-modal interactions to achieve close semantic alignment during the encoding stage and strengthen its capabilities in modeling auditory long-distance context dependencies. In addition, a two-stage training strategy is elaborately devised to derive semantics-aware acoustic representations effectively. The first stage focuses on pre-training the speech-text multimodal encoding module to enhance inter-modal semantic alignment and aural long-distance context dependencies. The second stage fine-tunes the entire network to bridge the input modality variation gap between the training and inference phases and boost generalization performance. Extensive experiments demonstrate the effectiveness of the proposed speech-text multimodal speech recognition method on the ATCC and AISHELL-1 datasets. It reduces the character error rate to 6.54% and 8.73%, respectively, and exhibits substantial performance gains of 28.76% and 23.82% compared with the best baseline model. The case studies indicate that the obtained semantics-aware acoustic representations aid in accurately recognizing terms with similar pronunciations but distinctive semantics. The research provides a novel modeling paradigm for semantics-aware speech recognition in air traffic control communications, which could contribute to the advancement of intelligent and efficient aviation safety management.展开更多
This study aims to address the deviation in downstream tasks caused by inaccurate recognition results when applying Automatic Speech Recognition(ASR)technology in the Air Traffic Control(ATC)field.This paper presents ...This study aims to address the deviation in downstream tasks caused by inaccurate recognition results when applying Automatic Speech Recognition(ASR)technology in the Air Traffic Control(ATC)field.This paper presents a novel cascaded model architecture,namely Conformer-CTC/Attention-T5(CCAT),to build a highly accurate and robust ATC speech recognition model.To tackle the challenges posed by noise and fast speech rate in ATC,the Conformer model is employed to extract robust and discriminative speech representations from raw waveforms.On the decoding side,the Attention mechanism is integrated to facilitate precise alignment between input features and output characters.The Text-To-Text Transfer Transformer(T5)language model is also introduced to handle particular pronunciations and code-mixing issues,providing more accurate and concise textual output for downstream tasks.To enhance the model’s robustness,transfer learning and data augmentation techniques are utilized in the training strategy.The model’s performance is optimized by performing hyperparameter tunings,such as adjusting the number of attention heads,encoder layers,and the weights of the loss function.The experimental results demonstrate the significant contributions of data augmentation,hyperparameter tuning,and error correction models to the overall model performance.On the Our ATC Corpus dataset,the proposed model achieves a Character Error Rate(CER)of 3.44%,representing a 3.64%improvement compared to the baseline model.Moreover,the effectiveness of the proposed model is validated on two publicly available datasets.On the AISHELL-1 dataset,the CCAT model achieves a CER of 3.42%,showcasing a 1.23%improvement over the baseline model.Similarly,on the LibriSpeech dataset,the CCAT model achieves a Word Error Rate(WER)of 5.27%,demonstrating a performance improvement of 7.67%compared to the baseline model.Additionally,this paper proposes an evaluation criterion for assessing the robustness of ATC speech recognition systems.In robustness evaluation experiments based on this criterion,the proposed model demonstrates a performance improvement of 22%compared to the baseline model.展开更多
Objective speech quality is difficult to be measured without the input reference speech.Mapping methods using data mining are investigated and designed to improve the output-based speech quality assessment algorithm.T...Objective speech quality is difficult to be measured without the input reference speech.Mapping methods using data mining are investigated and designed to improve the output-based speech quality assessment algorithm.The degraded speech is firstly separated into three classes(unvoiced,voiced and silence),and then the consistency measurement between the degraded speech signal and the pre-trained reference model for each class is calculated and mapped to an objective speech quality score using data mining.Fuzzy Gaussian mixture model(GMM)is used to generate the artificial reference model trained on perceptual linear predictive(PLP)features.The mean opinion score(MOS)mapping methods including multivariate non-linear regression(MNLR),fuzzy neural network(FNN)and support vector regression(SVR)are designed and compared with the standard ITU-T P.563 method.Experimental results show that the assessment methods with data mining perform better than ITU-T P.563.Moreover,FNN and SVR are more efficient than MNLR,and FNN performs best with 14.50% increase in the correlation coefficient and 32.76% decrease in the root-mean-square MOS error.展开更多
Based on fuzzy Gaussian mixture model （FGMM） and support vector regression （SVR）,an improved version of non-intrusive objective measurement for assessing quality of output speech without inputting clean speech is ...Based on fuzzy Gaussian mixture model （FGMM） and support vector regression （SVR）,an improved version of non-intrusive objective measurement for assessing quality of output speech without inputting clean speech is proposed for narrowband speech.Its perceptual linear predictive （PLP） features extracted from clean speech and clustered by FGMM are used as an artificial reference model.Input speech is separated into three classes,for each a consistency parameter between each feature pair from test speech signals and its counterpart in the pre-trained FGMM reference model is calculated and mapped to an objective speech quality score using SVR method.The correlation degree between subjective mean opinion score （MOS） and objective MOS is analyzed.Experimental results show that the proposed method offers an effective technique and can give better performances than the ITU-T P.563 method under most of the test conditions for narrowband speech.展开更多
Perceptual Objective Listening Quality Assessment (POLQA) and Perceptual <span>Evaluation of Speech Quality (PESQ) are commonly used objective standards for evaluating speech quality. These methods were develope...Perceptual Objective Listening Quality Assessment (POLQA) and Perceptual <span>Evaluation of Speech Quality (PESQ) are commonly used objective standards for evaluating speech quality. These methods were developed and trained on native </span>speakers’ speech sequences of some western languages. One can then wonder how these methods perform if they are applied to other languages or if the<span> speaker is non-native. This paper deals with the evaluation of PESQ and POLQA </span>on languages that were not been considered when setting up these methods, with emphasis on Moore and Dioula, two local languages of Burkina Faso. <span>Another aspect is the evaluation of these two methods in the case of non-native speakers. For this purpose, in the one hand, the Mean Opinion Score-Listening Quality Objective (MOS-LQO) of PESQ and POLQA, computed for Moore and Dioula, are compared to those of French and English. On the second hand, the </span><span>MOS-LQO scores of French and English are compared for native and</span><span> non-native speakers, to evaluate the effect of the accent of speakers.</span>展开更多
To improve the performance of human-computer interaction interfaces, emotion is considered to be one of the most important factors. The major objective of expressive speech synthesis is to inject various expressions r...To improve the performance of human-computer interaction interfaces, emotion is considered to be one of the most important factors. The major objective of expressive speech synthesis is to inject various expressions reflecting different emotions to the synthesized speech. To effectively model and control the emotion, emotion intensity is introduced for expressive speech synthesis model to generate speech conveyed the delicate and complicate emotional states. The system was composed of an emotion analysis module with the goal of extracting control emotion intensity vector and a speech synthesis module responsible for mapping text characters to speech waveform. The proposed continuous variable “perception vector” is a data-driven approach of controlling the model to synthesize speech with different emotion intensities. Compared with the system using a one-hot vector to control emotion intensity, this model using perception vector is able to learn the high-level emotion information from low-level acoustic features. In terms of the model controllability and flexibility, both the objective and subjective evaluations demonstrate perception vector outperforms one-hot vector.展开更多
Total laryngectomy affects the speaking functions of many patients.Speech deprivation has great impacts on the quality of life of patients,especially on self-efficacy.Learning esophageal speech represents a way to hel...Total laryngectomy affects the speaking functions of many patients.Speech deprivation has great impacts on the quality of life of patients,especially on self-efficacy.Learning esophageal speech represents a way to help laryngectomees speak again.The purpose of this study was to determine the influence of collective esophageal speech training on self-efficacy of laryngectomees.In this study,28 patients and 30 family members were included.The participants received information about training via telephone or a WeChat group.Collective esophageal speech training was used to educate laryngectomees on esophageal speech.Before and after collective esophageal speech training,all participants completed the General Self-Efficacy Scale(GSES)to assess their perceptions on self-efficacy.Through the training,laryngectomees recovered their speech.After the training,the self-efficacy scores of laryngectomees were higher than those before the training,with significant differences noted(P<0.05).However,family members'scores did not change significantly.In conclusion,collective esophageal speech training is not only convenient and economical,but also improves self-efficacy and confidence of laryngectomees.Greater self-efficacy is helpful for laryngectomees to master esophageal speech and improve their quality of life.In addition,more attention should be focused on improving the self-efficacy of family members and making them give full play to their talent and potential on laryngectomees'voice rehabilitation.展开更多
Distributed speech recognition (DSR) applications have certain QoS (Quality of service) requirements in terms of latency, packet loss rate, etc. To deliver quality guaranteed DSR application over wirelined or wireless...Distributed speech recognition (DSR) applications have certain QoS (Quality of service) requirements in terms of latency, packet loss rate, etc. To deliver quality guaranteed DSR application over wirelined or wireless links, some QoS mechanisms should be provided. We put forward a RTP/RSVP transmission scheme with DSR-specific payload and QoS parameters by modifying the present WAP protocol stack. The simulation result shows that this scheme will provide adequate network bandwidth to keep the real-time transport of DSR data over either wirelined or wireless channels.展开更多
In phase space reconstruction of time series, the selection of embedding dimension is important. Based on the idea of checking the behavior of near neighbors in the reconstruction dimension, a new method to determine ...In phase space reconstruction of time series, the selection of embedding dimension is important. Based on the idea of checking the behavior of near neighbors in the reconstruction dimension, a new method to determine proper minimum embedding dimension is constructed. This method has a sound theoretical basis and can lead to good result. It can indicate the noise level in the data to be reconstructed, and estimate the reconstruction quality. It is applied to speech signal reconstruction and the generic embedding dimension of speech signals is deduced.展开更多
A kind of Web voice browser based on improved synchronous linear predictive coding （ISLPC） and Text-toSpeech （TTS） algorithm and Internet application was proposed. The paper analyzes the features of TTS system wit...A kind of Web voice browser based on improved synchronous linear predictive coding （ISLPC） and Text-toSpeech （TTS） algorithm and Internet application was proposed. The paper analyzes the features of TTS system with ISLPC speech synthesis and discusses the design and implementation of ISLPC TTS-based Web voice browser. The browser integrates Web technology, Chinese information processing, artificial intelligence and the key technology of Chinese ISLPC speech synthesis. It＇s a visual and audible web browser that can improve information precision for network users. The evaluation results show that ISLPC-based TTS model has a better performance than other browsers in voice quality and capability of identifying Chinese characters.展开更多
基金This research was funded by Shenzhen Science and Technology Program(Grant No.RCBS20221008093121051)the General Higher Education Project of Guangdong Provincial Education Department(Grant No.2020ZDZX3085)+1 种基金China Postdoctoral Science Foundation(Grant No.2021M703371)the Post-Doctoral Foundation Project of Shenzhen Polytechnic(Grant No.6021330002K).
文摘In air traffic control communications (ATCC), misunderstandings between pilots and controllers could result in fatal aviation accidents. Fortunately, advanced automatic speech recognition technology has emerged as a promising means of preventing miscommunications and enhancing aviation safety. However, most existing speech recognition methods merely incorporate external language models on the decoder side, leading to insufficient semantic alignment between speech and text modalities during the encoding phase. Furthermore, it is challenging to model acoustic context dependencies over long distances due to the longer speech sequences than text, especially for the extended ATCC data. To address these issues, we propose a speech-text multimodal dual-tower architecture for speech recognition. It employs cross-modal interactions to achieve close semantic alignment during the encoding stage and strengthen its capabilities in modeling auditory long-distance context dependencies. In addition, a two-stage training strategy is elaborately devised to derive semantics-aware acoustic representations effectively. The first stage focuses on pre-training the speech-text multimodal encoding module to enhance inter-modal semantic alignment and aural long-distance context dependencies. The second stage fine-tunes the entire network to bridge the input modality variation gap between the training and inference phases and boost generalization performance. Extensive experiments demonstrate the effectiveness of the proposed speech-text multimodal speech recognition method on the ATCC and AISHELL-1 datasets. It reduces the character error rate to 6.54% and 8.73%, respectively, and exhibits substantial performance gains of 28.76% and 23.82% compared with the best baseline model. The case studies indicate that the obtained semantics-aware acoustic representations aid in accurately recognizing terms with similar pronunciations but distinctive semantics. The research provides a novel modeling paradigm for semantics-aware speech recognition in air traffic control communications, which could contribute to the advancement of intelligent and efficient aviation safety management.
基金This study was co-supported by the National Key R&D Program of China(No.2021YFF0603904)National Natural Science Foundation of China(U1733203)Safety Capacity Building Project of Civil Aviation Administration of China(TM2019-16-1/3).
文摘This study aims to address the deviation in downstream tasks caused by inaccurate recognition results when applying Automatic Speech Recognition(ASR)technology in the Air Traffic Control(ATC)field.This paper presents a novel cascaded model architecture,namely Conformer-CTC/Attention-T5(CCAT),to build a highly accurate and robust ATC speech recognition model.To tackle the challenges posed by noise and fast speech rate in ATC,the Conformer model is employed to extract robust and discriminative speech representations from raw waveforms.On the decoding side,the Attention mechanism is integrated to facilitate precise alignment between input features and output characters.The Text-To-Text Transfer Transformer(T5)language model is also introduced to handle particular pronunciations and code-mixing issues,providing more accurate and concise textual output for downstream tasks.To enhance the model’s robustness,transfer learning and data augmentation techniques are utilized in the training strategy.The model’s performance is optimized by performing hyperparameter tunings,such as adjusting the number of attention heads,encoder layers,and the weights of the loss function.The experimental results demonstrate the significant contributions of data augmentation,hyperparameter tuning,and error correction models to the overall model performance.On the Our ATC Corpus dataset,the proposed model achieves a Character Error Rate(CER)of 3.44%,representing a 3.64%improvement compared to the baseline model.Moreover,the effectiveness of the proposed model is validated on two publicly available datasets.On the AISHELL-1 dataset,the CCAT model achieves a CER of 3.42%,showcasing a 1.23%improvement over the baseline model.Similarly,on the LibriSpeech dataset,the CCAT model achieves a Word Error Rate(WER)of 5.27%,demonstrating a performance improvement of 7.67%compared to the baseline model.Additionally,this paper proposes an evaluation criterion for assessing the robustness of ATC speech recognition systems.In robustness evaluation experiments based on this criterion,the proposed model demonstrates a performance improvement of 22%compared to the baseline model.
基金Projects(61001188,1161140319)supported by the National Natural Science Foundation of ChinaProject(2012ZX03001034)supported by the National Science and Technology Major ProjectProject(YETP1202)supported by Beijing Higher Education Young Elite Teacher Project,China
文摘Objective speech quality is difficult to be measured without the input reference speech.Mapping methods using data mining are investigated and designed to improve the output-based speech quality assessment algorithm.The degraded speech is firstly separated into three classes(unvoiced,voiced and silence),and then the consistency measurement between the degraded speech signal and the pre-trained reference model for each class is calculated and mapped to an objective speech quality score using data mining.Fuzzy Gaussian mixture model(GMM)is used to generate the artificial reference model trained on perceptual linear predictive(PLP)features.The mean opinion score(MOS)mapping methods including multivariate non-linear regression(MNLR),fuzzy neural network(FNN)and support vector regression(SVR)are designed and compared with the standard ITU-T P.563 method.Experimental results show that the assessment methods with data mining perform better than ITU-T P.563.Moreover,FNN and SVR are more efficient than MNLR,and FNN performs best with 14.50% increase in the correlation coefficient and 32.76% decrease in the root-mean-square MOS error.
文摘Based on fuzzy Gaussian mixture model （FGMM） and support vector regression （SVR）,an improved version of non-intrusive objective measurement for assessing quality of output speech without inputting clean speech is proposed for narrowband speech.Its perceptual linear predictive （PLP） features extracted from clean speech and clustered by FGMM are used as an artificial reference model.Input speech is separated into three classes,for each a consistency parameter between each feature pair from test speech signals and its counterpart in the pre-trained FGMM reference model is calculated and mapped to an objective speech quality score using SVR method.The correlation degree between subjective mean opinion score （MOS） and objective MOS is analyzed.Experimental results show that the proposed method offers an effective technique and can give better performances than the ITU-T P.563 method under most of the test conditions for narrowband speech.
文摘Perceptual Objective Listening Quality Assessment (POLQA) and Perceptual <span>Evaluation of Speech Quality (PESQ) are commonly used objective standards for evaluating speech quality. These methods were developed and trained on native </span>speakers’ speech sequences of some western languages. One can then wonder how these methods perform if they are applied to other languages or if the<span> speaker is non-native. This paper deals with the evaluation of PESQ and POLQA </span>on languages that were not been considered when setting up these methods, with emphasis on Moore and Dioula, two local languages of Burkina Faso. <span>Another aspect is the evaluation of these two methods in the case of non-native speakers. For this purpose, in the one hand, the Mean Opinion Score-Listening Quality Objective (MOS-LQO) of PESQ and POLQA, computed for Moore and Dioula, are compared to those of French and English. On the second hand, the </span><span>MOS-LQO scores of French and English are compared for native and</span><span> non-native speakers, to evaluate the effect of the accent of speakers.</span>
基金the results of the research project funded by Natural Science Foundation of Hebei University of Economics and Business (No. 2016KYQ05).
文摘To improve the performance of human-computer interaction interfaces, emotion is considered to be one of the most important factors. The major objective of expressive speech synthesis is to inject various expressions reflecting different emotions to the synthesized speech. To effectively model and control the emotion, emotion intensity is introduced for expressive speech synthesis model to generate speech conveyed the delicate and complicate emotional states. The system was composed of an emotion analysis module with the goal of extracting control emotion intensity vector and a speech synthesis module responsible for mapping text characters to speech waveform. The proposed continuous variable “perception vector” is a data-driven approach of controlling the model to synthesize speech with different emotion intensities. Compared with the system using a one-hot vector to control emotion intensity, this model using perception vector is able to learn the high-level emotion information from low-level acoustic features. In terms of the model controllability and flexibility, both the objective and subjective evaluations demonstrate perception vector outperforms one-hot vector.
文摘Total laryngectomy affects the speaking functions of many patients.Speech deprivation has great impacts on the quality of life of patients,especially on self-efficacy.Learning esophageal speech represents a way to help laryngectomees speak again.The purpose of this study was to determine the influence of collective esophageal speech training on self-efficacy of laryngectomees.In this study,28 patients and 30 family members were included.The participants received information about training via telephone or a WeChat group.Collective esophageal speech training was used to educate laryngectomees on esophageal speech.Before and after collective esophageal speech training,all participants completed the General Self-Efficacy Scale(GSES)to assess their perceptions on self-efficacy.Through the training,laryngectomees recovered their speech.After the training,the self-efficacy scores of laryngectomees were higher than those before the training,with significant differences noted(P<0.05).However,family members'scores did not change significantly.In conclusion,collective esophageal speech training is not only convenient and economical,but also improves self-efficacy and confidence of laryngectomees.Greater self-efficacy is helpful for laryngectomees to master esophageal speech and improve their quality of life.In addition,more attention should be focused on improving the self-efficacy of family members and making them give full play to their talent and potential on laryngectomees'voice rehabilitation.
文摘Distributed speech recognition (DSR) applications have certain QoS (Quality of service) requirements in terms of latency, packet loss rate, etc. To deliver quality guaranteed DSR application over wirelined or wireless links, some QoS mechanisms should be provided. We put forward a RTP/RSVP transmission scheme with DSR-specific payload and QoS parameters by modifying the present WAP protocol stack. The simulation result shows that this scheme will provide adequate network bandwidth to keep the real-time transport of DSR data over either wirelined or wireless channels.
基金Supported by the Naltural Science Foundation of Hunan Province(97JJY1006)Open Foundation of Stalte Key Lab. of Theory and Chief Technology on ISN of Xidian University(991894102)
文摘In phase space reconstruction of time series, the selection of embedding dimension is important. Based on the idea of checking the behavior of near neighbors in the reconstruction dimension, a new method to determine proper minimum embedding dimension is constructed. This method has a sound theoretical basis and can lead to good result. It can indicate the noise level in the data to be reconstructed, and estimate the reconstruction quality. It is applied to speech signal reconstruction and the generic embedding dimension of speech signals is deduced.
基金Supported by the National High-Technology Re-search and Development Program(2005AA122210) the National Out-standing Youth Foundation (60325104)
文摘A kind of Web voice browser based on improved synchronous linear predictive coding （ISLPC） and Text-toSpeech （TTS） algorithm and Internet application was proposed. The paper analyzes the features of TTS system with ISLPC speech synthesis and discusses the design and implementation of ISLPC TTS-based Web voice browser. The browser integrates Web technology, Chinese information processing, artificial intelligence and the key technology of Chinese ISLPC speech synthesis. It＇s a visual and audible web browser that can improve information precision for network users. The evaluation results show that ISLPC-based TTS model has a better performance than other browsers in voice quality and capability of identifying Chinese characters.
文摘目的 分析音乐疗法对老年性聋患者负面情绪、言语识别能力的影响。方法 纳入2021年1月~2023年3月我院接纳的老年性聋患者80例,随机分为对照组(40例,言语识别能力训练)与音乐组(40例,言语识别能力训练+音乐疗法),训练前后比较两组患者听觉功能、负面情绪、认知功能及生活质量。结果 训练后音乐组平均听阈、老年听力障碍筛查量表(hearing handicap inventory for the elderly-screening,HHIE-S)评分显著低于对照组(P<0.05);音乐组焦虑自评量表(self-rating anxiety scale,SAS)、抑郁自评量表(self-rating depression scale,SDS)评分显著低于对照组(P<0.05);音乐组简易精神状态评价量表(mini-mentalstateexamination,MMSE)、蒙特利尔认知评估量表(montrealcognitive assessment,MoCA)评分显著高于对照组(P<0.05);音乐组生活质量综合评定问卷-74(generic quality of life inventory-74,GQOLI-74)各维度评分(物质、社会、躯体、心理)显著高于对照组(P<0.05)。结论 对老年性聋患者采用音乐疗法可显著改善言语识别能力,减轻患者负面情绪,提高其认知功能及生活质量。
