Rethinking multi-spatial information for transferable adversarial attacks on speaker recognition systems

Adversarial attacks have been posing significant security concerns to intelligent systems,such as speaker recognition systems(SRSs).Most attacks assume the neural networks in the systems are known beforehand,while black-box attacks are proposed without such information to meet practical situations.Existing black-box attacks improve trans-ferability by integrating multiple models or training on multiple datasets,but these methods are costly.Motivated by the optimisation strategy with spatial information on the perturbed paths and samples,we propose a Dual Spatial Momentum Iterative Fast Gradient Sign Method(DS-MI-FGSM)to improve the transferability of black-box at-tacks against SRSs.Specifically,DS-MI-FGSM only needs a single data and one model as the input;by extending to the data and model neighbouring spaces,it generates adver-sarial examples against the integrating models.To reduce the risk of overfitting,DS-MI-FGSM also introduces gradient masking to improve transferability.The authors conduct extensive experiments regarding the speaker recognition task,and the results demonstrate the effectiveness of their method,which can achieve up to 92%attack success rate on the victim model in black-box scenarios with only one known model.

An Efficient Text-Independent Speaker Identification Using Feature Fusion and Transformer Model

Automatic Speaker Identification(ASI)involves the process of distinguishing an audio stream associated with numerous speakers’utterances.Some common aspects,such as the framework difference,overlapping of different sound events,and the presence of various sound sources during recording,make the ASI task much more complicated and complex.This research proposes a deep learning model to improve the accuracy of the ASI system and reduce the model training time under limited computation resources.In this research,the performance of the transformer model is investigated.Seven audio features,chromagram,Mel-spectrogram,tonnetz,Mel-Frequency Cepstral Coefficients(MFCCs),delta MFCCs,delta-delta MFCCs and spectral contrast,are extracted from the ELSDSR,CSTRVCTK,and Ar-DAD,datasets.The evaluation of various experiments demonstrates that the best performance was achieved by the proposed transformer model using seven audio features on all datasets.For ELSDSR,CSTRVCTK,and Ar-DAD,the highest attained accuracies are 0.99,0.97,and 0.99,respectively.The experimental results reveal that the proposed technique can achieve the best performance for ASI problems.

Optical Ciphering Scheme for Cancellable Speaker Identification System

Most current security and authentication systems are based on personal biometrics.The security problem is a major issue in the field of biometric systems.This is due to the use in databases of the original biometrics.Then biometrics will forever be lost if these databases are attacked.Protecting privacy is the most important goal of cancelable biometrics.In order to protect privacy,therefore,cancelable biometrics should be non-invertible in such a way that no information can be inverted from the cancelable biometric templates stored in personal identification/verification databases.One methodology to achieve non-invertibility is the employment of non-invertible transforms.This work suggests an encryption process for cancellable speaker identification using a hybrid encryption system.This system includes the 3D Jigsaw transforms and Fractional Fourier Transform(FrFT).The proposed scheme is compared with the optical Double Random Phase Encoding(DRPE)encryption process.The evaluation of simulation results of cancellable biometrics shows that the algorithm proposed is secure,authoritative,and feasible.The encryption and cancelability effects are good and reveal good performance.Also,it introduces recommended security and robustness levels for its utilization for achieving efficient cancellable biometrics systems.

Voice Response Questionnaire System for Speaker Recognition Using Biometric Authentication Interface

The use of voice to perform biometric authentication is an importanttechnological development,because it is a non-invasive identification methodand does not require special hardware,so it is less likely to arouse user disgust.This study tries to apply the voice recognition technology to the speech-driveninteractive voice response questionnaire system aiming to upgrade the traditionalspeech system to an intelligent voice response questionnaire network so that thenew device may offer enterprises more precise data for customer relationshipmanagement(CRM).The intelligence-type voice response gadget is becominga new mobile channel at the current time,with functions of the questionnaireto be built in for the convenience of collecting information on local preferencesthat can be used for localized promotion and publicity.Authors of this study propose a framework using voice recognition and intelligent analysis models to identify target customers through voice messages gathered in the voice response questionnaire system;that is,transforming the traditional speech system to anintelligent voice complex.The speaker recognition system discussed hereemploys volume as the acoustic feature in endpoint detection as the computationload is usually low in this method.To correct two types of errors found in the endpoint detection practice because of ambient noise,this study suggests ways toimprove the situation.First,to reach high accuracy,this study follows a dynamictime warping(DTW)based method to gain speaker identification.Second,it isdevoted to avoiding any errors in endpoint detection by filtering noise from voicesignals before getting recognition and deleting any test utterances that might negatively affect the results of recognition.It is hoped that by so doing the recognitionrate is improved.According to the experimental results,the method proposed inthis research has a high recognition rate,whether it is on personal-level or industrial-level computers,and can reach the practical application standard.Therefore,the voice management system in this research can be regarded as Virtual customerservice staff to use.

Efficiency in Teaching Speaker and Listener Repertoires:Comparing Three Instructional Sequences in Autistic Children

Previous studies have investigated the efficiency in teaching listener and speaker repertoires in children diagnosed with autism spectrum disorder(ASD).Some investigations focused on listener responding by function,feature,and class(LRFFC)and intraverbal by function,feature,and class(FFC).For some children,teaching intraverbal FFC was more efficient because it resulted in a better emergence effect of a related untaught repertoire(LRFFC).For other children,teaching LRFFC along with tacting pictures was more efficient,resulting in a better emergence effect of a related untaught repertoire(intraverbal FFC).In these cases,it is not clear whether the tact increased the efficiency of LRFFC training because a comparison with a condition in which tacts were not required was not conducted.This investigation consisted of a replication with two children diagnosed with ASD.Three instructional sequences were compared:teaching LRFFC-probing intraverbal;teaching LRFFC+tacts-probing intraverbal;teaching intraverbal-probing LRFFC.For a child,all sequences were equally efficient because all related untaught repertoires emerged without errors.However,the acquisition of intraverbals during training occurred with variability.In the case of the second child,the most efficient sequence consisted of teaching intraverbals,resulting in the emergence of LRFFC without errors.In both cases of teaching LRFFC,the emergence of related intraverbals was partial and acquisition of the trained repertoires occurred with variability.The case that did not demand tact responses was slightly more efficient.Data were discussed in the sense that the best instructional sequence may vary from learner to learner.

面向域外说话人适应场景的多层级解耦个性化语音合成

个性化语音合成任务旨在合成特定说话人音色的语音。传统方法在合成域外说话人语音时,与真实语音存在明显音色差异,解耦说话人特征仍较为困难。本文提出面向训练时未出现的域外说话人适应场景下的多层级解耦个性化语音合成方法,通过不同粒度特征融合,有效提升零资源条件下域外说话人语音合成性能。本文方法采用快速傅里叶卷积提取说话人全局特征,以提高模型对域外说话人的泛化能力,实现句子粒度的说话人解耦;借助语音识别模型解耦音素粒度说话人特征,并通过注意力机制捕捉音素级音色特征,实现音素粒度的说话人解耦。实验结果表明:在公开数据集AISHELL3上,本文方法对域外说话人在客观评价指标说话人特征向量余弦相似度上达到0.697,相比基线模型提高6.25%,有效提升对域外说话人音色特征建模能力。

多说话人分离与目标说话人提取的研究现状与展望

语音分离作为语音信号处理领域的前沿技术,具有重要的研究价值和广阔的应用前景。通常,麦克风拾取的信号包含有多个说话人的语音、噪声和混响。为了提升用户的听觉体验以及后端设备的处理性能,需要对混合信号进行语音分离。语音分离起源于著名的鸡尾酒会问题,旨在从混合信号中分离出说话人的语音信号。近年来,研究人员提出了大量的语音分离方法,显著提升了分离性能。本文对这些语音分离方法进行了系统的归纳和总结。首先,根据目标说话人的辅助信息利用与否,将语音分离方法分为两大类,即多说话人分离与目标说话人提取;其次,从传统到基于深度学习的角度,分别对多说话人分离和目标说话人提取两类方法进行详细介绍;最后,讨论了当前语音分离领域面临的一些挑战,并对未来的研究方向进行展望。

融合信息扰动与特征解耦的单样本语音转换

单样本语音转换的特性是利用单条目标说话人的语音样本即可实现身份的转换,但由于声学特征呈现复杂的相互作用和动态变化,现有方法难以充分将单样本语音中的说话人音色与其他声学特征解耦,导致转换音频在听觉上仍与源说话人的音色特征相似,存在说话人音色泄露情况。为此提出一种融合信息扰动与特征解耦的单样本语音转换模型,即IPFD-VC模型。首先,引入信息扰动模块对语音信号进行三次扰动操作,去除输入内容和韵律编码器中的冗余信息;其次,将处理后的语音信号送入各编码器,并结合最小化互信息策略进一步解耦声学特征,降低不同特征与说话人音色特征的相关性;最后通过解码器及声码器输出转换音频。实验结果表明:IPFD-VC模型转换音频的语音自然度和说话人相似度分别达到3.72和3.68,与目前先进的UUVC模型相比,梅尔倒谱失真降低0.26 dB。该模型能够有效对声学特征进行解耦,捕获目标说话人音色特征,同时保持源语言内容和韵律变化,降低说话人音色泄露风险。

说话人音频攻击与对抗技术研究综述

文中概括了说话人音频攻击与对抗技术的最新进展。由于说话人音频攻击已经成为语音应用安全的严重威胁,以WaveNet、Transformer和GAN三种模型在音频攻击技术中的应用作为节点,分别介绍以其为基础的音频攻击技术。音频对抗技术则以涵盖的攻击技术分为3类,分别是基础音频攻击、重放攻击和深度伪造攻击。系统地阐述了音频攻击与对抗技术的最新研究成果,并分析比较了各算法在不同条件下的优劣,同时还介绍了音频技术常用的数据集。最后结合该领域目前的研究现状,提出了说话人音频攻防对抗技术研究中亟待关注与研究的问题。

语篇主观性表征研究

基于语言主观性理论,本研究重点考察语篇的主观性表征。研究发现,语篇主观性表征有词汇、句法和语用等方面,它们共同承载着说写者的情感、态度、认识等主观自我成分。本研究与语言主观性现象的微观研究互补,为语篇层面语言主观性现象的整体研究提供研究思路,有利于深入挖掘说写者隐藏在语篇中的主观自我表征方式。

科技领域的中国合规管理推动者研究——基于2018—2023年公开会议报告人分析

设定会议内容包含科技内容的就是科技领域合规管理会议,会议报告人即为合规管理推动者。根据此设定,收集确认2018—2023年55个中国本土举行的属于科技领域合规管理会议,并对确认的55个会议的高频次报告人、高频次报告人所在单位及属性、高频次报告人所在企业与行业、会议报告人报告内容主题等进行计量分析。研究聚焦科技领域,求解合规管理领域的领军人物、主要机构、主要研究成果与工作内容等问题,较为全面地展示目前中国国内聚焦科技领域的合规管理推动者、研究与服务工作现状和重点,为合规管理领域情报研究与服务提供参考。

基于可供性的智能音箱与用户互动的扎根研究

基于ICTs与AI演进和对网络内容生态、生活服务生态和物联网智能家居生态三大生态的链接,智能音箱实现与互联网、物理空间与个人行动的连接,在家庭生活网络中具有中心枢纽功能。本研究以可供性为理论框架,以24名智能音箱活跃用户的半结构化深度访谈为主要原始资料,导入Nvivo12软件,通过扎根理论的程序化法展开质化分析。分析结果表明,智能音箱与用户的互动行为主要表现在三大层面:互动功能层,技术可供性驱动互动行为发生;互动行为层,想象可供性赋能人性化交互;互动情感层,用户触发情感可供性。

基于主动交互的适老家居语音交互设计——以智能音箱为例

从主动交互发展及应用现状出发,对老年用户的认知特征进行分析,探索将主动交互应用在智能家居产品语音交互的设计途径。针对传统语音交互下老年用户的认知负担,明确主动交互对于老年用户的价值。通过对老年用户的实际调研,收窄智能音箱语音功能的范围,基于主动交互进行语音交互的创新设计,提出主动交互下适老化智能音箱的交互策略,解决了老年人在传统被动交互中存在的认知成本高、负荷大的问题,同时为智能家居语音交互提供了设计思路与流程参考。

基于概率球面判别分析的说话人识别信道补偿算法

在说话人识别任务中,概率线性判别分析(PLDA)模型是目前常用的分类后端,但由于高斯PLDA模型分布假设不能准确拟合真实说话人特征分布,导致基于高斯分布假设长度归一化的信道补偿方法会破坏说话人特征类内分布的独立性,使得高斯PLDA不能充分利用上游任务提取特征所包含的说话人信息,从而影响识别结果。针对这一问题,提出基于概率球面判别分析的信道补偿算法(CC-PSDA),通过引入冯·米塞斯-费希尔(VMF)分布假设的概率球面判别分析模型(PSDA)和特征变换方法代替高斯分布假设的概率线性判别分析方法,以避免信道补偿对说话人特征类内分布独立性的影响。首先,为了使说话人特征符合VMF分布先验假设拟合后端分类模型,在特征级利用非线性转换对说话人特征进行分布变换。之后,利用基于VMF分布假设的PLDA模型不会破坏说话人特征的类内分布结构的特点,将变换后的说话人特征定义到特定维度的超球面,最大化特征类间距离。所提算法通过期望最大化(EM)算法进行求解,最终完成分类任务。实验结果表明,改进算法在三个测试集上的识别等错误率相较于对比模型PSDA、高斯PLDA均最低。由此可见,所提模型可以有效区分说话人特征,提高识别性能。

说话人感知的交叉注意力说话人提取网络

目标说话人提取任务的目标是在一段混合音频中提取特定说话人的语音,任务设置上一般会给一段目标说话人注册音频作为辅助信息。现有的研究工作主要有以下不足:(1)说话人识别的辅助网络无法捕获学习注册音频中的关键信息;(2)缺乏混合音频嵌入和注册音频嵌入的交互学习机制。以上不足导致了现有研究工作在注册音频和目标音频之间存在较大差异时有说话人混淆问题。为了解决该问题,提出说话人感知的交叉注意力说话人提取网络(Speaker-aware Cross Attention Speaker Extraction Network,SACAN)。SACAN在说话人识别辅助网络引入基于注意力的说话人聚合模块,有效聚合目标说话人声音特性的关键信息和利用混合音频增强目标说话人嵌入。进一步地,SACAN通过交叉注意力构建交互学习机制促进说话人嵌入与混合音频嵌入融合学习,增强了模型的说话人感知能力。实验结果表明,SACAN相比基准方法在STOI和SI-SDRi分别提高了0.013 3、1.069 5 d B,并在说话人混淆相关评估和消融实验中验证了不同模块的有效性。

面向中文短语音的文本无关说话人确认新框架

相较于文本相关说话人确认,文本无关说话人确认由于验证文本内容不受限制,结合语音识别能够有效避免录音欺诈等常见攻击。然而,文本无关说话人确认系统在短语音验证上会出现严重的性能下降。为此,文章首先提出了一种改进的端到端模型,通过长、短语音说话人分类损失增强网络对不同时长语音段的说话人分类识别能力;同时,在嵌入码空间中增大同一说话人的短语音和长语音之间的相似度,减小不同说话人的短语音之间的相似度,增强网络对短语音的特征提取能力。此外,还提出了一种基于注意力机制的验证词选择方法,选择具有高注意力权重的中文词作为系统验证提示词。实验结果表明,文章提出的改进的端到端模型结合softmax预训练使得模型在短测试语音上的等错误率相对降低29%,基于注意力机制的验证词选择方法也能筛选出具有更好识别结果的验证词,二者结合能够有效提升说话人确认系统对于短中文语音的识别性能。

Emotional speaker recognition based on prosody transformation

A novel emotional speaker recognition system （ESRS） is proposed to compensate for emotion variability. First, the emotion recognition is adopted as a pre-processing part to classify the neutral and emotional speech. Then, the recognized emotion speech is adjusted by prosody modification. Different methods including Gaussian normalization, the Gaussian mixture model （GMM） and support vector regression （SVR） are adopted to define the mapping rules of F0s between emotional and neutral speech, and the average linear ratio is used for the duration modification. Finally, the modified emotional speech is employed for the speaker recognition. The experimental results show that the proposed ESRS can significantly improve the performance of emotional speaker recognition, and the identification rate （IR） is higher than that of the traditional recognition system. The emotional speech with F0 and duration modifications is closer to the neutral one.

基于深度学习的说话人确认方法研究现状及展望

随着深度学习的不断发展,说话人确认(Speaker verification)技术已经取得了长足的进步。该技术相较于其他生物特征识别技术,具有可远程操作、成本低和易于人机交互等优势,在公安刑侦、金融服务等领域展现出广泛的应用前景。本文系统综述了基于深度学习的说话人确认技术的发展脉络。首先,介绍了基于深度学习的说话人特征表示模型在模型输入与结构、池化层、有监督损失函数和自监督学习与预训练模型4个方面的发展历程和研究现状;其次,探讨了说话人确认技术在实际应用中面临的跨域不匹配问题,如噪声干扰、信道不匹配和远场语音等,并概述了相应的领域自适应和领域泛化方法;最后,指出了进一步的研究方向。

基于多尺度卷积编码器的说话人验证网络

说话人验证是一种有效的生物身份验证方法,说话人嵌入特征的质量在很大程度上影响着说话人验证系统的性能。最近,Transformer模型在自动语音识别领域展现出了巨大的潜力,但由于Transformer中传统的自注意力机制对局部特征的提取能力较弱,难以提取有效的说话人嵌入特征,因此Transformer模型在说话人验证领域的性能难以超越以往的基于卷积网络的模型。为了提高Transformer对局部特征的提取能力,文中提出了一种新的自注意力机制用于Transformer编码器,称为多尺度卷积自注意力编码器(Multi-scale Convolutional Self-Attention Encoder,MCAE)。利用不同尺度的卷积操作来提取多时间尺度信息,并通过融合时域和频域的特征,使模型获得更丰富的局部特征表示,这样的编码器设计对于说话人验证是更有效的。通过实验表明,在3个公开的测试集上,所提方法的综合性能表现更佳。与传统的Transformer编码器相比,MCAE也是更轻量级的,这更有利于模型的应用部署。

基于Conformer的实时多场景说话人识别模型

为解决在多场景(跨域、长时以及噪声干扰语音场景)下说话人确认系统性能较差的问题,提出了一种基于Conformer构建的、实时多场景鲁棒的说话人识别模型——PMS-Conformer。PMS-Conformer的设计灵感来自于先进的模型MFA-Conformer。PMS-Conformer对MFA-Conformer的声学特征提取器、网络组件和损失函数计算模块进行了改进,其具有新颖有效的声学特征提取器,以及鲁棒的、具有较强泛化能力的声纹嵌入码提取器。基于VoxCeleb1&2数据集实现了PMS-Conformer的训练;开展了PMS-Conformer与基线MFA-Conformer以及ECAPA-TDNN在说话人确认任务上的性能对比评估实验。实验结果表明在长语音SITW、跨域VoxMovies以及加噪处理的VoxCeleb-O测试集上,以PMS-Conformer构建的说话人确认系统的性能比用这两个基线构建的说话人确认系统更有竞争力;并且在声纹嵌入码提取器的可训练参数(Params)和推理速度(RTF)方面,PMS-Conformer明显优于ECAPA-TDNN。实验结果说明了PMS-Conformer在实时多场景下具有良好的性能。