BLIND SPEECH SEPARATION FOR ROBOTS WITH INTELLIGENT HUMAN-MACHINE INTERACTION

Speech recognition rate will deteriorate greatly in human-machine interaction when the speaker's speech mixes with a bystander's voice. This paper proposes a time-frequency approach for Blind Source Seperation (BSS) for intelligent Human-Machine Interaction(HMI). Main idea of the algorithm is to simultaneously diagonalize the correlation matrix of the pre-whitened signals at different time delays for every frequency bins in time-frequency domain. The prososed method has two merits: (1) fast convergence speed; (2) high signal to interference ratio of the separated signals. Numerical evaluations are used to compare the performance of the proposed algorithm with two other deconvolution algorithms. An efficient algorithm to resolve permutation ambiguity is also proposed in this paper. The algorithm proposed saves more than 10% of computational time with properly selected parameters and achieves good performances for both simulated convolutive mixtures and real room recorded speeches.

A Study on Teacher-Child Interaction from the Perspective of Children

Children’s perspective is based on their own cognitive level in understanding objective things.The study of children’s perspective is a bottom-up research process under the premise of having a full respect for a child’s view.With the change of views about children in recent years,“children’s perspective”has become a new research direction.At the same time,teacher-child interaction,as an important means of evaluating the quality of kindergarten education,requires a bottom-up perspective from children.This study hopes to understand children’s emotional experience in the process of teacher-child interaction as well as their understanding and evaluation of their own experience by exploring their perspectives on the interaction,so as to better improve the quality of teacher-child interaction in kindergarten.

A hybrid method for extraction of protein-protein interactions from literature

In this work, a hybrid method is proposed to eliminate the limitations of traditional protein-protein interactions （PPIs） extraction methods, such as pattern learning and machine learning. Each sentence from the biomedical literature containing a protein pair describes a PPI which is predicted by first learning syntax patterns typical of PPIs from training corpus and then using their presence as features, along with bag-of-word features in a maximum entropy model. Tested on the BioCreAtIve corpus, the PPIs extraction method, which achieved a precision rate of 64%, recall rate of 60%, improved the performance in terms of F1 value by 11% compared with the component pure pattern- based and bag-of-word methods. The results on this test set were also compared with other three extraction methods and found to improve the performance remarkably.

A novel speech emotion recognition algorithm based on combination of emotion data field and ant colony search strategy

In order to effectively conduct emotion recognition from spontaneous, non-prototypical and unsegmented speech so as to create a more natural human-machine interaction; a novel speech emotion recognition algorithm based on the combination of the emotional data field （EDF） and the ant colony search （ACS） strategy, called the EDF-ACS algorithm, is proposed. More specifically, the inter- relationship among the turn-based acoustic feature vectors of different labels are established by using the potential function in the EDF. To perform the spontaneous speech emotion recognition, the artificial colony is used to mimic the turn- based acoustic feature vectors. Then, the canonical ACS strategy is used to investigate the movement direction of each artificial ant in the EDF, which is regarded as the emotional label of the corresponding turn-based acoustic feature vector. The proposed EDF-ACS algorithm is evaluated on the continueous audio）＇visual emotion challenge （AVEC） 2012 dataset, which contains the spontaneous, non-prototypical and unsegmented speech emotion data. The experimental results show that the proposed EDF-ACS algorithm outperforms the existing state-of-the-art algorithm in turn-based speech emotion recognition.

Multi-scale discrepancy adversarial network for cross-corpus speech emotion recognition

Background One of the most critical issues in human-computer interaction applications is recognizing human emotions based on speech.In recent years,the challenging problem of cross-corpus speech emotion recognition(SER)has generated extensive research.Nevertheless,the domain discrepancy between training data and testing data remains a major challenge to achieving improved system performance.Methods This paper introduces a novel multi-scale discrepancy adversarial(MSDA)network for conducting multiple timescales domain adaptation for cross-corpus SER,i.e.,integrating domain discriminators of hierarchical levels into the emotion recognition framework to mitigate the gap between the source and target domains.Specifically,we extract two kinds of speech features,i.e.,handcraft features and deep features,from three timescales of global,local,and hybrid levels.In each timescale,the domain discriminator and the feature extrator compete against each other to learn features that minimize the discrepancy between the two domains by fooling the discriminator.Results Extensive experiments on cross-corpus and cross-language SER were conducted on a combination dataset that combines one Chinese dataset and two English datasets commonly used in SER.The MSDA is affected by the strong discriminate power provided by the adversarial process,where three discriminators are working in tandem with an emotion classifier.Accordingly,the MSDA achieves the best performance over all other baseline methods.Conclusions The proposed architecture was tested on a combination of one Chinese and two English datasets.The experimental results demonstrate the superiority of our powerful discriminative model for solving cross-corpus SER.

“谢谢”的行为反语功能

本文讨论“谢谢”类施为短句回应抵触性引发行为、实施言语行为对抗的隐性否定新用法,即行为反语。“谢谢”的行为对抗功能,朝着交际非当面、回应对象非可控角度发展。从日常对话向网络虚拟对话演进,其行为对抗强度、负面情绪态度发生强化,形成“我谢谢你”等负面对抗性固化表达。“谢谢”的行为对抗功能填补了不礼貌抵触类行为配对的交际空位,满足了互动平衡中“文明对抗”的交际需要。

面向便携式野战医疗装备语音交互的小尺寸双麦前端系统设计

目的:针对野战噪声条件下便携式野战医疗装备的语音交互性能受到影响的问题,设计一种小尺寸双麦前端系统。方法:该系统基于最小二乘准则实现小尺寸双麦波速形成,进而实现前端语音增强。系统硬件主要由双麦、信号预处理模块、嵌入式处理器、模拟数字转换器(analog to digital converter,ADC)、数字模拟转换器(digital to analog converter,DAC)、供电模块等组成。其中,双麦采用2个贴片式微机电系统(micro electro mechanical system,MEMS)麦克风,信号预处理模块、ADC、DAC内置在通用音频编码器WM8978中,嵌入式处理器采用STM32F405系列处理器,供电模块采用LM1117电压调节器芯片。系统软件采用KeilμVision4开发软件编译和测试。为验证该系统的性能,进行指向性实验和语音增强实验。结果:指向性实验结果表明,在0.5~2.0 kHz频率范围内,该系统在各频点的指向性一致性较好;语音增强实验结果表明,在枪声、监护仪报警、医疗器皿碰撞3类非平稳噪声条件下,该系统可有效提升语音的音质及识别率。结论:该系统能实现语音增强,可为便携式野战医疗装备的语音交互提供有效的支持。

中文学习者多模态互动能力及教学策略研究

以往中文学习者语言交际能力培养中,重点是语言交际能力的培养,其他模态交际能力的培养力度较弱。结合多模态互动分析法,研究以泰国留学生致歉言语行为作为个案,突破以往基于单一语言模态语料的局限,描述了不同水平泰国留学生致歉中多模态表达手段使用上的异同点及其多模态互动能力的差异。结果显示,中文学习者多模态互动能力有助于完成交际意图,提升互动效果;引入“互动变异观”来指导课堂教学有助于提升中文学习者多模态语言交际能力。

言语策略、隐含话意、人际礼貌的互动研究——以赞扬和批评为例

赞扬/批评和人际礼貌可以从强度上划分为过度、适度、弱度三个级阶,过度或者弱度取代适度是赞扬/批评和人际礼貌策略的核心路径。人际礼貌策略是言语策略调整的重要推手。言语策略致使的隐含话意是策略社会交往的应景产品。隐含话意、言语策略和人际礼貌策略立体互动,共同成就了策略社会交往。

基于句法依赖增强图的方面级情感分析

方面级情感分析旨在分析句子中特定方面的情感极性,现有研究侧重于利用图神经网络建模上下文与方面的依赖信息,忽略了对上下文中情感词及其词性的挖掘和利用。为此,提出一种基于句法依赖的增强图(syntactic dependency enhancement graph, SDEG)模型,在原始句法依赖图上引入情感知识和词性信息,增强情感词权重和相关词性单词在上下文中的作用。使用双向长短期记忆网络和卷积神经网络捕捉句子的重点语义信息,通过图卷积神经网络建模句法依赖增强图,通过交互注意力机制生成特定方面的上下文语义和语法表示以进行情感极性分类。在多个公共基准数据集上的实验结果表明,所提模型在性能上有明显提升。

面向YSU-Ⅱ下肢康复机器人语音交互系统的指令文本校对模型

针对YSU-Ⅱ下肢康复机器人语音交互系统存在指令误识的问题,构建了基于双向门控循环单元的Seq2Seq模型来检测并纠正指令文本中的错误字符,提出一种基于指令上下文和关键字的注意力机制(CK Attention),用于捕获指令文本的上下文语义和关键字信息,以提升模型的文本校对能力。面向康复机器人的训练任务自行建立了语料库,并采用5次5折交叉验证法在该语料库上开展了相关实验,以客观评估模型性能。实验结果表明,所建模型适用于指令文本校对任务,CK Attention机制能够有效提升模型的文本校对性能,其检错F_(1)值和纠错F_(1)值分别达到97.72%和93.89%,对常用指令文本的校对时长在0.156 s~0.391 s之间。

基于语音交互的四足机器人多模式控制实验设计

针对四足机器人全向运动控制、步态与行为切换、人员跟随、自主导航等多模式运动中的人机交互难题,设计了1套基于语音识别与指令解析的四足机器人交互控制系统。明确了四足机器人语音交互系统的硬件组成,并搭建了包含语音采集、识别、解析、播报一体的交互模块;基于SOFM网络设计了语音识别算法;基于ROS建立了语音模块与机器人运动控制模块的通信交互机制,实现了语音指令下达与机器人状态信息播报反馈。通过一系列四足机器人语音交互控制实验,验证了在动态场景下机器人的语音识别准确率高于85%,而在静态场景下则超过95%,保障了四足机器人不同运动模式的准确运动控制。

基于Unity3D的数字虚拟人交互技术研究与应用

目前,数字虚拟人交互技术虽然能够实现与用户的基本交互,但仍然存在着语言理解偏误、缺乏情感表达能力等一系列问题,导致用户的交互体验感不足。在此背景下,本研究首先分析了数字虚拟人技术的发展现状和存在的问题,进而探究了基于Unity3D的数字虚拟人交互技术,并提出了一种由文本直接生成带有情感特征语音的方法。基于此,将其与ChatGPT语言理解与文本生成、文本情感分析和改进后的VITS语音合成技术结合,并使用Kinect 2.0设备模拟全息交互效果,最终构建了一款能够进行准确理解并模拟情感回应的数字虚拟人交互应用。结果表明,该技术可有效提高数字虚拟人的理解与表达能力,为用户提供更好的交互体验,对于数字虚拟人技术的应用和发展具有参考价值。

分布式麦克风阵列拾音理论与应用方法综述

经过数十年的发展,麦克风阵列技术日益成熟,并广泛应用于视频会议、智能电视、移动通话和助听器等人机交互系统。然而,现实噪声或远距离交互场景中,限定阵型结构的传统麦克风阵列的拾音质量难以保证。随着无线智能终端设备的广泛使用,分布式麦克风阵列(或称无线声传感器网络)为提升复杂开放域语音交互系统的拾音质量提供了更多可能性,并在阵列组织、应用体验和声场覆盖度上更有优势。近年来,分布式麦克风阵列在很多语音交互任务上展现出良好的应用潜力,基本实现了对传统麦克风阵列语音任务的全覆盖。本文将重点总结现阶段分布式麦克风阵列的拾音理论和应用方法,包括阵列组织原理、麦克风节点效用评估,以及结合下游语音任务阐述其应用方法。最后,将简要论述分布式麦克风阵列走向实用的关键挑战与发展趋势。

Interaction between auditory and motor systems in speech perception

Based on the Motor Theory of speech perception, the interaction between the auditory and motor systems plays an essential role in speech perception. Since the Motor Theory was proposed, it has received remarkable attention in the field. However, each of the three hypotheses of the theory still needs further verification. In this review, we focus on how the auditory-motor anatomical and functional associations play a role in speech perception and discuss why previous studies could not reach an agreement and particularly whether the motor system involvement in speech perception is task-load dependent. Finally, we suggest that the function of the auditory-motor link is particularly useful for speech perception under adverse listening conditions and the further revised Motor Theory is a potential solution to the ＂cocktail-party＂ problem.

基于语音增强的改进型英语课堂交互系统的研究

针对复杂噪声环境下英语课堂交互系统语音识别性能下降的问题,文章提出将双路Transformer语音增强网络作为降噪模块引入该系统,通过对噪声环境下的英语语音数据进行降噪增强,提升英语交互系统的语音识别准确性。搭建实验环境对改进的方式进行验证,实验结果表明,改进后的方式能够有效提升复杂噪声环境下英语课堂交互系统的语音识别准确性。

智能语音识别技术在车机交互系统中的应用

随着我国新能源汽车朝着智能化方向快速发展,具备良好交互性能的车机系统成为影响用户用车体验的关键因素。车机系统的良好交互性能建立在对用户语音指令的准确识别上。因此,从智能语音识别技术的概念与优势出发,对其在车机交互系统中应用的意义与原则进行详细分析,进而阐述智能语音识别技术在车机交互系统中高效应用的策略,为我国新能源汽车产业智能化发展提供参考。

Speech interaction with a rat

This paper presents the first report of a system of human's speech interaction with rats via integration of brain–machine interfaces and automatic speech recognition technologies. We propose a novel human–rat speech interaction paradigm by incorporating speech translator module, which translates human's speech commands into suitable electrical brain stimulation to steer the rat to induce expected locomotor behaviors. The preliminary results show that we can guide a rat's movement by speech commands. We further look into the future application scenarios together with forthcoming challenges facing this newly evolved cyborg intelligent system. This work will pave the way for natural interaction with animal robots.

基于语音信号的跨语种交互翻译机器人语义纠错方法

传统的跨语种交互翻译机器人语义纠错方法通常是单向的,效率较低,导致识别错误率较高。为此,文章提出基于语音信号的跨语种交互翻译机器人语义纠错方法。在基础语音识别的基础上,通过交互标定和特征提取来修正语义错误位置,并设计语音信号翻译机器人的语义纠错模型,采用随时间反向传播(Backpropagation Through Time,BPTT)循环训练核验方式,以确保纠错的准确性。测试结果显示,经过3个阶段测试,选定的5段语音材料的纠错识别率成功控制在10%以下,表明基于语音信号的跨语种交互翻译机器人语义纠错方法高效,具有实际应用价值。

语音交互技术在仿真雷达伺服控制系统研制中的应用研究

在介绍语音交互技术的基础上,设计基于语音交互技术的仿真雷达伺服控制系统,并分析详细的设计流程。实验结果表明,该系统在语音识别、指令解析、实时响应及伺服控制等方面均取得优异的表现,验证了语音交互技术在提升雷达控制系统性能方面的显著效果。