A Novel Locomotion Rule Rmbedding Long Short-Term Memory Network with Attention for Human Locomotor Intent Classification Using Multi-Sensors Signals

Locomotor intent classification has become a research hotspot due to its importance to the development of assistive robotics and wearable devices.Previous work have achieved impressive performance in classifying steady locomotion states.However,it remains challenging for these methods to attain high accuracy when facing transitions between steady locomotion states.Due to the similarities between the information of the transitions and their adjacent steady states.Furthermore,most of these methods rely solely on data and overlook the objective laws between physical activities,resulting in lower accuracy,particularly when encountering complex locomotion modes such as transitions.To address the existing deficiencies,we propose the locomotion rule embedding long short-term memory(LSTM)network with Attention(LREAL)for human locomotor intent classification,with a particular focus on transitions,using data from fewer sensors(two inertial measurement units and four goniometers).The LREAL network consists of two levels:One responsible for distinguishing between steady states and transitions,and the other for the accurate identification of locomotor intent.Each classifier in these levels is composed of multiple-LSTM layers and an attention mechanism.To introduce real-world motion rules and apply constraints to the network,a prior knowledge was added to the network via a rule-modulating block.The method was tested on the ENABL3S dataset,which contains continuous locomotion date for seven steady and twelve transitions states.Experimental results showed that the LREAL network could recognize locomotor intents with an average accuracy of 99.03%and 96.52%for the steady and transitions states,respectively.It is worth noting that the LREAL network accuracy for transition-state recognition improved by 0.18%compared to other state-of-the-art network,while using data from fewer sensors.

A deep Koopman operator-based modelling approach for long-term prediction of dynamics with pixel-level measurements

Although previous studies have made some clear leap in learning latent dynamics from high-dimensional representations,the performances in terms of accuracy and inference time of long-term model prediction still need to be improved.In this study,a deep convolutional network based on the Koopman operator(CKNet)is proposed to model non-linear systems with pixel-level measurements for long-term prediction.CKNet adopts an autoencoder network architecture,consisting of an encoder to generate latent states and a linear dynamical model(i.e.,the Koopman operator)which evolves in the latent state space spanned by the encoder.The decoder is used to recover images from latent states.According to a multi-step ahead prediction loss function,the system matrices for approximating the Koopman operator are trained synchronously with the autoencoder in a mini-batch manner.In this manner,gradients can be synchronously transmitted to both the system matrices and the autoencoder to help the encoder self-adaptively tune the latent state space in the training process,and the resulting model is time-invariant in the latent space.Therefore,the proposed CKNet has the advantages of less inference time and high accuracy for long-term prediction.Experiments are per-formed on OpenAI Gym and Mujoco environments,including two and four non-linear forced dynamical systems with continuous action spaces.The experimental results show that CKNet has strong long-term prediction capabilities with sufficient precision.

3D Reconstruction for Motion Blurred Images Using Deep Learning-Based Intelligent Systems

The 3D reconstruction using deep learning-based intelligent systems can provide great help for measuring an individual’s height and shape quickly and accurately through 2D motion-blurred images.Generally,during the acquisition of images in real-time,motion blur,caused by camera shaking or human motion,appears.Deep learning-based intelligent control applied in vision can help us solve the problem.To this end,we propose a 3D reconstruction method for motion-blurred images using deep learning.First,we develop a BF-WGAN algorithm that combines the bilateral filtering(BF)denoising theory with a Wasserstein generative adversarial network(WGAN)to remove motion blur.The bilateral filter denoising algorithm is used to remove the noise and to retain the details of the blurred image.Then,the blurred image and the corresponding sharp image are input into the WGAN.This algorithm distinguishes the motion-blurred image from the corresponding sharp image according to the WGAN loss and perceptual loss functions.Next,we use the deblurred images generated by the BFWGAN algorithm for 3D reconstruction.We propose a threshold optimization random sample consensus(TO-RANSAC)algorithm that can remove the wrong relationship between two views in the 3D reconstructed model relatively accurately.Compared with the traditional RANSAC algorithm,the TO-RANSAC algorithm can adjust the threshold adaptively,which improves the accuracy of the 3D reconstruction results.The experimental results show that our BF-WGAN algorithm has a better deblurring effect and higher efficiency than do other representative algorithms.In addition,the TO-RANSAC algorithm yields a calculation accuracy considerably higher than that of the traditional RANSAC algorithm.

Deep-learning-based motion-correction algorithm in optical resolution photoacoustic microscopy

In this study,we propose a deep-learning-based method to correct motion artifacts in optical resolution photoacoustic microscopy(OR-PAM).The method is a convolutional neural network that establishes an end-to-end map from input raw data with motion artifacts to output corrected images.First,we performed simulation studies to evaluate the feasibility and effectiveness of the proposed method.Second,we employed this method to process images of rat brain vessels with multiple motion artifacts to evaluate its performance for in vivo applications.The results demonstrate that this method works well for both large blood vessels and capillary networks.In comparison with traditional methods,the proposed method in this study can be easily modified to satisfy different scenarios of motion corrections in OR-PAM by revising the training sets.

Research on the Effects of in-Line Oscillatory Flow on the Vortex-Induced Motions of A Deep Draft Semi-Submersible in Currents

A Deep Draft Semi-submersible （DDS） under certain flow conditions could be subjected to Vortex-Induced Motions （VIM）, which significantly influences the loads on and life fatigue of the moorings and the risers. To investigate the VIM of a DDS with four rectangular section columns in waves coupled with a uniform current, a numerical study using the computational fluid dynamics （CFD） method was conducted. The issues of the VIM of multi-column floaters can be con＇,＇eniently converted to the issues of oscillating cylinders in fluid cross flows. This paper looks into the CFD numerical simulation of infinite cylinders having rectangular sections in a two-dimensional sinusoidal time- dependent flow field coupled with a uniform current. The resulted hydrodynamic forces and motion responses in different oscillatory flows plus currents both aligned in the same direction for the incidence of 135° of the DDS relative to the flow are compared with the ones in current only cases. The results show that the VIM response of this geometric arrangement of a DDS with four rectangular columns in a current combined with oscillatory flows is more evident than that in the current only case. The oscillatory flows and waves have the significant influence on the VIM response, forces and trajectory, in-plane motions of the DDS.

Amplification effect of near-field ground motion around deep tunnels based on finite fracturing seismic source model

Dynamic failure of rock masses around deep tunnels,such as fault-slip rockburst and seismic-induced collapse,can pose a significant threat to tunnel construction safety.One of the most significant factors that control the accuracy of its risk assessment is the estimation of the ground motion around a tunnel caused by seismicity events.In general,the characteristic parameters of ground motion are estimated in terms of empirical scaling laws.However,these scaling laws make it difficult to accurately estimate the near-field ground motion parameters because the roles of control factors,such as tunnel geometry,damage zone distribution,and seismic source parameters,are not considered.For this,the finite fracturing seismic source model(FFSSM)proposed in this study is used to simulate the near-field ground motion characteristics around deep tunnels.Then,the amplification effects of ground motion caused by the interaction between seismic waves and deep tunnels and corresponding control factors are studied.The control effects of four factors on the near-field ground motion amplification effect are analyzed,including the main seismic source wavelength,tunnel span,tunnel shape,and range of damage zones.An empirical formula for the maximum amplification factor(a_(m))of the near-field ground motion around deep tunnels is proposed,which consists of four control factors,i.e.the wavelength control factor(F_(λ)),tunnel span factor(F_(D)),tunnel shape factor(F_(s))and excavation damage factor(F_(d)).This empirical formula provides an easy approach for accurately estimating the ground motion parameters in seismicityprone regimes and the rock support design of deep tunnels under dynamic loads.

Motion simulation of moorings using optimized LSTM neural network

Mooring arrays have been widely deployed in sustained ocean observation in high resolution to measure finer dynamic features of marine phenomena.However,the irregular posture changes and nonlinear response of moorings under the effect of ocean currents face huge challenges for the deployment of mooring arrays,which may cause the deviations of measurements and yield a vacuum of observation in the upper ocean.We developed a data-driven mooring simulation model based on LSTM(long short-term memory)neural network,coupling the ocean current with position data from moorings to predict the motion of moorings,including single-step output prediction and multi-step prediction.Based on the predictive information,the formation of the mooring array can be adjusted to improve the accuracy and integrity of measurements.Moreover,we proposed the cuckoo search(CS)optimization algorithm to tune the parameters of LSTM,which improves the robustness and generalization of the model.We utilize the datasets observed from moorings anchored in the Kuroshio Extension region to train and validate the simulation model.The experimental results demonstrate that the model can remarkably improve prediction accuracy and yield stable performance.Moreover,compared with other optimization algorithms,CS is more efficient and performs better in simulating the motion of moorings.

Advanced 4-dimensional cone-beam computed tomography reconstruction by combining motion estimation, motioncompensated reconstruction, biomechanical modeling and deep learning

4-Dimensional cone-beam computed tomography(4D-CBCT)offers several key advantages over conventional 3DCBCT in moving target localization/delineation,structure de-blurring,target motion tracking,treatment dose accumulation and adaptive radiation therapy.However,the use of the 4D-CBCT in current radiation therapy practices has been limited,mostly due to its sub-optimal image quality from limited angular sampling of conebeam projections.In this study,we summarized the recent developments of 4D-CBCT reconstruction techniques for image quality improvement,and introduced our developments of a new 4D-CBCT reconstruction technique which features simultaneous motion estimation and image reconstruction(SMEIR).Based on the original SMEIR scheme,biomechanical modeling-guided SMEIR(SMEIR-Bio)was introduced to further improve the reconstruction accuracy of fine details in lung 4D-CBCTs.To improve the efficiency of reconstruction,we recently developed a U-net-based deformation-vector-field(DVF)optimization technique to leverage a population-based deep learning scheme to improve the accuracy of intra-lung DVFs(SMEIR-Unet),without explicit biomechanical modeling.Details of each of the SMEIR,SMEIR-Bio and SMEIR-Unet techniques were included in this study,along with the corresponding results comparing the reconstruction accuracy in terms of CBCT images and the DVFs.We also discussed the application prospects of the SMEIR-type techniques in image-guided radiation therapy and adaptive radiation therapy,and presented potential schemes on future developments to achieve faster and more accurate 4D-CBCT imaging.

Double Deep Q-Network Decoder Based on EEG Brain-Computer Interface

Brain-computer interfaces(BCI)use neural activity as a control signal to enable direct communication between the human brain and external devices.The electrical signals generated by the brain are captured through electroencephalogram(EEG)and translated into neural intentions reflecting the user’s behavior.Correct decoding of the neural intentions then facilitates the control of external devices.Reinforcement learning-based BCIs enhance decoders to complete tasks based only on feedback signals(rewards)from the environment,building a general framework for dynamic mapping from neural intentions to actions that adapt to changing environments.However,using traditional reinforcement learning methods can have challenges such as the curse of dimensionality and poor generalization.Therefore,in this paper,we use deep reinforcement learning to construct decoders for the correct decoding of EEG signals,demonstrate its feasibility through experiments,and demonstrate its stronger generalization on motion imaging(MI)EEG data signals with high dynamic characteristics.

动态场景的三维重建研究综述

随着静态场景三维重建算法的不断成熟,动态场景三维重建算法成为近年来的研究热点和研究难点。现有的静态场景三维重建算法对静止的对象有较好的重建效果,一旦场景中对象出现变形或者是相对运动,其重建效果不太理想,因此发展对动态场景的三维重建研究工作是相当重要的。简要介绍三维重建的相关概念及基本知识、静态场景三维重建和动态场景三维重建的研究分类及研究现状;全面总结了动态场景三维重建研究最新进展,将动态场景三维重建按照基于RGB数据源的动态三维重建和基于RGB-D数据源的动态三维重建进行分类,其中RGB数据源下又可划分为基于模板的动态三维重建、基于非刚性运动恢复结构的动态三维重建和RGB数据源下基于学习的动态三维重建,RGB-D数据源下主要总结归纳基于学习的动态三维重建,对各类典型重建算法进行了介绍和对比分析;介绍了动态场景三维重建在医学、智能制造、虚拟现实与增强现实、交通等领域的应用;提出了动态场景三维重建的未来研究方向,并对这个快速发展领域中的各个方向研究进行了展望。

基于深度强化学习的多自动导引车运动规划

为解决移动机器人仓储系统中的多自动导引车(AGV)无冲突运动规划问题,建立了Markov决策过程模型,提出一种新的基于深度Q网络(DQN)的求解方法。将AGV的位置作为输入信息,利用DQN估计该状态下采取每个动作所能获得的最大期望累计奖励,并采用经典的深度Q学习算法进行训练。算例计算结果表明,该方法可以有效克服AGV车队在运动中的碰撞问题,使AGV车队能够在无冲突的情况下完成货架搬运任务。与已有启发式算法相比,该方法求得的AGV运动规划方案所需要的平均最大完工时间更短。

深海AUV无动力垂直下潜运动特性研究

深海AUV(自主水下机器人)采用无动力下潜方式能有效节约能源,但是水平或小纵倾角姿态下潜用时较长,水平偏移大。为提高下潜速度,减小下潜偏移距离,对AUV无动力垂直下潜过程进行研究。使用四元数代替欧拉角得到AUV运动方程,并加入洋流和浮力数学模型,通过Simulink仿真模块建立AUV垂直下潜运动仿真方法,解决垂直下潜产生的姿态角解算奇异性问题。通过设置不同仿真工况,得到AUV垂直下潜运动状态量受负浮力、重浮心距离、舵角、初始纵倾角及环境扰动力的作用规律。结果表明,AUV采用垂直下潜方式,可实现较大的下潜速度,同时减小水平偏移距离;洋流扰动是垂直下潜运动产生水平偏移的主要原因;下潜深度6000 m时,浮力变化扰动导致末端下潜速度减小30%,下潜时间增加21%。

表面肌电与三轴信息融合的运动判断实验

为了提高基于表面肌电与三轴加速度信号的运动识别准确率,提出了一套多源信息融合处理的实验流程与方法。该方法利用5层离散小波变换对表面肌电信号进行分解,充分提取不同运动产生的肌电信号中各频域的特征信息;再将分解后的表面肌电信号与三轴加速度信号通过滑动窗口的方法进行特征融合,构造融合肌电与空间运动特征的特征图;最后用融合特征图对深度学习模型进行训练,并结合自动状态机进行最终运动状态的识别。实验结果表明,多源信息融合处理方法可以提高运动识别的准确性,总体识别精度分别达到了95.4%和89.2%。该方法在实时性与准确性上均有良好表现。

基于运动特征的骨骼行为识别方法

针对现有的骨骼行为识别方法对人体行为的运动信息利用不足的问题,提出一种基于运动特征的时空注意力图卷积(STA-GCN)行为识别模型。对动作捕捉设备采集到的关节点运动轨迹和速度信息进行建模,在时间和空间构建注意力权重矩阵,结合图卷积网络进行特征提取,能够关注到具有判别力的关节点和时间帧。通过在自建动作捕捉数据集和NTU-RGB+D数据集的CS和CV标准上进行实验,其结果表明,该模型增强了对人体骨骼行为信息的理解能力,验证了模型对行为识别的有效性。

Evolutionary Decision-Making and Planning for Autonomous Driving Based on Safe and Rational Exploration and Exploitation

Decision-making and motion planning are extremely important in autonomous driving to ensure safe driving in a real-world environment.This study proposes an online evolutionary decision-making and motion planning framework for autonomous driving based on a hybrid data-and model-driven method.First,a data-driven decision-making module based on deep reinforcement learning(DRL)is developed to pursue a rational driving performance as much as possible.Then,model predictive control(MPC)is employed to execute both longitudinal and lateral motion planning tasks.Multiple constraints are defined according to the vehicle’s physical limit to meet the driving task requirements.Finally,two principles of safety and rationality for the self-evolution of autonomous driving are proposed.A motion envelope is established and embedded into a rational exploration and exploitation scheme,which filters out unreasonable experiences by masking unsafe actions so as to collect high-quality training data for the DRL agent.Experiments with a high-fidelity vehicle model and MATLAB/Simulink co-simulation environment are conducted,and the results show that the proposed online-evolution framework is able to generate safer,more rational,and more efficient driving action in a real-world environment.

融合Kinect和IMU多模态数据的多阶段运动去噪网络

运动数据去噪在影视特效、游戏和康复医疗等动作捕捉应用中起着重要作用.为提高低成本动作捕捉设备的精确度和鲁棒性,提出一种融合Kinect和惯性测量单元(IMU)两种模态运动数据的多阶段去噪网络MMCapNet,利用特征提取器从两种模态数据中提取并融合运动特征,使用关节位置估计器分阶段预测关键关节、身体关节和手部关节坐标位置.为了提高方法的泛化能力,在现有2180332帧多模态数据的基础上,采集了227160帧包含高噪声的多模态运动数据集.实验结果表明,在日常运动和高噪声多模态数据集上输出结果的关节点位置精度均有提升.与BRA、DIP和STTrans方法相比,在日常运动数据集上全身估计误差分别降低78.5%、87.1%和31%,在高噪声数据上的估计结果更加合理.本文通过特征提取,融合多模态数据和多阶段预测,在降低位置估计误差的同时增强了对高噪声数据的处理能力.

基于混合运动激励和时序增强的篮球运动员动作识别算法

为了解决在背景相似的篮球视频中提取特征级运动信息不充分和捕获长时序依赖关系困难等问题,从局部和全局的角度出发,提出一种混合运动激励和时序增强网络(mixed motion excitation and temporal enhancement network,MTE-Net),该网络由在时间建模上互补的混合运动激励(mixed motion excitation,MME)模块和时序增强(temporal enhancement,TE)模块构成。混合运动激励模块通过计算短距离视频帧之间混合的特征级差分来充分表征局部运动信息,并显性地对运动敏感通道进行激励。时序增强模块对长距离视频帧使用自注意力机制来构建时序关联函数并捕获时序之间的全局依赖关系,增强视频中的重要帧序列。在不额外引入光流和过多参数的情况下,在SpaceJam篮球动作数据集上的实验结果表明,与其他主流的动作识别算法相比,所提模型对篮球运动员动作识别的准确率更高。

无信号交叉口处基于深度强化学习的智能网联车辆运动规划

为了兼顾无信号交叉口处智能网联车辆通行效率和舒适性要求,提出基于深度强化学习的车辆运动规划算法.结合时间卷积网络(TCN)和Transformer算法构建周围车辆行驶意图预测模型,通过多层卷积和自注意力机制提高车辆运动特征捕捉能力;利用双延迟深度确定性策略梯度(TD3)强化学习算法构建车辆运动规划模型,综合考虑周围车辆行驶意图、驾驶风格、交互风险以及自车舒适性等因素设计状态空间和奖励函数以增强对动态环境的理解;通过延迟策略更新和平滑目标策略提高算法稳定性,实时输出期望加速度.实验结果表明,所提运动规划算法能够根据周围车辆的行驶意图实时感知潜在的交互风险,生成的运动规划策略满足通行效率、安全性和舒适性要求,且对不同风格的周围车辆和密集交互场景均有良好的适应能力,不同场景下成功率均高于92.1%.

横向振动立管上升流中球形单颗粒运动特征

深海采矿过程中,输送矿石颗粒的立管在复杂海洋环境作用下会产生运动响应,这会对管道中矿石颗粒的运动行为产生重要影响,从而影响矿石提升效率,甚至可能危及整个采矿系统的安全.结合颗粒运动方程和软球碰撞模型,对横向振动立管中的球形单颗粒运动特征进行研究,主要分析了立管振动参数、颗粒与流体密度比以及颗粒与立管直径比对立管中单颗粒运动的影响.研究表明,随振动频率和幅度、颗粒与流体密度比以及颗粒与立管直径比的增加,颗粒垂向平均速度减小.颗粒与管壁之间无碰撞发生时,颗粒与立管之间横向相对速度幅值、颗粒运动与立管之间横向速度的相位差以及颗粒垂向速度波动幅值,与立管振动的频率和振幅、颗粒与流体密度比以及颗粒与立管直径比呈正相关.然而,当颗粒与立管之间有碰撞发生时,颗粒与立管间的横向速度的相位差减小,而颗粒垂向速度波动幅值显著增大.另外,随着密度比和直径比的增大,颗粒与管壁之间更容易发生碰撞,而碰撞会减弱密度比和直径比对颗粒横向速度和垂向速度的影响.

深吃水圆筒型浮式核能平台涡激运动数值模拟

深吃水圆筒型浮式核能平台是一种新型多功能高效平台,可有效解决南海岛礁开发过程中的能源供给问题。在一定来流速度下,尾流区交替泄涡进而诱导平台发生涡激运动(vortex induced motions,VIM),这将严重加速系泊和立管系统疲劳损害,同时对平台内部核反应堆运行产生不利影响。基于改进的延迟分离涡方法(improved delayed detached eddy simulation,IDDES)对平台在不同折合速度下的横荡、纵荡、艏摇运动响应进行数值模拟,并从水平面内质心运动轨迹、运动频率、三维流场特性等角度分析涡激运动关键特征。研究结果表明:当折合速度5.45<Ur<9.08时,平台横荡、艏摇振幅均逐渐增加且运动轨迹类似“香蕉”形,横荡与艏摇运动频率基本一致且横荡未出现明显的“锁定”区间变化;当7.26<Ur<9.08时,艏摇振幅近似线性递增且运动频率中主频附近出现多个峰值,运动轨迹在顺流方向上逐渐变宽;三维流场中发现尾流区三维漩涡结构相当复杂且平台底部柱靴结构对表面流动分离造成干扰,柱靴结构具有一定的减涡效果。