Unsupervised multi-modal image translation based on the squeeze-and-excitation mechanism and feature attention module

The unsupervised multi-modal image translation is an emerging domain of computer vision whose goal is to transform an image from the source domain into many diverse styles in the target domain.However,the multi-generator mechanism is employed among the advanced approaches available to model different domain mappings,which results in inefficient training of neural networks and pattern collapse,leading to inefficient generation of image diversity.To address this issue,this paper introduces a multi-modal unsupervised image translation framework that uses a generator to perform multi-modal image translation.Specifically,firstly,the domain code is introduced in this paper to explicitly control the different generation tasks.Secondly,this paper brings in the squeeze-and-excitation(SE)mechanism and feature attention(FA)module.Finally,the model integrates multiple optimization objectives to ensure efficient multi-modal translation.This paper performs qualitative and quantitative experiments on multiple non-paired benchmark image translation datasets while demonstrating the benefits of the proposed method over existing technologies.Overall,experimental results have shown that the proposed method is versatile and scalable.

Robust Symmetry Prediction with Multi-Modal Feature Fusion for Partial Shapes

In geometry processing,symmetry research benefits from global geo-metric features of complete shapes,but the shape of an object captured in real-world applications is often incomplete due to the limited sensor resolution,single viewpoint,and occlusion.Different from the existing works predicting symmetry from the complete shape,we propose a learning approach for symmetry predic-tion based on a single RGB-D image.Instead of directly predicting the symmetry from incomplete shapes,our method consists of two modules,i.e.,the multi-mod-al feature fusion module and the detection-by-reconstruction module.Firstly,we build a channel-transformer network(CTN)to extract cross-fusion features from the RGB-D as the multi-modal feature fusion module,which helps us aggregate features from the color and the depth separately.Then,our self-reconstruction net-work based on a 3D variational auto-encoder(3D-VAE)takes the global geo-metric features as input,followed by a prediction symmetry network to detect the symmetry.Our experiments are conducted on three public datasets:ShapeNet,YCB,and ScanNet,we demonstrate that our method can produce reliable and accurate results.

Improving VQA via Dual-Level Feature Embedding Network

Visual Question Answering(VQA)has sparked widespread interest as a crucial task in integrating vision and language.VQA primarily uses attention mechanisms to effectively answer questions to associate relevant visual regions with input questions.The detection-based features extracted by the object detection network aim to acquire the visual attention distribution on a predetermined detection frame and provide object-level insights to answer questions about foreground objects more effectively.However,it cannot answer the question about the background forms without detection boxes due to the lack of fine-grained details,which is the advantage of grid-based features.In this paper,we propose a Dual-Level Feature Embedding(DLFE)network,which effectively integrates grid-based and detection-based image features in a unified architecture to realize the complementary advantages of both features.Specifically,in DLFE,In DLFE,firstly,a novel Dual-Level Self-Attention(DLSA)modular is proposed to mine the intrinsic properties of the two features,where Positional Relation Attention(PRA)is designed to model the position information.Then,we propose a Feature Fusion Attention(FFA)to address the semantic noise caused by the fusion of two features and construct an alignment graph to enhance and align the grid and detection features.Finally,we use co-attention to learn the interactive features of the image and question and answer questions more accurately.Our method has significantly improved compared to the baseline,increasing accuracy from 66.01%to 70.63%on the test-std dataset of VQA 1.0 and from 66.24%to 70.91%for the test-std dataset of VQA 2.0.

A Further Study on an Extended Nonlinear Ocean-Atmosphere Coupled Hydrodynamic Characteristic System and the Abrupt Feature of ENSO Events

An extended ocean-atmosphere coupled characteristic system including thermodynamic physical processes in ocean mixed layer is formulated in order to describe SST explicitly and remove possible limitation of ocean-atmosphere coupling assumption in hydrodynamic ENSO models. It is revealed that there is a kind of abrupt nonlinear characteristic behaviour, which relates to rapid onset and intermittency of El Nino events, on the second order slow time scale due to the nonlinear interaction between a linear unstable low-frequency primary eigen component of ocean-atmosphere coupled Kelvin wave and its higher order harmonic components under a strong ocean-atmosphere coupling background. And, on the other hand, there is a kind of finite amplitude nonlinear characteristic behaviour on the second order slow time scale due to the nonlinear interaction between the linear unstable primary eigen component and its higher order harmonic components under a weak ocean-atmosphere coupling background in this model system.

Multi-modal face parts fusion based on Gabor feature for face recognition

A novel face recognition method, which is a fusion of muhi-modal face parts based on Gabor feature （MMP-GF）, is proposed in this paper. Firstly, the bare face image detached from the normalized image was convolved with a family of Gabor kernels, and then according to the face structure and the key-points locations, the calculated Gabor images were divided into five parts： Gabor face, Gabor eyebrow, Gabor eye, Gabor nose and Gabor mouth. After that multi-modal Gabor features were spatially partitioned into non-overlapping regions and the averages of regions were concatenated to be a low dimension feature vector, whose dimension was further reduced by principal component analysis （PCA）. In the decision level fusion, match results respectively calculated based on the five parts were combined according to linear discriminant analysis （LDA） and a normalized matching algorithm was used to improve the performance. Experiments on FERET database show that the proposed MMP-GF method achieves good robustness to the expression and age variations.

Adaptive multi-modal feature fusion for far and hard object detection

In order to solve difficult detection of far and hard objects due to the sparseness and insufficient semantic information of LiDAR point cloud,a 3D object detection network with multi-modal data adaptive fusion is proposed,which makes use of multi-neighborhood information of voxel and image information.Firstly,design an improved ResNet that maintains the structure information of far and hard objects in low-resolution feature maps,which is more suitable for detection task.Meanwhile,semantema of each image feature map is enhanced by semantic information from all subsequent feature maps.Secondly,extract multi-neighborhood context information with different receptive field sizes to make up for the defect of sparseness of point cloud which improves the ability of voxel features to represent the spatial structure and semantic information of objects.Finally,propose a multi-modal feature adaptive fusion strategy which uses learnable weights to express the contribution of different modal features to the detection task,and voxel attention further enhances the fused feature expression of effective target objects.The experimental results on the KITTI benchmark show that this method outperforms VoxelNet with remarkable margins,i.e.increasing the AP by 8.78%and 5.49%on medium and hard difficulty levels.Meanwhile,our method achieves greater detection performance compared with many mainstream multi-modal methods,i.e.outperforming the AP by 1%compared with that of MVX-Net on medium and hard difficulty levels.

RDH12-associated retinal degeneration caused by a homozygous pathogenic variant of 146C>T and literature review

AIM:To describe the clinical,electrophysiological,and genetic features of an unusual case with an RDH12 homozygous pathogenic variant and reviewed the characteristics of the patients reported with the same variant.METHODS:The patient underwent a complete ophthalmologic examination including best-corrected visual acuity,anterior segment and dilated fundus,visual field,spectral-domain optical coherence tomography(OCT)and electroretinogram(ERG).The retinal disease panel genes were sequenced through chip capture high-throughput sequencing and Sanger sequencing was used to confirm the result.Then we reviewed the characteristics of the patients reported with the same variant.RESULTS:A 30-year male presented with severe early retinal degeneration who complained night blindness,decreased visual acuity,vitreous floaters and amaurosis fugax.The best corrected vision was 0.04 OD and 0.12 OS,respectively.The fundus photo and OCT showed bilateral macular atrophy but larger areas of macular atrophy in the left eye.Autofluorescence shows bilateral symmetrical hypo-autofluorescence.ERG revealed that the amplitudes of a-and b-wave were severely decreased.Multifocal ERG showed decreased amplitudes in the local macular area.A homozygous missense variant c.146C>T(chr14:68191267)was found.The clinical characteristics of a total of 13 patients reported with the same pathologic variant varied.CONCLUSION:An unusual patient with a homozygous pathogenic variant in the c.146C>T of RDH12 which causes late-onset and asymmetric retinal degeneration are reported.The clinical manifestations of the patient with multimodal retinal imaging and functional examinations have enriched our understanding of this disease.

基于XFEM的大体积结构波动传播规律及裂纹反演方法

大体积混凝土结构被广泛应用于土木、水利等领域的重大工程中,而混凝土抗拉强度低的力学特性决定了其易产生裂纹,因此,发展高效的检测方法,识别大体积混凝土结构中的裂纹信息十分必要.论文提出了一种新的方法,通过提取响应信号频谱中特定频率的幅值特征,基于BP人工神经网络建立幅值特征与裂纹信息间的映射关系,从而有效识别出裂纹信息.首先采用扩展有限元法(eXtended Finite Element Methods, XFEM)和人工吸收边界模型,分别模拟了单裂纹和双裂纹情形下,大量不同裂纹信息下特定位置传感器的响应,分析其频谱曲线并提取特征,建立频谱特征—裂尖位置数据集,以训练人工神经网络,测试集的反演效果显示,该方法具有较好的准确度,可有效识别出裂纹信息.

基于改进Faster RCNN的PCB表面缺陷检测研究

印刷电路板(PCB)在制造过程中不可避免地存在焊点缺焊、短路、毛刺、缺口、开路、余铜等微小缺陷。传统的基于机器视觉检测的缺陷检测方法存在检测速度慢、误检率和漏检率高、抗干扰能力弱等问题。为解决上述问题,提出一种基于改进快速区域卷积神经网络(Faster RCNN)的PCB表面缺陷检测方法。首先,在传统Faster RCNN框架的基础上,融入扩展特征金字塔网络(EFPN)以实现特征提取与融合,并进行多尺度检测,从而尽可能保留图像细节信息以提高检测性能。其次,利用K-means算法结合交并比(IoU)优化区域建议网络(RPN)结构中的锚框参数,使得生成的锚框方案更有针对性。试验结果表明,改进Faster RCNN在PCB缺陷数据集上的全类平均正确率(mAP)值达到93.4%、检测速度达到每秒21.79帧。所提方法可推广应用至芯片、光学器件表面微小缺陷在线检测,从而提升工业生产效率。

基于特征匹配的暂态稳定紧急控制策略快速生成

“在线计算、实时匹配”的紧急控制模式是降低控制策略失配风险的有效途径和技术发展趋势。为进一步提高在线紧急控制策略生成的快速性,提出了一种基于特征匹配的暂态稳定紧急控制策略快速生成方法。借助扩展等面积准则对暂态稳定性以及电网故障下轨迹时变程度的量化分析能力,综合利用电网暂态稳定轨迹特征和稳态潮流关键特征量,建立运行方式特征量匹配指标,在历史方式中匹配最接近方式,实现紧急控制策略的快速生成,通过校核验证后下发装置执行。最后,基于实际电网验证了所提方法的有效性。

Motion estimation based feature selection for visual SLAM

Feature selection is always an important issue in the visual SLAM （simultaneous location and mapping） literature. Considering that the location estimation can be improved by tracking features with larger value of visible time, a new feature selection method based on motion estimation is proposed. First, a k-step iteration algorithm is presented for visible time estimation using an affme motion model; then a delayed feature detection method is introduced for efficiently detecting features with the maximum visible time. As a means of validation for the proposed method, both simulation and real data experiments are carded out. Results show that the proposed method can improve both the estimation performance and the computational performance compared with the existing random feature selection method.

Speed-up Multi-modal Near Duplicate Image Detection

Near-duplicate image detection is a necessary operation to refine image search results for efficient user exploration. The existences of large amounts of near duplicates require fast and accurate automatic near-duplicate detection methods. We have designed a coarse-to-fine near duplicate detection framework to speed-up the process and a multi-modal integra-tion scheme for accurate detection. The duplicate pairs are detected with both global feature (partition based color his-togram) and local feature (CPAM and SIFT Bag-of-Word model). The experiment results on large scale data set proved the effectiveness of the proposed design.

基于扩展卡尔曼滤波的5-Dof圆位姿估计算法

现有圆位姿估计方法对输入帧进行独立处理,忽略了有价值的目标动态信息,圆位姿估计精度有提升空间,提出一种基于EKF的高精度5-Dof圆位姿估计方法,圆位姿由5自由度向量ξ=(X,Y,Z,α,β)^(T)表示。该方法引入贝叶斯框架捕获视频连续帧的时间信息,优化圆位姿估计系统。首先,为了与2D椭圆轮廓交互,算法构造出5自由度向量ξ表示的空间圆投影轮廓5-Dof模型,进而设计非线性测量函数。其次,将该测量函数与扩展卡尔曼滤波(EKF)算法相结合,用于圆位姿优化。此外,使用简单的线性卡尔曼滤波算法(KF)对圆位姿估计值进行修正。实验表明,针对含有不同方差噪声的图像序列,算法利用图像序列的时间相关性,有效提高圆位姿估计精度。

机器视觉中小目标检测实验优化模型设计与实现

目标检测是计算机视觉课程的重要实验内容之一,但现有模型对小目标检测能力普遍较弱。为了加深学生对现有模型结构和缺陷的深入理解,掌握模型的优化方法,基于实验中常用的SSD模型,引入了轻量级秩扩展网络ReXNet,重新设计了特征融合与过滤模块。特征融合模块在深浅层特征融合之前,先对深层特征图进行特征抽取,减少无效语义信息对浅层特征的干扰,增强了模型对小目标语义特征的表征能力。特征过滤模块则分别在分类和回归时,引入通道注意力和空间注意力的双路结构,提高分类与回归的精度。在VOC和COCO数据集上的实验结果表明,改进后的模型不仅提高了对小目标的检测性能,保留了较快的检测速度,而且改善了原始模型存在的漏检问题。通过模型设计的优化,加深了学生对于目标检测模型架构的理解,提高了学生的综合实践能力,促进了计算机视觉课程的实验教学内容建设。

基于激光雷达点云的目标识别与跟踪方法

对复杂驾驶环境中移动目标的准确识别与跟踪是驾驶辅助系统需要解决的重要问题之一。本文提出了一种基于激光雷达点云的目标识别与跟踪方法,首先对每一帧点云数据进行聚类并提取目标线段,利用采集的大量实验数据获取各类目标的特征值参考区间,然后通过特征匹配的方式完成目标类别的识别,最后基于扩展卡尔曼滤波(EKF)对不同类别的目标状态进行预测与更新。实验结果表明,相较于既有的基于特征匹配的目标识别方法,本文的方法能够明显提升目标识别性能,并能实现稳定的目标跟踪。

YOLOv5定位多特征融合的车标识别

为解决智能交通系统中车标识别的问题,提出YOLOv5s网络车标定位多特征融合的车标图像识别方案。车标定位阶段选择YOLOv5s网络以满足对车标定位速度与精度等的需求。车标识别阶段通过调整扩展高斯差分中的参数得到具有不同效果的车标边缘,设计一组二维Gabor滤波器对边缘检测后的车标图像进行滤波处理并提取出对应的车标图像特征向量,通过计算待测车标图像特征与标准比对库中特征向量的欧几里德距离,取距离最小者对应的标签索引作为分类识别结果,该方案的最佳识别正确率为96.91%。采用随机森林算法进行分类后的最佳识别正确率可达99.33%。该方案的车标定位与识别最佳整体正确率超过了YOLOv5s网络直接一步到位识别车标的方案,且相较于传统图像处理方法有明显提升。

基于拓展分析方法的便民消毒装置研究

2003年前,我国消毒行业没有受到足够的重视,消毒药剂和消毒设备只在一些专业领域配备,并没有实现便民化。自2019年年末,普通的消毒服务和消毒装置需求量倍增,导致专业领域的消杀服务、消杀设备满负荷运转,不能够满足突如其来的大量需求。因此,大量非专业领域或非专业人员也投入对消杀行业的研究。各式各样的消毒装置应运而生,种类繁多,利弊也随之而来。文章主要针对便民消毒装置进行分析研究,在此过程中运用了一种拓展分析方法对这些装置进行量化表达,为研究者提供一种新的创新路径。同时,在对这些消毒装置进行研究的过程中积极响应国家对消杀行业的政策要求,也要避免虚假宣传、虚假产品的推出,坚持以使用安全性、消毒可靠性、运行环保性为主要研究因素,秉承创新、便捷、绿色环保的理念设计便民消毒装置。

唐宋延寿命菩萨探微

延寿命菩萨是唐宋时期新出现的题材,图式简单,主要分布于甘肃、四川、新疆等地,流行于9—11世纪。该菩萨产生过程漫长而复杂,严格意义上的延寿命菩萨出自大本《佛说延寿命经》,主要功能是延长信众现世寿命。延寿命菩萨是顺应佛教本土化的延寿信仰需求而产生的,形成过程反映出佛教中国化、世俗化的繁杂脉络之一象。

结合扩展卡尔曼滤波的快速判别尺寸空间滤波跟踪算法

为解决跟踪过程中目标发生尺度变换和遮挡导致的目标丢失问题,文中提出一种结合扩展卡尔曼滤波器的快速多尺度相关滤波跟踪算法。利用主成分分析法对提取的融合方向梯度直方图和颜色特征进行降维,再进行特征融合;根据基于对数极坐标变换和混合高斯模型的消失判断机制判断目标是否被遮挡,当目标被遮挡,用扩展卡尔曼滤波器估计目标遮挡的当前位置;用快速判别尺寸空间滤波器计算目标的尺度。文中选取5组具有代表性的视频序列测试文中算法。实验结果表明:与经典算法相比,文中算法在跟踪具有遮挡和尺度变化属性的视频序列时,中心位置误差降低了21.662 pixel,距离精度提高0.1606,可以很好地解决遮挡和尺度变化问题,同时保持着高帧率。

Face anti-spoofing based on multi-modal and multi-scale features fusion

Face anti-spoofing is used to assist face recognition system to judge whether the detected face is real face or fake face. In the traditional face anti-spoofing methods, features extracted by hand are used to describe the difference between living face and fraudulent face. But these handmade features do not apply to different variations in an unconstrained environment. The convolutional neural network(CNN) for face deceptions achieves considerable results. However, most existing neural network-based methods simply use neural networks to extract single-scale features from single-modal data, while ignoring multi-scale and multi-modal information. To address this problem, a novel face anti-spoofing method based on multi-modal and multi-scale features fusion(MMFF) is proposed. Specifically, first residual network(Resnet)-34 is adopted to extract features of different scales from each modality, then these features of different scales are fused by feature pyramid network(FPN), finally squeeze-and-excitation fusion(SEF) module and self-attention network(SAN) are combined to fuse features from different modalities for classification. Experiments on the CASIA-SURF dataset show that the new method based on MMFF achieves better performance compared with most existing methods.