An Improved Deep Fusion CNN for Image Recognition

With the development of Deep Convolutional Neural Networks(DCNNs),the extracted features for image recognition tasks have shifted from low-level features to the high-level semantic features of DCNNs.Previous studies have shown that the deeper the network is,the more abstract the features are.However,the recognition ability of deep features would be limited by insufficient training samples.To address this problem,this paper derives an improved Deep Fusion Convolutional Neural Network(DF-Net)which can make full use of the differences and complementarities during network learning and enhance feature expression under the condition of limited datasets.Specifically,DF-Net organizes two identical subnets to extract features from the input image in parallel,and then a well-designed fusion module is introduced to the deep layer of DF-Net to fuse the subnet’s features in multi-scale.Thus,the more complex mappings are created and the more abundant and accurate fusion features can be extracted to improve recognition accuracy.Furthermore,a corresponding training strategy is also proposed to speed up the convergence and reduce the computation overhead of network training.Finally,DF-Nets based on the well-known ResNet,DenseNet and MobileNetV2 are evaluated on CIFAR100,Stanford Dogs,and UECFOOD-100.Theoretical analysis and experimental results strongly demonstrate that DF-Net enhances the performance of DCNNs and increases the accuracy of image recognition.

A deep multimodal fusion and multitasking trajectory prediction model for typhoon trajectory prediction to reduce flight scheduling cancellation

Natural events have had a significant impact on overall flight activity,and the aviation industry plays a vital role in helping society cope with the impact of these events.As one of the most impactful weather typhoon seasons appears and continues,airlines operating in threatened areas and passengers having travel plans during this time period will pay close attention to the development of tropical storms.This paper proposes a deep multimodal fusion and multitasking trajectory prediction model that can improve the reliability of typhoon trajectory prediction and reduce the quantity of flight scheduling cancellation.The deep multimodal fusion module is formed by deep fusion of the feature output by multiple submodal fusion modules,and the multitask generation module uses longitude and latitude as two related tasks for simultaneous prediction.With more dependable data accuracy,problems can be analysed rapidly and more efficiently,enabling better decision-making with a proactive versus reactive posture.When multiple modalities coexist,features can be extracted from them simultaneously to supplement each other’s information.An actual case study,the typhoon Lichma that swept China in 2019,has demonstrated that the algorithm can effectively reduce the number of unnecessary flight cancellations compared to existing flight scheduling and assist the new generation of flight scheduling systems under extreme weather.

The Fusion of Temporal Sequence with Scene Priori Information in Deep Learning Object Recognition

For some important object recognition applications such as intelligent robots and unmanned driving, images are collected on a consecutive basis and associated among themselves, besides, the scenes have steady prior features. Yet existing technologies do not take full advantage of this information. In order to take object recognition further than existing algorithms in the above application, an object recognition method that fuses temporal sequence with scene priori information is proposed. This method first employs YOLOv3 as the basic algorithm to recognize objects in single-frame images, then the DeepSort algorithm to establish association among potential objects recognized in images of different moments, and finally the confidence fusion method and temporal boundary processing method designed herein to fuse, at the decision level, temporal sequence information with scene priori information. Experiments using public datasets and self-built industrial scene datasets show that due to the expansion of information sources, the quality of single-frame images has less impact on the recognition results, whereby the object recognition is greatly improved. It is presented herein as a widely applicable framework for the fusion of information under multiple classes. All the object recognition algorithms that output object class, location information and recognition confidence at the same time can be integrated into this information fusion framework to improve performance.

一种耦合DeepLab与Transformer的农作物种植类型遥感精细分类方法

如何精细遥感监测复杂的不同类型农田作物种植情况,是智慧农业农村领域实现农耕面积调查与农作物估产的关键。目前的高分辨率影像的作物种植像素级语义分割中,深度卷积神经网络难以兼顾空间多尺度全局特征和局部细节特征,从而导致各类农田地块之间边界轮廓模糊和同类农田区域内部完整性不高等问题。针对这些不足,本文提出了一种耦合DeepLabv3+和Transformer编码器的双分支并行特征融合网络FDTNet,以实现农作物种植类型的精细遥感监测。首先,在FDTNet中并行嵌入DeepLabv3+和Transformer分别捕获农田影像的局部特征和全局特征;其次,应用耦合注意力融合模块CAFM有效融合两者的特征;然后,在解码器阶段应用卷积注意力模块CBAM增强卷积层有效特征的权重;最后,采用渐进式多层特征融合策略将编码器和解码器中的有效特征全面融合并输出特征图,以实现晚稻、中稻、藕田、菜地和大棚的高精度分类识别。为了验证FDTNet网络模型在高分辨率作物分类应用的有效性,本文选择不同高分辨率的Yuhu数据集和Zhejiang数据集验证,mIoU分别达到74.7%和81.4%。相比于已有的UNet、DeepLabv3、DeepLabv3+、ResT和Res-Swin等深度学习方法,FDTNet的mIoU可分别高2.2%和3.6%。结果表明,FDTNet在纹理单一、大样本量,以及纹理多样、小样本量的两类农田场景中同时表现出优于对比方法的性能,具有较全面的多类别农作物有效特征提取能力。

Deep Bimodal Fusion Approach for Apparent Personality Analysis

Personality distinguishes individuals’ patterns of feeling, thinking,and behaving. Predicting personality from small video series is an excitingresearch area in computer vision. The majority of the existing research concludespreliminary results to get immense knowledge from visual and Audio(sound) modality. To overcome the deficiency, we proposed the Deep BimodalFusion (DBF) approach to predict five traits of personality-agreeableness,extraversion, openness, conscientiousness and neuroticism. In the proposedframework, regarding visual modality, the modified convolution neural networks(CNN), more specifically Descriptor Aggregator Model (DAN) areused to attain significant visual modality. The proposed model extracts audiorepresentations for greater efficiency to construct the long short-termmemory(LSTM) for the audio modality. Moreover, employing modality-based neuralnetworks allows this framework to independently determine the traits beforecombining them with weighted fusion to achieve a conclusive prediction of thegiven traits. The proposed approach attains the optimal mean accuracy score,which is 0.9183. It is achieved based on the average of five personality traitsand is thus better than previously proposed frameworks.

Method of Multi-Mode Sensor Data Fusion with an Adaptive Deep Coupling Convolutional Auto-Encoder

To address the difficulties in fusing multi-mode sensor data for complex industrial machinery, an adaptive deep coupling convolutional auto-encoder (ADCCAE) fusion method was proposed. First, the multi-mode features extracted synchronously by the CCAE were stacked and fed to the multi-channel convolution layers for fusion. Then, the fused data was passed to all connection layers for compression and fed to the Softmax module for classification. Finally, the coupling loss function coefficients and the network parameters were optimized through an adaptive approach using the gray wolf optimization (GWO) algorithm. Experimental comparisons showed that the proposed ADCCAE fusion model was superior to existing models for multi-mode data fusion.

Seismic velocity inversion based on CNN-LSTM fusion deep neural network

Based on the CNN-LSTM fusion deep neural network,this paper proposes a seismic velocity model building method that can simultaneously estimate the root mean square(RMS)velocity and interval velocity from the common-midpoint(CMP)gather.In the proposed method,a convolutional neural network(CNN)Encoder and two long short-term memory networks(LSTMs)are used to extract spatial and temporal features from seismic signals,respectively,and a CNN Decoder is used to recover RMS velocity and interval velocity of underground media from various feature vectors.To address the problems of unstable gradients and easily fall into a local minimum in the deep neural network training process,we propose to use Kaiming normal initialization with zero negative slopes of rectifi ed units and to adjust the network learning process by optimizing the mean square error(MSE)loss function with the introduction of a freezing factor.The experiments on testing dataset show that CNN-LSTM fusion deep neural network can predict RMS velocity as well as interval velocity more accurately,and its inversion accuracy is superior to that of single neural network models.The predictions on the complex structures and Marmousi model are consistent with the true velocity variation trends,and the predictions on fi eld data can eff ectively correct the phase axis,improve the lateral continuity of phase axis and quality of stack section,indicating the eff ectiveness and decent generalization capability of the proposed method.

基于声音与视觉特征多级融合的鱼类行为识别模型U-FusionNet-ResNet50+SENet

为解决在光线昏暗、声音与视觉噪声干扰等复杂条件下,单模态鱼类行为识别准确率和召回率低的问题,提出了基于声音和视觉特征多级融合的鱼类行为识别模型U-FusionNet-ResNet50+SENet,该方法采用ResNet50模型提取视觉模态特征,通过MFCC+RestNet50模型提取声音模态特征,并在此基础上设计一种U型融合架构,使不同维度的鱼类视觉和声音特征充分交互,在特征提取的各阶段实现特征融合,最后引入SENet构成关注通道信息特征融合网络,并通过对比试验,采用多模态鱼类行为的合成加噪试验数据验证算法的有效性。结果表明:U-FusionNet-ResNet50+SENet对鱼类行为识别准确率达到93.71%,F1值达到93.43%,召回率达到92.56%,与效果较好的已有模型Intermediate-feature-level deep model相比,召回率、F1值和准确率分别提升了2.35%、3.45%和3.48%。研究表明,所提出的U-FusionNet-ResNet50+SENet识别方法,可有效解决单模态鱼类行为识别准确率低的问题,提升了鱼类行为识别的整体效果,可以有效识别复杂条件下鱼类的游泳、摄食等行为,为真实生产条件下的鱼类行为识别研究提供了新思路和新方法。

基于T-Fusion的TFP3D人体行为识别算法

针对当前人体行为识别算法中双流卷积神经网络时效性差、3D卷积神经网络参数多、算法的复杂度高等不足,提出了基于3D卷积网络和时空融合网络的时空融合伪3D卷积神经网络模型TFP3D。首先,使用3D卷积拆分减少3D卷积核带来的庞大参数量;其次,增加时空融合模块T-Fusion,保证人体行为信息时空特征的有效传递;最后,使用Kinetics数据集对深层模型进行预训练,在保证准确率的前提下提升网络速率。在常见的人体行为识别数据集UCFl01上进行了大量的实验分析,并将识别的结果和当前流行的算法进行比较,结果证明所设计的TFP3D优于其他方法,平均识别率相比其他方法有较大的提高。

Application of Attributes Fusion Technology in Prediction of Deep Reservoirs in Paleogene of Bohai Sea

1 Introduction The Paleogene strata(with a depth of more than 2500m)in the Bohai sea is complex(Xu Changgui,2006),the reservoir buried deeply,the reservoir prediction is difficult(LAI Weicheng,XU Changgui,2012),and more

MFC-DeepLabV3+:一种多特征级联融合裂缝缺陷检测网络模型

道路裂缝对道路安全存在很大威胁,确保道路的安全性离不开对裂缝的准确检测。针对常规的人工检测方法和传统机器学习检测方法泛化性低且在复杂背景下裂缝分割检测准确率低等诸多问题,提出一种新型道路裂缝缺陷检测模型MFC-DeepLabV3+(MultiFeatureCascade-DeepLabV3+,多特征级联-DeepLabV3+)。首先,针对裂缝图像拓扑结构复杂,非均匀性强等问题,对主干特征提取网络进行改进,提出采用通道维度的分组卷积和分离注意力模块增强模型对裂缝图像特征提取能力,同时引入位置信息注意力机制提升对裂缝目标结构特征的精准定位,扩大网络各层特征信息的利用率。其次,加入多分支共享密集连接改进ASPP(AtrousSpatialPyramidPooling,空洞空间金字塔池化)模块,使其模仿人类视觉行为感知,在感受野保持均衡的同时生成密集覆盖裂缝尺度范围的特征语义信息。最后,在模型特征融合阶段增加多重边缘细化融合机制,使模型加大对高低阶特征信息的利用,提升模型对裂缝边缘精确分割的能力,防止裂缝轮廓边缘像素缺失。为验证MFC-DeepLabV3+模型的有效性,在公开路面裂缝数据集CRACK500与DeepCrack上进行实验,相较其他分割模型,在平均交并比上分别达到79.63%和76.99%,同时在主观视觉对比上预测出的裂缝分割图像边缘更加清晰,区域更加完整,表明该模型具有良好的工程应用价值。

以ChatGPT为代表的AI技术与高等教育的深度融合

随着人工智能技术飞速发展,以ChatGPT为代表的人工智能技术已经逐渐成为人们关注的焦点。旨在探讨ChatGPT技术与高等教育的深度融合研究,分析其对高等教育的影响和挑战,并提出相应的解决方案。首先,介绍了ChatGPT技术的基本原理和特点,包括其语言生成模型的构建、预训练和优化等;然后,通过分析ChatGPT技术在高等教育中的应用场景,如智能教育、在线教学、个性化推荐等,探讨了其对高等教育的影响和挑战;最后,提出了一些解决方案,比如加强ChatGPT技术与高等教育的深度融合、建立完善的教育监管机制等,以应对ChatGPT技术在高等教育中面临的挑战。

共生理论视角下应用型高校校企合作的现实困境及路径优化

全面深化应用型高校校企合作既是实现应用型高校高质量发展的重要举措,也是服务技能型社会建设的关键路径。应用型高校校企合作是一种以创新要素高效流动为核心,实现资源跨界整合和价值增值的双向投资和合作过程。根据共生理论的分析框架,当前应用型高校在深化校企合作过程中,共生单元价值取向分离,利益本位思想固化;共生模式内生动力不足,主体间合作互动疏离;共生环境支持不力,环境建设支撑度不足;共生界面信息传导受阻,资源要素流动通道不畅。为了进一步促进应用型高校校企深度融合,需完善校企合作共生单元,优化利益主体共生路径;创新校企合作共生模式,优化主体协同合作共生路径;营造校企合作共生环境,大力提升环境支撑力度;打通校企合作共生界面,优化资源要素共生路径。

虹膜与眼周深度特征融合网络模型

虹膜识别的识别率虽然很高,但单模态识别时受环境、欺骗攻击等影响,并且在远距离或移动端、较少约束等场景下,识别率会大大下降。利用位置与虹膜相近并且被研究证明鉴别性很强的眼周生物特征,将虹膜和眼周进行双模态融合识别是较好的思路。为实现精确自适应的融合识别,本文提出新颖的虹膜与眼周深度特征融合网络模型(MultipleFusionNet)。根据特征通道注意力和通道分组注意力的思想,设计自动权值生成网络,通过网络学习自动获得虹膜与眼周的权值。权值与卷积神经网络(CNN)生成的虹膜与眼周深度特征加权计算,可实现两个模态的深度特征动态精确融合,从而提高识别准确率。本文网络模型中融合部分可作为通用的深度特征融合模块使用,该模块可灵活地嵌入在任何CNN主干网络中,轻便且易于实现。在中国科学院公开的远距离虹膜库图像库CASIA-Iris-Distance和近距离光照变化虹膜图像库CASIA-Iris-Lamp上进行了实验验证,多种方法的对比实验和距离度量实验结果显示:本文的特征融合模型准确率最高为99.56%,采用余弦距离度量的等误率(EER)最低为0.002 7,优于单模态方法和相关的特征融合方法;计算复杂度方面,参数量和计算量比单模型的两倍少1.5%,计算量只比基准融合方法高1%,这表明该融合模型计算复杂度低,具有良好的性能。

Deep Convolutional Feature Fusion Model for Multispectral Maritime Imagery Ship Recognition

Combining both visible and infrared object information, multispectral data is a promising source data for automatic maritime ship recognition. In this paper, in order to take advantage of deep convolutional neural network and multispectral data, we model multispectral ship recognition task into a convolutional feature fusion problem, and propose a feature fusion architecture called Hybrid Fusion. We fine-tune the VGG-16 model pre-trained on ImageNet through three channels single spectral image and four channels multispectral images, and use existing regularization techniques to avoid over-fitting problem. Hybrid Fusion as well as the other three feature fusion architectures is investigated. Each fusion architecture consists of visible image and infrared image feature extraction branches, in which the pre-trained and fine-tuned VGG-16 models are taken as feature extractor. In each fusion architecture, image features of two branches are firstly extracted from the same layer or different layers of VGG-16 model. Subsequently, the features extracted from the two branches are flattened and concatenated to produce a multispectral feature vector, which is finally fed into a classifier to achieve ship recognition task. Furthermore, based on these fusion architectures, we also evaluate recognition performance of a feature vector normalization method and three combinations of feature extractors. Experimental results on the visible and infrared ship (VAIS) dataset show that the best Hybrid Fusion achieves 89.6% mean per-class recognition accuracy on daytime paired images and 64.9% on nighttime infrared images, and outperforms the state-of-the-art method by 1.4% and 3.9%, respectively.

基于特征强化U⁃Net的地震速度反演方法

基于深度神经网络的地震速度反演方法面临的挑战是:时间域地震数据与空间域模型信息间语义映射的弱对应关系导致多解性;神经网络将地震数据映射到速度模型过程中缺少有效引导,易受噪声干扰,影响反演精度。为此,提出一种基于特征强化U‑Net的地震速度反演方法。首先,通过多炮地震数据特征叠加使输入网络的地震时间序列信号与对应速度模型之间的空间关系更加明确;其次,基于多尺度特征融合的思想设计具有不同尺寸卷积核的模块,以增强网络对有效特征的学习能力;然后,利用注意力门引导网络,增强网络重点关注的特征;最后,结合瓶颈残差和预激活的思想,在网络中加入预激活瓶颈残差,避免梯度消失和网络退化。实验表明,该方法在地震速度反演方面具有更高的精度,并在抗噪声测试中效果较好,具有一定的泛化能力。

基于自适应纹理特征融合的纹理图像分类方法

现有基于深度学习的图像分类方法普遍缺少纹理特征的针对性,分类精度较低,难以同时适用于简单纹理和复杂纹理分类。提出一种基于自适应纹理特征融合的深度学习模型,能够结合类间差异性纹理特征做出分类决策。首先,根据纹理特征的最大类间差异性,构建图像的纹理特征图像;然后,采用原始图像与特征鲜明的纹理特征图像并行训练改进的双线性模型,获取双通道特征;最后,基于决策融合构建自适应分类模块,连接原图与纹理集的平均池化特征图进行通道权重提取,根据通道权重融合2个并行神经网络模型的分类向量,得到最优融合分类结果。在KTH-TIPS,KTH-TIPS-2b, UIUC和DTD 4个公共纹理数据集上对模型的分类性能进行评估,分别得到了99.98%、99.95%、99.99%和67.09%的准确率,表明所提模型具有普遍高效的识别性能。

雷达信号与遥感地图融合的深度学习低慢小目标检测算法

雷达复杂环境低慢小目标检测是一项具有挑战性的任务,而利用深度学习以及数据特征融合等方法是解决这一难题的有效手段。本文在雷达地图融合检测网络(Radar Map fusion Detection Network,RMDN)的基础上进行了优化,主要优化方向为将雷达与地图信息在检测过程中进行重要性程度区分,具体优化内容为减少地图特征提取模块的网络深度,加入通道注意力机制,让神经网络自主学习雷达信息与地图信息特征的权重,使神经网能够更好地利用地图信息对雷达目标进行辅助检测。在此优化基础上,本文重新设计出了雷达地图融合检测网络RMDN-V2。算法的主要思想为利用卫星遥感地图来提供背景环境信息,作为雷达信号检测的辅助,通过将目标背景中的特征信息融入检测决策中,提高目标检测的准确性和鲁棒性,减少对强杂波和移动物体的干扰敏感性,改善目标检测算法在复杂环境下的表现。最后的无人机雷达实测数据实验结果表明,本文所做的针对性优化是有效的,RMDN-V2的检测性能优于原始的RMDN,同时本文算法检测性能远超传统的雷达检测算法,同时也优于目前主流的一些深度学习雷达目标检测算法。本文为解决当下低慢小目标检测的难题提出了新的算法。

基于改进YOLOv网络的外观检测研究

外观检测涉及对图像或视频中的物体进行准确和高效的识别和定位,为了解决物体表面小尺寸目标检测的问题,研究通过优化YOLOv3网络模型,引入多尺度检测和深度可分离卷积技术来提高检测精度和模型效率,以增强对小尺寸目标的识别能力,再采用深度可分离卷积技术来减少计算量,并提高模型的训练效果;实验结果表明,研究模型在物体表面小尺寸检测方面取得显著提升;与其他金属表面损伤检测算法相比,优化后的YOLOv3实现了71.52%的检测精度,超越Faster R-CNN 6.83%;尽管Faster R-CNN在准确性方面优异但速度慢,SSD速度较快但不及YOLOv2;而YOLOv2虽速度快但精度稍低;相对于原始模型,研究算法的平均精度提升了7.77个百分点,达到了79.21%;虽然网络深度的提升稍增计算量,略有检测速率下降,但引入深度可分离卷积后,检测速度达到36.2帧/秒,仅较原模型稍低2.4帧/秒;研究可以优化算法,提高小尺寸目标检测的准确性和鲁棒性,推动其在计算机视觉领域的广泛应用。

基于SE模块的神经协同过滤

基于传统推荐方法对辅助信息利用不足,为优化用户、项目间内在联系挖掘有限等问题,采用将Squeeze-and-Excitation Networks结构嵌入神经协同过滤的方法提出SE-NCF模型,利用SE模块学习权重,去除不同特征中权重较低的噪声来实现特征融合,通过神经协同过滤层获得用户-项目间的线性与非线性关系,实现模型优化。通过茶评与Amazon_Food两公开数据集对同类推荐方法进行实验,实验结果表明,相比于原神经协同过滤,SE-NCF模型在两数据集中MSE指标与NDCG指标均得到改善,在茶评数据集下MSE降低10%,NDCG提升5.1%;在Amazon_Food下MSE降低4.3%,NDCG提升9.3%。