A Lightweight Convolutional Neural Network with Hierarchical Multi-Scale Feature Fusion for Image Classification

Convolutional neural networks (CNNs) are widely used in image classification tasks, but their increasing model size and computation make them challenging to implement on embedded systems with constrained hardware resources. To address this issue, the MobileNetV1 network was developed, which employs depthwise convolution to reduce network complexity. MobileNetV1 employs a stride of 2 in several convolutional layers to decrease the spatial resolution of feature maps, thereby lowering computational costs. However, this stride setting can lead to a loss of spatial information, particularly affecting the detection and representation of smaller objects or finer details in images. To maintain the trade-off between complexity and model performance, a lightweight convolutional neural network with hierarchical multi-scale feature fusion based on the MobileNetV1 network is proposed. The network consists of two main subnetworks. The first subnetwork uses a depthwise dilated separable convolution (DDSC) layer to learn imaging features with fewer parameters, which results in a lightweight and computationally inexpensive network. Furthermore, depthwise dilated convolution in DDSC layer effectively expands the field of view of filters, allowing them to incorporate a larger context. The second subnetwork is a hierarchical multi-scale feature fusion (HMFF) module that uses parallel multi-resolution branches architecture to process the input feature map in order to extract the multi-scale feature information of the input image. Experimental results on the CIFAR-10, Malaria, and KvasirV1 datasets demonstrate that the proposed method is efficient, reducing the network parameters and computational cost by 65.02% and 39.78%, respectively, while maintaining the network performance compared to the MobileNetV1 baseline.

An infrared and visible image fusion method based upon multi-scale and top-hat transforms

The high-frequency components in the traditional multi-scale transform method are approximately sparse, which can represent different information of the details. But in the low-frequency component, the coefficients around the zero value are very few, so we cannot sparsely represent low-frequency image information. The low-frequency component contains the main energy of the image and depicts the profile of the image. Direct fusion of the low-frequency component will not be conducive to obtain highly accurate fusion result. Therefore, this paper presents an infrared and visible image fusion method combining the multi-scale and top-hat transforms. On one hand, the new top-hat-transform can effectively extract the salient features of the low-frequency component. On the other hand, the multi-scale transform can extract highfrequency detailed information in multiple scales and from diverse directions. The combination of the two methods is conducive to the acquisition of more characteristics and more accurate fusion results. Among them, for the low-frequency component, a new type of top-hat transform is used to extract low-frequency features, and then different fusion rules are applied to fuse the low-frequency features and low-frequency background; for high-frequency components, the product of characteristics method is used to integrate the detailed information in high-frequency. Experimental results show that the proposed algorithm can obtain more detailed information and clearer infrared target fusion results than the traditional multiscale transform methods. Compared with the state-of-the-art fusion methods based on sparse representation, the proposed algorithm is simple and efficacious, and the time consumption is significantly reduced.

A novel image fusion algorithm based on 2D scale-mixing complex wavelet transform and Bayesian MAP estimation for multimodal medical images

In this paper,we propose a new image fusion algorithm based on two-dimensional Scale-Mixing Complex Wavelet Transform(2D-SMCWT).The fusion of the detail 2D-SMCWT cofficients is performed via a Bayesian Maximum a Posteriori(MAP)approach by considering a trivariate statistical model for the local neighboring of 2D-SMCWT coefficients.For the approx imation coefficients,a new fusion rule based on the Principal Component Analysis(PCA)is applied.We conduct several experiments using three different groups of multimodal medical images to evaluate the performance of the proposed method.The obt ained results prove the superiority of the proposed method over the state of the art fusion methods in terms of visual quality and several commonly used metrics.Robustness of the proposed method is further tested against different types of noise.The plots of fusion met rics establish the accuracy of the proposed fusion method.

DS-TransFusion:基于改进Swin Transformer的视网膜血管自动分割

视网膜血管的准确分割在辅助筛查各种疾病方面具有重大意义.然而,当前流行的模型仍存在细小血管的分割不清晰,以及眼底血管分支末端与背景的对比度较低等问题.针对这些问题,本文提出了一种全新的视网膜血管分割模型,命名为Dual Swin Transformer Fusion(DS-TransFusion).首先,DS-TransFusion采用基于Swin Transformer的双尺度编码器子网络,以提取视网膜血管的粗粒度和细粒度特征.其次,在跳跃连接处引入了Transformer交互融合注意力(TIFA)模块,用于丰富跨视图上下文建模和语义依赖,同时捕获来自不同图像视图的数据之间的长期相关性.最后,在编码器和解码器之间,DS-TransFusion采用了多尺度注意力(MA),用于收集多尺度特征表示的全局对应关系,进一步优化模型的分割效果.实验结果表明,DS-TransFusion在公共数据集STARE、CHASEDB1和DRIVE上表现出色,准确率分别达到了96.50%、97.22%和97.80%,灵敏度达到84.10%、84.55%和83.17%.实验表明DS-TransFusion能有效提高视网膜血管分割的精度,准确分割出细小血管.对视网膜血管分割的准确度、灵敏度和特异性都有大幅提高,与现有的SOTA方法相比具有更好的分割性能.

融合Multi-scale CNN和Bi-LSTM的人脸表情识别研究

为了有效改善现有人脸表情识别模型中存在信息丢失严重、特征信息之间联系不密切的问题,提出一种融合多尺度卷积神经网络(Multi-scale CNN)和双向长短期记忆(Bi-LSTM)的模型。Bi-LSTM可以增强特征信息间的联系与信息的维持,在Multi-scale CNN中通过不同尺度的卷积核可以提取到更加丰富的特征信息,并通过加入批标准化(BN)层与特征融合处理,从而加快网络的收敛速度,有利于特征信息的重利用,再将两者提取到的特征信息进行融合,最后将改进的正则化方法应用到目标函数中,减小网络复杂度和过拟合。在JAFFE和FER-2013公开数据集上进行实验,准确率分别达到了95.455%和74.115%,由此证明所提算法的有效性和先进性。

Triple Multimodal Cyclic Fusion and Self-Adaptive Balancing for Video Q&A Systems

Performance of Video Question and Answer(VQA)systems relies on capturing key information of both visual images and natural language in the context to generate relevant questions’answers.However,traditional linear combinations of multimodal features focus only on shallow feature interactions,fall far short of the need of deep feature fusion.Attention mechanisms were used to perform deep fusion,but most of them can only process weight assignment of single-modal information,leading to attention imbalance for different modalities.To address above problems,we propose a novel VQA model based on Triple Multimodal feature Cyclic Fusion(TMCF)and Self-AdaptiveMultimodal Balancing Mechanism(SAMB).Our model is designed to enhance complex feature interactions among multimodal features with cross-modal information balancing.In addition,TMCF and SAMB can be used as an extensible plug-in for exploring new feature combinations in the visual image domain.Extensive experiments were conducted on MSVDQA and MSRVTT-QA datasets.The results confirm the advantages of our approach in handling multimodal tasks.Besides,we also provide analyses for ablation studies to verify the effectiveness of each proposed component.

Lumbar spine localisation method based on feature fusion

To eliminate unnecessary background information,such as soft tissues in original CT images and the adverse impact of the similarity of adjacent spines on lumbar image segmentation and surgical path planning,a two‐stage approach for localising lumbar segments is proposed.First,based on the multi‐scale feature fusion technology,a non‐linear regression method is used to achieve accurate localisation of the overall spatial region of the lumbar spine,effectively eliminating useless background information,such as soft tissues.In the second stage,we directly realised the precise positioning of each segment in the lumbar spine space region based on the non‐linear regression method,thus effectively eliminating the interference caused by the adjacent spine.The 3D Intersection over Union(3D_IOU)is used as the main evaluation indicator for the positioning accuracy.On an open dataset,3D_IOU values of 0.8339�0.0990 and 0.8559�0.0332 in the first and second stages,respectively is achieved.In addition,the average time required for the proposed method in the two stages is 0.3274 and 0.2105 s respectively.Therefore,the proposed method performs very well in terms of both pre-cision and speed and can effectively improve the accuracy of lumbar image segmentation and the effect of surgical path planning.

Maintenance time of sedative effects after an intravenous infusion of diazepam: A guide for endoscopy using diazepam

AIM: To examine whether the sedative effects assessed by psychomotor tests would depend on the cytochrome P450 (CYP ) 2C19 genotypes after an infusion regimen of diazepam commonly used forgastrointestinal endoscopy in Japan. METHODS: Fifteen healthy Japanese volunteers consisting of three different CYP2C19 genotype groups underwent a critical ? icker fusion test, an eye movement analysis and a postural sway test as a test for physical sedative effects, and a visual analog scale (VAS) symptom assessment method as a test for mental sedative effects during the 336 h period after the intravenous infusion of diazepam (5 mg). RESULTS: The physical sedative effects assessed by the critical flicker test continued for 1 h (t values of 5 min, 30 min and 60 min later: 4.35, 5.00 and 3.19, respectively) and those by the moving radial area of a postural sway test continued for 3 h (t values of 5 h, 30 h, 60 min and 3 h later: -4.05, -3.42, -2.17 and -2.58, respectively), which changed significantly compared with the baseline level before infusion (P < 0.05). On the other hand, the mental sedative effects by the VAS method improved within 1 h. The CYP2C19 genotype-dependent differences in the postinfusion sedative effects were not observed in any of the four psychomotor function tests. CONCLUSION: With the psychomotor tests, the objective sedative effects of diazepam continued for 1 h to 3 h irrespective of CYP2C19 genotype status and the subjective sedative symptoms improved within 1 h. Up to 3 h of clinical care appears to be required after the infusion of diazepam, although patients feel subjectively improved.

基于改进DETR的机器人铆接缺陷检测方法研究

铆接作为铁道车辆结构件的主要连接方式,合格的铆接质量是车辆安全稳定运行的重要保证。针对现有铆接缺陷检测方法存在检测精度低、检测点位少、检测智能化水平不高等问题,提出一种基于改进DETR的机器人铆接缺陷检测方法。首先,搭建铆接缺陷检测系统,依次采集工件尺寸大、铆钉尺寸小工况下的铆接缺陷图像。其次,为了增强DETR模型在小目标中的图像特征提取能力和检测性能,以EfficientNet作为DETR中的主干特征提取网络,并将3-D权重注意力机制SimAM引入EfficientNet网络,从而有效保留图像特征层的镦头形态信息和铆点区域的空间信息。然后,在颈部网络中引入加权双向特征金字塔模块,以EfficientNet网络的输出作为特征融合模块的输入对各尺度特征信息进行聚合,增大不同铆接缺陷的类间差异。最后,利用Smooth L1和DIoU的线性组合改进原模型预测网络的回归损失函数,提高模型的检测精度和收敛速度。结果表明,改进模型表现出较高的检测性能,对于铆接缺陷的平均检测精度mAP为97.12%,检测速度FPS为25.4帧/s,与Faster RCNN、YOLOX等其他主流检测模型相比,在检测精度和检测速度方面均具有较大优势。研究结果能够满足实际工况中大型铆接件的小尺寸铆钉铆接缺陷实时在线检测的需求,为视觉检测技术在铆接工艺中的应用提供一定的参考价值。

三维成矿预测关键问题

三维成矿预测是当前深部找矿预测和勘查的重要方法和手段,其方法体系和实践应用均已取得大量成果,但同时存在若干关键科学技术问题,导致其进一步发展受到制约。本文从多尺度三维成矿预测方法体系不完善、不确定性分析与优化研究薄弱、三维成矿预测要素挖掘存在瓶颈、缺少针对三维成矿预测的三维深度学习模型和方法等关键问题出发,对目前三维成矿预测领域相关方面的研究进展进行综合分析,并提出针对上述关键问题可能的解决方案和研究方向。预期未来三维成矿预测领域的研究工作将创新发展出多种方法,实现对三维预测信息的深度挖掘;构建形成适用的三维深度学习模型和训练方法,有效增强三维成矿预测结果的预测能力;通过系统性地开展三维成矿预测不确定性研究,进一步优化预测过程和结果,有效提高三维成矿预测方法的可靠性和准确性;形成面向多尺度三维成矿预测的方法体系,更有效地指导矿集区-矿田-勘查区块(矿床)等不同级别的深部矿产资源找矿勘查工作。相关关键问题的解决将进一步深化和完善三维成矿预测理论和方法体系,促进三维成矿预测理论方法的实践应用,显著提升深部找矿预测和勘查工作的效率与水平,助力深部找矿突破。

一种时频尺度下的多元短期电力负荷组合预测方法

随机因素的增加导致电力负荷数据成分日渐复杂,使短期负荷预测的难度逐渐增大。针对该问题,提出一种时频尺度下的时间卷积网络与多元线性回归相融合的组合预测模型。利用自适应噪声完备集合经验模态分解(complete ensemble empirical mode decomposition with adaptive noise,CEEMDAN)在时频域上将负荷数据分解为若干个频率特征不同的本征模态分量,在模糊熵准则下聚类为随机项和趋势项。采用皮尔逊系数从诸多影响因素中筛选出与电力负荷高度相关的特征,鉴于小时间尺度分析更易于挖掘局部细节特征,分别构建了随机项与趋势项的细颗粒度特征集。利用具有强非线性处理能力的时间卷积网络(temporal convolutional network,TCN)去预测随机项,利用结构简单及线性拟合效果好的多元线性回归(multiplelinearregression,MLR)去预测趋势项,将二者的预测结果进行叠加重构后获得最终预测值。在新加坡和比利时两组数据集上的实验结果证明:所提模型具有较高的预测精度、较好的泛化性能及鲁棒性。

一种基于SAM-MSFF网络的低照度目标检测方法

由于低照度图像具有对比度低、细节丢失严重、噪声大等缺点,现有的目标检测算法对低照度图像的检测效果不理想.为此,本文提出一种结合空间感知注意力机制和多尺度特征融合(Spatial-aware Attention Mechanism and Multi-Scale Feature Fusion,SAM-MSFF)的低照度目标检测方法 .该方法首先通过多尺度交互内存金字塔融合多尺度特征,增强低照度图像特征中的有效信息,并设置内存向量存储样本的特征,捕获样本之间的潜在关联性;然后,引入空间感知注意力机制获取特征在空间域的长距离上下文信息和局部信息,从而增强低照度图像中的目标特征,抑制背景信息和噪声的干扰;最后,利用多感受野增强模块扩张特征的感受野,对具有不同感受野的特征进行分组重加权计算,使检测网络根据输入的多尺度信息自适应地调整感受野的大小.在ExDark数据集上进行实验,本文方法的平均精度(mean Average Precision,mAP)达到77.04%,比现有的主流目标检测方法提高2.6%~14.34%.

多尺度融合与动态自适应图的公交客流预测模型

公交客流预测是公共交通规划和管理中的重要问题。虽然时空图卷积在地铁客流预测任务中获得了很好的预测效果,但是面对公交更复杂的线路、大规模的节点数据,现有的基于图卷积的空间建模方法将带来巨大的空间内存消耗。同时,公交客流量短时间范围内更可能受到瞬时交通状况的影响。为了解决这些挑战,提出了一种多尺度融合和动态自适应图的公交客流预测模型(MFDAG)。该模型融合客流、时刻和周信息以增加数据的特征维度,用动态自适应图的方法来学习不同站点之间的关系。进一步提出了一种多尺度融合传播的方法来表示复杂的空间依赖关系,同时设计了一种多尺度卷积传播的方法来学习不同尺度的时间依赖关系。在两个真实的客流数据集上进行了实验,并与其他交通预测方法进行了比较。实验结果表明,所提出的多尺度融合和动态自适应图的公交客流预测方法具有更高的预测准确度。

基于改进的YOLOv5安全帽佩戴检测算法

针对安全帽佩戴检测中存在的误检和漏检的问题,提出一种基于YOLOv5模型改进的安全帽佩戴检测算法。改进模型引入多尺度加权特征融合网络,即在YOLOv5的网络结构中增加一个浅层检测尺度,并引入特征权重进行加权融合,构成新的四尺检测结构,有效地提升图像浅层特征的提取及融合能力;在YOLOv5的Neck网络的BottleneckCSP结构中加入SENet模块,使模型更多地关注目标信息忽略背景信息;针对大分辨率的图像,添加图像切割层,避免多倍下采样造成的小目标特征信息大量丢失。对YOLOv5模型进行改进之后,通过自制的安全帽数据集进行训练检测,mAP和召回率分别达到97.06%、92.54%,与YOLOv5相比较分别提升了4.74%和4.31%。实验结果表明:改进的YOLOv5算法可有效提升安全帽佩戴的检测性能,能够准确识别施工人员的安全帽佩戴情况,从而大大降低施工现场的安全风险。

基于改进DeepLabv3+的轻量化作物杂草识别方法

为在存储资源与计算能力有限的设备上实现田间作物和杂草的识别,本文提出一种基于改进DeepLabv3+的轻量化语义分割网络。首先,以MobileNet v2作为DeepLabv3+的特征提取骨干网络,提出双分支残差模块替换倒残差模块,并删除后两层卷积以降低模型参数量。其次,在空洞空间金字塔池化(Atrous Spatial Pyramid Pooling,ASPP)模块中引入分组逐点卷积,使用深度扩张卷积替换标准卷积,并将卷积后的特征图进行多尺度特征融合增强对作物和杂草深层特征的提取能力。最后,将原有的非线性激活函数替换为Leaky ReLU激活函数来提升分割精度。实验结果表明:改进后网络的mIOU达到86.75%,参数量仅为0.69M,FPS达到了98,与原始DeepLabv3+以及3个典型轻量化语义分割网络的相比,参数量最小,在对比的轻量化网络中具有最高的分割精度。

基于多尺度注意力特征融合的场景文本检测

针对目前文本检测中小尺度文本和长文本检测精度低的问题,提出了一种基于多尺度注意力特征融合的场景文本检测算法。该方法以Mask R-CNN为基线模型,引入Swin_Transformer作为骨干网络提取底层特征。在特征金字塔(feature pyramid networks,FPN)中,通过将多尺度注意力热图与底层特征通过横向连接相融合,使检测器的不同层级专注于特定尺度的目标,并利用相邻层注意力热图之间的关系实现了FPN结构中的纵向特征共享,避免了不同层之间梯度计算的不一致性问题。实验结果表明:在ICDAR2015数据集上,该方法的准确率、召回率和F值分别达到了88.3%、83.07%和85.61%,在CTW1500和Total-Text弯曲文本数据集上相较现有方法均有良好表现。

基于多尺度特征信息融合的时间序列异常检测

目前大多数的时间序列都缺少相应的异常标签,且现有基于重构的异常检测算法不能很好地捕获到多维数据间复杂的潜在相关性和时间依赖性,为了构建特征丰富的时间序列,提出一种多尺度特征信息融合的异常检测模型。该模型首先通过卷积神经网络对滑动窗口内的不同序列进行特征卷积来获取不同尺度下的局部上下文信息。然后,利用Transformer中的位置编码对卷积后的时间序列窗口进行位置嵌入,增强滑动窗口中每一个时间序列和邻近序列之间的位置联系,并引入时间注意力获取数据在时间维度上的自相关性,并进一步通过多头自注意力自适应地为窗口内不同时间序列分配不同的权重。最后,对反卷积过程中上采样得到的窗口数据与不同尺度下得到的局部特征和时间上下文信息进行逐步融合,从而准确重构原始时间序列,并将重构误差作为最终的异常得分进行异常判定。实验结果表明,所构建模型在SWaT和SMD数据集上与基线模型相比F1分数均有所提升。在数据维度高且均衡性较差的WADI数据集上与GDN模型相比F1分数降低了1.66%。

基于改进YOLOv网络的外观检测研究

外观检测涉及对图像或视频中的物体进行准确和高效的识别和定位,为了解决物体表面小尺寸目标检测的问题,研究通过优化YOLOv3网络模型,引入多尺度检测和深度可分离卷积技术来提高检测精度和模型效率,以增强对小尺寸目标的识别能力,再采用深度可分离卷积技术来减少计算量,并提高模型的训练效果;实验结果表明,研究模型在物体表面小尺寸检测方面取得显著提升;与其他金属表面损伤检测算法相比,优化后的YOLOv3实现了71.52%的检测精度,超越Faster R-CNN 6.83%;尽管Faster R-CNN在准确性方面优异但速度慢,SSD速度较快但不及YOLOv2;而YOLOv2虽速度快但精度稍低;相对于原始模型,研究算法的平均精度提升了7.77个百分点,达到了79.21%;虽然网络深度的提升稍增计算量,略有检测速率下降,但引入深度可分离卷积后,检测速度达到36.2帧/秒,仅较原模型稍低2.4帧/秒;研究可以优化算法,提高小尺寸目标检测的准确性和鲁棒性,推动其在计算机视觉领域的广泛应用。

结合轻量化与多尺度融合的交通标志检测算法

交通标志检测在自动驾驶领域具有重要的应用价值,及时准确地检测交通目标对提高驾驶安全性和预防交通事故具有重要意义。针对交通标志尺寸小,易受遮挡,在复杂环境下容易出现漏检、错检等问题,在YOLOv8的结构基础上提出一种结合轻量化与多尺度融合的交通标志检测网络架构M-YOLO,构建M-YOLOs模型来应对高精度需求的检测任务,并调整网络深度得到更轻量化的M-YOLOn模型来解决不同环境下的检测需求。首先针对交通标志目标尺寸小、图像特征流失的问题,通过增加小目标检测层,保留更多的特征信息,提高网络对于小目标的特征学习能力。提出高效多尺度特征金字塔融合网络MPANet,将浅层特征图进行降维与跳跃连接,从而融合更多的图像特征信息。然后提出融合稀疏注意力和空间注意力的BRSA注意力模块,有效提取全局和局部的位置信息,减少复杂背景下对于关键信息的干扰。最后设计两种轻量高效的BBot模块和C2fGhost模块,以提高模型运算速度并减少参数量。实验结果表明,M-YOLO相较于YOLOv8,参数量降低约1/3。在TT100K数据集和GTSDB数据集上,M-YOLOs检测精度分别提升了9.7和2.1个百分点,M-YOLOn检测精度分别提升了14.5和2.6个百分点,在轻量化的同时具备更高的检测效果。M-YOLO架构解决了浅层特征图在特征提取过程中信息丢失的问题,并显著降低模型特征提取过程中冗余的计算开销,在实景采集的数据集上证实效果有效,表明在交通标志检测任务中具有应用价值。

考虑风电不确定性的电气综合能源系统混合尺度调控

为改善电-气互联综合能源系统中风电出力不确定性和多能传输差异对调控过程的影响,提出了基于改进小波融合算法的混合尺度调控方法.首先采用区间数学的方法,对系统中风电功率不确定性进行表示并给出风电处理策略.其次,考虑到不同能源传输特性的差异,提出了改进的小波融合算法,即先对电力网络中传感器信号数据进行多个不同小波基的多尺度分解,再对天然气系统信号数据中使用相同小波基分解的信号在混合尺度上实施加权数据融合,进行不同小波基的逆变换后得到融合信号.最后基于所搭建仿真模型,对比分析了不同调控方法的调控效果.结果表明本文所提方法的调控结果优于DMPC(分布式模型预测控制)滚动优化调控结果,且在改善了系统运行经济性的同时也提高了系统稳定性.