A Lightweight Convolutional Neural Network with Hierarchical Multi-Scale Feature Fusion for Image Classification

Convolutional neural networks (CNNs) are widely used in image classification tasks, but their increasing model size and computation make them challenging to implement on embedded systems with constrained hardware resources. To address this issue, the MobileNetV1 network was developed, which employs depthwise convolution to reduce network complexity. MobileNetV1 employs a stride of 2 in several convolutional layers to decrease the spatial resolution of feature maps, thereby lowering computational costs. However, this stride setting can lead to a loss of spatial information, particularly affecting the detection and representation of smaller objects or finer details in images. To maintain the trade-off between complexity and model performance, a lightweight convolutional neural network with hierarchical multi-scale feature fusion based on the MobileNetV1 network is proposed. The network consists of two main subnetworks. The first subnetwork uses a depthwise dilated separable convolution (DDSC) layer to learn imaging features with fewer parameters, which results in a lightweight and computationally inexpensive network. Furthermore, depthwise dilated convolution in DDSC layer effectively expands the field of view of filters, allowing them to incorporate a larger context. The second subnetwork is a hierarchical multi-scale feature fusion (HMFF) module that uses parallel multi-resolution branches architecture to process the input feature map in order to extract the multi-scale feature information of the input image. Experimental results on the CIFAR-10, Malaria, and KvasirV1 datasets demonstrate that the proposed method is efficient, reducing the network parameters and computational cost by 65.02% and 39.78%, respectively, while maintaining the network performance compared to the MobileNetV1 baseline.

HOG-VGG:VGG Network with HOG Feature Fusion for High-Precision PolSAR Terrain Classification

This article proposes a VGG network with histogram of oriented gradient(HOG) feature fusion(HOG-VGG) for polarization synthetic aperture radar(PolSAR) image terrain classification.VGG-Net has a strong ability of deep feature extraction,which can fully extract the global deep features of different terrains in PolSAR images,so it is widely used in PolSAR terrain classification.However,VGG-Net ignores the local edge & shape features,resulting in incomplete feature representation of the PolSAR terrains,as a consequence,the terrain classification accuracy is not promising.In fact,edge and shape features play an important role in PolSAR terrain classification.To solve this problem,a new VGG network with HOG feature fusion was specifically proposed for high-precision PolSAR terrain classification.HOG-VGG extracts both the global deep semantic features and the local edge & shape features of the PolSAR terrains,so the terrain feature representation completeness is greatly elevated.Moreover,HOG-VGG optimally fuses the global deep features and the local edge & shape features to achieve the best classification results.The superiority of HOG-VGG is verified on the Flevoland,San Francisco and Oberpfaffenhofen datasets.Experiments show that the proposed HOG-VGG achieves much better PolSAR terrain classification performance,with overall accuracies of 97.54%,94.63%,and 96.07%,respectively.

Grid Side Distributed Energy Storage Cloud Group End Region Hierarchical Time-Sharing Configuration Algorithm Based onMulti-Scale and Multi Feature Convolution Neural Network

There is instability in the distributed energy storage cloud group end region on the power grid side.In order to avoid large-scale fluctuating charging and discharging in the power grid environment and make the capacitor components showa continuous and stable charging and discharging state,a hierarchical time-sharing configuration algorithm of distributed energy storage cloud group end region on the power grid side based on multi-scale and multi feature convolution neural network is proposed.Firstly,a voltage stability analysis model based onmulti-scale and multi feature convolution neural network is constructed,and the multi-scale and multi feature convolution neural network is optimized based on Self-OrganizingMaps(SOM)algorithm to analyze the voltage stability of the cloud group end region of distributed energy storage on the grid side under the framework of credibility.According to the optimal scheduling objectives and network size,the distributed robust optimal configuration control model is solved under the framework of coordinated optimal scheduling at multiple time scales;Finally,the time series characteristics of regional power grid load and distributed generation are analyzed.According to the regional hierarchical time-sharing configuration model of“cloud”,“group”and“end”layer,the grid side distributed energy storage cloud group end regional hierarchical time-sharing configuration algorithm is realized.The experimental results show that after applying this algorithm,the best grid side distributed energy storage configuration scheme can be determined,and the stability of grid side distributed energy storage cloud group end region layered timesharing configuration can be improved.

Hierarchical particle filter tracking algorithm based on multi-feature fusion

A hierarchical particle filter（HPF） framework based on multi-feature fusion is proposed.The proposed HPF effectively uses different feature information to avoid the tracking failure based on the single feature in a complicated environment.In this approach,the Harris algorithm is introduced to detect the corner points of the object,and the corner matching algorithm based on singular value decomposition is used to compute the firstorder weights and make particles centralize in the high likelihood area.Then the local binary pattern（LBP） operator is used to build the observation model of the target based on the color and texture features,by which the second-order weights of particles and the accurate location of the target can be obtained.Moreover,a backstepping controller is proposed to complete the whole tracking system.Simulations and experiments are carried out,and the results show that the HPF algorithm with the backstepping controller achieves stable and accurate tracking with good robustness in complex environments.

Medical image fusion based on pulse coupled neural networks and multi-feature fuzzy clustering

Medical image fusion plays an important role in clinical applications such as image-guided surgery, image-guided radiotherapy, noninvasive diagnosis, and treatment planning. In order to retain useful information and get more reliable results, a novel medical image fusion algorithm based on pulse coupled neural networks (PCNN) and multi-feature fuzzy clustering is proposed, which makes use of the multi-feature of image and combines the advantages of the local entropy and variance of local entropy based PCNN. The results of experiments indicate that the proposed image fusion method can better preserve the image details and robustness and significantly improve the image visual effect than the other fusion methods with less information distortion.

Feature Layer Fusion of Linear Features and Empirical Mode Decomposition of Human EMG Signal

To explore the influence of the fusion of different features on recognition,this paper took the electromyography(EMG)signals of rectus femoris under different motions(walk,step,ramp,squat,and sitting)as samples,linear features(time-domain features(variance(VAR)and root mean square(RMS)),frequency-domain features(mean frequency(MF)and mean power frequency(MPF)),and nonlinear features(empirical mode decomposition(EMD))of the samples were extracted.Two feature fusion algorithms,the series splicing method and complex vector method,were designed,which were verified by a double hidden layer(BP)error back propagation neural network.Results show that with the increase of the types and complexity of feature fusions,the recognition rate of the EMG signal to actions is gradually improved.When the EMG signal is used in the series splicing method,the recognition rate of time-domain+frequency-domain+empirical mode decomposition(TD+FD+EMD)splicing is the highest,and the average recognition rate is 92.32%.And this rate is raised to 96.1%by using the complex vector method,and the variance of the BP system is also reduced.

Digital modulation classification using multi-layer perceptron and time-frequency features

Considering that real communication signals corrupted by noise are generally nonstationary, and timefrequency distributions are especially suitable for the analysis of nonstationary signals, time-frequency distributions are introduced for the modulation classification of communication signals： The extracted time-frequency features have good classification information, and they are insensitive to signal to noise ratio （SNR） variation. According to good classification by the correct rate of a neural network classifier, a multilayer perceptron （MLP） classifier with better generalization, as well as, addition of time-frequency features set for classifying six different modulation types has been proposed. Computer simulations show that the MLP classifier outperforms the decision-theoretic classifier at low SNRs, and the classification experiments for real MPSK signals verify engineering significance of the MLP classifier.

Multi-Feature Fusion Based Relative Pose Adaptive Estimation for On-Orbit Servicing of Non-Cooperative Spacecraft

On-orbit servicing, such as spacecraft maintenance, on-orbit assembly, refueling, and de-orbiting, can reduce the cost of space missions, improve the performance of spacecraft, and extend its life span. The relative state between the servicing and target spacecraft is vital for on-orbit servicing missions, especially the final approaching stage. The major challenge of this stage is that the observed features of the target are incomplete or are constantly changing due to the short distance and limited Field of View (FOV) of camera. Different from cooperative spacecraft, non-cooperative target does not have artificial feature markers. Therefore, contour features, including triangle supports of solar array, docking ring, and corner points of the spacecraft body, are used as the measuring features. To overcome the drawback of FOV limitation and imaging ambiguity of the camera, a "selfie stick" structure and a self-calibration strategy were implemented, ensuring that part of the contour features could be observed precisely when the two spacecraft approached each other. The observed features were constantly changing as the relative distance shortened. It was difficult to build a unified measurement model for different types of features, including points, line segments, and circle. Therefore, dual quaternion was implemented to model the relative dynamics and measuring features. With the consideration of state uncertainty of the target, a fuzzy adaptive strong tracking filter( FASTF) combining fuzzy logic adaptive controller (FLAC) with strong tracking filter(STF) was designed to robustly estimate the relative states between the servicing spacecraft and the target. Finally, the effectiveness of the strategy was verified by mathematical simulation. The achievement of this research provides a theoretical and technical foundation for future on-orbit servicing missions.

A deep learning fusion model for accurate classification of brain tumours in Magnetic Resonance images

Detecting brain tumours is complex due to the natural variation in their location, shape, and intensity in images. While having accurate detection and segmentation of brain tumours would be beneficial, current methods still need to solve this problem despite the numerous available approaches. Precise analysis of Magnetic Resonance Imaging (MRI) is crucial for detecting, segmenting, and classifying brain tumours in medical diagnostics. Magnetic Resonance Imaging is a vital component in medical diagnosis, and it requires precise, efficient, careful, efficient, and reliable image analysis techniques. The authors developed a Deep Learning (DL) fusion model to classify brain tumours reliably. Deep Learning models require large amounts of training data to achieve good results, so the researchers utilised data augmentation techniques to increase the dataset size for training models. VGG16, ResNet50, and convolutional deep belief networks networks extracted deep features from MRI images. Softmax was used as the classifier, and the training set was supplemented with intentionally created MRI images of brain tumours in addition to the genuine ones. The features of two DL models were combined in the proposed model to generate a fusion model, which significantly increased classification accuracy. An openly accessible dataset from the internet was used to test the model's performance, and the experimental results showed that the proposed fusion model achieved a classification accuracy of 98.98%. Finally, the results were compared with existing methods, and the proposed model outperformed them significantly.

DS-TransFusion:基于改进Swin Transformer的视网膜血管自动分割

视网膜血管的准确分割在辅助筛查各种疾病方面具有重大意义.然而,当前流行的模型仍存在细小血管的分割不清晰,以及眼底血管分支末端与背景的对比度较低等问题.针对这些问题,本文提出了一种全新的视网膜血管分割模型,命名为Dual Swin Transformer Fusion(DS-TransFusion).首先,DS-TransFusion采用基于Swin Transformer的双尺度编码器子网络,以提取视网膜血管的粗粒度和细粒度特征.其次,在跳跃连接处引入了Transformer交互融合注意力(TIFA)模块,用于丰富跨视图上下文建模和语义依赖,同时捕获来自不同图像视图的数据之间的长期相关性.最后,在编码器和解码器之间,DS-TransFusion采用了多尺度注意力(MA),用于收集多尺度特征表示的全局对应关系,进一步优化模型的分割效果.实验结果表明,DS-TransFusion在公共数据集STARE、CHASEDB1和DRIVE上表现出色,准确率分别达到了96.50%、97.22%和97.80%,灵敏度达到84.10%、84.55%和83.17%.实验表明DS-TransFusion能有效提高视网膜血管分割的精度,准确分割出细小血管.对视网膜血管分割的准确度、灵敏度和特异性都有大幅提高,与现有的SOTA方法相比具有更好的分割性能.

Multi-layer perception approach to identification of compound information

Presents a novel approach of multi layer sensing for perception of high level environmental information related to many conventional physical quantities, such as temperature, humidity and brightness, which focuses on the processing of multi functional variables in a multi layer framework, and consists of multi functional sensing and multi layer fusion. Concerning the first aspect, a CdS and Fe 3O 4 materials based multi function sensor has been developed to measure the three quantities, and provides a possible solution to the sensor multi functional measurement equations, especially when the sensor processes more than three quantities, and proposes ways to evaluate the concerned environment as degree of comfort, Quantity Creditability Tactics (QCT) of multi layer data fusion.

A Multi-Channel Fusion Based Newborn Seizure Detection

We propose and compare two multi-channel fusion schemes to utilize the information extracted from simultaneously recorded multiple newborn electroencephalogram (EEG) channels for seizure detection. The first approach is known as the multi-channel feature fusion. It involves concatenating EEG feature vectors independently obtained from the different EEG channels to form a single feature vector. The second approach, called the multi-channel decision/classifier fusion, is achieved by combining the independent decisions of the different EEG channels to form an overall decision as to the existence of a newborn EEG seizure. The first approach suffers from the large dimensionality problem. In order to overcome this problem, three different dimensionality reduction techniques based on the sum, Fisher’s linear discriminant and symmetrical uncertainty (SU) were considered. It was found that feature fusion based on SU technique outperformed the other two techniques. It was also shown that feature fusion, which was developed on the basis that there was inter-dependence between recorded EEG channels, was superior to the independent decision fusion.

Multi-Sensor Image Fusion: A Survey of the State of the Art

Image fusion has been developing into an important area of research. In remote sensing, the use of the same image sensor in different working modes, or different image sensors, can provide reinforcing or complementary information. Therefore, it is highly valuable to fuse outputs from multiple sensors (or the same sensor in different working modes) to improve the overall performance of the remote images, which are very useful for human visual perception and image processing task. Accordingly, in this paper, we first provide a comprehensive survey of the state of the art of multi-sensor image fusion methods in terms of three aspects: pixel-level fusion, feature-level fusion and decision-level fusion. An overview of existing fusion strategies is then introduced, after which the existing fusion quality measures are summarized. Finally, this review analyzes the development trends in fusion algorithms that may attract researchers to further explore the research in this field.

CNN-Transformer特征融合多目标跟踪算法

在卷积神经网络(CNN)中,卷积运算能高效地提取目标的局部特征,却难以捕获全局表示;而在视觉Transformer中,注意力机制可以捕获长距离的特征依赖,但会忽略局部特征细节。针对以上问题,提出一种基于CNN-Transformer双分支主干网络进行特征提取和融合的多目标跟踪算法CTMOT(CNN-transformer multi-object tracking)。使用基于CNN和Transformer双分支并行的主干网络分别提取图像的局部和全局特征。使用双向桥接模块(two-way braidge module,TBM)对两种特征进行充分融合。将融合后的特征输入两组并行的解码器进行处理。将解码器输出的检测框和跟踪框进行匹配,完成多目标跟踪任务。在多目标跟踪数据集MOT17、MOT20、KITTI以及UADETRAC上进行评估,CTMOT算法的MOTP和IDs指标在四个数据集上均达到了SOTA效果,MOTA指标分别达到了76.4%、66.3%、92.36%和88.57%,在MOT数据集上与SOTA方法效果相当,在KITTI数据集上达到SOTA效果。由于同时完成目标检测和关联,能够端到端进行目标跟踪,跟踪速度可达35 FPS,表明CTMOT算法在跟踪的实时性和准确性上达到了较好的平衡,具有较大潜力。

DM-L Based Feature Extraction and Classifier Ensemble for Object Recognition

Deep Learning is a powerful technique that is widely applied to Image Recognition and Natural Language Processing tasks amongst many other tasks. In this work, we propose an efficient technique to utilize pre-trained Convolutional Neural Network (CNN) architectures to extract powerful features from images for object recognition purposes. We have built on the existing concept of extending the learning from pre-trained CNNs to new databases through activations by proposing to consider multiple deep layers. We have exploited the progressive learning that happens at the various intermediate layers of the CNNs to construct Deep Multi-Layer (DM-L) based Feature Extraction vectors to achieve excellent object recognition performance. Two popular pre-trained CNN architecture models i.e. the VGG_16 and VGG_19 have been used in this work to extract the feature sets from 3 deep fully connected multiple layers namely “fc6”, “fc7” and “fc8” from inside the models for object recognition purposes. Using the Principal Component Analysis (PCA) technique, the Dimensionality of the DM-L feature vectors has been reduced to form powerful feature vectors that have been fed to an external Classifier Ensemble for classification instead of the Softmax based classification layers of the two original pre-trained CNN models. The proposed DM-L technique has been applied to the Benchmark Caltech-101 object recognition database. Conventional wisdom may suggest that feature extractions based on the deepest layer i.e. “fc8” compared to “fc6” will result in the best recognition performance but our results have proved it otherwise for the two considered models. Our experiments have revealed that for the two models under consideration, the “fc6” based feature vectors have achieved the best recognition performance. State-of-the-Art recognition performances of 91.17% and 91.35% have been achieved by utilizing the “fc6” based feature vectors for the VGG_16 and VGG_19 models respectively. The recognition performance has been achieved by considering 30 sample images per class whereas the proposed system is capable of achieving improved performance by considering all sample images per class. Our research shows that for feature extraction based on CNNs, multiple layers should be considered and then the best layer can be selected that maximizes the recognition performance.

自适应特征融合的多模态实体对齐研究

多模态数据间交互式任务的兴起对于综合利用不同模态的知识提出了更高的要求,因此融合不同模态知识的多模态知识图谱应运而生.然而,现有多模态知识图谱存在图谱知识不完整的问题,严重阻碍对信息的有效利用.缓解此问题的有效方法是通过实体对齐进行知识图谱补全.当前多模态实体对齐方法以固定权重融合多种模态信息,在融合过程中忽略不同模态信息贡献的差异性.为解决上述问题,设计一套自适应特征融合机制,根据不同模态数据质量动态融合实体结构信息和视觉信息.此外,考虑到视觉信息质量不高、知识图谱之间的结构差异也影响实体对齐的效果,本文分别设计提升视觉信息有效利用率的视觉特征处理模块以及缓和结构差异性的三元组筛选模块.在多模态实体对齐任务上的实验结果表明,提出的多模态实体对齐方法的性能优于当前最好的方法.

基于改进DETR的机器人铆接缺陷检测方法研究

铆接作为铁道车辆结构件的主要连接方式,合格的铆接质量是车辆安全稳定运行的重要保证。针对现有铆接缺陷检测方法存在检测精度低、检测点位少、检测智能化水平不高等问题,提出一种基于改进DETR的机器人铆接缺陷检测方法。首先,搭建铆接缺陷检测系统,依次采集工件尺寸大、铆钉尺寸小工况下的铆接缺陷图像。其次,为了增强DETR模型在小目标中的图像特征提取能力和检测性能,以EfficientNet作为DETR中的主干特征提取网络,并将3-D权重注意力机制SimAM引入EfficientNet网络,从而有效保留图像特征层的镦头形态信息和铆点区域的空间信息。然后,在颈部网络中引入加权双向特征金字塔模块,以EfficientNet网络的输出作为特征融合模块的输入对各尺度特征信息进行聚合,增大不同铆接缺陷的类间差异。最后,利用Smooth L1和DIoU的线性组合改进原模型预测网络的回归损失函数,提高模型的检测精度和收敛速度。结果表明,改进模型表现出较高的检测性能,对于铆接缺陷的平均检测精度mAP为97.12%,检测速度FPS为25.4帧/s,与Faster RCNN、YOLOX等其他主流检测模型相比,在检测精度和检测速度方面均具有较大优势。研究结果能够满足实际工况中大型铆接件的小尺寸铆钉铆接缺陷实时在线检测的需求,为视觉检测技术在铆接工艺中的应用提供一定的参考价值。

基于参数优化VMD和改进LSSVM的道岔故障诊断方法

为了解决道岔设备智能故障诊断中特征指标难以提取以及模型训练时间较长的问题,以ZDJ9型转辙机带动的道岔设备为研究对象,以转辙机功率曲线为数据基础,提出一种基于参数优化变分模态分解(Variational Mode Decomposition,VMD)和改进最小二乘支持向量机(Least Squares Support Vector Machines,LSSVM)的道岔故障诊断方法。首先,采用鲸鱼优化算法(Whale Optimization Algorithm,WOA)优化VMD参数,得到模态(Intrinsic Mode Functions,IMF)分量个数和惩罚因子的最优参数组合。其次,计算IMF分量与功率曲线的相关系数,优选相关性较大的前3阶IMF分量,并计算功率谱熵、模糊熵及包络熵值,建立多特征融合样本数据库。最后,针对麻雀搜索算法(Sparrow Search Algorithm,SSA)易陷入局部最优的问题,通过改进Tent混沌映射初始化策略随机生成种群,正余弦算法(Sine Cosine Algorithm,SCA)更新追随者的位置,并采用改进SSA优化LSSVM算法的惩罚因子和核函数方差,构建基于TSSSA-LSSVM的道岔故障诊断模型。实验结果表明:所提道岔故障诊断方法是可行的,采用多特征融合能够更加全面地提取道岔典型故障特征,反映道岔的真实运行状态,提高了故障诊断准确率,而且较TSSSA-SVM,PSO-LSSVM,GWO-LSSVM以及SSA-LSSVM等方法具有较高的故障诊断准确率、召回率以及较低的漏报率,减少了模型训练时间,完全满足现场道岔故障导向安全的原则,具有更好的故障诊断性能,对现场道岔设备的故障维修具有一定的指导意义。