Temporal sequence Object-based CNN(TS-OCNN) for crop classification from fine resolution remote sensing image time-series

Accurate crop distribution mapping is required for crop yield prediction and field management. Due to rapid progress in remote sensing technology, fine spatial resolution(FSR) remotely sensed imagery now offers great opportunities for mapping crop types in great detail. However, within-class variance can hamper attempts to discriminate crop classes at fine resolutions. Multi-temporal FSR remotely sensed imagery provides a means of increasing crop classification from FSR imagery, although current methods do not exploit the available information fully. In this research, a novel Temporal Sequence Object-based Convolutional Neural Network(TS-OCNN) was proposed to classify agricultural crop type from FSR image time-series. An object-based CNN(OCNN) model was adopted in the TS-OCNN to classify images at the object level(i.e., segmented objects or crop parcels), thus, maintaining the precise boundary information of crop parcels. The combination of image time-series was first utilized as the input to the OCNN model to produce an ‘original’ or baseline classification. Then the single-date images were fed automatically into the deep learning model scene-by-scene in order of image acquisition date to increase successively the crop classification accuracy. By doing so, the joint information in the FSR multi-temporal observations and the unique individual information from the single-date images were exploited comprehensively for crop classification. The effectiveness of the proposed approach was investigated using multitemporal SAR and optical imagery, respectively, over two heterogeneous agricultural areas. The experimental results demonstrated that the newly proposed TS-OCNN approach consistently increased crop classification accuracy, and achieved the greatest accuracies(82.68% and 87.40%) in comparison with state-of-the-art benchmark methods, including the object-based CNN(OCNN)(81.63% and85.88%), object-based image analysis(OBIA)(78.21% and 84.83%), and standard pixel-wise CNN(79.18%and 82.90%). The proposed approach is the first known attempt to explore simultaneously the joint information from image time-series with the unique information from single-date images for crop classification using a deep learning framework. The TS-OCNN, therefore, represents a new approach for agricultural landscape classification from multi-temporal FSR imagery. Besides, it is readily generalizable to other landscapes(e.g., forest landscapes), with a wide application prospect.

基于双目立体视觉的多分辨率图像匹配方法研究

在双目立体视觉系统中,面对复杂场景时噪声会损害图像特征,增加提取难度,导致匹配精度和鲁棒性下降。因此,文中提出基于双目立体视觉的多分辨率图像匹配方法,旨在从不同尺度图像中有效获取信息并实现高精度匹配。该方法利用双目立体视觉模型的双目旋转相机扫描目标并进行成像,根据内、外空间标定提升双目旋转相机的位置精度,保证目标的多分辨率成像效果;将其输入金字塔立体匹配网络中,通过网络中的类金字塔多空洞卷积操作提取双目图像特征,在此基础上,基于可变卷积增强其纹理特征细节;结合细粒度特征和互注意力机制完成双目图像匹配。测试结果显示,空间标定后,左、右两个相机的成像误差最小值分别为0.6 Pixel和0.4 Pixel;匹配点坐标偏差均值和坐标偏差方差值分别低于0.012和0.011,匹配效果良好。

A Survey on Deep Learning-based Fine-grained Object Classification and Semantic Segmentation

The deep learning technology has shown impressive performance in various vision tasks such as image classification, object detection and semantic segmentation. In particular, recent advances of deep learning techniques bring encouraging performance to fine-grained image classification which aims to distinguish subordinate-level categories, such as bird species or dog breeds. This task is extremely challenging due to high intra-class and low inter-class variance. In this paper, we review four types of deep learning based fine-grained image classification approaches, including the general convolutional neural networks （CNNs）, part detection based, ensemble of networks based and visual attention based fine-grained image classification approaches. Besides, the deep learning based semantic segmentation approaches are also covered in this paper. The region proposal based and fully convolutional networks based approaches for semantic segmentation are introduced respectively.

Zero-shot Fine-grained Classification by Deep Feature Learning with Semantics

Fine-grained image classification, which aims to distinguish images with subtle distinctions, is a challenging task for two main reasons: lack of sufficient training data for every class and difficulty in learning discriminative features for representation. In this paper, to address the two issues, we propose a two-phase framework for recognizing images from unseen fine-grained classes, i.e., zeroshot fine-grained classification. In the first feature learning phase, we finetune deep convolutional neural networks using hierarchical semantic structure among fine-grained classes to extract discriminative deep visual features. Meanwhile, a domain adaptation structure is induced into deep convolutional neural networks to avoid domain shift from training data to test data. In the second label inference phase, a semantic directed graph is constructed over attributes of fine-grained classes. Based on this graph, we develop a label propagation algorithm to infer the labels of images in the unseen classes. Experimental results on two benchmark datasets demonstrate that our model outperforms the state-of-the-art zero-shot learning models. In addition, the features obtained by our feature learning model also yield significant gains when they are used by other zero-shot learning models, which shows the flexility of our model in zero-shot finegrained classification.

Classification of Hydrocarbon-Bearing Fine-Grained Sedimentary Rocks

Fine-grained sedimentary rocks are defined as rocks which mainly compose of fine grains（〈62.5 μm）. The detailed studies on these rocks have revealed the need of a more unified, comprehensive and inclusive classification. The study focuses on fine-grained rocks has turned from the differences of inorganic mineral components to the significance of organic matter and microorganisms. The proposed classification is based on mineral composition, and it is noted that organic matters have been taken as a very important parameter in this classification scheme. Thus, four parameters, the TOC content, silica（quartz plus feldspars）, clay minerals and carbonate minerals, are considered to divide the fine-grained sedimentary rocks into eight categories, and the further classification within every category is refined depending on subordinate mineral composition. The nomenclature consists of a root name preceded by a primary adjective. The root names reflect mineral constituent of the rock, including low organic（TOC〈2%）, middle organic（2%4%） claystone, siliceous mudstone, limestone, and mixed mudstone. Primary adjectives convey structure and organic content information, including massive or limanited. The lithofacies are closely related to the reservoir storage space, porosity, permeability, hydrocarbon potential and shale oil/gas sweet spot, and are the key factor for the shale oil and gas exploration. The classification helps to systematically and practicably describe variability within fine-grained sedimentary rocks, what＇s more, it helps to guide the hydrocarbon exploration.

Fine-grained P2P traffic classification by simply counting flows

The continuous emerging of peer-to-peer(P2P) applications enriches resource sharing by networks, but it also brings about many challenges to network management. Therefore, P2 P applications monitoring, in particular,P2 P traffic classification, is becoming increasingly important. In this paper, we propose a novel approach for accurate P2 P traffic classification at a fine-grained level. Our approach relies only on counting some special flows that are appearing frequently and steadily in the traffic generated by specific P2 P applications. In contrast to existing methods, the main contribution of our approach can be summarized as the following two aspects. Firstly, it can achieve a high classification accuracy by exploiting only several generic properties of flows rather than complicated features and sophisticated techniques. Secondly, it can work well even if the classification target is running with other high bandwidth-consuming applications, outperforming most existing host-based approaches, which are incapable of dealing with this situation. We evaluated the performance of our approach on a real-world trace. Experimental results show that P2 P applications can be classified with a true positive rate higher than 97.22% and a false positive rate lower than 2.78%.

Relation Enhanced Neural Model for Type Classification of EntityMentions with a Fine-Grained Taxonomy

Inferring semantic types of the entity mentions in a sentence is a necessary yet challenging task. Most of existing methods employ a very coarse-grained type taxonomy, which is too general and not exact enough for many tasks. However, the performances of the methods drop sharply when we extend the type taxonomy to a fine-grained one with several hundreds of types. In this paper, we introduce a hybrid neural network model for type classification of entity mentions with a fine-grained taxonomy. There are four components in our model, namely, the entity mention component, the context component, the relation component, the already known type component, which are used to extract features from the target entity mention, context, relations and already known types of the entity mentions in surrounding context respectively. The learned features by the four components are concatenated and fed into a softmax layer to predict the type distribution. We carried out extensive experiments to evaluate our proposed model. Experimental results demonstrate that our model achieves state-of-the-art performance on the FIGER dataset. Moreover, we extracted larger datasets from Wikipedia and DBpedia. On the larger datasets, our model achieves the comparable performance to the state-of-the-art methods with the coarse-grained type taxonomy, but performs much better than those methods with the fine-grained type taxonomy in terms of micro-F1, macro-F1 and weighted-F1.

联合判别区域特征的细粒度视觉分类方法

细粒度视觉分类方法的核心是定位图像中的判别区域。现有研究通过利用与改进视觉Transformer方法增强了判别区域特征的远距离依赖关系,但是大多数方法仅局限于增强显著判别区域的注意力,忽略了次显著的判别区域中可以联合提取的特征信息,导致具有相似局部特征的不同类别区分难度大,分类准确率较低。因此,提出了一种联合判别区域的提取特征方法,在自注意力模块的前端划分特征图的候选判别区域,引导模型提取不同显著程度的判别区域特征;通过双线性融合自注意力模块对多个不同显著程度的判别区域进行联合特征的提取,获取更加全面的判别区域特征信息。实验结果表明,引入联合判别区域方法的视觉Transformer网络在CUB-200-2011数据集上的准确率达92.7%,较标准视觉Transformer方法提升了2.4个百分点,并且在其余的基准数据集上均超越了当前最优的细粒度视觉分类方法。

细粒度南海数字资源多标签分类算法的比较与应用

[研究目的]面对海量多模态的南海数字资源,数据分类在构建南海证据链关联模型中起着至关重要的作用。通过系统地分类和整理数据,可以揭示资源之间的潜在关联,为后续的数据处理和分析提供坚实依据。[研究方法]应用文献计量工具COOC,全面描述南海数字资源和数据分类的研究现状。从409篇南海数字资料中抽取21174条细粒度南海数字资源,分别应用并比较RNN、CNN、DNN、LSTM、Bi-LSTM、Attention、BERT模型的细粒度多标签分类效果。最后,使用知识图谱技术实现细粒度南海数字资源的分类检索和可视化。[研究结果/结论]基于BERT的细粒度南海数字资源多标签分类效果最佳。同时,知识图谱能够直观展示南海数字资源的分类关系。该研究为后续的南海数字资源管理和多标签分类应用提供了坚实的理论支持和技术保障。

多区域注意力的细粒度图像分类网络

目前细粒度图像分类的难点在于如何精准定位图像中高度可辨的局部区域以及其他辅助判别特征。提出一种多区域注意力的细粒度图像分类网络来解决这个问题。首先使用Inception-V3对图像特征进行提取,通过重复使用注意力擦除的方法使模型关注次要特征;然后通过背景去除以及上采样的方法获取图像更精准的局部图像,对提取到的局部特征进行位置统计,并以矩形框的方式获取图像整体,减少细节信息丢失;最后对局部与整体图像进行更加细致的学习。此外,设计联合损失函数,通过动态平衡难易样本和缩小类内差距的方法改善模型的识别效果。实验结果表明,该方法在公开的细粒度图像数据集CUB-200-2011、Stanford-Cars和FGVC-Aircraft上的准确率分别达到89.2%、94.8%、94.0%,相较于对比方法性能更优。

基于深度主动学习与CBAM的细粒度菊花表型识别

针对菊花种类繁多,花型差别细微,准确标注比较困难的问题,基于深度主动学习与混合注意力机制模块(Convolutional block attention module,CBAM),提出了一种标号数据不足情况下的菊花表型智能识别方法和框架。首先,通过主动学习策略基于最优标号和次优标号法(Best vs second-best,BvSB)在未标记菊花样本中选取信息量较大的样本进行标记,并将标记后的样本放入训练样本中;其次,使用深度卷积神经网络ResNet50作为本文的主干网络训练标记样本,引入混合注意力机制模块CBAM,使模型能够更为准确地提取细粒度图像中的高层语义信息;最后,用更新后的训练样本继续训练分类模型,直到模型达到迭代次数后停止。实验结果表明,该方法在少量菊花标记样本下,精确率、召回率和F1值分别达到93.66%、93.15%和93.41%。本文方法可为标号数据不足情况下的菊花等花卉智能化识别提供技术支撑。

基于细粒度特征的面料图像检索

面料图像检索对于纺织工厂面料库存和样品管理意义重大,但面料外观的多样性以及织物纹理的精细性,使得在面料检索时面料的特征提取较困难。该研究提出一种基于细粒度特征的面料图像检索算法。该算法使用坐标注意(coordinate attention,CA)模块来提取图像的精准位置信息,并将缩放系数法用于在宽度和高度方面整体缩放MobileNetV3的网络结构以减少模型参数数量,达到减少网络训练时间的目的。据此筛选出提取面料图像细粒度特征的最佳模型,在面料图像数据集(fabric image dataset,FID)上进行面料检索实验。结果表明,该算法有效提高了面料图像细粒度特征提取的准确性,检索精度达到91.82%,浮点运算数达到175.34 MB。检索精度比MobileNetV3原模型提高了13.49个百分点,同时减少了网络训练时间,速度提高了25.14%。该算法具有实际应用价值。

基于渐进式生成对抗网络的农作物病虫害细粒度分类

随着深度学习应用的普及和飞速发展,基于深度学习的图像识别方法广泛应用于农作物病虫害领域,但大部分的神经网络重视识别准确率的提高,却忽略神经网络庞大的参数计算量。为解决这个问题,基于渐进式生成对抗网络判别器模型和卷积注意力模块,提出一种改进的渐进式生成对抗网络判别器CPDM网络模型对农作物病虫害进行识别。通过对渐进式生成对抗网络判别器网络结构的调整,采用均衡学习率、像素级特征向量归一化和卷积注意力模块增强CPDM网络模型的特征提取能力,提高对真实图片的识别准确率。试验在PlantVillage数据集上进行,将该模型与VGG16、VGG19和ResNet18进行比较,得到TOP-1准确率分别为99.06%、96.50%、96.65%、98.86%,分别提高2.56%、2.41%、0.2%,且参数量仅为8.2 M。试验证明提出的CPDM网络模型满足在保证分类准确率的基础上,有效控制神经网络参数计算量的目的。

基于改进的Transformer细粒度图像识别算法研究

针对细粒度图像识别存在类间差异小、难以区分等问题,本文通过提升网络对图像细节特征的表达能力,来改善这一问题。为此,设计了一种基于改进的Transformer细粒度识别算法。首先,可变形卷积令牌嵌入通过自适应调整采样点的位置,来改变卷积操作范围及其卷积核的形状,从而增强网络模型对空间信息的感知能力,以获取更为精准的空间信息;其次,高效相关通道注意力机制通过对通道的自动选择,将通道注意力的计算从通道相邻转换成语义相似,来捕获语义相似的通道信息。而精准的空间信息和语义相似的通道信息将有效提升网络模型局部特征感知能力。实验结果表明,与基线算法相比,本文方法在CUB-200-2011、StanfordCars和StanfordDogs三个数据集上的识别结果分别提升了1.5%、2.4%、1.5%。结果表明,本文提出的方法通过提升细粒度图像细节特征的表达能力,从而有效提高了细粒度图像识别的有效性。

改进注意力模型的食品图像识别方法

随着人们对健康饮食需求的日益增加,各种饮食评估辅助软件应运而生,食品图像识别问题受到越来越多的关注。食品图像识别属于细粒度图像识别问题,较其他图像识别难度更大。目前主流的食品图像数据集,如ISIA Food-500、ETH Food-101、Vireo Food-172等所包含的图像数量偏少,难以很好地训练图像识别系统,进一步增大了图像识别难度。提出一种基于注意力机制的图像识别方法,该方法在自注意力的基础上引入局部注意力的概念,用于描绘图像细粒度特征,提高图像识别的准确率。此外,还提出一种图像自监督预训练算法,缓解食品图像训练样本不足的问题。实验结果表明,所提方法在ISIA Food-500数据集的Top-1和Top-5准确率分别达到65.58%和90.03%,性能优于现有的其他算法。

基于多粒度融合和双注意力的细粒度图像分类

针对现有模型对细粒度图像关键信息精准识别较难,分类指标较单一且特征利用不充分的问题,提出一个新的细粒度图像分类网络模型.该模型在网络训练步骤中嵌入双注意力网络以强化中层特征与深度特征的相关性,根据网络不同层的感受野大小不同将数据剪裁后再拼接成新的样本数据作为下一层输入,采用支持向量机分类器将中层和深度特征输出结果一同作为最终分类指标.在3个经典数据集CUB-200-2011、Stanford Cars和102 Category Flower上的实验结果表明,其分类准确率分别达89.56%,95.00%,96.05%,相比于其他网络模型有较好的分类准确率和泛化能力.

基于细粒度识别的即插性多粒度特征融合算法

为丰富卷积神经网络下细粒度图像特征的表征信息,进一步扩大类间差异缩小类内差异,提出一种基于多种粒度图像训练的特征融合方式,挖掘图像的细化特征。通过逐渐改变输入图像的粒度值,构造一个包含多种粒度信息的网络模型;提取多粒度多尺度图像特征,与原始特征进行融合,完成最终的分类预测。无需引入其它辅助网络,在没有显著增加模型参数的情况下,融合不同粒度特征。实验结果表明,其分类准确度高于只包含单一粒度图像的训练结果,验证了该方法能有效丰富特征信息。

融合注意力机制的人体部位细分类方法

针对传统人体部位体型分类方法费时费力、成本较高的问题,设计一种融合注意力机制的体型分类网络(Attention Body Classification Net,A_BCN)。该网络由弱监督的注意力学习和数据增强两个模块组成,其中:弱监督的注意力学习模块通过注意力机制获得注意力图;数据增强模块通过注意力图指导图像的数据增强,包括注意力裁剪、注意力丢弃和注意力平均。将增强后的图像重新输入到网络中得到特征图,将得到的特征图和注意力图融合进行分类。在后续自制的人体图像数据集中,该算法准确率为90.52%,提高了分类准确率并节省了成本。

基于改进MMAL的细粒度图像分类研究

针对细粒度图像分类中目标区域难以精准定位及其内部细粒度特征难以识别的问题,提出了一种基于改进MMAL的细粒度图像分类方法。首先,利用形变卷积的感知区域可变性原理,动态地感知样本图像中不同尺度和形状的目标区域特征,从而增强网络对目标区域位置的感知能力。随后,采用GradCAM梯度回流的方法生成网络注意力热图,以减小特征背景噪声的干扰,实现对图像目标区域的精准定位。最后,提出位置感知空间注意力模块,通过融合坐标位置和双尺度空间信息,显著提升了网络对目标区域细粒度特征的提取能力。实验结果表明,与基线算法相比,该方法在CUB-200-2011、Stanford Car和FGVC-Aircraft三个公共数据集上分类精度分别提升了1.4%、1.5%、1.9%,该结果验证了所提方法的有效性。

注意力机制和多尺度特征融合的细粒度图像分类

针对细粒度图像分类易受背景干扰、关键区域定位不准确以及模型参数量大的问题,提出了一种注意力机制和多尺度特征融合的分类网络(networks of combine attention mechanisms and multi-scale features,AM-Net)。首先,以YOLOv7网络为基础,使用Ghost BottleNeck模块重新搭建轻量级主干网络,并使用GhostConv替换颈部网络中的Conv,实现模型的轻量化。其次,引入无参的SimAM注意力机制,通过考虑空间和通道维度的相关性推断特征图的三维注意力权重,表征局部显著特征,抑制无用特征,提高目标区域信息的有效性。最后,构建可特征选择的金字塔池化模块(fast spatial pyramid pooling with feature selection and convolutions,SPPFC),帮助网络模型更好地捕捉和处理目标的多尺度特征,提高模型的感知能力。通过实验可知,AM-Net在Stanford Dogs数据集上的准确率、精确率、召回率和F 1分数分别达到88.9%、83.6%、85.7%和84.6%,模型参数量为26.53 MB,每秒帧率达到89.3帧,在Stanford Cars数据集上的准确率、精确率和召回率分别达到95.2%、93.7%和94.9%。实验结果表明,AM-Net可以在轻量化网络的同时提高细粒度图像的分类精度,相比于其他网络模型性能有较大提升。