FPN算法在视觉感知机器人抓取控制的应用研究

针对视觉感知机器人对物体抓取的准确性控制,在抓取姿势估计基础上使用密集连接的特征金字塔网络(FPN)作为特征提取器,将语义更强的高级特征图与分辨率更高的低级特征图融合,将机器人物体抓取过程分为两个阶段,第一个阶段生成待抓取区域,第二阶段对抓取区域进行细化以预测抓取姿势。模型在Cornell抓取数据集和Jacquard数据集上训练,验证了所提算法在抓取姿势估计的有效性。设计了两种不同真实场景的物体抓取控制实验,结果表明所提模型能有效提高机器人抓取各种不同尺寸物体的能力。

基于SSD-MobileNetv2和FPN的人脸检测

随着人工智能技术的不断发展,人脸检测与识别技术以其广泛的应用性成为学术研究的重点。提出了SSD-MobileNetv2-FPN人脸检测模型,首先用轻量级的MobileNetv2代替SSD中的VGG-16主干网络,减少模型训练参数以提高模型的检测速度,然后引入FPN网络提取多尺度特征信息使得模型更利于小目标人脸的检测,增加检测精度。最后引入Focal loss损失函数解决模型在训练过程中出现前景和背景类分布不平衡问题,提高模型性能。实验表明上述模型在Pascal Voc 2012人脸部分数据集中准确率为92.5%,且处理速度快,满足实时需求。

基于NCM-FPN的古建筑修缮阶段施工安全综合评价

为有效预防古建筑修缮阶段施工安全事故,提出了基于正态云模型(Normal Cloud Model, NCM)与模糊Petri网(Fuzzy Petri Net, FPN)的施工安全综合评价方法。首先,分析古建筑修缮阶段施工特点及风险特性,建立多因素耦合作用下的三脚架事故致因模型(Tripod-Delta),构建指标体系。然后,将指标体系转换为施工安全多因素耦合FPN网络结构,采用NCM确定FPN指标初始状态,通过逆向搜索策略约简FPN冗余指标节点,并运用模糊推理算法与障碍因子诊断模型得出评价结果。结果表明,实例评价结果与现场情况基本一致,协同管理与材料设备是影响古建筑修缮阶段施工安全的关键因素。所提方法能充分表达古建筑修缮阶段施工安全风险的耦合特性,并确定安全管理的关键因素,评价结果客观准确。

基于NRF2/FPN1通路探究黄芪甲苷减轻糖尿病心肌梗死大鼠心肌损伤的作用机制

目的探讨基于核因子E2相关因子2(NRF2)/膜铁转运蛋白1(FPN1)通路对黄芪甲苷(AS)减轻糖尿病(DM)心肌梗死大鼠心肌损伤的作用机制。方法通过高糖饲料及链脲佐菌素(STZ)建立DM大鼠模型,将DM造模成功的50只大鼠通过结扎左冠状动脉前降支构建DM心肌梗死大鼠模型,将造模成功的50只大鼠随机分为梗死组、AS低剂量(AS-L,20 mg/kg AS)组、AS中剂量(AS-M,40 mg/kg AS)组、AS高剂量(AS-H,80 mg/kg AS)组、AS-H+NRF2抑制剂(ML385,80 mg/kg AS+30 mg/kg ML385)组,同时以10只开胸大鼠设为假手术组。干预结束后,检测心功能[左心室射血分数(LVEF)、肌酸激酶同工酶-MB(CK-MB)、肌酸激酶(CK)]指标;2,3,5-氯化三苯基四唑(TTC)检测心肌梗死体积比;试剂盒检测铁含量水平;HE染色检测大鼠心肌组织病理变化;qRT-PCR及Western blot分析心肌组织中NRF2、谷胱甘肽过氧化酶4(GPX4)、FPN1mRNA及蛋白表达。结果与假手术组相比,梗死组大鼠病理损伤严重,LVEF、NRF2、GPX4、FPN1mRNA及蛋白表达显著降低,梗死体积比、铁离子含量、CK-MB、CK显著增加(均P<0.05);与梗死组相比,AS-L组、AS-M组、AS-H组大鼠病理损伤得到改善,LVEF、NRF2、GPX4、FPN1mRNA及蛋白表达显著增加,梗死体积比、铁离子含量、CK-MB、CK显著降低,呈现剂量依赖性(均P<0.05);与AS-H组相比,AS-H+ML385组病理损伤相对严重,LVEF、NRF2、GPX4、FPN1mRNA及蛋白表达显著降低,梗死体积比、铁离子含量、CK-MB、CK显著增加(均P<0.05)。结论AS可减轻DM心肌梗死大鼠心肌损伤,可能通过激活NRF2/FPN1通路抑制铁死亡实现。

Object-based classification of cloudy coastal areas using medium-resolution optical and SAR images for vulnerability assessment of marine disaster

Efficient and accurate access to coastal land cover information is of great significance for marine disaster prevention and mitigation.Although the popular and common sensors of land resource satellites provide free and valuable images to map the land cover,coastal areas often encounter significant cloud cover,especially in tropical areas,which makes the classification in those areas non-ideal.To solve this problem,we proposed a framework of combining medium-resolution optical images and synthetic aperture radar(SAR)data with the recently popular object-based image analysis(OBIA)method and used the Landsat Operational Land Imager(OLI)and Phased Array type L-band Synthetic Aperture Radar(PALSAR)images acquired in Singapore in 2017 as a case study.We designed experiments to confirm two critical factors of this framework:one is the segmentation scale that determines the average object size,and the other is the classification feature.Accuracy assessments of the land cover indicated that the optimal segmentation scale was between 40 and 80,and the features of the combination of OLI and SAR resulted in higher accuracy than any individual features,especially in areas with cloud cover.Based on the land cover generated by this framework,we assessed the vulnerability of the marine disasters of Singapore in 2008 and 2017 and found that the high-vulnerability areas mainly located in the southeast and increased by 118.97 km2 over the past decade.To clarify the disaster response plan for different geographical environments,we classified risk based on altitude and distance from shore.The newly increased high-vulnerability regions within 4 km offshore and below 30 m above sea level are at high risk;these regions may need to focus on strengthening disaster prevention construction.This study serves as a typical example of using remote sensing techniques for the vulnerability assessment of marine disasters,especially those in cloudy coastal areas.

Temporal sequence Object-based CNN(TS-OCNN) for crop classification from fine resolution remote sensing image time-series

Accurate crop distribution mapping is required for crop yield prediction and field management. Due to rapid progress in remote sensing technology, fine spatial resolution(FSR) remotely sensed imagery now offers great opportunities for mapping crop types in great detail. However, within-class variance can hamper attempts to discriminate crop classes at fine resolutions. Multi-temporal FSR remotely sensed imagery provides a means of increasing crop classification from FSR imagery, although current methods do not exploit the available information fully. In this research, a novel Temporal Sequence Object-based Convolutional Neural Network(TS-OCNN) was proposed to classify agricultural crop type from FSR image time-series. An object-based CNN(OCNN) model was adopted in the TS-OCNN to classify images at the object level(i.e., segmented objects or crop parcels), thus, maintaining the precise boundary information of crop parcels. The combination of image time-series was first utilized as the input to the OCNN model to produce an ‘original’ or baseline classification. Then the single-date images were fed automatically into the deep learning model scene-by-scene in order of image acquisition date to increase successively the crop classification accuracy. By doing so, the joint information in the FSR multi-temporal observations and the unique individual information from the single-date images were exploited comprehensively for crop classification. The effectiveness of the proposed approach was investigated using multitemporal SAR and optical imagery, respectively, over two heterogeneous agricultural areas. The experimental results demonstrated that the newly proposed TS-OCNN approach consistently increased crop classification accuracy, and achieved the greatest accuracies(82.68% and 87.40%) in comparison with state-of-the-art benchmark methods, including the object-based CNN(OCNN)(81.63% and85.88%), object-based image analysis(OBIA)(78.21% and 84.83%), and standard pixel-wise CNN(79.18%and 82.90%). The proposed approach is the first known attempt to explore simultaneously the joint information from image time-series with the unique information from single-date images for crop classification using a deep learning framework. The TS-OCNN, therefore, represents a new approach for agricultural landscape classification from multi-temporal FSR imagery. Besides, it is readily generalizable to other landscapes(e.g., forest landscapes), with a wide application prospect.

基于ConA-FPN的肝脏肿瘤检测算法

深度学习方法在病灶检测任务中被广泛应用,但因肝脏肿瘤较小和样本较少的问题,导致无法达到辅助诊断的准确率要求。针对以上问题,提出基于ConA-FPN的肝脏肿瘤检测算法,具体过程为:使用融合ResNet和注意力机制的特征金字塔替换Faster R-CNN中的特征提取网络;使用融合特征解决特征金字塔中的高层模块通道信息损失问题,通过添加CAG注意力机制解决了特征融合带来的特征混叠问题,增强模型对小肿瘤的检测能力;使用迁移学习和数据增强提升模型在小数据集上的检测能力和泛化能力。实验结果表明,ConA-FPN在LITS2017和3D-IRCADB上的平均精度达到87.43%,明显优于主流检测模型。

Development of a Generic Model for the Detection of Roof Materials Based on an Object-Based Approach Using WorldView-2 Satellite Imagery

The detection of impervious surface (IS) in heterogeneous urban areas is one of the most challenging tasks in urban remote sensing. One of the limitations in IS detection at the parcel level is the lack of sufficient training data. In this study, a generic model of spatial distribution of roof materials is considered to overcome this limitation. A generic model that is based on spectral, spatial and textural information which is extracted from available training data is proposed. An object-based approach is used to extract the information inherent in the image. Furthermore, linear discriminant analysis is used for dimensionality reduction and to discriminate between different spatial, spectral and textural attributes. The generic model is composed of a discriminant function based on linear combinations of the predictor variables that provide the best discrimination among the groups. The discriminate analysis result shows that of the 54 attributes extracted from the WorldView-2 image, only 13 attributes related to spatial, spectral and textural information are useful for discriminating different roof materials. Finally, this model is applied to different WorldView-2 images from different areas and proves that this model has good potential to predict roof materials from the WorldView-2 images without using training data.

Integration of SAR Polarimetric Features and Multi-spectral Data for Object-Based Land Cover Classification

An object-based approach is proposed for land cover classification using optimal polarimetric parameters.The ability to identify targets is effectively enhanced by the integration of SAR and optical images.The innovation of the presented method can be summarized in the following two main points:①estimating polarimetric parameters(H-A-Alpha decomposition)through the optical image as a driver;②a multi-resolution segmentation based on the optical image only is deployed to refine classification results.The proposed method is verified by using Sentinel-1/2 datasets over the Bakersfield area,California.The results are compared against those from pixel-based SVM classification using the ground truth from the National Land Cover Database(NLCD).A detailed accuracy assessment complied with seven classes shows that the proposed method outperforms the conventional approach by around 10%,with an overall accuracy of 92.6%over regions with rich texture.

Object-Based vs. Pixel-Based Classification of Mangrove Forest Mapping in Vien An Dong Commune, Ngoc Hien District, Ca Mau Province Using VNREDSat-1 Images

Many researches have been performed comparing object-based classification (OBC) and pixel-based classification (PBC), particularly in classifying high-resolution satellite images. VNREDSat-1 is the first optical remote sensing satellite of Vietnam with resolution of 2.5 m (Panchromatic) and 10 m (Multispectral). The objective of this research is to compare two classification approaches using VNREDSat-1 image for mapping mangrove forest in Vien An Dong commune, Ngoc Hien district, Ca Mau province. ISODATA algorithm (in PBC method) and membership function classifier (in OBC method) were chosen to classify the same image. The results show that the overall accuracies of OBC and PBC are 73% and 62.16% respectively, and OBC solved the “salt and pepper” which is the main issue of PBC as well. Therefore, OBC is supposed to be the better approach to classify VNREDSat-1 for mapping mangrove forest in Ngoc Hien commune.

Faster RCNN Target Detection Algorithm Integrating CBAM and FPN

Small targets and occluded targets will inevitably appear in the image during the shooting process due to the influence of angle,distance,complex scene,illumination intensity,and other factors.These targets have few effective pixels,few features,and no apparent features,which makes extracting their efficient features difficult and easily leads to false detection,missed detection,and repeated detection,affecting the performance of target detection models.An improved faster region convolutional neural network(RCNN)algorithm(CF-RCNN)integrating convolutional block attention module(CBAM)and feature pyramid networks(FPN)is proposed to improve the detection and recognition accuracy of small-size objects,occluded or truncated objects in complex scenes.Firstly,the CBAM mechanism is integrated into the feature extraction network to improve the detection ability of occluded or truncated objects.Secondly,the FPN-featured pyramid structure is introduced to obtain high-resolution and vital semantic data to enhance the detection effect of small-size objects.The experimental results show that the mean average precision of target detection of the improved algorithm on PASCAL VOC2012 is improved to 76.1%,which is 13.8 percentage points higher than that of the commonly used Faster RCNN and other algorithms.Furthermore,it is better than the commonly used small sample target detection algorithm.

OBH-RSI:Object-Based Hierarchical Classification Using Remote Sensing Indices for Coastal Wetland

With the deterioration of the environment,it is imperative to protect coastal wetlands.Using multi-source remote sensing data and object-based hierarchical classification to classify coastal wetlands is an effective method.The object-based hierarchical classification using remote sensing indices(OBH-RSI)for coastal wetland is proposed to achieve fine classification of coastal wetland.First,the original categories are divided into four groups according to the category characteristics.Second,the training and test maps of each group are extracted according to the remote sensing indices.Third,four groups are passed through the classifier in order.Finally,the results of the four groups are combined to get the final classification result map.The experimental results demonstrate that the overall accuracy,average accuracy and kappa coefficient of the proposed strategy are over 94%using the Yellow River Delta dataset.

Object-based Analysis for Extraction of Dominant Tree Species

As forest is of great significance for our whole development and the sustainable plan is so focus on it. It is very urgent for us to have the whole distribution,stock volume and other related information about that. So the forest inventory program is on our schedule. Aiming at dealing with the problem in extraction of dominant tree species,we tested the highly hot method-object-based analysis. Based on the ALOS image data,we combined multi-resolution in e Cognition software and fuzzy classification algorithm. Through analyzing the segmentation results,we basically extract the spruce,the pine,the birch and the oak of the study area. Both the spectral and spatial characteristics were derived from those objects,and with the help of GLCM,we got the differences of each species. We use confusion matrix to do the Classification accuracy assessment compared with the actual ground data and this method showed a comparatively good precision as 87% with the kappa coefficient 0. 837.

Object-based image analysis for mapping geomorphic zones of coral reefs in the Xisha Islands, China

Mapping regional spatial patterns of coral reef geomorphology provides the primary information to understand the constructive processes in the reef ecosystem. However, this work is challenged by the pixel-based image classification method for its comparatively low accuracy. In this paper, an object-based image analysis(OBIA)method was presented to map intra-reef geomorphology of coral reefs in the Xisha Islands, China using Landsat 8satellite imagery. Following the work of the Millennium Coral Reef Mapping Project, a regional reef class hierarchy with ten geomorphic classes was first defined. Then, incorporating the hierarchical concept and integrating the spectral and additional spatial information such as context, shape and contextual relationships, a large-scale geomorphic map was produced by OBIA with accuracies generally more than 80%. Although the robustness of OBIA has been validated in the applications of coral reef mapping from individual reefs to reef system in this paper, further work is still required to improve its transferability.

基于YOLOv5-EA-FPNs的芯片缺陷检测方法研究

针对芯片缺陷检测中,缺陷尺寸跨度大、特征相似、小目标难识别、漏检等问题,本文提出基于YOLOv5改进的缺陷检测方法。针对小目标缺陷检测中出现的漏检、误检等问题,提出新增小目标特征检测器(small target feature detector,S-Detector),提升模型对小目标缺陷的学习能力;针对缺陷尺寸跨度大、特征相似等问题,提出具有高效聚焦学习能力的特征金字塔结构(efficient attention feature pyramid networks,EA-FPNs),提升模型对不同尺寸缺陷的检测能力;针对预测阶段冗余框较多导致时间开销大的问题,提出基于面积的边界框融合算法(bounding box fusion algorithm,BFA),减少冗余框。实验结果表明,本文方法相较于改进前,检测精确度提升1.2%,小目标缺陷精确度提升1.6%;采用BFA消除冗余框的同时,平均检测时长为26.8μs/张,较使用BFA前减少了5.2μs。本文所提方法具有良好性能,能够提升检测效率。

Geographic Object-Based Image Analysis of Changes in Land Cover in the Coastal Zones of the Red River Delta (Vietnam)

The majority of the population and economic activity of the northern half of Vietnam is clustered in the Red River Delta and about half of the country’s rice production takes place here. There are significant problems associated with its geographical position and the intensive exploitation of resources by an overabundant population (population density of 962 inhabitants/km2). Some thirty years after the economic liberalization and the opening of the country to international markets, agricultural land use patterns in the Red River Delta, particularly in the coastal area, have undergone many changes. Remote sensing is a particularly powerful tool in processing and providing spatial information for monitoring land use changes. The main methodological objective is to find a solution to process the many heterogeneous coastal land use parameters, so as to describe it in all its complexity, specifically by making use of the latest European satellite data (Sentinel-2). This complexity is due to local variations in ecological conditions, but also to anthropogenic factors that directly and indirectly influence land use dynamics. The methodological objective was to develop a new Geographic Object-based Image Analysis (GEOBIA) approach for mapping coastal areas using Sentinel-2 data and Landsat 8. By developing a new segmentation, accuracy measure, in this study was determined that segmentation accuracies decrease with increasing segmentation scales and that the negative impact of under-segmentation errors significantly increases at a large scale. An Estimation of Scale Parameter (ESP) tool was then used to determine the optimal segmentation parameter values. A popular machine learning algorithms (Random Forests-RFs) is used. For all classifications algorithm, an increase in overall accuracy was observed with the full synergistic combination of available data sets.

FPN-MSTCN模型在学生专注度评价中的应用

为了提高智慧教育场景下的学生专注度评价准确率,针对小样本类别难以识别的问题,提出一种FPN-MSTCN模型进行专注度评价,该模型通过改进的FPN网络对单帧人脸进行多尺度的特征提取,解决了在图像中人脸特征无法完整提取的问题。然后,通过融合了SimGNN模块的MSTCN网络对图像序列进行分类,并通过SimGNN模块解决了图像标签与视频标签不一致的问题。采用DAiSEE和EmotiW数据集进行实验。由于DAiSEE和EmotiW数据集的分布严重不均衡,使用代价敏感损失函数作为该模型的损失函数,解决了过拟合问题,测试集准确率分别提高了3.8%和3.1%。

基于3D FPN的多时相遥感影像农作物分割

为了解决目前面向多时相遥感影像农作物分割中存在的研究较少、分割精准度较低等问题,设计了一种基于3D特征金字塔网络(FPN)结构的模型,并引入空洞空间卷积池化金字塔(ASPP)模块,使得该网络能够捕获图像的多尺度特征,融合图像更多的上下文语义信息.此外针对数据集类别数量和难易区分程度不平衡的问题,引入了Focal loss函数,通过设置权重因子,使得网络更多关注数量少或难区分的类别.以2019年齐齐哈尔市的多时相遥感影像为数据源,对玉米、大豆、水稻以及其他四种类别进行分割.实验结果显示,总精准度达到了93.6%,平均召回率达到了93.2%,平均准确率达到了94.0%,平均F1得分达到了93.5%,平均交并比(IoU)达到了88.6%,优于3D FCN、3D Unet、2D Unet+CLSTM等农作物分割方法,证明了提出的模型的有效性.

基于改进FPN与SVM的树障检测方法

针对目前无人机搭载传感器的树障检测方法无法实现自动检测的问题,提出一种基于改进的FPN(特征金字塔网络)与SVM(支持向量机)的树障检测算法。在传统的FPN基础上,进行自下而上的反向侧边连接并融合,采用ResNet 50(深度残差网络)和改进的FPN作为特征提取网络得到特征向量,并将其输入基于遗传算法的SVM中进行二分类,进而判断所检测图像中是否存在树障隐患。实验结果表明,本算法用于树障检测的准确率达到93.4%,处理图像的平均速度达到每秒11张,漏检率和误检率较低,具有较强的泛化能力。

基于模糊Petri网的引航员作业舒适度评价

为提高引航员的作业舒适度,提出一种基于模糊Petri网(FPN)的模糊推理算法(FRA)下的组合评价方法。首先,针对作业舒适度影响因子的不确定性信息,建立多因素耦合的FPN拓扑结构;然后,采用博弈论组合赋权法确定最优组合权重,提出融合层间相关性判断临界重要性、层次分析法和FRA,建立基于主客观权重的FRA,通过迭代求解库所可信度和状态矩阵;最后,结合上海港船舶引航的场景数据,基于FPN的FRA应用,评价引航员作业舒适度。结果表明:环境与引航设备是影响其作业舒适度的关键因素,冬季和夏季的引航作业舒适度评价等级对应“较不舒适”,其中,5月份为“较舒适”。所提方法充分体现系统舒适度影响因素的耦合特性。