Full Scale-Aware Balanced High-Resolution Network for Multi-Person Pose Estimation

Scale variation is amajor challenge inmulti-person pose estimation.In scenes where persons are present at various distances,models tend to perform better on larger-scale persons,while the performance for smaller-scale persons often falls short of expectations.Therefore,effectively balancing the persons of different scales poses a significant challenge.So this paper proposes a newmulti-person pose estimation model called FSANet to improve themodel’s performance in complex scenes.Our model utilizes High-Resolution Network(HRNet)as the backbone and feeds the outputs of the last stage’s four branches into the DCB module.The dilated convolution-based(DCB)module employs a parallel structure that incorporates dilated convolutions with different rates to expand the receptive field of each branch.Subsequently,the attention operation-based(AOB)module performs attention operations at both branch and channel levels to enhance high-frequency features and reduce the influence of noise.Finally,predictions are made using the heatmap representation.The model can recognize images with diverse scales and more complex semantic information.Experimental results demonstrate that FSA Net achieves competitive results on the MSCOCO and MPII datasets,validating the effectiveness of our proposed approach.

图像级高光谱影像高分辨率特征网络分类方法

基于深度学习的高光谱影像分类方法通常将高光谱影像切分为局部方块作为模型的输入,这不但限制了长距离空-谱信息关联的获取,还带来了大量额外的计算开销。以全局图像作为输入的图像级分类方法能够有效避免这些缺陷,然而,现有的基于全卷积神经网络特征串行流动模式的图像级分类方法在信息恢复时的细节损失会导致分类精度低、分类图视觉效果差等问题。因此,本文提出一种基于HRNet的图像级高光谱影像快速分类方法,在全程保持高分辨率特征的基础上对影像的多重分辨率特征进行并行计算与交叉融合,从而缓解了传统特征串行流动模式造成的信息损失问题。同时,提出多分辨率特征联合监督和投票分类策略,进一步提升了模型分类性能。利用4组开源高光谱影像数据集对本文方法进行验证,试验结果表明,与现有的先进分类方法相比,本文方法能够取得具有竞争性的分类结果,同时显著减少训练和分类时长,在实际应用时更具时效性。为了保证方法的复现性,笔者将代码开源于https://github.com/sssssyf/fast-image-level-vote。

Aquaculture area extraction and vulnerability assessment in Sanduao based on richer convolutional features network model

Sanduao is an important sea-breeding bay in Fujian,South China and holds a high economic status in aquaculture.Quickly and accurately obtaining information including the distribution area,quantity,and aquaculture area is important for breeding area planning,production value estimation,ecological survey,and storm surge prevention.However,as the aquaculture area expands,the seawater background becomes increasingly complex and spectral characteristics differ dramatically,making it difficult to determine the aquaculture area.In this study,we used a high-resolution remote-sensing satellite GF-2 image to introduce a deep-learning Richer Convolutional Features(RCF)network model to extract the aquaculture area.Then we used the density of aquaculture as an assessment index to assess the vulnerability of aquaculture areas in Sanduao.The results demonstrate that this method does not require land and water separation of the area in advance,and good extraction can be achieved in the areas with more sediment and waves,with an extraction accuracy>93%,which is suitable for large-scale aquaculture area extraction.Vulnerability assessment results indicate that the density of aquaculture in the eastern part of Sanduao is considerably high,reaching a higher vulnerability level than other parts.

Research on Facial Expression Capture Based on Two-Stage Neural Network

To generate realistic three-dimensional animation of virtual character,capturing real facial expression is the primary task.Due to diverse facial expressions and complex background,facial landmarks recognized by existing strategies have the problem of deviations and low accuracy.Therefore,a method for facial expression capture based on two-stage neural network is proposed in this paper which takes advantage of improved multi-task cascaded convolutional networks(MTCNN)and high-resolution network.Firstly,the convolution operation of traditional MTCNN is improved.The face information in the input image is quickly filtered by feature fusion in the first stage and Octave Convolution instead of the original ones is introduced into in the second stage to enhance the feature extraction ability of the network,which further rejects a large number of false candidates.The model outputs more accurate facial candidate windows for better landmarks recognition and locates the faces.Then the images cropped after face detection are input into high-resolution network.Multi-scale feature fusion is realized by parallel connection of multi-resolution streams,and rich high-resolution heatmaps of facial landmarks are obtained.Finally,the changes of facial landmarks recognized are tracked in real-time.The expression parameters are extracted and transmitted to Unity3D engine to drive the virtual character’s face,which can realize facial expression synchronous animation.Extensive experimental results obtained on the WFLW database demonstrate the superiority of the proposed method in terms of accuracy and robustness,especially for diverse expressions and complex background.The method can accurately capture facial expression and generate three-dimensional animation effects,making online entertainment and social interaction more immersive in shared virtual space.

一种新的基于深度学习的遥感影像变化检测算法——H-BIT方法的提出与应用

对遥感影像变化检测进行了研究,提出了一种融合HRNet与BIT的H-BIT方法,能兼顾遥感影像高分辨率与高语义特征,在场景复杂、目标尺度跨度大时表现优异。该方法经HRNet网络、词元分析器处理后,引入Transformer,通过注意力机制,能从更大的感受野解译变化结果。在LEVIR-CD数据集上进行了实验,结果表明H-BIT方法能完整提取目标,对目标边缘的处理更平滑,变化检测的精确率、召回度、F1得分和总体精度分别为86.95%、90.24%、87.93%和96.62%,均高于原始BIT算法的表现,说明该方法能适应多尺度目标与复杂场景,变化检测精度高、计算速度快、鲁棒性高、泛化性强。

基于高分辨率网络和图卷积网络的三维人体重建模型

针对单目图像重建人体时出现的头部姿态翻转和图像特征间隐式空间线索缺失的问题,提出了一种基于高分辨率网络(HRNet)和图卷积网络(GCN)的三维人体重建模型。首先利用HRNet和残差块作为主干网络从原始图像中提取丰富的人体特征信息,然后使用GCN来捕获特征之间隐式的空间线索以获得空间精确的特征表示,最后使用此特征来预测多人线性蒙皮模型(SMPL)的参数以得到更加准确的重建结果;同时为了有效解决人体头部姿态翻转的问题,对SMPL的关节点重新进行了定义,在原有关节的基础上增加对头部关节点的定义。实验结果表明,所提模型能够准确地重建出三维人体,在2D数据集LSP上的重建准确率达到了92.41%,在3D数据集MPI-INF-3DHP上的关节误差和重建误差也大幅降低,平均误差仅分别为97.73 mm和64.63 mm,验证了所提模型在人体重建领域的有效性。

基于高分辨率网络的轻量型人体姿态估计方法

在高分辨率网络(HRNet)的基础上,提出一种融合Ghost卷积的轻量型高分辨率网络(GLHRNet)。首先使用Ghost卷积模块和极化自注意力(PSA)模块在HRNet中构建新的残差块结构,新的残差块结构可以在减少网络模型参数量和计算量的同时,建模高分辨率图像的长距离依赖关系。接着在新网络模型中引入IBN-Net的设计思想,在新网络模型的浅层同时使用批量归一化和实例归一化,为网络模型引入外观不变性,减小光照变化问题对模型的影响。算法在COCO人体姿态估计数据集上的实验结果表明,与HRNet相比新网络模型的参数量降低了36.1%,计算量降低了35.2%,人体姿态估计的平均准确率提高了1.4个百分点。

基于改进RetinaNet的行人检测算法

为提高实际应用场景中行人的检测精度,提出了使用高分辨率特征提取网络HRNet(high-resolution representation network)并引入Guided Anchoring机制对RetinaNet算法进行改进,维持了特征图在特征提取过程中的高分辨率信息,同时使网络中的锚框自适应生成,提高了算法的检测精度。结果表明:该改进算法在Caltech行人数据集上取得了0.905的平均精度均值(mean average precision,mAP),相比于标准的RetinaNet算法提高了6.0%,在每帧图像尺寸为1280×720像素的视频上检测速度达到了19 FPS(FPS为每秒检测帧数),达到了检测精度与检测速度的均衡。

二维人体姿态估计研究进展

人体姿态估计一直是计算机视觉领域的研究热点,随着人体姿态估计方法的性能和精度不断提升,目前可以广泛应用于人机交互、智能监控和人体活动分析等领域。人体姿态估计属于强应用相关的研究领域,现有研究成果均不同程度地涉及方法、模型和应用层面,亟待对其进行系统性归纳和总结。文中综述了大量二维人体姿态估计的研究成果,以供研究人员参考。具体包括:单人和多人姿态估计方法,基于ResNet,Hourglass和HRNet的姿态估计模型,以及姿态估计在人机交互和智能监控领域的应用。文中提出的关于移动设备中的人体姿态估计、拥挤场景下的人体姿态估计和装备人群的姿态估计等研究问题和研究思路,是现有研究的良好补充,为研究人员提供了广阔的研究空间。

基于注意力机制的轻量型人体姿态估计

针对高分辨率人体姿态估计网络存在参数量大、运算复杂度高等问题,提出一种基于高分辨率网络(HRNet)的轻量型沙漏坐标注意力网络(SCANet)用于人体姿态估计。首先引入沙漏(Sandglass)模块和坐标注意力(CoordAttention)模块;然后在此基础上构建了沙漏坐标注意力瓶颈(SCAneck)模块和沙漏坐标注意力基础(SCAblock)模块两种轻量型模块,在降低模型参数量和运算复杂度的同时,获取特征图空间方向的长程依赖和精确位置信息。实验结果显示,在相同图像分辨率和环境配置的情况下,在COCO(Common Objects in COntext)校验集上,SCANet模型与HRNet模型相比参数量降低了52.6%,运算复杂度降低了60.6%;在MPII(Max Planck Institute for Informatics)校验集上,SCANet模型与HRNet模型相比参数量和运算复杂度分别降低了52.6%和61.1%;与常见的人体姿态估计网络如堆叠沙漏网络(Hourglass)、级联金字塔网络(CPN)和SimpleBaseline相比,SCANet模型在拥有更少的参数量与运算复杂度的情况下,仍能实现对人体关键点的高准确度预测。

基于非局部高分辨率网络的人体姿态估计方法

人体姿态估计是计算机视觉中的基础任务,其可应用于动作识别、游戏、动画制作等。受非局部均值方法的启发,设计了非局部高分辨率网络(non-local high-resolution,NLHR),在原始图像1/32分辨率的网络阶段融合非局部网络模块的,使网络有了获取全局特征的能力,从而提高人体姿态估计的准确率。NLHR网络在MPII数据集上训练,在MPII验证集上测试,PCKh@0.5评价标准下的平均准确率为90.5%,超过HRNet基线0.2个百分点;在COCO人体关键点检测数据集上训练,在COCO验证集上测试,平均准确率为76.7%,超过HRNet基线2.3个百分点。通过3组消融实验,验证NLHR网络针对人体姿态估计在精度上能够超过现有的人体姿态估计网络。

A Remote Sensing Image Semantic Segmentation Method by Combining Deformable Convolution with Conditional Random Fields

Currently,deep convolutional neural networks have made great progress in the field of semantic segmentation.Because of the fixed convolution kernel geometry,standard convolution neural networks have been limited the ability to simulate geometric transformations.Therefore,a deformable convolution is introduced to enhance the adaptability of convolutional networks to spatial transformation.Considering that the deep convolutional neural networks cannot adequately segment the local objects at the output layer due to using the pooling layers in neural network architecture.To overcome this shortcoming,the rough prediction segmentation results of the neural network output layer will be processed by fully connected conditional random fields to improve the ability of image segmentation.The proposed method can easily be trained by end-to-end using standard backpropagation algorithms.Finally,the proposed method is tested on the ISPRS dataset.The results show that the proposed method can effectively overcome the influence of the complex structure of the segmentation object and obtain state-of-the-art accuracy on the ISPRS Vaihingen 2D semantic labeling dataset.

Real-Time Safety Behavior Detection Technology of Indoors Power Personnel Based on Human Key Points

Safety production is of great significance to the development of enterprises and society.Accidents often cause great losses because of the particularity environment of electric power.Therefore,it is important to improve the safety supervision and protection in the electric power environment.In this paper,we simulate the actual electric power operation scenario by monitoring equipment and propose a real-time detection method of illegal actions based on human body key points to ensure safety behavior in real time.In this method,the human body key points in video frames were first extracted by the high-resolution network,and then classified in real time by spatial-temporal graph convolutional network.Experimental results show that this method can effectively detect illegal actions in the simulated scene.

Fusion of Convolutional Self-Attention and Cross-Dimensional Feature Transformationfor Human Posture Estimation

Human posture estimation is a prominent research topic in the fields of human-com-puter interaction,motion recognition,and other intelligent applications.However,achieving highaccuracy in key point localization,which is crucial for intelligent applications,contradicts the lowdetection accuracy of human posture detection models in practical scenarios.To address this issue,a human pose estimation network called AT-HRNet has been proposed,which combines convolu-tional self-attention and cross-dimensional feature transformation.AT-HRNet captures significantfeature information from various regions in an adaptive manner,aggregating them through convolu-tional operations within the local receptive domain.The residual structures TripNeck and Trip-Block of the high-resolution network are designed to further refine the key point locations,wherethe attention weight is adjusted by a cross-dimensional interaction to obtain more features.To vali-date the effectiveness of this network,AT-HRNet was evaluated using the COCO2017 dataset.Theresults show that AT-HRNet outperforms HRNet by improving 3.2%in mAP,4.0%in AP75,and3.9%in AP^(M).This suggests that AT-HRNet can offer more beneficial solutions for human posture estimation.

语义一致性引导的多任务拼接篡改检测

目的 随着数字图像及编辑软件的广泛应用,伪造图像层出不穷,对新闻传播、法律取证等行业造成了影响。拼接伪造是一种常见的伪造方式,这种伪造方式往往会向原始图像中添加新的对象,导致原始图像语义受到改变、曲解。现有很多基于卷积神经网络的篡改检测方法都更关注篡改痕迹的特征提取,但忽略了伪造图像中的语义不一致。针对拼接伪造中原始图像发生的语义变化,提出了一种以篡改检测为主任务,语义分割和噪声重建为辅助任务的多分辨率全卷积神经网络。方法 通过多任务策略将语义分割和噪声重建作为辅助任务。语义分割任务旨在捕捉拼接伪造图像过程中产生的语义不一致现象,噪声重建任务允许网络获得更全面的图像噪声分布。为了使网络获取更全面、准确的特征,网络中的RGB流、噪声流和融合模块都使用多分辨率思想从多个分辨率上提取处理不同形状和大小的拼接对象。结果 本文与其他几种先进的篡改检测网络和基于HRNet(high-resolution network)的基线网络进行了对比实验,在Fantastic Reality和Spliced Dataset两个数据集中,本文方法均取得了最优性能,F1分数分别为0.946和0.961。对JPEG(joint photographic experts group)压缩、亮度调节、对比度调节和添加噪声进行鲁棒性实验,结果表明,本文方法针对常见的图像后处理手段具有良好的鲁棒性。结论 提出的语义一致性引导的多任务多分辨率拼接篡改检测网络检测更加准确,具有良好的鲁棒性,拓展了数字图像取证研究新思路。

Monitoring the green evolution of vernacular buildings based on deep learning and multi-temporal remote sensing images

The increasingly mature computer vision(CV)technology represented by convolutional neural networks(CNN)and available high-resolution remote sensing images(HR-RSIs)provide opportunities to accurately measure the evolution of natural and artificial environments on Earth at a large scale.Based on the advanced CNN method high-resolution net(HRNet)and multi-temporal HR-RSIs,a framework is proposed for monitoring a green evolution of courtyard buildings characterized by their courtyards being roofed(CBR).The proposed framework consists of an expert module focusing on scenes analysis,a CV module for automatic detection,an evaluation module containing thresholds,and an output module for data analysis.Based on this,the changes in the adoption of different CBR technologies(CBRTs),including light-translucent CBRTs(LT-CBRTs)and non-lighttranslucent CBRTs(NLT-CBRTs),in 24 villages in southern Hebei were identified from 2007 to 2021.The evolution of CBRTs was featured as an inverse S-curve,and differences were found in their evolution stage,adoption ratio,and development speed for different villages.LT-CBRTs are the dominant type but are being replaced and surpassed by NLT-CBRTs in some villages,characterizing different preferences for the technology type of villages.The proposed research framework provides a reference for the evolution monitoring of vernacular buildings,and the identified evolution laws enable to trace and predict the adoption of different CBRTs in a particular village.This work lays a foundation for future exploration of the occurrence and development mechanism of the CBR phenomenon and provides an important reference for the optimization and promotion of CBRTs.

Deep Learning Based Single Image Super-resolution:A Survey

Single image super-resolution has attracted increasing attention and has a wide range of applications in satellite imaging, medical imaging, computer vision, security surveillance imaging, remote sensing, objection detection, and recognition. Recently, deep learning techniques have emerged and blossomed, producing " the state-of-the-art” in many domains. Due to their capability in feature extraction and mapping, it is very helpful to predict high-frequency details lost in low-resolution images. In this paper, we give an overview of recent advances in deep learning-based models and methods that have been applied to single image super-resolution tasks. We also summarize, compare and discuss various models from the past and present for comprehensive understanding and finally provide open problems and possible directions for future research.

多分辨率特征注意力融合行人再识别

目的行人再识别是实现跨摄像头识别同一行人的关键技术,面临外观、光照、姿态、背景等问题,其中区别行人个体差异的核心是行人整体和局部特征的表征。为了高效地表征行人,提出一种多分辨率特征注意力融合的行人再识别方法。方法借助注意力机制,基于主干网络HRNet(high-resolution network),通过交错卷积构建4个不同的分支来抽取多分辨率行人图像特征,既对行人不同粒度特征进行抽取,也对不同分支特征进行交互,对行人进行高效的特征表示。结果在Market1501、CUHK03以及Duke MTMC-ReID这3个数据集上验证了所提方法的有效性,rank1分别达到95. 3%、72. 8%、90. 5%,mAP(mean average precision)分别达到89. 2%、70. 4%、81. 5%。在Market1501与Duke MTMC-ReID两个数据集上实验结果超越了当前最好表现。结论本文方法着重提升网络提取特征的能力,得到强有力的特征表示,可用于行人再识别、图像分类和目标检测等与特征提取相关的计算机视觉任务,显著提升行人再识别的准确性。

Water consumption and biodiversity:Responses to global emergency events

Given that it was a once-in-a-century emergency event,the confinement measures related to the coronavirus disease 2019(COVID-19)pandemic caused diverse disruptions and changes in life and work patterns.These changes significantly affected water consumption both during and after the pandemic,with direct and indirect consequences on biodiversity.However,there has been a lack of holistic evaluation of these responses.Here,we propose a novel framework to study the impacts of this unique global emergency event by embedding an environmentally extended supply-constrained global multi-regional input-output model(MRIO)into the drivers-pressure-state-impact-response(DPSIR)framework.This framework allowed us to develop scenarios related to COVID-19 confinement measures to quantify country-sector-specific changes in freshwater consumption and the associated changes in biodiversity for the period of 2020-2025.The results suggest progressively diminishing impacts due to the implementation of COVID-19 vaccines and the socio-economic system’s self-adjustment to the new normal.In 2020,the confinement measures were estimated to decrease global water consumption by about 5.7% on average across all scenarios when compared with the baseline level with no confinement measures.Further,such a decrease is estimated to lead to a reduction of around 5% in the related pressure on biodiversity.Given the interdependencies and interactions across global supply chains,even those countries and sectors that were not directly affected by the COVID-19 shocks experienced significant impacts:Our results indicate that the supply chain propagations contributed to 79% of the total estimated decrease in water consumption and 84%of the reduction in biodiversity loss on average.Our study demonstrates that the MRIO-enhanced DSPIR framework can help quantify resource pressures and the resultant environmental impacts across supply chains when facing a global emergency event.Further,we recommend the development of more locally based water conservation measures—to mitigate the effects of trade disruptions—and the explicit inclusion of water resources in post-pandemic recovery schemes.In addition,innovations that help conserve natural resources are essential for maintaining environmental gains in the post-pandemic world.