Towards complex scenes: A deep learning-based camouflaged people detection method for snapshot multispectral images

Camouflaged people are extremely expert in actively concealing themselves by effectively utilizing cover and the surrounding environment. Despite advancements in optical detection capabilities through imaging systems, including spectral, polarization, and infrared technologies, there is still a lack of effective real-time method for accurately detecting small-size and high-efficient camouflaged people in complex real-world scenes. Here, this study proposes a snapshot multispectral image-based camouflaged detection model, multispectral YOLO(MS-YOLO), which utilizes the SPD-Conv and Sim AM modules to effectively represent targets and suppress background interference by exploiting the spatial-spectral target information. Besides, the study constructs the first real-shot multispectral camouflaged people dataset(MSCPD), which encompasses diverse scenes, target scales, and attitudes. To minimize information redundancy, MS-YOLO selects an optimal subset of 12 bands with strong feature representation and minimal inter-band correlation as input. Through experiments on the MSCPD, MS-YOLO achieves a mean Average Precision of 94.31% and real-time detection at 65 frames per second, which confirms the effectiveness and efficiency of our method in detecting camouflaged people in various typical desert and forest scenes. Our approach offers valuable support to improve the perception capabilities of unmanned aerial vehicles in detecting enemy forces and rescuing personnel in battlefield.

Intelligent Sensing and Control of Road Construction Robot Scenes Based on Road Construction

Automatic control technology is the basis of road robot improvement,according to the characteristics of construction equipment and functions,the research will be input type perception from positioning acquisition,real-world monitoring,the process will use RTK-GNSS positional perception technology,by projecting the left side of the earth from Gauss-Krueger projection method,and then carry out the Cartesian conversion based on the characteristics of drawing;steering control system is the core of the electric drive unmanned module,on the basis of the analysis of the composition of the steering system of unmanned engineering vehicles,the steering system key components such as direction,torque sensor,drive motor and other models are established,the joint simulation model of unmanned engineering vehicles is established,the steering controller is designed using the PID method,the simulation results show that the control method can meet the construction path demand for automatic steering.The path planning will first formulate the construction area with preset values and realize the steering angle correction during driving by PID algorithm,and never realize the construction-based path planning,and the results show that the method can control the straight path within the error of 10 cm and the curve error within 20 cm.With the collaboration of various modules,the automatic construction simulation results of this robot show that the design path and control method is effective.

Autonomous landing scene recognition based on transfer learning for drones

In this paper, we study autonomous landing scene recognition with knowledge transfer for drones. Considering the difficulties in aerial remote sensing, especially that some scenes are extremely similar, or the same scene has different representations in different altitudes, we employ a deep convolutional neural network(CNN) based on knowledge transfer and fine-tuning to solve the problem. Then, LandingScenes-7 dataset is established and divided into seven classes. Moreover, there is still a novelty detection problem in the classifier, and we address this by excluding other landing scenes using the approach of thresholding in the prediction stage. We employ the transfer learning method based on ResNeXt-50 backbone with the adaptive momentum(ADAM) optimization algorithm. We also compare ResNet-50 backbone and the momentum stochastic gradient descent(SGD) optimizer. Experiment results show that ResNeXt-50 based on the ADAM optimization algorithm has better performance. With a pre-trained model and fine-tuning, it can achieve 97.845 0% top-1 accuracy on the LandingScenes-7dataset, paving the way for drones to autonomously learn landing scenes.

Traffic Scene Captioning with Multi-Stage Feature Enhancement

Traffic scene captioning technology automatically generates one or more sentences to describe the content of traffic scenes by analyzing the content of the input traffic scene images,ensuring road safety while providing an important decision-making function for sustainable transportation.In order to provide a comprehensive and reasonable description of complex traffic scenes,a traffic scene semantic captioningmodel withmulti-stage feature enhancement is proposed in this paper.In general,the model follows an encoder-decoder structure.First,multilevel granularity visual features are used for feature enhancement during the encoding process,which enables the model to learn more detailed content in the traffic scene image.Second,the scene knowledge graph is applied to the decoding process,and the semantic features provided by the scene knowledge graph are used to enhance the features learned by the decoder again,so that themodel can learn the attributes of objects in the traffic scene and the relationships between objects to generate more reasonable captions.This paper reports extensive experiments on the challenging MS-COCO dataset,evaluated by five standard automatic evaluation metrics,and the results show that the proposed model has improved significantly in all metrics compared with the state-of-the-art methods,especially achieving a score of 129.0 on the CIDEr-D evaluation metric,which also indicates that the proposed model can effectively provide a more reasonable and comprehensive description of the traffic scene.

Object detection in crowded scenes via joint prediction

Detecting highly-overlapped objects in crowded scenes remains a challenging problem,especially for one-stage detector.In this paper,we extricate YOLOv4 from the dilemma in a crowd by fine-tuning its detection scheme,named YOLO-CS.Specifically,we give YOLOv4 the power to detect multiple objects in one cell.Center to our method is the carefully designed joint prediction scheme,which is executed through an assignment of bounding boxes and a joint loss.Equipped with the derived joint-object augmentation(DJA),refined regression loss(RL)and Score-NMS(SN),YOLO-CS achieves competitive detection performance on CrowdHuman and CityPersons benchmarks compared with state-of-the-art detectors at the cost of little time.Furthermore,on the widely used general benchmark COCO,YOLOCS still has a good performance,indicating its robustness to various scenes.

A Lightweight Road Scene Semantic Segmentation Algorithm

In recent years,with the continuous deepening of smart city construction,there have been significant changes and improvements in the field of intelligent transportation.The semantic segmentation of road scenes has important practical significance in the fields of automatic driving,transportation planning,and intelligent transportation systems.However,the current mainstream lightweight semantic segmentation models in road scene segmentation face problems such as poor segmentation performance of small targets and insufficient refinement of segmentation edges.Therefore,this article proposes a lightweight semantic segmentation model based on the LiteSeg model improvement to address these issues.The model uses the lightweight backbone network MobileNet instead of the LiteSeg backbone network to reduce the network parameters and computation,and combines the Coordinate Attention(CA)mechanism to help the network capture long-distance dependencies.At the same time,by combining the dependencies of spatial information and channel information,the Spatial and Channel Network(SCNet)attention mechanism is proposed to improve the feature extraction ability of the model.Finally,a multiscale transposed attention encoding(MTAE)module was proposed to obtain features of different resolutions and perform feature fusion.In this paper,the proposed model is verified on the Cityscapes dataset.The experimental results show that the addition of SCNet and MTAE modules increases the mean Intersection over Union(mIoU)of the original LiteSeg model by 4.69%.On this basis,the backbone network is replaced with MobileNet,and the CA model is added at the same time.At the cost of increasing the minimum model parameters and computing costs,the mIoU of the original LiteSeg model is increased by 2.46%.This article also compares the proposed model with some current lightweight semantic segmentation models,and experiments show that the comprehensive performance of the proposed model is the best,especially in achieving excellent results in small object segmentation.Finally,this article will conduct generalization testing on the KITTI dataset for the proposed model,and the experimental results show that the proposed algorithm has a certain degree of generalization.

Exploiting Human Pose and Scene Information for Interaction Detection

Identifying human actions and interactions finds its use in manyareas, such as security, surveillance, assisted living, patient monitoring, rehabilitation,sports, and e-learning. This wide range of applications has attractedmany researchers to this field. Inspired by the existing recognition systems,this paper proposes a new and efficient human-object interaction recognition(HOIR) model which is based on modeling human pose and scene featureinformation. There are different aspects involved in an interaction, includingthe humans, the objects, the various body parts of the human, and the backgroundscene. Themain objectives of this research include critically examiningthe importance of all these elements in determining the interaction, estimatinghuman pose through image foresting transform (IFT), and detecting the performedinteractions based on an optimizedmulti-feature vector. The proposedmethodology has six main phases. The first phase involves preprocessing theimages. During preprocessing stages, the videos are converted into imageframes. Then their contrast is adjusted, and noise is removed. In the secondphase, the human-object pair is detected and extracted from each image frame.The third phase involves the identification of key body parts of the detectedhumans using IFT. The fourth phase relates to three different kinds of featureextraction techniques. Then these features are combined and optimized duringthe fifth phase. The optimized vector is used to classify the interactions in thelast phase. TheMSRDaily Activity 3D dataset has been used to test this modeland to prove its efficiency. The proposed system obtains an average accuracyof 91.7% on this dataset.

Scene image recognition with knowledge transfer for drone navigation

In this paper,we study scene image recognition with knowledge transfer for drone navigation.We divide navigation scenes into three macro-classes,namely outdoor special scenes(OSSs),the space from indoors to outdoors or from outdoors to indoors transitional scenes(TSs),and others.However,there are difficulties in how to recognize the TSs,to this end,we employ deep convolutional neural network(CNN)based on knowledge transfer,techniques for image augmentation,and fine tuning to solve the issue.Moreover,there is still a novelty detection prob-lem in the classifier,and we use global navigation satellite sys-tems(GNSS)to solve it in the prediction stage.Experiment results show our method,with a pre-trained model and fine tun-ing,can achieve 91.3196%top-1 accuracy on Scenes21 dataset,paving the way for drones to learn to understand the scenes around them autonomously.

Multi-Modal Scene Matching Location Algorithm Based on M2Det

In recent years,many visual positioning algorithms have been proposed based on computer vision and they have achieved good results.However,these algorithms have a single function,cannot perceive the environment,and have poor versatility,and there is a certain mismatch phenomenon,which affects the positioning accuracy.Therefore,this paper proposes a location algorithm that combines a target recognition algorithm with a depth feature matching algorithm to solve the problem of unmanned aerial vehicle(UAV)environment perception and multi-modal image-matching fusion location.This algorithm was based on the single-shot object detector based on multi-level feature pyramid network(M2Det)algorithm and replaced the original visual geometry group(VGG)feature extraction network with the ResNet-101 network to improve the feature extraction capability of the network model.By introducing a depth feature matching algorithm,the algorithm shares neural network weights and realizes the design of UAV target recognition and a multi-modal image-matching fusion positioning algorithm.When the reference image and the real-time image were mismatched,the dynamic adaptive proportional constraint and the random sample consensus consistency algorithm(DAPC-RANSAC)were used to optimize the matching results to improve the correct matching efficiency of the target.Using the multi-modal registration data set,the proposed algorithm was compared and analyzed to verify its superiority and feasibility.The results show that the algorithm proposed in this paper can effectively deal with the matching between multi-modal images(visible image–infrared image,infrared image–satellite image,visible image–satellite image),and the contrast,scale,brightness,ambiguity deformation,and other changes had good stability and robustness.Finally,the effectiveness and practicability of the algorithm proposed in this paper were verified in an aerial test scene of an S1000 sixrotor UAV.

Robust Counting in Overcrowded Scenes Using Batch-Free Normalized Deep ConvNet

The analysis of overcrowded areas is essential for flow monitoring,assembly control,and security.Crowd counting’s primary goal is to calculate the population in a given region,which requires real-time analysis of congested scenes for prompt reactionary actions.The crowd is always unexpected,and the benchmarked available datasets have a lot of variation,which limits the trained models’performance on unseen test data.In this paper,we proposed an end-to-end deep neural network that takes an input image and generates a density map of a crowd scene.The proposed model consists of encoder and decoder networks comprising batch-free normalization layers known as evolving normalization(EvoNorm).This allows our network to be generalized for unseen data because EvoNorm is not using statistics from the training samples.The decoder network uses dilated 2D convolutional layers to provide large receptive fields and fewer parameters,which enables real-time processing and solves the density drift problem due to its large receptive field.Five benchmark datasets are used in this study to assess the proposed model,resulting in the conclusion that it outperforms conventional models.

Study on Recognition Method of Similar Weather Scenes in Terminal Area

Weather is a key factor affecting the control of air traffic.Accurate recognition and classification of similar weather scenes in the terminal area is helpful for rapid decision-making in air trafficflow management.Current researches mostly use traditional machine learning methods to extract features of weather scenes,and clustering algorithms to divide similar scenes.Inspired by the excellent performance of deep learning in image recognition,this paper proposes a terminal area similar weather scene classification method based on improved deep convolution embedded clustering(IDCEC),which uses the com-bination of the encoding layer and the decoding layer to reduce the dimensionality of the weather image,retaining useful information to the greatest extent,and then uses the combination of the pre-trained encoding layer and the clustering layer to train the clustering model of the similar scenes in the terminal area.Finally,term-inal area of Guangzhou Airport is selected as the research object,the method pro-posed in this article is used to classify historical weather data in similar scenes,and the performance is compared with other state-of-the-art methods.The experi-mental results show that the proposed IDCEC method can identify similar scenes more accurately based on the spatial distribution characteristics and severity of weather;at the same time,compared with the actualflight volume in the Guangz-hou terminal area,IDCEC's recognition results of similar weather scenes are con-sistent with the recognition of experts in thefield.

多场景下基于传感器的行为识别

针对基于传感器的行为识别任务中识别场景单一且固定的问题,提出一种多场景下基于传感器的行为识别迁移模型,由基于传感器的动态感知算法(dynamic perception algorithm,DPA)和自适应场景的行为识别迁移方法(adaptive scene human recognition,AHR)两部分组成,解决在固定场景下对传感器的依赖性以及在场景转换时识别模型失效的问题。DPA提出两阶段迁移模式,将行为识别阶段和模型迁移阶段同步推进,保证模型在传感器异动发生后仍能持续拥有识别能力。进一步提出AHR场景迁移方法,实现模型在多场景下的行为识别能力。实验验证该模型具有更优的适应性和可扩展性。

Development Strategy of Urban Cultural Scene to Promote the Upgrading of Regional Cultural Consumption in Shaanxi

Regional cultural patterns and characteristics play a positive role in economic and social development.By planning and constructing cultural amenities and creating cultural scenes,the spatial quality and quality of life in a region can be enhanced,facilitating the expansion of cultural consumption.Shaanxi,with its rich historical and cultural resources,positions the capital city of Xi’an as a“world historical city”,boasting a vast number of cultural amenities represented by“cultural facilities”,“cultural activities”,“cultural experiences”,and“cultural services”.The development of urban cultural scene,with the aim of promoting the upgrading of regional cultural consumption in Shaanxi,requires comprehensive planning and a multifaceted approach,particularly in integrating provincial cultural scenes,clarifying the positioning of cultural scenes,innovating cultural scene experience projects,creating cultural scene intellectual property(IP),and empowering cultural scenes through the application of science and technology.

Analysis of the Writing Characteristics of the Title Music,Using“Scenes of Childhood”and“Children’s Garden”as Examples

In European thought and culture,there exists a group of passionate artists who are fascinated by the intention,passion,and richness of artistic expression.They strive to establish connections between different art forms.Musicians not only attempt to represent masterpieces through the language of music but also aim to convey subjective experiences of emotions and personal imagination to listeners by adding titles to their musical works.This study examines two pieces,“Scenes of Childhood”and“Children’s Garden”,and analyzes the different approaches employed by the composers in portraying similar content.

虚拟现实场景下的精准化采摘机器人作业研究

利用虚拟现实场景仿真技术,对葡萄采摘机器人的精准作业控制过程进行分析,通过构建虚拟现实环境下葡萄采摘机器人作业场景模型,以机器人采摘运动柔顺、无碰撞和不损伤采摘对象为目标,对采摘作业过程的夹紧、托举以及剪切动作进行虚拟仿真。虚拟现实场景下20次采摘动作仿真结果中,有3次采摘发生碰撞、17次采摘成功。这表明,在设计阶段利用虚拟现实场景进行精准化作业仿真和控制算法测试,能够有效缩短采摘机器人研制过程中的调试和优化时间。

动态场景的三维重建研究综述

随着静态场景三维重建算法的不断成熟,动态场景三维重建算法成为近年来的研究热点和研究难点。现有的静态场景三维重建算法对静止的对象有较好的重建效果,一旦场景中对象出现变形或者是相对运动,其重建效果不太理想,因此发展对动态场景的三维重建研究工作是相当重要的。简要介绍三维重建的相关概念及基本知识、静态场景三维重建和动态场景三维重建的研究分类及研究现状;全面总结了动态场景三维重建研究最新进展,将动态场景三维重建按照基于RGB数据源的动态三维重建和基于RGB-D数据源的动态三维重建进行分类,其中RGB数据源下又可划分为基于模板的动态三维重建、基于非刚性运动恢复结构的动态三维重建和RGB数据源下基于学习的动态三维重建,RGB-D数据源下主要总结归纳基于学习的动态三维重建,对各类典型重建算法进行了介绍和对比分析;介绍了动态场景三维重建在医学、智能制造、虚拟现实与增强现实、交通等领域的应用;提出了动态场景三维重建的未来研究方向,并对这个快速发展领域中的各个方向研究进行了展望。

多光源照射下目标图像实时生成方法

由于地理位置、太阳、大气环境等因素限制,无法获取空间目标在各种姿态、光照条件、特别是激光、太阳和背景光共同作用下的实际成像。提出一种多光源照射下目标图像实时生成方法。该方法基于计算机图形学中纹理映射思想,采用现代图形显卡编程技术和帧缓存对象特性,在GPU(Graphics Processing Unit)端采用着色器语言实现多光源作用下目标亮度值高效计算和真实感增强;采用开源三维图形引擎OSG(Open SceneGraph)支持多种格式三维模型文件,提高与国产麒麟操作系统及常用战场态势显示软件的兼容性。仿真实验验证了该方法的有效性和优越性。

面向无人机绝对定位的遥感影像快速检索方法

针对在复杂环境下无人机景象匹配导航中的视觉绝对定位问题,提出了一种聚合深度学习特征的实时影像快速检索方法。首先,引入可训练软分配深度学习框架—NetVLAD,结合VGG16网络提取并聚合生成影像稳定的全局特征表达向量;其次,在初始检索阶段,使用KD树结构对影像全局特征向量构建检索索引,在不损失检索精度的前提下提高检索速度;最后,使用皮尔逊积矩相关系数对初始检索结果进行快速预判断,自动过滤初始检索结果,对于需要重排序的影像则采用特征学习匹配算法——图神经网络SuperGlue进行匹配重排序。所提方法在公开的夏季和冬季遥感影像数据集分组进行实验,实验结果表明:未重排序条件下,初始检索结果第一张影像平均准确率达到了58.27%,部分特征较好地区准确率达到了85%,对不同时相遥感影像也有很好的适应性,平均检索一张影像耗时3.7 s,可为无人机景象匹配导航的初始定位提供参考。

航迹先验融合特征的车载雷达实例分割算法

点云实例分割是场景感知中的基本任务。近年来,随着车载毫米波雷达分辨能力的提高,大量基于毫米波雷达散射点的实例分割方案被提出。实例分割的结果可作为跟踪的输入,跟踪得到各个实例的航迹信息,为后续的车辆决策与路径规划提供数据支持。然而,面向毫米波雷达的实例分割方法仍存在以下挑战。一方面,相较于激光雷达,毫米波雷达观测下的散射点更稀疏,信息量较少。当同一实例的散射点距离较远或者多个相邻实例密集分布时,分割性能显著下降;另一方面,雷达穿透性有限,路面障碍物或交通参与者对实例造成部分遮挡时,分割算法无法对实例进行正确分割和判别。考虑到实际行车场景的时间连续性,利用交通参与者的航迹先验信息,即该参与者上一时刻和当前时刻的位置信息,可以克服上述问题。因此,本文提出了一种利用航迹先验融合上一帧散射点特征的车载雷达点云分割算法。该算法利用航迹的连续性,在相邻两帧之间计算实例和散射点的对应关系并基于上述关系完成散射点特征融合。相较于单帧,融合后的高质量特征不仅信息更丰富,不同实例间的特征差异更明显,而且能弥补由于遮挡导致的信息缺失。实验结果显示,所提算法的平均覆盖率和平均精度指标分别优于基于单帧的分割算法6.19%和4.54%。该结果表明,所提算法优于文献中其他方法,能有效解决上述分割算法存在的问题。此外,与基于单帧的分割方案在典型场景的可视化对比中,所提方法也凸显了其有效性和潜力。未来,我们将进一步挖掘轨迹先验信息,以加强特征提取,同时深入探讨分割性能与帧数之间的关系。

基于动作条件交互的高效行人过街意图预测

城市化的进程不断加速,人车冲突问题已成为现代社会亟待解决的重大难题。复杂交通场景下,行人横穿马路行为导致交通事故频发,准确、实时地预测行人过街意图对避免人车冲突、提高驾驶安全系数和保障行人安全至关重要。本文提出基于动作条件交互的高效行人过街意图预测框架(efficient action-conditioned interaction pedestrian crossing intention anticipation framework,EAIPF)来预测行人过街意图。EAIPF引入行人动作编码模块增强多模态动作模式下的表征能力,挖掘深层骨架上下文信息。同时,引入场景对象交互模块挖掘与对象交互信息,理解交通场景中高级语义线索。最后,意图预测模块融合行人动作特征和对象交互特征,实现行人过街意图的鲁棒预测。所提出的方法在两个公共数据集JAAD和PIE上验证算法性能,准确率分别达到了89%和90%,表明本文方法可以在复杂交通场景下准确预测行人穿越意图。