Real-Time Detection and Instance Segmentation of Strawberry in Unstructured Environment

The real-time detection and instance segmentation of strawberries constitute fundamental components in the development of strawberry harvesting robots.Real-time identification of strawberries in an unstructured envi-ronment is a challenging task.Current instance segmentation algorithms for strawberries suffer from issues such as poor real-time performance and low accuracy.To this end,the present study proposes an Efficient YOLACT(E-YOLACT)algorithm for strawberry detection and segmentation based on the YOLACT framework.The key enhancements of the E-YOLACT encompass the development of a lightweight attention mechanism,pyramid squeeze shuffle attention(PSSA),for efficient feature extraction.Additionally,an attention-guided context-feature pyramid network(AC-FPN)is employed instead of FPN to optimize the architecture’s performance.Furthermore,a feature-enhanced model(FEM)is introduced to enhance the prediction head’s capabilities,while efficient fast non-maximum suppression(EF-NMS)is devised to improve non-maximum suppression.The experimental results demonstrate that the E-YOLACT achieves a Box-mAP and Mask-mAP of 77.9 and 76.6,respectively,on the custom dataset.Moreover,it exhibits an impressive category accuracy of 93.5%.Notably,the E-YOLACT also demonstrates a remarkable real-time detection capability with a speed of 34.8 FPS.The method proposed in this article presents an efficient approach for the vision system of a strawberry-picking robot.

FIR-YOLACT:Fusion of ICIoU and Res2Net for YOLACT on Real-Time Vehicle Instance Segmentation

Autonomous driving technology has made a lot of outstanding achievements with deep learning,and the vehicle detection and classification algorithm has become one of the critical technologies of autonomous driving systems.The vehicle instance segmentation can perform instance-level semantic parsing of vehicle information,which is more accurate and reliable than object detection.However,the existing instance segmentation algorithms still have the problems of poor mask prediction accuracy and low detection speed.Therefore,this paper proposes an advanced real-time instance segmentation model named FIR-YOLACT,which fuses the ICIoU(Improved Complete Intersection over Union)and Res2Net for the YOLACT algorithm.Specifically,the ICIoU function can effectively solve the degradation problem of the original CIoU loss function,and improve the training convergence speed and detection accuracy.The Res2Net module fused with the ECA(Efficient Channel Attention)Net is added to the model’s backbone network,which improves the multi-scale detection capability and mask prediction accuracy.Furthermore,the Cluster NMS(Non-Maximum Suppression)algorithm is introduced in the model’s bounding box regression to enhance the performance of detecting similarly occluded objects.The experimental results demonstrate the superiority of FIR-YOLACT to the based methods and the effectiveness of all components.The processing speed reaches 28 FPS,which meets the demands of real-time vehicle instance segmentation.

Lane Line Detection Based on Improved PINet

Accurate perception of lane line information is one of the basic requirements of unmanned driving technology, which is related to the localization of the vehicle and the determination of the forward direction. In this paper, multi-level constraints are added to the lane line detection model PINet, which is used to improve the perception of lane lines. Predicted lane lines in the network are predicted to have real and imaginary attributes, which are used to enhance the perception of features around the lane lines, with pixel-level constraints on the lane lines;images are converted to bird’s-eye views, where the parallelism between lane lines is reconstructed, with lane line-level constraints on the predicted lane lines;and vanishing points are used to focus on the image hierarchy, with image-level constraints on the lane lines. The model proposed in this paper meets both accuracy (96.44%) and real-time (30 + FPS) requirements, has been tested on the highway on the ground, and has performed stably.

A New Algorithmic Approach for Detection and Identification of Vehicle Plate Numbers

This work proposes a method for the detection and identification of parked vehicles stationed. This technique composed many algorithms for the detection, localization, segmentation, extraction and recognition of number plates in images. It is acts of a technology of image processing used to identify the vehicles by their number plates. Knowing that we work on images whose level of gray is sampled with (120×180), resulting from a base of abundant data by PSA. We present two algorithms allowing the detection of the horizontal position of the vehicle: the classical method “horizontal gradients” and our approach “symmetrical method”. In fact, a car seen from the front presents a symmetry plan and by detecting its axis, that one finds its position in the image. A phase of localization is treated using the parameter MGD (Maximum Gradient Difference) which allows locating all the segments of text per horizontal scan. A specific technique of filtering, combining the method of symmetry and the localization by the MGD allows eliminating the blocks which don’t pass by the axis of symmetry and thus find the good block containing the number plate. Once we locate the plate, we use four algorithms that must be realized in order to allow our system to identify a license plate. The first algorithm is adjusting the intensity and the contrast of the image. The second algorithm is segmenting the characters on the plate using profile method. Then extracting and resizing the characters and finally recognizing them by means of optical character recogni-tion OCR. The efficiency of these algorithms is shown using a database of 350 images for the tests. We find a rate of lo-calization of 99.6% on a basis of 350 images with a rate of false alarms (wrong block text) of 0.88% by image.

Resource Efficient Hardware Implementation for Real-Time Traffic Sign Recognition

Traffic sign recognition (TSR, or Road Sign Recognition, RSR) is one of the Advanced Driver Assistance System (ADAS) devices in modern cars. To concern the most important issues, which are real-time and resource efficiency, we propose a high efficiency hardware implementation for TSR. We divide the TSR procedure into two stages, detection and recognition. In the detection stage, under the assumption that most German traffic signs have red or blue colors with circle, triangle or rectangle shapes, we use Normalized RGB color transform and Single-Pass Connected Component Labeling (CCL) to find the potential traffic signs efficiently. For Single-Pass CCL, our contribution is to eliminate the “merge-stack” operations by recording connected relations of region in the scan phase and updating the labels in the iterating phase. In the recognition stage, the Histogram of Oriented Gradient (HOG) is used to generate the descriptor of the signs, and we classify the signs with Support Vector Machine (SVM). In the HOG module, we analyze the required minimum bits under different recognition rate. The proposed method achieves 96.61% detection rate and 90.85% recognition rate while testing with the GTSDB dataset. Our hardware implementation reduces the storage of CCL and simplifies the HOG computation. Main CCL storage size is reduced by 20% comparing to the most advanced design under typical condition. By using TSMC 90 nm technology, the proposed design operates at 105 MHz clock rate and processes in 135 fps with the image size of 1360 × 800. The chip size is about 1 mm2 and the power consumption is close to 8 mW. Therefore, this work is resource efficient and achieves real-time requirement.

改进EDLines算法暨在高空输电线识别与巡检中的应用

针对无人机高空输电线实时巡线识别速度慢、精度低等问题,提出基于图像特征检测识别EDLines算法,以去除背景干扰提高输电线识别精度。通过改进锚点判定的方式搜索锚节点,搜索邻域梯度最大值作为绘制边缘的路径,利用随机抽样一致性(RANSAC)算法拟合直线段,以亥姆霍兹原理对虚假线段进行滤波处理,采用K均值聚类筛选出输电线特征直线,设计方向模板二次滤波定位输电线的位置。实验结果表明:本文算法识别输电线准确率提高,抗干扰能力增强,运行效率高,具有工程实践意义。

Real-time estimation of the structural utilization level of segmental tunnel lining

Over the last decades,an expansion of the underground network has been taking place to cope with the increasing amount of moving people and freight.As a consequence,it is of vital importance to guarantee the full functionality of the tunnel network by means of preventive maintenance and the monitoring of the tunnel lining state over time.A new method has been developed for the real-time prediction of the utilization level in tunnel segmental linings based on input monitoring data.The new concept is founded on a framework,which encompasses an offline and an online stage.In the former,the generation of feedforward neural networks is accomplished by employing synthetically produced data.Finite element simulations of the lining structure are conducted to analyze the structural response under multiple loading conditions.The scenarios are generated by assuming ranges of variation of the model input parameters to account for the uncertainty due to the not fully determined in situ conditions.Input and target quantities are identified to better assess the structural utilization of the lining.The latter phase consists in the application of the methodological framework on input monitored data,which allows for a real-time prediction of the physical quantities deployed for the estimation of the lining utilization.The approach is validated on a full-scale test of segmental lining,where the predicted quantities are compared with the actual measurements.Finally,it is investigated the influence of artificial noise added to the training data on the overall prediction performances and the benefits along with the limits of the concept are set out.

基于EDLines的遥感影像直线快速提取方法研究

随着现代科学技术的发展,大数据已经成为这个时代的标志。因此快速、准确地采集图像数据变得十分重要。本文研究的是实时线段检测算法EDLines,首先通过实验对比分析及所给公式,可以获得梯度幅度和方向、线验证参数、梯度阈值、锚点阈值和扫描间隔参数值,基于所获得的参数值,在处理不同类型图像时不需要改变其参数就能获取准确的结果,然后对所研究的算法EDLines与现今已知运行较快的直线提取算法进行对比分析,得到本文所研究的EDLines算法相比较其他直线提取算法都具有最优性。最后,根据本文研究的EDLines算法对遥感影像进行直线提取,获得清晰且假阳性较少的直线提取图像。超高的精度和计算速度使实时计算机视觉和图像处理的应用迈出了崭新的一步。

Lane departure warning systems and lane line detection methods based on image processing and semantic segmentation:A review

Recently,the development and application of lane line departure warning systems have been in the market.For any of the systems,the key part of lane line tracking,lane line identification,or lane line departure warning is whether it can accurately and quickly detect lane lines.Since 1990 s,they have been studied and implemented for the situations defined by the good viewing conditions and the clear lane markings on road.After then,the accuracy for particular situations,the robustness for a wide range of scenarios,time efficiency and integration into higher-order tasks define visual lane line detection and tracking as a continuing research subject.At present,these kinds of lane marking line detection methods based on machine vision and image processing can be divided into two categories:the traditional image processing and semantic segmentation(includes deep learning)methods.The former mainly involves feature-based and model-based steps,and which can be classified into similarity-and discontinuity-based ones;and the model-based step includes different parametric straight line,curve or pattern models.The semantic segmentation includes different machine learning,neural network and deep learning methods,which is the new trend for the research and application of lane line departure warning systems.This paper describes and analyzes the lane line departure warning systems,image processing algorithms and semantic segmentation methods for lane line detection.

融合边缘感知与统计纹理知识的输电线路金具锈蚀检测

针对输电线路金具目标小、背景环境复杂和锈蚀区域不规则等问题,提出了一种融合边缘感知与统计纹理知识的输电线路金具锈蚀检测算法。首先通过改进YOLOv7模型检测金具,然后利用改进Res-UNet模型对检测的金具进行锈蚀分割,加入SE(squeeze-excitation)注意力提高模型的稳定性,引入统计纹理知识模块(statistical texture knowledge module,STM)和边缘感知模块(edge-aware module,EAM),提出一种知识融合模块对边缘感知和统计纹理知识进行融合,提高对锈蚀分割精度。实验结果表明,检测和分割模型mAP分别提高了2.8百分点和7.7百分点。

基于改进HRNetV2的高寒区隧道衬砌冻害检测方法

针对修建在高寒区的隧道衬砌存在的所处环境恶劣、冻害频发、衬砌图像干扰因素多、冻害目标尺度不一致及传统人工目视检测方法效率低下且成本昂贵等问题,提出了基于HRNetV2的高寒区隧道衬砌冻害检测方法。首先以HRNetV2为基础模型,提出改进模型,在主干特征提取网络结合迁移学习的知识,在结构中引入注意力机制以加强模型对于冻害特征的学习能力,并使用Focalloss作为损失函数以解决类别不平衡问题。为验证改进后模型的性能,使用高清摄像头采集高寒区隧道衬砌冻害图像,经过裁剪及数据增强等手段,建立一个包含2800张图像的冻害数据集。实验结果表明,改进后的模型在冻害数据集上的平均交并比(mean intersection over union,mIoU)可达到89.05%,相比原始模型提升了5.41%,在面对复杂形态冻害时展现出较好的鲁棒性,可直接应用于高分辨率原图;且在综合性能上优于DeeplabV3+、U-Net、PSPNet三种模型。所提方法可准确、安全地实现衬砌冻害智能检测,可为高寒区隧道智能化运维提供一定技术支持。

基于直线段检测和LT描述符的矿井图像线特征匹配算法

图像匹配是同步定位与地图构建(SLAM)技术中极为重要的一环,用于根据图像之间的变换关系确定相机位姿。基于线特征的图像匹配方法具有较强的鲁棒性和抗噪能力,更加适用于井下图像匹配,基于深度学习的线描述符对线段遮挡等场景具有较高的鲁棒性,性能优于传统描述符,但卷积神经网络架构的描述符将可变长度线段抽象为固定维进行描述,不利于线段长度及视差变化较大图像的匹配。针对上述问题,提出一种基于直线段检测和线描述符的矿井图像线特征匹配算法。在频域利用单参数同态滤波降低图像的照射分量,并增强反射分量,提升亮度及对比度;在YUV空间利用对比度受限的自适应直方图均衡化(CLAHE)算法对亮度分量进行均衡,使亮度分布更加均匀;变换至RGB空间提取直线段检测(LSD)线,引入一种基于Transformer架构的LT描述符构建LSD线的特征向量,最后完成线特征匹配。实验结果表明:该算法结合了同态滤波和CLAHE算法的优点,增强后图像的亮度适中,对比度良好,灰度分布均匀,增强效果优于单参数同态滤波算法、EnlightenGAN算法;该算法提取的线特征数较原图平均提升了32.92%,在不同相似纹理占比、不同程度旋转与平移变化的井下图像匹配中鲁棒性好,平均正确匹配数为61.75对,平均精度为86.83%,优于线二进制描述符(LBD)算法、LBD_NNDR算法、LT算法,能够满足矿井图像稳健匹配的需求。

基于改进HT-LCNN线段检测模型的隧洞施工活动时间信息智能提取方法

施工活动时间信息的有效提取是隧洞施工进度分析与决策的重要前提,目前常采用的一种隧洞施工活动时间记录方式是绘制横道图线段。针对传统依赖于人工统计的方式存在效率低、易出错等问题,提出基于改进深度霍夫线条先验网络(Hough Transform-Line Convolutional Neural Network,HT-LCNN)线段检测模型的隧洞施工活动时间信息智能提取方法。首先,采用单应性变换手段进行施工日志图像预处理,解决原始图像存在的倾斜、旋转、扭曲等问题;其次,利用全局上下文注意力模块(Global Context Network,GCNet)改进HT-LCNN模型的残差模块,通过建立和共享全局注意力图,获得目标线段在特征图和通道间的长距离依赖关系,提高模型对目标手绘线段的注意力,克服原有HT-LCNN方法容易受到表格线段和文字干扰的不足,实现手绘线段的高精度智能检测;进一步地,建立施工时刻-活动坐标系,根据所检测的手绘横道图线段的端点坐标位置特征,将其自动转化为施工活动时间信息。将该方法应用于某长距离引水隧洞TBM施工日志活动时间提取,本文提出的改进HT-LCNN模型的检测精度AP_(5)、AP_(10)、AP_(15)值分别为94.7%、95.0%、95.1%,均高于HT-LCNN和LCNN;基于本文方法自动提取的施工活动时间与人工提取结果相比,平均绝对误差仅为1.82 min。本研究为隧洞施工活动时间信息准确高效提取提供了新思路。

激光雷达点云特征线检测与结构化研究进展与展望

实景三维(3D)建设是国家新型基础测绘建设的重要组成部分,是对在3D地理场景上承载结构化、语义化、支持人机兼容理解和物联实时感知的地理实体进行构建。而结构化是实现地理实体单体分割并获取其几何轮廓及组成结构的过程。随着大范围、城市级激光点云数据采集成本逐渐降低,3D点云的应用也越来越广泛。基于3D点云能完成生成地物的结构化表达,涉及线、面、体重建及语义分割、实例分割等。点云数据处理的核心内容是点云特征线检测、特征线线段化及特征线结构化。因此,点云特征线对点云处理和行业应用具有理论和实践价值。进行城市场景点云自适应特征线检测和精细结构化研究具有重要意义;研究点云特征线智能感知理论及结构化表达方法能为点云数据处理提供新的视角,对各类工程实践具有重要的理论指导意义,可服务于实景3D建设、二维(2D)、3D高清地图生成等具体应用。本文从点云特征线检测、点云特征线线段化、点云特征线结构化角度进行国内外研究现状综述,总结归纳点云特征线检测与结构化研究存在问题,并明确下一步研究思路。

基于L0梯度平滑与图像分块聚类的海天线检测

海天线检测在海洋工程安防活动中具有重要的意义,真实海洋环境中的海天线检测易受云朵、海浪、光照变化、目标遮挡物、边界模糊等外界干扰。为了实现对真实海洋环境中海天线的检测,本研究提出一种基于L0梯度平滑和图像分块聚类的海天线检测算法。首先,对图像进行L0梯度平滑滤波,以增强海天线边缘,弱化非海天线因素干扰;接着,将图像沿着竖直方向分割成若干等宽图像块,以降低整体环境干扰,加强局部海天线检测效果;然后,通过Canny算子和霍夫变换提取每个分割图像块中的直线段;最后,采取K-means聚类算法提取每个图像块中的海天线段,拟合生成完整海天线。实验结果表明,在真实的海天线数据集中,本研究方法获取的矩形框重叠率平均精度为93.22%,角度差平均精度为7.66%,均高于文中选取的近年典型对比算法。满足实际海天线检测抗干扰强、准确率高、适应性广等要求。

融合多尺度特征的残差车道线检测网络

针对车道线分布范围广、占像素少、特征不易提取的问题,本文构建了一种基于多尺度特征融合的残差车道线检测网络。首先,以残差双边网络为基础,采用双边特征聚合模块,利用语义分支的上下文信息指导同一阶段的细节分支的特征响应,并融合两分支的信息;然后,针对不同阶段具有不同尺度,使用多尺度自适应特征对齐融合模块,构建采样前后偏移向量索引表,降低因简单采样而造成的细节信息缺失;最后,引入空间注意力机制,增强模型的长距离特征捕捉能力。试验结果表明,本文模型在3个公开数据集上均取得了良好效果,其中在CULane数据集上的准确度达77.89%,比目前主流算法高2%。

基于线激光传感器的盾构管片自动拼装方法

针对盾构管片自动化拼装中的管片位姿检测问题,提出了一种无需事先标记管片且适用于几何参数未知的盾构管片自动化拼装方案,并推导了管片位姿检测算法。该方案利用安装在拼装机器人末端执行器抓取板上的5个线激光传感器分别对管片的直边和弧边进行照射,通过检测线激光照射在管片边缘而产生的偏折点的3D坐标,逐步计算出管片在拼装过程中相对于拼装机器人的基坐标系的位姿,再据此计算出机器人将待拼装管片拼装至期望位姿时,机器人末端执行器相对于基坐标系的位姿,从而实现管片的自动化拼装。为了验证该算法的有效性,在实验室环境中搭建了基于线激光传感器的管片自动拼装实验平台,并利用该平台进行了管片自动拼装实验。实验结果表明,提出的盾构管片自动拼装方法在管片定位方面表现出较小的误差,且管片拼装最大错台为1.50 mm,最大间隙为0.96 mm,符合工程要求。这为盾构管片的自动化拼装提供了一种有效的技术解决方案。

基于直线生成的卷积霍夫线段检测

直线检测作为计算机视觉的上游任务,为下游包括工业视觉、遥感图像分析等任务提供支撑。直线检测的一大方向是霍夫直线检测,但现有霍夫检测基于近似原理设计计票器,直线检测准确度不高。为提高霍夫变换线段检测的准确度,利用卷积改进霍夫直线检测的计票器并提出基于直线生成的卷积霍夫线段检测方法。利用中值滤波对原始图像中的复杂纹理平滑处理后检测图像中的边界;通过按位异或卷积去除边界检测结果中的噪点并保留候选的线段端点;将候选的线段端点两两组合并使用布雷森汉姆算法进行线段生成,由利用卷积改进的计票器判断生成的线段是否存在于边界上;确认端点所构成线段位于边界后求取端点对之间的参数并合并参数相似的加检测结果,得到最终线段检测结果。对比实验中该方法的F1指标为0.7626,优于对比方法中最高的0.6523,证明该方法保留了霍夫变换较高鲁棒性的同时提高了检测结果的准确性。

红外架空导线图像的分割与检测方法

针对架空高压电线巡检难度高、检测效率低等问题,提出了一种架空高压导线分割与检测的方法。该方法利用直方图双峰法、感兴趣区域提取、滤波器与图像差分相结合的导线优化方法来剔除掉图像中云层、杆塔、地面等干扰项。通过优化了投票与筛选机制的改进的霍夫直线检测算法提取导线,最后通过绝缘子与导线间的拓扑关系截取需要的导线部分。实验结果表明:本文算法检测性能良好,平均交并比为94.4%,导线检测正确率92.8%。满足实际工业生产要求。

基于语义分割网络的植保机器人视觉伺服控制方法

为实现稳定可靠的植保机器人视觉伺服控制,提出了一种基于语义分割网络的作物行特征检测方法。基于语义分割网络ESNet实现农田场景图像像素级带状区域检测,并利用最小二乘算法拟合得到每条行作物线特征;在此基础上通过设计一种主导航线提取算法获取导航路径,并利用卡尔曼滤波对主导航线几何参数进行平滑处理,有效抑制了不平整地面导致的机器人运动颠簸与视觉图像测量噪声引起的导航参数波动。继而构建机器人前轮转向、后轮差速的阿克曼运动学模型;在图像空间坐标下设计纯追踪控制器实现植保机器人的伺服运动控制。大田环境下的现场实验结果为:总体横向偏差为0.092 m,验证了本文方法的有效性。