苹果的可见光谱目标的高效、精准识别是实现果园测产或机器自动采摘作业的关键,由于绿色目标果实与枝叶背景颜色较为相近,因此绿色苹果的识别成为新的挑战。再由于果园实际复杂环境因素影响,如光照、阴雨、枝叶遮挡、目标重叠等情况,现...苹果的可见光谱目标的高效、精准识别是实现果园测产或机器自动采摘作业的关键,由于绿色目标果实与枝叶背景颜色较为相近,因此绿色苹果的识别成为新的挑战。再由于果园实际复杂环境因素影响,如光照、阴雨、枝叶遮挡、目标重叠等情况,现有的目标果实识别方案难以满足测产或自动采摘的实时、精准作业需求。为更好地实现果园自然环境中绿色目标果实识别问题,提出一种新的核密度估计优化的聚类分割算法(kernel density clustering,KDC)。新算法首先利用简单的迭代聚类(simple linear iterative cluster,SLIC)算法将目标图像分割成不规则块,集结小区域内近似像素点组成超像素区域,计算单元由像素点转变为超像素区域,有效降低数据复杂度,且SLIC算法简化图像数据时可有效避免目标果实轮廓模糊;基于超像素构造R-B区域均值和G-B区域均值的二维特征分量,建立针对聚类分析的青苹果颜色特征空间。然后借助密度峰值聚类中心计算绿色苹果图像每个数据点的局部密度和局部差异度,为解决分割边界模糊问题,在计算过程中利用核密度估计计算局部密度,确保局部密度在不同复杂场景中的清晰准确表达,以更精准找出被低密度区域分割的高密度区域,实现任意形状的聚类。最后以局部密度和距离构造寻找聚类中心的决策图,该研究采用双排序算法实现聚类中心的自动选择,完成目标果实的高效分割。新算法通过SLIC算法获得图像的超像素区域表示,数据点的局部密度通过核密度估计得到,大幅降低算法的计算量,实现目标图像的高效、精准分割。为更好地验证新算法性能,实验采集多光照、阴雨等环境下的遮挡、重叠等复杂目标图像,以分割效率、分割有效性、假阳性、假阴性等指标进行评价,通过对比k-means聚类算法、meanshift聚类算法、FCM算法和DPCA算法,该研究提出的新算法分割性能均最优。展开更多
To model a true three-dimensional(3D)display system,we introduced the method of voxel molding to obtain the stereoscopic imaging space of the system.For the distribution of each voxel,we proposed a four-dimensional(4D...To model a true three-dimensional(3D)display system,we introduced the method of voxel molding to obtain the stereoscopic imaging space of the system.For the distribution of each voxel,we proposed a four-dimensional(4D)Givone–Roessor(GR)model for state-space representation—that is,we established a local state-space model with the 3D position and one-dimensional time coordi-nates to describe the system.First,we extended the original elementary operation approach to a 4D condition and proposed the implementation steps of the realiza-tion matrix of the 4D GR model.Then,we described the working process of a true 3D display system,analyzed its real-time performance,introduced the fixed-point quantization model to simplify the system matrix,and derived the conditions for the global asymptotic stability of the system after quantization.Finally,we provided an example to prove the true 3D display system’s feasibility by simulation.The GR-model-representation method and its implementation steps proposed in this paper simplified the system’s mathematical expression and facilitated the microcon-troller software implementation.Real-time and stability analyses can be used widely to analyze and design true 3D display systems.展开更多
Visual simultaneous localisation and mapping(vSLAM)finds applications for indoor and outdoor navigation that routinely subjects it to visual complexities,particularly mirror reflections.The effect of mirror presence(t...Visual simultaneous localisation and mapping(vSLAM)finds applications for indoor and outdoor navigation that routinely subjects it to visual complexities,particularly mirror reflections.The effect of mirror presence(time visible and its average size in the frame)was hypothesised to impact localisation and mapping performance,with systems using direct techniques expected to perform worse.Thus,a dataset,MirrEnv,of image sequences recorded in mirror environments,was collected,and used to evaluate the performance of existing representative methods.RGBD ORB-SLAM3 and BundleFusion appear to show moderate degradation of absolute trajectory error with increasing mirror duration,whilst the remaining results did not show significantly degraded localisation performance.The mesh maps generated proved to be very inaccurate,with real and virtual reflections colliding in the reconstructions.A discussion is given of the likely sources of error and robustness in mirror environments,outlining future directions for validating and improving vSLAM performance in the presence of planar mirrors.The MirrEnv dataset is available at https://doi.org/10.17035/d.2023.0292477898.展开更多
Bagging is an essential skill that humans perform in their daily activities.However,deformable objects,such as bags,are complex for robots to manipulate.A learning-based framework that enables robots to learn bagging ...Bagging is an essential skill that humans perform in their daily activities.However,deformable objects,such as bags,are complex for robots to manipulate.A learning-based framework that enables robots to learn bagging is presented.The novelty of this framework is its ability to learn and perform bagging without relying on simulations.The learning process is accomplished through a reinforcement learning(RL)algorithm introduced and designed to find the best grasping points of the bag based on a set of compact state representations.The framework utilises a set of primitive actions and represents the task in five states.In our experiments,the framework reached 60% and 80% success rates after around 3 h of training in the real world when starting the bagging task from folded and unfolded states,respectively.Finally,the authors test the trained RL model with eight more bags of different sizes to evaluate its generalisability.展开更多
Despite of significant achievements made in the detection of target fruits,small fruit detection remains a great challenge,especially for immature small green fruits with a few pixels.The closeness of color between th...Despite of significant achievements made in the detection of target fruits,small fruit detection remains a great challenge,especially for immature small green fruits with a few pixels.The closeness of color between the fruit skin and the background greatly increases the difficulty of locating small target fruits in the natural orchard environment.In this paper,we propose a balanced feature pyramid network(BFP Net)for small apple detection.展开更多
To better address the difficulties in designing green fruit recognition techniques in machine vision systems,a new fruit detection model is proposed.This model is an optimization of the FCOS(full convolution one-stage...To better address the difficulties in designing green fruit recognition techniques in machine vision systems,a new fruit detection model is proposed.This model is an optimization of the FCOS(full convolution one-stage object detection)algorithm,incorporating LSC(level scales,spaces,channels)attention blocks in the network structure,and named FCOS-LSC.The method achieves efficient recognition and localization of green fruit images affected by overlapping occlusions,lighting conditions,and capture angles.Specifically,the improved feature extraction network ResNet50 with added deformable convolution is used to fully extract green fruit feature information.The feature pyramid network(FPN)is employed to fully fuse low-level detail information and high-level semantic information in a cross-connected and top-down connected way.Next,the attention mechanisms are added to each of the 3 dimensions of scale,space(including the height and width of the feature map),and channel of the generated multiscale feature map to improve the feature perception capability of the network.Finally,the classification and regression subnetworks of the model are applied to predict the fruit category and bounding box.In the classification branch,a new positive and negative sample selection strategy is applied to better distinguish supervised signals by designing weights in the loss function to achieve more accurate fruit detection.The proposed FCOS-LSC model has 38.65M parameters,38.72G floating point operations,and mean average precision of 63.0%and 75.2%for detecting green apples and green persimmons,respectively.In summary,FCOS-LSC outperforms the state-of-the-art models in terms of precision and complexity to meet the accurate and efficient requirements of green fruit recognition using intelligent agricultural equipment.Correspondingly,FCOS-LSC can be used to improve the robustness and generalization of the green fruit detection models.展开更多
基金Focus on Research and Development Plan in Shandong Province(2019GNC106115)China Postdoctoral Science Foundation(2018M630797)+1 种基金Shandong Province Higher Educational Science and Technology Program(J18KA308)National Nature Science Foundation of China(62072289)。
文摘苹果的可见光谱目标的高效、精准识别是实现果园测产或机器自动采摘作业的关键,由于绿色目标果实与枝叶背景颜色较为相近,因此绿色苹果的识别成为新的挑战。再由于果园实际复杂环境因素影响,如光照、阴雨、枝叶遮挡、目标重叠等情况,现有的目标果实识别方案难以满足测产或自动采摘的实时、精准作业需求。为更好地实现果园自然环境中绿色目标果实识别问题,提出一种新的核密度估计优化的聚类分割算法(kernel density clustering,KDC)。新算法首先利用简单的迭代聚类(simple linear iterative cluster,SLIC)算法将目标图像分割成不规则块,集结小区域内近似像素点组成超像素区域,计算单元由像素点转变为超像素区域,有效降低数据复杂度,且SLIC算法简化图像数据时可有效避免目标果实轮廓模糊;基于超像素构造R-B区域均值和G-B区域均值的二维特征分量,建立针对聚类分析的青苹果颜色特征空间。然后借助密度峰值聚类中心计算绿色苹果图像每个数据点的局部密度和局部差异度,为解决分割边界模糊问题,在计算过程中利用核密度估计计算局部密度,确保局部密度在不同复杂场景中的清晰准确表达,以更精准找出被低密度区域分割的高密度区域,实现任意形状的聚类。最后以局部密度和距离构造寻找聚类中心的决策图,该研究采用双排序算法实现聚类中心的自动选择,完成目标果实的高效分割。新算法通过SLIC算法获得图像的超像素区域表示,数据点的局部密度通过核密度估计得到,大幅降低算法的计算量,实现目标图像的高效、精准分割。为更好地验证新算法性能,实验采集多光照、阴雨等环境下的遮挡、重叠等复杂目标图像,以分割效率、分割有效性、假阳性、假阴性等指标进行评价,通过对比k-means聚类算法、meanshift聚类算法、FCM算法和DPCA算法,该研究提出的新算法分割性能均最优。
基金This work was supported by the Key Research and Development Projects of Science and Technology Development Plan of Jilin Provincial Department of Science and Technology(20180201090gx).
文摘To model a true three-dimensional(3D)display system,we introduced the method of voxel molding to obtain the stereoscopic imaging space of the system.For the distribution of each voxel,we proposed a four-dimensional(4D)Givone–Roessor(GR)model for state-space representation—that is,we established a local state-space model with the 3D position and one-dimensional time coordi-nates to describe the system.First,we extended the original elementary operation approach to a 4D condition and proposed the implementation steps of the realiza-tion matrix of the 4D GR model.Then,we described the working process of a true 3D display system,analyzed its real-time performance,introduced the fixed-point quantization model to simplify the system matrix,and derived the conditions for the global asymptotic stability of the system after quantization.Finally,we provided an example to prove the true 3D display system’s feasibility by simulation.The GR-model-representation method and its implementation steps proposed in this paper simplified the system’s mathematical expression and facilitated the microcon-troller software implementation.Real-time and stability analyses can be used widely to analyze and design true 3D display systems.
基金funded by the UK EPSRC through a Doctoral Training Partnership No.EP/T517951/1(2435656).
文摘Visual simultaneous localisation and mapping(vSLAM)finds applications for indoor and outdoor navigation that routinely subjects it to visual complexities,particularly mirror reflections.The effect of mirror presence(time visible and its average size in the frame)was hypothesised to impact localisation and mapping performance,with systems using direct techniques expected to perform worse.Thus,a dataset,MirrEnv,of image sequences recorded in mirror environments,was collected,and used to evaluate the performance of existing representative methods.RGBD ORB-SLAM3 and BundleFusion appear to show moderate degradation of absolute trajectory error with increasing mirror duration,whilst the remaining results did not show significantly degraded localisation performance.The mesh maps generated proved to be very inaccurate,with real and virtual reflections colliding in the reconstructions.A discussion is given of the likely sources of error and robustness in mirror environments,outlining future directions for validating and improving vSLAM performance in the presence of planar mirrors.The MirrEnv dataset is available at https://doi.org/10.17035/d.2023.0292477898.
基金This work was partially supported by Consejo Nacional de Humanidades,Ciencias y Tecnologías(CONAHCyT)the Engineering and Physical Sciences Research Council(grant No.EP/X018962/1).
文摘Bagging is an essential skill that humans perform in their daily activities.However,deformable objects,such as bags,are complex for robots to manipulate.A learning-based framework that enables robots to learn bagging is presented.The novelty of this framework is its ability to learn and perform bagging without relying on simulations.The learning process is accomplished through a reinforcement learning(RL)algorithm introduced and designed to find the best grasping points of the bag based on a set of compact state representations.The framework utilises a set of primitive actions and represents the task in five states.In our experiments,the framework reached 60% and 80% success rates after around 3 h of training in the real world when starting the bagging task from folded and unfolded states,respectively.Finally,the authors test the trained RL model with eight more bags of different sizes to evaluate its generalisability.
基金This work is supported by the Natural Science Foundation of Shandong Province in China(No.:ZR2020MF076)the National Nature Science Foundation of China(No.:62072289)+2 种基金the Focus on Research and Development Plan in Shandong Province(No.:2019GNC106115)the Taishan Scholar Program of Shandong Province of Chinathe New Twentieth Items of Universities in Jinan(2021GXRC049).
文摘Despite of significant achievements made in the detection of target fruits,small fruit detection remains a great challenge,especially for immature small green fruits with a few pixels.The closeness of color between the fruit skin and the background greatly increases the difficulty of locating small target fruits in the natural orchard environment.In this paper,we propose a balanced feature pyramid network(BFP Net)for small apple detection.
基金supported by the National Nature Science Foundation of China(no.62072289)Natural Science Foundation of Shandong Province in China(no.ZR2020MF076)+1 种基金New Twentieth Items of Universities in Jinan(2021GXRC049)Taishan Scholar Program of Shandong Province of China.
文摘To better address the difficulties in designing green fruit recognition techniques in machine vision systems,a new fruit detection model is proposed.This model is an optimization of the FCOS(full convolution one-stage object detection)algorithm,incorporating LSC(level scales,spaces,channels)attention blocks in the network structure,and named FCOS-LSC.The method achieves efficient recognition and localization of green fruit images affected by overlapping occlusions,lighting conditions,and capture angles.Specifically,the improved feature extraction network ResNet50 with added deformable convolution is used to fully extract green fruit feature information.The feature pyramid network(FPN)is employed to fully fuse low-level detail information and high-level semantic information in a cross-connected and top-down connected way.Next,the attention mechanisms are added to each of the 3 dimensions of scale,space(including the height and width of the feature map),and channel of the generated multiscale feature map to improve the feature perception capability of the network.Finally,the classification and regression subnetworks of the model are applied to predict the fruit category and bounding box.In the classification branch,a new positive and negative sample selection strategy is applied to better distinguish supervised signals by designing weights in the loss function to achieve more accurate fruit detection.The proposed FCOS-LSC model has 38.65M parameters,38.72G floating point operations,and mean average precision of 63.0%and 75.2%for detecting green apples and green persimmons,respectively.In summary,FCOS-LSC outperforms the state-of-the-art models in terms of precision and complexity to meet the accurate and efficient requirements of green fruit recognition using intelligent agricultural equipment.Correspondingly,FCOS-LSC can be used to improve the robustness and generalization of the green fruit detection models.