The point pair feature(PPF)is widely used for 6D pose estimation.In this paper,we propose an efficient 6D pose estimation method based on the PPF framework.We introduce a well-targeted down-sampling strategy that focu...The point pair feature(PPF)is widely used for 6D pose estimation.In this paper,we propose an efficient 6D pose estimation method based on the PPF framework.We introduce a well-targeted down-sampling strategy that focuses on edge areas for efficient feature extraction for complex geometry.A pose hypothesis validation approach is proposed to resolve ambiguity due to symmetry by calculating the edge matching degree.We perform evaluations on two challenging datasets and one real-world collected dataset,demonstrating the superiority of our method for pose estimation for geometrically complex,occluded,symmetrical objects.We further validate our method by applying it to simulated punctures.展开更多
In unstructured environments such as disaster sites and mine tunnels,it is a challenge for robots to estimate the poses of objects under complex lighting backgrounds,which limit their operation.Owing to the shadows pr...In unstructured environments such as disaster sites and mine tunnels,it is a challenge for robots to estimate the poses of objects under complex lighting backgrounds,which limit their operation.Owing to the shadows produced by a point light source,the brightness of the operation scene is seriously unbalanced,and it is difficult to accurately extract the features of objects.It is especially difficult to accurately label the poses of objects with weak corners and textures.This study proposes an automatic pose annotation method for such objects,which combine 3D-2D matching projection and rendering technology to improve the efficiency of dataset annotation.A 6D object pose estimation method under low-light conditions(LP_TGC)is then proposed,including(1)a light preprocessing neural network model based on a low-light preprocessing module(LPM)to balance the brightness of a picture and improve its quality;and(2)a 6D pose estimation model(TGC)based on the keypoint matching.Four typical datasets are constructed to verify our method,the experimental results validated and demonstrated the effectiveness of the proposed LP_TGC method.The estimation model based on the preprocessed image can accurately estimate the pose of the object in the mentioned unstructured environments,and it can improve the accuracy by an average of~3%based on the ADD metric.展开更多
We propose a feature-fusion network for pose estimation directly from RGB images without any depth information in this study.First,we introduce a two-stream architecture consisting of segmentation and regression strea...We propose a feature-fusion network for pose estimation directly from RGB images without any depth information in this study.First,we introduce a two-stream architecture consisting of segmentation and regression streams.The segmentation stream processes the spatial embedding features and obtains the corresponding image crop.These features are further coupled with the image crop in the fusion network.Second,we use an efficient perspective-n-point(E-PnP)algorithm in the regression stream to extract robust spatial features between 3D and 2D keypoints.Finally,we perform iterative refinement with an end-to-end mechanism to improve the estimation performance.We conduct experiments on two public datasets of YCB-Video and the challenging Occluded-LineMOD.The results show that our method outperforms state-of-the-art approaches in both the speed and the accuracy.展开更多
The 6D pose estimation is important for the safe take-off and landing of the aircraft using a single RGB image. Due to the large scene and large depth, the exiting pose estimation methods have unstratified performance...The 6D pose estimation is important for the safe take-off and landing of the aircraft using a single RGB image. Due to the large scene and large depth, the exiting pose estimation methods have unstratified performance on the accuracy. To achieve precise 6D pose estimation of the aircraft, an end-to-end method using an RGB image is proposed. In the proposed method, the2D and 3D information of the keypoints of the aircraft is used as the intermediate supervision,and 6D pose information of the aircraft in this intermediate information will be explored. Specifically, an off-the-shelf object detector is utilized to detect the Region of the Interest(Ro I) of the aircraft to eliminate background distractions. The 2D projection and 3D spatial information of the pre-designed keypoints of the aircraft is predicted by the keypoint coordinate estimator(Kp Net).The proposed method is trained in an end-to-end fashion. In addition, to deal with the lack of the related datasets, this paper builds the Aircraft 6D Pose dataset to train and test, which captures the take-off and landing process of three types of aircraft from 11 views. Compared with the latest Wide-Depth-Range method on this dataset, our proposed method improves the average 3D distance of model points metric(ADD) and 5° and 5 m metric by 86.8% and 30.1%, respectively. Furthermore, the proposed method gets 9.30 ms, 61.0% faster than YOLO6D with 23.86 ms.展开更多
A localization method based on distance function of projected features is presented to solve the accuracy reduction or failure problem due to occlusion and blurring caused by smog, when dealing with vision based local...A localization method based on distance function of projected features is presented to solve the accuracy reduction or failure problem due to occlusion and blurring caused by smog, when dealing with vision based localization for target oil and gas wellhead （OGWH）. Firstly, the target OGWH is modeled as a cylinder with marker, and a vector with redundant parameter is used to describe its pose. Secondly, the explicit mapping relationship between the pose vector with redundant parameter and projected features is derived. Then, a 2D-point-to-feature distance function is proposed, as well as its derivative. Finally, based on this distance function and its derivative, an algorithm is proposed to estimate the pose of target OGWH directly according to the 2D image information, and the validity of the method is verified by both synthetic data and real image experiments. The results show that this method is able to accomplish the localization in the case of occlusion and blurring, and its anti-noise ability is good especially with noise ratio of less than 70%.展开更多
基金This work was supported in part by the National Key R&D Program of China(2018AAA0102200)National Natural Science Foundation of China(62132021,62102435,61902419,62002375,62002376)+2 种基金Natural Science Foundation of Hunan Province of China(2021JJ40696)Huxiang Youth Talent Support Program(2021RC3071)NUDT Research Grants(ZK19-30,ZK22-52).
文摘The point pair feature(PPF)is widely used for 6D pose estimation.In this paper,we propose an efficient 6D pose estimation method based on the PPF framework.We introduce a well-targeted down-sampling strategy that focuses on edge areas for efficient feature extraction for complex geometry.A pose hypothesis validation approach is proposed to resolve ambiguity due to symmetry by calculating the edge matching degree.We perform evaluations on two challenging datasets and one real-world collected dataset,demonstrating the superiority of our method for pose estimation for geometrically complex,occluded,symmetrical objects.We further validate our method by applying it to simulated punctures.
基金supported by the National Key Research and Development Program of China(Grant No.2018YFB1305300)the China Postdoctoral Science Foundation(Grant Nos.2020TQ0039 and 2021M700425)the National Natural Science Foundation of China(Grant Nos.61733001,62103054,U2013602,61873039,U1913211 and U1713215)。
文摘In unstructured environments such as disaster sites and mine tunnels,it is a challenge for robots to estimate the poses of objects under complex lighting backgrounds,which limit their operation.Owing to the shadows produced by a point light source,the brightness of the operation scene is seriously unbalanced,and it is difficult to accurately extract the features of objects.It is especially difficult to accurately label the poses of objects with weak corners and textures.This study proposes an automatic pose annotation method for such objects,which combine 3D-2D matching projection and rendering technology to improve the efficiency of dataset annotation.A 6D object pose estimation method under low-light conditions(LP_TGC)is then proposed,including(1)a light preprocessing neural network model based on a low-light preprocessing module(LPM)to balance the brightness of a picture and improve its quality;and(2)a 6D pose estimation model(TGC)based on the keypoint matching.Four typical datasets are constructed to verify our method,the experimental results validated and demonstrated the effectiveness of the proposed LP_TGC method.The estimation model based on the preprocessed image can accurately estimate the pose of the object in the mentioned unstructured environments,and it can improve the accuracy by an average of~3%based on the ADD metric.
基金the National Key Research and Development Program of China under Grant No.2021YFB1715900the National Natural Science Foundation of China under Grant Nos.12022117 and 61802406+2 种基金the Beijing Natural Science Foundation under Grant No.Z190004the Beijing Advanced Discipline Fund under Grant No.115200S001Alibaba Group through Alibaba Innovative Research Program.
文摘We propose a feature-fusion network for pose estimation directly from RGB images without any depth information in this study.First,we introduce a two-stream architecture consisting of segmentation and regression streams.The segmentation stream processes the spatial embedding features and obtains the corresponding image crop.These features are further coupled with the image crop in the fusion network.Second,we use an efficient perspective-n-point(E-PnP)algorithm in the regression stream to extract robust spatial features between 3D and 2D keypoints.Finally,we perform iterative refinement with an end-to-end mechanism to improve the estimation performance.We conduct experiments on two public datasets of YCB-Video and the challenging Occluded-LineMOD.The results show that our method outperforms state-of-the-art approaches in both the speed and the accuracy.
基金co-supported by the Key research and development plan project of Sichuan Province,China(No.2022YFG0153).
文摘The 6D pose estimation is important for the safe take-off and landing of the aircraft using a single RGB image. Due to the large scene and large depth, the exiting pose estimation methods have unstratified performance on the accuracy. To achieve precise 6D pose estimation of the aircraft, an end-to-end method using an RGB image is proposed. In the proposed method, the2D and 3D information of the keypoints of the aircraft is used as the intermediate supervision,and 6D pose information of the aircraft in this intermediate information will be explored. Specifically, an off-the-shelf object detector is utilized to detect the Region of the Interest(Ro I) of the aircraft to eliminate background distractions. The 2D projection and 3D spatial information of the pre-designed keypoints of the aircraft is predicted by the keypoint coordinate estimator(Kp Net).The proposed method is trained in an end-to-end fashion. In addition, to deal with the lack of the related datasets, this paper builds the Aircraft 6D Pose dataset to train and test, which captures the take-off and landing process of three types of aircraft from 11 views. Compared with the latest Wide-Depth-Range method on this dataset, our proposed method improves the average 3D distance of model points metric(ADD) and 5° and 5 m metric by 86.8% and 30.1%, respectively. Furthermore, the proposed method gets 9.30 ms, 61.0% faster than YOLO6D with 23.86 ms.
基金supported by National Natural Science Foundation of China(No.61403226)the State Key Laboratory of Tribology of China(No.SKLT09A03)
文摘A localization method based on distance function of projected features is presented to solve the accuracy reduction or failure problem due to occlusion and blurring caused by smog, when dealing with vision based localization for target oil and gas wellhead （OGWH）. Firstly, the target OGWH is modeled as a cylinder with marker, and a vector with redundant parameter is used to describe its pose. Secondly, the explicit mapping relationship between the pose vector with redundant parameter and projected features is derived. Then, a 2D-point-to-feature distance function is proposed, as well as its derivative. Finally, based on this distance function and its derivative, an algorithm is proposed to estimate the pose of target OGWH directly according to the 2D image information, and the validity of the method is verified by both synthetic data and real image experiments. The results show that this method is able to accomplish the localization in the case of occlusion and blurring, and its anti-noise ability is good especially with noise ratio of less than 70%.
