嵌入式开发需要良好的软硬件环境,目前ARM公司的开发工具ADS、RealView以及Keil与ARM核处理器结合的较好,但硬件开发平台昂贵.Proteus软件较好的解决了硬件仿真的问题,它支持外围数字电路和模拟电路与处理器协同仿真,可以随意搭建硬件...嵌入式开发需要良好的软硬件环境,目前ARM公司的开发工具ADS、RealView以及Keil与ARM核处理器结合的较好,但硬件开发平台昂贵.Proteus软件较好的解决了硬件仿真的问题,它支持外围数字电路和模拟电路与处理器协同仿真,可以随意搭建硬件虚拟仿真平台.针对Keil for ARM编译器,结合Proteus软件,以ARM处理器LPC2124介绍了如何构建ARM嵌入式开发平台进行源代码级调试的方法.为嵌入式系统学习提出了一种新的思路和方法.展开更多
As the agricultural internet of things(IoT)technology has evolved,smart agricultural robots needs to have both flexibility and adaptability when moving in complex field environments.In this paper,we propose the concep...As the agricultural internet of things(IoT)technology has evolved,smart agricultural robots needs to have both flexibility and adaptability when moving in complex field environments.In this paper,we propose the concept of a vision-based navigation system for the agricultural IoT and a binocular vision navigation algorithm for smart agricultural robots,which can fuse the edge contour and the height information of rows of crop in images to extract the navigation parameters.First,the speeded-up robust feature(SURF)extracting and matching algorithm is used to obtain featuring point pairs from the green crop row images observed by the binocular parallel vision system.Then the confidence density image is constructed by integrating the enhanced elevation image and the corresponding binarized crop row image,where the edge contour and the height information of crop row are fused to extract the navigation parameters(θ,d)based on the model of a smart agricultural robot.Finally,the five navigation network instruction sets are designed based on the navigation angleθand the lateral distance d,which represent the basic movements for a certain type of smart agricultural robot working in a field.Simulated experimental results in the laboratory show that the algorithm proposed in this study is effective with small turning errors and low standard deviations,and can provide a valuable reference for the further practical application of binocular vision navigation systems in smart agricultural robots in the agricultural IoT system.展开更多
In this paper,we present a robot vision based system for coordinate measurement of feature points on large scale automobile parts.Our system consists of an industrial 6-DOF robot mounted with a CCD camera and a PC.The...In this paper,we present a robot vision based system for coordinate measurement of feature points on large scale automobile parts.Our system consists of an industrial 6-DOF robot mounted with a CCD camera and a PC.The system controls the robot into the area of feature points.The images of measuring feature points are acquired by the camera mounted on the robot.3D positions of the feature points are obtained from a model based pose estimation that applies to the images.The measured positions of all feature points are then transformed to the reference coordinate of feature points whose positions are obtained from the coordinate measuring machine（CMM）.Finally,the point-to-point distances between the measured feature points and the reference feature points are calculated and reported.The results show that the root mean square error（RMSE） of measure values obtained by our system is less than 0.5 mm.Our system is adequate for automobile assembly and can perform faster than conventional methods.展开更多
Remarkable progress has been made in self-supervised monocular depth estimation (SS-MDE) by exploring cross-view consistency, e.g., photometric consistency and 3D point cloud consistency. However, they are very vulner...Remarkable progress has been made in self-supervised monocular depth estimation (SS-MDE) by exploring cross-view consistency, e.g., photometric consistency and 3D point cloud consistency. However, they are very vulnerable to illumination variance, occlusions, texture-less regions, as well as moving objects, making them not robust enough to deal with various scenes. To address this challenge, we study two kinds of robust cross-view consistency in this paper. Firstly, the spatial offset field between adjacent frames is obtained by reconstructing the reference frame from its neighbors via deformable alignment, which is used to align the temporal depth features via a depth feature alignment (DFA) loss. Secondly, the 3D point clouds of each reference frame and its nearby frames are calculated and transformed into voxel space, where the point density in each voxel is calculated and aligned via a voxel density alignment (VDA) loss. In this way, we exploit the temporal coherence in both depth feature space and 3D voxel space for SS-MDE, shifting the “point-to-point” alignment paradigm to the “region-to-region” one. Compared with the photometric consistency loss as well as the rigid point cloud alignment loss, the proposed DFA and VDA losses are more robust owing to the strong representation power of deep features as well as the high tolerance of voxel density to the aforementioned challenges. Experimental results on several outdoor benchmarks show that our method outperforms current state-of-the-art techniques. Extensive ablation study and analysis validate the effectiveness of the proposed losses, especially in challenging scenes. The code and models are available at https://github.com/sunnyHelen/RCVC-depth.展开更多
3D shape recognition has drawn much attention in recent years.The view-based approach performs best of all.However,the current multi-view methods are almost all fully supervised,and the pretraining models are almost a...3D shape recognition has drawn much attention in recent years.The view-based approach performs best of all.However,the current multi-view methods are almost all fully supervised,and the pretraining models are almost all based on ImageNet.Although the pretraining results of ImageNet are quite impressive,there is still a significant discrepancy between multi-view datasets and ImageNet.Multi-view datasets naturally retain rich 3D information.In addition,large-scale datasets such as ImageNet require considerable cleaning and annotation work,so it is difficult to regenerate a second dataset.In contrast,unsupervised learning methods can learn general feature representations without any extra annotation.To this end,we propose a three-stage unsupervised joint pretraining model.Specifically,we decouple the final representations into three fine-grained representations.Data augmentation is utilized to obtain pixel-level representations within each view.And we boost the spatial invariant features from the view level.Finally,we exploit global information at the shape level through a novel extract-and-swap module.Experimental results demonstrate that the proposed method gains significantly in 3D object classification and retrieval tasks,and shows generalization to cross-dataset tasks.展开更多
The irregular domain and lack of ordering make it challenging to design deep neural networks for point cloud processing.This paper presents a novel framework named Point Cloud Transformer(PCT)for point cloud learning....The irregular domain and lack of ordering make it challenging to design deep neural networks for point cloud processing.This paper presents a novel framework named Point Cloud Transformer(PCT)for point cloud learning.PCT is based on Transformer,which achieves huge success in natural language processing and displays great potential in image processing.It is inherently permutation invariant for processing a sequence of points,making it well-suited for point cloud learning.To better capture local context within the point cloud,we enhance input embedding with the support of farthest point sampling and nearest neighbor search.Extensive experiments demonstrate that the PCT achieves the state-of-the-art performance on shape classification,part segmentation,semantic segmentation,and normal estimation tasks.展开更多
Recently, a qualitative approach was proposed for 3-D shape recovery based on a hybrid object representation[1]. In this approach, aspect recovery is the most important stage which binds regions in the image into mean...Recently, a qualitative approach was proposed for 3-D shape recovery based on a hybrid object representation[1]. In this approach, aspect recovery is the most important stage which binds regions in the image into meaningful aspects to support 3-D primitive recovery. There is no known polynondal time algo-rithm to solve this problem. The previous approach dealt with this problem by using a heuristic method based on the conditional probability. Unlike the previous method, this paper presents a novel parallel voting scheme to conquer the problem for efficiency. For this purpose) the previous global aspect rep-resentation is replaced with a distributed representation of aspects. Based on this representation, a three-layer parallel voting network for aspect recovery is proposed. For evaluating likelihood, a continuous Hopfield net is employed so that all aspect coverings in decreasing order of likelihood can be enumerated.The paper describes this method in detail and demonstrates its usefulness with simulation.展开更多
A significant performance boost has been achieved in point cloud semantic segmentation by utilization of the encoder-decoder architecture and novel convolution operations for point clouds.However,co-occurrence relatio...A significant performance boost has been achieved in point cloud semantic segmentation by utilization of the encoder-decoder architecture and novel convolution operations for point clouds.However,co-occurrence relationships within a local region which can directly influence segmentation results are usually ignored by current works.In this paper,we propose a neighborhood co-occurrence matrix(NCM)to model local co-occurrence relationships in a point cloud.We generate target NCM and prediction NCM from semantic labels and a prediction map respectively.Then,Kullback-Leibler(KL)divergence is used to maximize the similarity between the target and prediction NCMs to learn the co-occurrence relationship.Moreover,for large scenes where the NCMs for a sampled point cloud and the whole scene differ greatly,we introduce a reverse form of KL divergence which can better handle the difference to supervise the prediction NCMs.We integrate our method into an existing backbone and conduct comprehensive experiments on three datasets:Semantic3D for outdoor space segmentation,and S3DIS and ScanNet v2 for indoor scene segmentation.Results indicate that our method can significantly improve upon the backbone and outperform many leading competitors.展开更多
Ceramic structural parts are one of the most widely utilized structural parts in the industry. However, they usually contain defects following the pressing process, such as burrs. Therefore, additional trimming is usu...Ceramic structural parts are one of the most widely utilized structural parts in the industry. However, they usually contain defects following the pressing process, such as burrs. Therefore, additional trimming is usually required, despite the deformation challenges and difficulty in positioning. This paper proposes an ultrafast laser processing system for trimming complex ceramic structural parts. Opto-electromechanical cooperative control software is developed to control the laser processing system. The trimming problem of the ceramic cores used in aero engines is studied. The regional registration method is introduced based on the iterative closest point algorithm to register the path extracted from the computer-aided design model with the deformed ceramic core. A zonal and layering processing method for three-dimensional contours on complex surfaces is proposed to generate the working data of high-speed scanning galvanometer and the computer numerical control machine tool, respectively. The results show that the laser system and the method proposed in this paper are suitable for trimming complex non-datum parts such as ceramic cores. Compared with the results of manual trimming, the method proposed in this paper has higher accuracy, efficiency, and yield. The method mentioned above has been used in practical application with satisfactory results.展开更多
Three-dimensional geovisualizations are currently pushed both by technological development and by the demands of experts in various applied areas.In the presented empirical study,we compared the features of real 3D(st...Three-dimensional geovisualizations are currently pushed both by technological development and by the demands of experts in various applied areas.In the presented empirical study,we compared the features of real 3D(stereoscopic)versus pseudo 3D(monoscopic)geovisualizations in static and interactive digital elevation models.We tested 39 high-school students in their ability to identify the correct terrain profile from digital elevation models.Students’performance was recorded and further analysed with respect to their spatial abilities,which were measured by a psychological mental rotation test and think aloud protocol.The results of the study indicated that the influence of the type of 3D visualization(monoscopic/stereoscopic)on the performance of the users is not clear,the level of navigational interactivity has significant influence on the usability of a particular 3D visualization,and finally no influences of the spatial abilities on the performance of the user within the 3D environment were identified.展开更多
文摘嵌入式开发需要良好的软硬件环境,目前ARM公司的开发工具ADS、RealView以及Keil与ARM核处理器结合的较好,但硬件开发平台昂贵.Proteus软件较好的解决了硬件仿真的问题,它支持外围数字电路和模拟电路与处理器协同仿真,可以随意搭建硬件虚拟仿真平台.针对Keil for ARM编译器,结合Proteus软件,以ARM处理器LPC2124介绍了如何构建ARM嵌入式开发平台进行源代码级调试的方法.为嵌入式系统学习提出了一种新的思路和方法.
基金the National Natural Science Foundationof China(No.31760345).
文摘As the agricultural internet of things(IoT)technology has evolved,smart agricultural robots needs to have both flexibility and adaptability when moving in complex field environments.In this paper,we propose the concept of a vision-based navigation system for the agricultural IoT and a binocular vision navigation algorithm for smart agricultural robots,which can fuse the edge contour and the height information of rows of crop in images to extract the navigation parameters.First,the speeded-up robust feature(SURF)extracting and matching algorithm is used to obtain featuring point pairs from the green crop row images observed by the binocular parallel vision system.Then the confidence density image is constructed by integrating the enhanced elevation image and the corresponding binarized crop row image,where the edge contour and the height information of crop row are fused to extract the navigation parameters(θ,d)based on the model of a smart agricultural robot.Finally,the five navigation network instruction sets are designed based on the navigation angleθand the lateral distance d,which represent the basic movements for a certain type of smart agricultural robot working in a field.Simulated experimental results in the laboratory show that the algorithm proposed in this study is effective with small turning errors and low standard deviations,and can provide a valuable reference for the further practical application of binocular vision navigation systems in smart agricultural robots in the agricultural IoT system.
基金wsupported by the Thailand Research Fund and Solimac Automation Co.,Ltd.under the Research and Researchers for Industry Program(RRI)under Grant No.MSD56I0098Office of the Higher Education Commission under the National Research University Project of Thailand
文摘In this paper,we present a robot vision based system for coordinate measurement of feature points on large scale automobile parts.Our system consists of an industrial 6-DOF robot mounted with a CCD camera and a PC.The system controls the robot into the area of feature points.The images of measuring feature points are acquired by the camera mounted on the robot.3D positions of the feature points are obtained from a model based pose estimation that applies to the images.The measured positions of all feature points are then transformed to the reference coordinate of feature points whose positions are obtained from the coordinate measuring machine（CMM）.Finally,the point-to-point distances between the measured feature points and the reference feature points are calculated and reported.The results show that the root mean square error（RMSE） of measure values obtained by our system is less than 0.5 mm.Our system is adequate for automobile assembly and can perform faster than conventional methods.
文摘Remarkable progress has been made in self-supervised monocular depth estimation (SS-MDE) by exploring cross-view consistency, e.g., photometric consistency and 3D point cloud consistency. However, they are very vulnerable to illumination variance, occlusions, texture-less regions, as well as moving objects, making them not robust enough to deal with various scenes. To address this challenge, we study two kinds of robust cross-view consistency in this paper. Firstly, the spatial offset field between adjacent frames is obtained by reconstructing the reference frame from its neighbors via deformable alignment, which is used to align the temporal depth features via a depth feature alignment (DFA) loss. Secondly, the 3D point clouds of each reference frame and its nearby frames are calculated and transformed into voxel space, where the point density in each voxel is calculated and aligned via a voxel density alignment (VDA) loss. In this way, we exploit the temporal coherence in both depth feature space and 3D voxel space for SS-MDE, shifting the “point-to-point” alignment paradigm to the “region-to-region” one. Compared with the photometric consistency loss as well as the rigid point cloud alignment loss, the proposed DFA and VDA losses are more robust owing to the strong representation power of deep features as well as the high tolerance of voxel density to the aforementioned challenges. Experimental results on several outdoor benchmarks show that our method outperforms current state-of-the-art techniques. Extensive ablation study and analysis validate the effectiveness of the proposed losses, especially in challenging scenes. The code and models are available at https://github.com/sunnyHelen/RCVC-depth.
基金This work was supported in part by National Natural Science Foundation of China(No.61976095)the Science and Technology Planning Project of Guangdong Province,China(No.2018B030323026).
文摘3D shape recognition has drawn much attention in recent years.The view-based approach performs best of all.However,the current multi-view methods are almost all fully supervised,and the pretraining models are almost all based on ImageNet.Although the pretraining results of ImageNet are quite impressive,there is still a significant discrepancy between multi-view datasets and ImageNet.Multi-view datasets naturally retain rich 3D information.In addition,large-scale datasets such as ImageNet require considerable cleaning and annotation work,so it is difficult to regenerate a second dataset.In contrast,unsupervised learning methods can learn general feature representations without any extra annotation.To this end,we propose a three-stage unsupervised joint pretraining model.Specifically,we decouple the final representations into three fine-grained representations.Data augmentation is utilized to obtain pixel-level representations within each view.And we boost the spatial invariant features from the view level.Finally,we exploit global information at the shape level through a novel extract-and-swap module.Experimental results demonstrate that the proposed method gains significantly in 3D object classification and retrieval tasks,and shows generalization to cross-dataset tasks.
基金supported by the National Natural Science Foundation of China(Project Number 61521002)the Joint NSFC–DFG Research Program(Project Number 61761136018).
文摘The irregular domain and lack of ordering make it challenging to design deep neural networks for point cloud processing.This paper presents a novel framework named Point Cloud Transformer(PCT)for point cloud learning.PCT is based on Transformer,which achieves huge success in natural language processing and displays great potential in image processing.It is inherently permutation invariant for processing a sequence of points,making it well-suited for point cloud learning.To better capture local context within the point cloud,we enhance input embedding with the support of farthest point sampling and nearest neighbor search.Extensive experiments demonstrate that the PCT achieves the state-of-the-art performance on shape classification,part segmentation,semantic segmentation,and normal estimation tasks.
文摘Recently, a qualitative approach was proposed for 3-D shape recovery based on a hybrid object representation[1]. In this approach, aspect recovery is the most important stage which binds regions in the image into meaningful aspects to support 3-D primitive recovery. There is no known polynondal time algo-rithm to solve this problem. The previous approach dealt with this problem by using a heuristic method based on the conditional probability. Unlike the previous method, this paper presents a novel parallel voting scheme to conquer the problem for efficiency. For this purpose) the previous global aspect rep-resentation is replaced with a distributed representation of aspects. Based on this representation, a three-layer parallel voting network for aspect recovery is proposed. For evaluating likelihood, a continuous Hopfield net is employed so that all aspect coverings in decreasing order of likelihood can be enumerated.The paper describes this method in detail and demonstrates its usefulness with simulation.
基金support of the National Natural Science Foundation of China(61972157)the Natural Science Foundation of Shanghai(20ZR1417700)+2 种基金the National Key R&D Program of China(2019YFC1521104,2020AAA0108301)Shanghai Municipal Commission of Economy and Information(XX-RGZN-01-19-6348)the Art Major Project of National Social Science Fund(I8ZD22).
文摘A significant performance boost has been achieved in point cloud semantic segmentation by utilization of the encoder-decoder architecture and novel convolution operations for point clouds.However,co-occurrence relationships within a local region which can directly influence segmentation results are usually ignored by current works.In this paper,we propose a neighborhood co-occurrence matrix(NCM)to model local co-occurrence relationships in a point cloud.We generate target NCM and prediction NCM from semantic labels and a prediction map respectively.Then,Kullback-Leibler(KL)divergence is used to maximize the similarity between the target and prediction NCMs to learn the co-occurrence relationship.Moreover,for large scenes where the NCMs for a sampled point cloud and the whole scene differ greatly,we introduce a reverse form of KL divergence which can better handle the difference to supervise the prediction NCMs.We integrate our method into an existing backbone and conduct comprehensive experiments on three datasets:Semantic3D for outdoor space segmentation,and S3DIS and ScanNet v2 for indoor scene segmentation.Results indicate that our method can significantly improve upon the backbone and outperform many leading competitors.
基金the National Key R&D Program of China(Grant No.2016YFB1102500)the Key R&D Project in Shaanxi Province(Grant No.2019ZDLGY01-07)the Science and Technology Program of Jiangsu Province,China(Grant No.SBK2019041271).
文摘Ceramic structural parts are one of the most widely utilized structural parts in the industry. However, they usually contain defects following the pressing process, such as burrs. Therefore, additional trimming is usually required, despite the deformation challenges and difficulty in positioning. This paper proposes an ultrafast laser processing system for trimming complex ceramic structural parts. Opto-electromechanical cooperative control software is developed to control the laser processing system. The trimming problem of the ceramic cores used in aero engines is studied. The regional registration method is introduced based on the iterative closest point algorithm to register the path extracted from the computer-aided design model with the deformed ceramic core. A zonal and layering processing method for three-dimensional contours on complex surfaces is proposed to generate the working data of high-speed scanning galvanometer and the computer numerical control machine tool, respectively. The results show that the laser system and the method proposed in this paper are suitable for trimming complex non-datum parts such as ceramic cores. Compared with the results of manual trimming, the method proposed in this paper has higher accuracy, efficiency, and yield. The method mentioned above has been used in practical application with satisfactory results.
基金This research was funded by projects‘Influence of cartographic visualization methods on the success of solving practical and educational spatial tasks’[grant number MUNI/M/0846/2015]‘Integrated research on environmental changes in the landscape sphere of Earth II’[grant number MUNI/A/1419/2016],both awarded by Masaryk University,Czech Republic.
文摘Three-dimensional geovisualizations are currently pushed both by technological development and by the demands of experts in various applied areas.In the presented empirical study,we compared the features of real 3D(stereoscopic)versus pseudo 3D(monoscopic)geovisualizations in static and interactive digital elevation models.We tested 39 high-school students in their ability to identify the correct terrain profile from digital elevation models.Students’performance was recorded and further analysed with respect to their spatial abilities,which were measured by a psychological mental rotation test and think aloud protocol.The results of the study indicated that the influence of the type of 3D visualization(monoscopic/stereoscopic)on the performance of the users is not clear,the level of navigational interactivity has significant influence on the usability of a particular 3D visualization,and finally no influences of the spatial abilities on the performance of the user within the 3D environment were identified.
