利用RGB-Depth相机的机械模型建模

从真实机械模型出发,提出了一种利用RGB-Depth相机重建机械模型的方法。该方法流程如下:首先,利用RGB-Depth相机捕获真实的机械模型,获取机械模型的RGB图像和深度数据,注册出模型的粗糙点云数据;其次,用户通过简单的交互,快速将点云分割成多个部分;再次,利用机械模型构造基元简单、规整性好、复用性高的特点,从零件数据库中选取合适的基元,匹配分割的子部分;最后,通过模型部件之间位置关系的优化,重建出多零件机械模型的网格数据。实验结果表明,这种建模方法可以为机械模型重建出具有良好几何特性且符合机械模型构成规则的网格数据,为机械模型的建模技术提供一种新的选择。

A Review of RGB-D Camera Calibration Methods

RGB-D camera is a new type of sensor,which can obtain the depth and texture information in an unknown 3D scene simultaneously,and they have been applied in various fields widely.In fact,when implementing such kinds of applications using RGB-D camera,it is necessary to calibrate it first.To the best of our knowledge,at present,there is no existing a systemic summary related to RGB-D camera calibration methods.Therefore,a systemic review of RGB-D camera calibration is concluded as follows.Firstly,the mechanism of obtained measurement and the related principle of RGB-D camera calibration methods are presented.Subsequently,as some specific applications need to fuse depth and color information,the calibration methods of relative pose between depth camera and RGB camera are introduced in Section 2.Then the depth correction models within RGB-D cameras are summarized and compared respectively in Section 3.Thirdly,considering that the angle of the view field of RGB-D camera is smaller and limited to some specific applications,we discuss the calibration models of relative pose among multiple RGB-D cameras in Section 4.At last,the direction and trend of RGB-D camera calibration are prospected and concluded.

Survey of 3D modeling using depth cameras

Three-dimensional(3D)modeling is an important topic in computer graphics and computer vision.In recent years,the introduction of consumer-grade depth cameras has resulted in profound advances in 3D modeling.Starting with the basic data structure,this survey reviews the latest developments of 3D modeling based on depth cameras,including research works on camera tracking,3D object and scene reconstruction,and high-quality texture reconstruction.We also discuss the future work and possible solutions for 3D modeling based on the depth camera.

Markerless Respiratory Motion Tracking Using Single Depth Camera

The aim of this study is to propose a novel system that has an ability to detect intra-fractional motion during radiotherapy treatment in real-time using three-dimensional surface taken by a depth camera, Microsoft Kinect v1. Our approach introduces three new aspects for three-dimensional surface tracking in radiotherapy treatment. The first aspect is a new algorithm for noise reduction of depth values. Ueda’s algorithm was implemented and enabling a fast least square regression of depth values. The second aspect is an application for detection of patient’s motion at multiple points in thracoabdominal regions. The third aspect is an estimation of three-dimensional surface from multiple depth values. For evaluation of noise reduction by Ueda’s algorithm, two respiratory patterns are measured by the Kinect as well as a laser range meter. The resulting cross correlation coefficients between the laser range meter and the Kinect were 0.982 for abdominal respiration and 0.995 for breath holding. Moreover, the mean cross correlation coefficients between the signals of our system and the signals of Anzai with respect to participant’s respiratory motion were 0.90 for thoracic respiration and 0.93 for abdominal respiration, respectively. These results proved that the performance of the developed system was comparable to existing motion monitoring devices. Reconstruction of three-dimensional surface also enabled us to detect the irregular motion and breathing arrest by comparing the averaged depth with predefined threshold values.

High performance hardware architecture for depth measurement by using binocular-camera

High performance hardware architecture for depth measurement by using binocular-camera is proposed.In the system,at first,video streams of the target are captured by left and right charge-coupled device(CCD)cameras to obtain an image including the target.Then,two different images with two different view points are obtained,and they are used in calculating the position deviation of the image's pixels based on triangular measurement.Finally,the three-dimensional coordinate of the object is reconstructed.All the video data is processed by using field-programmable gate array(FPGA)in real-time.Hardware implementation speeds up the performance and reduces the power,thus,this hardware architecture can be applied in the portable environment.

Relationship between the Alignment of a Non-Mydriatic Fundus Camera,Anterior Chamber Depth and Axial Length

Purpose:To evaluate the relationship between the position of the focal adjustment knob of a fundus camera and refractive error and biometric data as measured in the same eye.Methods:Normal eyes of patients presenting to clinics at the Beijing Tongren Hospital were examined with a non-mydriatic fundus camera.The position on the focal scale of a knob adjusting the distance between the camera lens and film plane,used to adjust focus the image of the patients fundus relative to the refractive power of the eye,was recorded in degrees.Ocular biometry and refractometry were performed on the same eyes.Results:The study included 136 subjects with a mean age of 36.5 ±19.6 years and a mean refractive error of-1.31 ±2.77 diopters.In univariate analysis,the position of the adjustment knob was significantly associated with refractive error.(P < 0.001;correlation coefficient r=-0.77),axial length.(P<0.001;r=0.65) and anterior chamber depth (P<0.001;r=0.48).After adjustment for age,anterior chamber depth decreased by 0.01 mm(95% confidence interval:0.003,0.017) for change per degree in the position of the adjustment knob.Conclusion:A fundus camera can be used to estimate anterior chamber depth,axial length and refractive error.In a screening setting,a fundus camera operated by a technician may be helpful to detect a shallow anterior chamber and evaluate a potential risk factor for primary angle closure.

基于RGB-D深度相机的智能导盲系统

针对现有的导盲杖和导盲犬在实际工作中存在作用有限,使用效果差,成本高,普及率低的问题,开发了一种基于RGB-D深度相机的智能导盲系统,通过使用RGB-D深度相机实现对障碍物的探测,通过GPS远程导航实现全局路径规划,结合A算法实现局部障碍物规避,通过上位机计算处理信号,下位机系统控制振动阵列和发出语音信号提示盲人。在实际测试中表现出精准的定位效果和良好的障碍物规避能力,语音提示和振动提示及时准确,基于RGB-D深度相机的障碍物距离探测误差小于3%,基本满足系统设计需要。

基于RGB-D点云的田间原位玉米株高测量试验研究

为满足田间原位玉米株高的测量需求,避免破坏性取样后进行株高测量难以捕捉植株在自然环境下生长的真实情况与表型的动态变化,提出了一种基于RGB-D相机的田间原位玉米株高的测量方法。首先,通过RGB-D相机同时获取可见光图像和其对应的深度信息,计算相机的内参,得到玉米植株的三维点云数据;其次,通过基于欧几里得距离的统计滤波算法和随机采样一致性算法(RANSAC)的阈值分割快速去除三维点云中的离群点、环境噪声及复杂的自然环境背景(包括土壤面和滴灌管等),并通过OBB包围盒验证滤波效果;最后,通过单株玉米植株分割,提出了一种基于俯视视角下玉米株高的测量方法,并计算出田间原位玉米的株高参数。试验中,在玉米快速生长期中选取两天的试验数据,其试验用5个品种的平均测量误差分别为1.47cm和2.70cm,均方根误差(RSME)分别为1.68cm和2.80cm,人工实测结果和算法测量结果进行线性拟合后得到待测系数R2分别为0.9831和0.9797。试验结果表明:利用RGB-D相机对田间原位玉米的表型测量与株高分析具有可行性,所提出的测量与计算方法最后获得的玉米株高参数具有较高的准确性,可以为玉米表型参数提取提供更为有效地技术手段。

一种基于线特征的RGB-D视觉里程计算法

基于特征点的视觉里程计在点特征稀少的环境下难以得到足够的匹配点对,从而导致相机运动估计失败,因而提出采集人造环境中特征明显的边缘作为线特征来提高视觉里程计算法的稳定性。采用深度相机获取的RGB图像进行LSD线特征提取,推断线特征对应的图像位置的深度信息,避免深度缺失,将线段上的2D点反投影为3D点,拟合3D点为三维直线,利用线特征匹配关系进行位姿估计。此外在位姿优化部分进行改进,利用拟合直线过程中的最佳过点,以及重投影的线段与观测线段的角度误差信息,推导了误差关于位姿扰动的雅克比矩阵,在图优化时利用重投影误差优化相机位姿,拓展了传统的优化方法。基于TUM缺少点特征的数据集的实验结果表明所提出的线特征视觉里程计方法相比ORB-SLAM2的轨迹估计精度提高63%,并能完整地跟踪轨迹。实验结果表明所提出算法在欠特征点环境中表现出了较高的精度和稳定性。

基于RGB-D相机的实时人数统计方法

针对传统人数统计方法因遮挡、光照变化导致准确率低的问题,提出一种适用于深度图的模拟降水分水岭算法(Depth map based Rainfalling Watershed Segmentation,D-RWS)。修复深度图并用混合高斯背景建模提取前景。利用D-RWS算法分割深度图中感兴趣的行人头部区域(Region Of Interest,ROI)。采用质心欧式距离最短法关联各帧中同一目标并跟踪计数。实验结果表明:提出的方法准确率能够达到98%以上,平均每帧处理时间为25 ms(40 f/s),准确率和实时性可满足实际应用的要求。

基于RGB-D序列的人体动态建模方法

人体建模是计算机视觉研究领域的重要研究课题。人体建模被广泛应用于科研、动画、游戏、服装设计、工业等领域,具有非常广阔的应用前景。传统的建模方法可以在大体上还原人体的姿态,但细节上会有偏差。本文提出一种基于RGB-D序列的人体动态建模方法。人体在场景中自然活动,利用廉价的深度摄像设备Kinect可以获取人体的骨架信息和三维点云。利用获得的骨架信息将模板人体分段刚性地变形到目标位置,使用ICP算法将变形后的模型与Kinect获取的点云进行更精确的配准,使用TPS变形获得一个平滑的柔性形变人体。

基于深度学习的RGB-D深度信息补全方法研究

目前的深度相机往往不能感知高光、透明和较远距离的深度信息,导致原始获得的深度图像存在一定的信息丢失.为了解决这个问题,采用了一种基于深度学习的方法,从大量彩色图像中训练出能够估计局部法向和遮挡边界的模型,并结合深度相机提供的原始深度信息,通过构造全局优化函数,实现对原始深度的补全.通过对多视角的RGB-D数据集进行实验对比,方法较好地估计出整体轮廓和局部细节的深度,解决了原始深度信息缺失的问题.

基于重投影深度差累积图与静态概率的动态RGB-D SLAM算法

为了提高同时定位与建图(SLAM)系统在动态场景下的定位精度和鲁棒性,提出新的RGB-D SLAM算法.建立基于重投影深度差值的累积模型,分割图像的动静态区域;为了避免动态区域过分割,先剔除与匹配地图点欧氏距离过大的动态区域特征点,再根据t分布估计其余特征点的静态概率;将静态区域特征点和动态区域的疑似静态点以不同权重加入位姿优化,得到提纯后的位姿.在公开数据集上的实验结果表明,所提算法在动态场景下较改进前的RGB-D ORB-SLAM2算法的定位精度提升96.1%,较其他动态SLAM算法提升31.2%,有效提高了视觉SLAM系统在动态环境下的定位精度和鲁棒性.

基于RGB-D相机的移动机器人定位分析与实现

同时定位与地图创建在移动机器人导航系统中至关重要。本文以RGB-D深度相机为视觉传感器,分析移动机器人定位方法,利用特征提取与匹配实现位姿估计,实现定位功能,以Turtlebot为载体的实验平台,运用成熟的ORB-SLAM视觉算法实现定位与地图构建,拟完善中国在核电设施中移动机器人室内定位问题,自主构建出厂房地图,或者搭载于已有的核装备机器人上完成自身定位与地图构建。

Comparison of corneal measurements in keratoconic eyes using rotating Scheimpflug camera and scanning-slit topography

AIM: To compare the anterior segment measurements obtained by rotating Scheimpflug camera(Pentacam) and Scanning-slit topography(Orbscan IIz) in keratoconic eyes.METHODS: A total of 121 patients, 71 males(58.7%)and 50 females(41.3%)(214 eyes) with the diagnosis of keratoconus(KC) were enrolled in this study. Following diagnosis of KC by slit-lamp biomicroscopic examination, central corneal thickness(CCT), thinnest corneal thickness(TCT), anterior chamber depth(ACD),and pupil diameter(PD) were measured by a single examiner using successive instrumentation by Pentacam and Orbscan.RESULTS: There was no significant difference between the two instruments for the measurement of CCT and TCT. In contrast, scanning-slit topography measured ACD(3.46±0.40 mm vs 3.38±0.33 mm, P =0.019) and PD(4.97 ±1.26 mm vs 4.08 ±1.19 mm, P 【0.001) significantly larger than rotating Scheimpflug camera.The two devices made similar measurements for CCT(95% CI:-2.94 to5.06, P =0.602). However, the mean difference for TCT was-6.28(95% CI:-10.51 to-2.06, P =0.004) showing a thinner measurement by Orbscan than by Pentacam. In terms of the ACD, the mean difference was 0.08 mm(95%CI: 0.04 to 0.12, P 【0.001) with Orbscan giving a slightly larger value than Pentacam. Similarly, Orbscan measurement for PD was longer than Pentacam(95% CI:0.68 to 1.08, P 【0.001).CONCLUSION: A good agreement was found between Pentacam and Orbscan concerning CCT measurement while comparing scanning-slit topography and rotatingScheimpflug camera there was an underestimation for TCT and overestimation for ACD and PD.

基于RGB-D深度相机的下一代虚拟演播室构建

虚拟演播室已经成为常见的电视节目制作形式,通过真实人物与虚拟场景的融合,大大增强了视觉体验。传统色键抠像技术需要蓝室场景,无法做到自然场景的前景提取,限制了虚拟演播室技术在更广泛的节目形式中应用。此外,目前的虚拟演播室录制过程中,缺乏现场的人机交互机制,不利于人物与虚拟场景的实时互动以及。基于RGB-D深度相机,研究了自然场景和复杂场景下多个人物的实时抠像方法,让虚拟演播室不再受制于蓝室环境;结合人体识别技术实现了人物与虚拟场景的现场交互方法。最后,结合RGB-D深度相机和机器视觉人机交互等新技术新方法,对下一代虚拟演播室的构建进行了讨论。

RGB-D相机的三维重建综述

三维重建是计算机视觉、计算机图像学重要的研究方向之一。三维重建将相机采集的二维图像信息进行计算处理,生成物体的三维立体信息并加以展示。随着移动端传感器的不断集成,尤其是集成了3D深度相机,为场景三维重建提供了深度信息,可直接采用几何关系得到点云数据,加快了重建速度,可实现实时重建。对RGB-D相机在三维重建方面的研究工作进行总结和分析,论述了传感器、重建流程及算法,对三维重建进行了总结及展望。

基于深度信息改进的RGB-D图像拼接技术

针对RGB-D相机拍摄目标的深度信息复杂导致RGB-D图像拼接出现错误的现象,提出一种基于深度信息改进的RGB-D图像拼接技术。首先使用像素比重与点云可视化密度联合的深度相机最优测量范围估计方法确定RGB-D相机拍摄目标的最优测量范围;其次设计一种深度值改进的特征匹配算法对RGB-D图像进行拼接;最后对拼接后的RGB-D图像进行质量分析并与使用传统拼接技术拼接后的图像进行对比。实验结果表明,改进后的图像拼接技术比传统拼接技术高效、准确,能较好地展现拍摄目标的纹理及深度信息。

Survey on depth and RGB image-based 3D hand shape and pose estimation

The field of vision-based human hand three-dimensional(3D)shape and pose estimation has attracted significant attention recently owing to its key role in various applications,such as natural human computer interactions.With the availability of large-scale annotated hand datasets and the rapid developments of deep neural networks(DNNs),numerous DNN-based data-driven methods have been proposed for accurate and rapid hand shape and pose estimation.Nonetheless,the existence of complicated hand articulation,depth and scale ambiguities,occlusions,and finger similarity remain challenging.In this study,we present a comprehensive survey of state-of-the-art 3D hand shape and pose estimation approaches using RGB-D cameras.Related RGB-D cameras,hand datasets,and a performance analysis are also discussed to provide a holistic view of recent achievements.We also discuss the research potential of this rapidly growing field.

Co-axial depth sensor with an extended depth range for AR/VR applications

Background Depth sensor is an essential element in virtual and augmented reality devices to digitalize users'environment in real time.The current popular technologies include the stereo,structured light,and Time-of-Flight(ToF).The stereo and structured light method require a baseline separation between multiple sensors for depth sensing,and both suffer from a limited measurement range.The ToF depth sensors have the largest depth range but the lowest depth map resolution.To overcome these problems,we propose a co-axial depth map sensor which is potentially more compact and cost-effective than conventional structured light depth cameras.Meanwhile,it can extend the depth range while maintaining a high depth map resolution.Also,it provides a high-resolution 2 D image along with the 3 D depth map.Methods This depth sensor is constructed with a projection path and an imaging path.Those two paths are combined by a beamsplitter for a co-axial design.In the projection path,a cylindrical lens is inserted to add extra power in one direction which creates an astigmatic pattern.For depth measurement,the astigmatic pattern is projected onto the test scene,and then the depth information can be calculated from the contrast change of the reflected pattern image in two orthogonal directions.To extend the depth measurement range,we use an electronically focus tunable lens at the system stop and tune the power to implement an extended depth range without compromising depth resolution.Results In the depth measurement simulation,we project a resolution target onto a white screen which is moving along the optical axis and then tune the focus tunable lens power for three depth measurement subranges,namely,near,middle and far.In each sub-range,as the test screen moves away from the depth sensor,the horizontal contrast keeps increasing while the vertical contrast keeps decreasing in the reflected image.Therefore,the depth information can be obtained by computing the contrast ratio between features in orthogonal directions.Conclusions The proposed depth map sensor could implement depth measurement for an extended depth range with a co-axial design.