Robust and Accurate Monocular Visual Navigation Combining IMU for a Quadrotor

In this paper, we present a multi-sensor fusion based monocular visual navigation system for a quadrotor with limited payload, power and computational resources. Our system is equipped with an inertial measurement unit (IMU), a sonar and a monocular down-looking camera. It is able to work well in GPS-denied and markerless environments. Different from most of the keyframe-based visual navigation systems, our system uses the information from both keyframes and keypoints in each frame. The GPU-based speeded up robust feature (SURF) is employed for feature detection and feature matching. Based on the flight characteristics of quadrotor, we propose a refined preliminary motion estimation algorithm combining IMU data. A multi-level judgment rule is then presented which is beneficial to hovering conditions and reduces the error accumulation effectively. By using the sonar sensor, the metric scale estimation problem has been solved. We also present the novel IMU+3P (IMU with three point correspondences) algorithm for accurate pose estimation. This algorithm transforms the 6-DOF pose estimation problem into a 4-DOF problem and can obtain more accurate results with less computation time. We perform the experiments of monocular visual navigation system in real indoor and outdoor environments. The results demonstrate that the monocular visual navigation system performing in real-time has robust and accurate navigation results of the quadrotor. © 2014 Chinese Association of Automation.

Central Retinal Microperimetry by Scanning Laser Ophthalmoscopy in Chiasmal Patients with Visual Inattention Temporally at Monocular Visual Acuity Testing

Background: In a previous study, a specific visual behaviour was described in four chiasmal patients. It manifested as inattention/ignorance within the temporal visual space (“visual blocking”) at monocular visual acuity testing. Moreover, in 3 out of the 4, the process of reading a text appeared cognitively impaired. Methods: As a supplement to conventional visual field testing, the present analysis focus was on microperimetry by Scanning Laser Ophthalmocopy (SLO). Our aim was to identify the perceptual retinal counterparts to the temporally located visual field defects as caused by the visual pathway lesion on a chiasmal level, possibly also to indicate the apparently ineffective saccadic movements underlying that only part of the line on the chart could be given. Results and Conclusions: The central retinal areas with lacking recognition by SLO testing were given by black rectangles expressing scotomatous points;they clustered in a zone extending nasally from the fixation area, as expected. The methodology further depicted an orderly fixation in two cases, and only minor deviation in two. Fixation saccades thus were considered by and large within normal and with only exceptional outlier points recorded. All considered, the shortcomings of the perceptual mechanisms underlying the lateralised visual inattention as observed in the four patients have remained without a satisfactory explanation.

Relationship between monocularly deprivation and amblyopia rats and visual system development

Objective:To explore the changes of lateral geniculate body and visual cortex in monocular strabismus and form deprived amblyopic rat,and visual development plastic stage and visual plasticity in adult rats.Methods:A total of 60 SD rats ages 13 d were randomly divided into A,B,C three groups with 20 in each group,group A was set as the normal control group without any processing,group B was strabismus amblyopic group,using the unilateral extraocular rectus resection to establish the strabismus amblyopia model,group C was monocular form deprivation amblyopia group using unilateral eyelid edge resection+lid suture.At visual developmental early phase(P2S),meta phase(P3S),late phase(P45)and adult phase(P120),the lateral geniculate body and visual cortex area 17 of five rats in each group were exacted for C-fos Immunocytochemistry.Neuron morphological changes in lateral geniculate body and visual cortex was observed,the positive neurons differences of C-fos expression induced by light stimulation was measured in each group,and the condition of radiation development of P120 amblyopic adult rats was observed.Results:In groups B and C,C-fos positive cells were significantly lower than the control group at P25(P<0.05),there was no statistical difference of C-fos protein positive cells between group B and group A(P>0.05),C-fos protein positive cells level of group B was significantly lower than that of group A(P<0.05).The binoculus C-fos protein positive cells level of groups B and C were significantly higher than that of control group at P35,P4S and P120 with statistically significant differences(P<0.05).Conclusions:The increasing of C-fos expression in geniculate body and visual cortex neurons of adult amblyopia suggests the visual cortex neurons exist a certain degree of visual plasticity.

The Effect of Monocular Blur on the Binocular Visual Field

Purpose/Aim: We aimed to investigate the effect of monocular blur on the binocular visual field. Materials and Methods: A total of 13 healthy young volunteers participated in this study. The mean subjective refractive error of the dominant eye (DE) was -3.33 ± 1.65D, and the non-dominant eye (NDE) was -3.15 ± 2.84D. The DE was determined by using the hole-in-the-card test. The visual field was examined by the Humphrey Visual Field Analyzer using the 30-2 SITA Standard program. The visual field was measured while wearing soft contact lens under three conditions;① both eyes: near vision correction;② DE: near vision correction +3.00D added, NDE: near vision correction;and ③ DE: near vision correction, NDE: near vision correction +3.00D added. The foveal threshold, mean deviation (MD), and pattern standard deviation (PSD) values were investigated. Results: The foveal threshold value (dB) at ①, ②, and ③ was 41.2, 37.8, and 38.1, respectively. The values at ② and ③ were both significantly lower than that at ① (p Conclusion: These results suggest that monocular blur reduced the sensitivity within the binocular visual field.

Bags of tricks for learning depth and camera motion from monocular videos

Background Based on the seminal work proposed by Zhou et al., much of the recent progress in learning monocular visual odometry, i.e., depth and camera motion from monocular videos, can be attributed to the tricks in the training procedure, such as data augmentation and learning objectives. Methods Herein, we categorize a collection of such tricks through the theoretical examination and empirical evaluation of their effects on the final accuracy of the visual odometry. Results/Conclusions By combining the aforementioned tricks, we were able to significantly improve a baseline model adapted from SfMLearner without additional inference costs. Furthermore, we analyzed the principles of these tricks and the reason for their success. Practical guidelines for future research are also presented.

Monocular VO Scale Ambiguity Resolution Using an Ultra Low-Cost Spike Rangefinder

Monocular visual odometry (VO) is the process of determining a user’s trajectory through a series of consecutive images taken by a single camera. A major problem that affects the accuracy of monocular visual odometry, however, is the scale ambiguity. This research proposes an innovative augmentation technique, which resolves the scale ambiguity problem of monocular visual odometry. The proposed technique augments the camera images with range measurements taken by an ultra-low-cost laser device known as the Spike. The size of the Spike laser rangefinder is small and can be mounted on a smartphone. Two datasets were collected along precisely surveyed tracks, both outdoor and indoor, to assess the effectiveness of the proposed technique. The coordinates of both tracks were determined using a total station to serve as a ground truth. In order to calibrate the smartphone’s camera, seven images of a checkerboard were taken from different positions and angles and then processed using a MATLAB-based camera calibration toolbox. Subsequently, the speeded-up robust features (SURF) method was used for image feature detection and matching. The random sample consensus (RANSAC) algorithm was then used to remove the outliers in the matched points between the sequential images. The relative orientation and translation between the frames were computed and then scaled using the spike measurements in order to obtain the scaled trajectory. Subsequently, the obtained scaled trajectory was used to construct the surrounding scene using the structure from motion (SfM) technique. Finally, both of the computed camera trajectory and the constructed scene were compared with ground truth. It is shown that the proposed technique allows for achieving centimeter-level accuracy in monocular VO scale recovery, which in turn leads to an enhanced mapping accuracy.

面向空间目标近距离操作的相对位姿测量系统设计

面向空间近距离操作的相对位姿测量系统(RPS)是对目标进行距离测量、位姿测量、三维重构及光学成像的有效方式。针对高精度一体化位姿测量需求,文章提出了融合激光雷达测距的单目视觉位姿测量系统,可同时具备目标距离、方位测量及三维成像等能力。经过在轨应用验证,结果表明:方位测角精度和位姿测量精度等关键指标均优于预期值,该系统主要功能性能可满足空间操作实际应用需求,可以为未来空间机动平台的目标测量系统研制提供参考。

无人船红外图像单目视觉检测与跟踪研究

无人船可用于海洋探测、军事侦查、军事打击等领域,结合GPS、AIS、无线通信技术以及图像视觉技术使得无人船实现智能化控制。无人船红外图像的单目视觉检测对实现目标检测以及跟踪具有至关重要的作用。本文提出使用YOLOV7算法对红外图像目标进行识别,确定红外图像训练集,对图像进行预处理后完成模型的训练及更新,最终实现对红外图像目标的识别。将KCF算法应用于红外图像目标跟踪,研究KCF算法对目标的跟踪流程,使用KCF算法和DCF算法进行仿真分析发现,同等情况下,KCF算法的跟踪准确率为78%,优于DCF算法。

Experience-dependent expression of Nogo-A and Nogo receptor in the developing rat visual cortex

Nogo-A and Nogo receptor （NgR） expression in the visual cortex following a critical developmental period （postnatal days 20-60） has been previously shown. However, little is known regarding Nogo-A and NgR expression between postnatal day 0 and initiation of the critical period. The present study analyzed Nogo-A and NgR expression at four different time points： postnatal day 0 （P0）, before critical period （P14）, during critical period （P28）, and after critical period （P60）. Results showed significantly increased Nogo-A mRNA and protein expression levels in the visual cortex following birth, and expression levels remained steady between P28 and P60. NgR mRNA or protein expression was dramatically upregulated with age and peaked at P14 or P28, respectively, and maintained high expression to P60. In addition, Nogo-A and NgR expression was analyzed in each visual cortex layer in normal developing rats and rats with monocular deprivation. Monocular deprivation decreased Nogo-A and NgR mRNA and protein expression in the rat visual cortex, in particular in layers Ⅱ-Ⅲ and Ⅳ in the visual cortex contralateral to the deprived eye. These findings suggested that Nogo-A and NgR regulated termination of the critical period in experience- dependent visual cortical plasticity.

走廊场景下辅助视觉里程计初始化的单目深度恢复方法

单目相机由于缺少尺度信息在视觉里程计等应用场景中性能受限。现有研究大多通过基于深度学习的方法解决这一问题,但其推理速度慢,难以实时运行。针对这一问题,提出了一种走廊场景下基于非线性优化进行快速单目深度恢复的显式方法。采用虚拟相机假设,简化对相机姿态角的求解;通过最小化几何残差,将深度估计问题转换为优化问题;设计一种深度平面构建方法,对空间点深度进行分类,实现走廊等封闭结构场景下的快速深度估计;最后,将所提方法在单目视觉里程计初始化中进行应用,使得单目视觉里程计可以获得真实的尺度信息,并提升其定位精度。实验结果表明:所提方法在走廊场景3m范围内深度估计的相对误差小于8.4%,在Intel Core i5-7300HQCPU处理器中能以20FPS的速度实时运行。

二维主动位姿引导的单目空间姿态测量方法

针对空间物体姿态快速测量问题,以构建视觉最小系统需求为依据,研究了一种基于二维主动位姿引导的单目视觉空间姿态测量方法,建立了单目相机、二维载台与倾角仪之间的姿态测量模型,实现了空间物体的姿态角的测量。该方法以大地倾角仪坐标系统一测量系统的测量基准,由精密二维载台引导单目相机覆盖地空大视野三维空间,通过前期标定设计完成了单目相机与二维载台之间的工装校准;建立了载台坐标系、摄像机坐标系以及大地倾角仪坐标系之间的姿态测量传递模型,实现了定轴旋转双视角拍照下的空间物体的姿态解算和角度测量。构建了实验验证环境,测角实验结果表明:在系统测量基准坐标系下,其俯仰角的测量误差≤0.82°,测量相对误差≤6.1%;其横滚角的测量误差≤0.43°,测量相对误差≤3.4%。

水面无人艇单目视觉伺服自主控制研究综述

单目视觉伺服采用相机模仿人眼视觉功能,感知周围环境和测量运动状态,是提高无人艇航行感知与控制自主性的重要手段。首先,从基本原理出发,简述视觉伺服技术分类、相机透视投影原理和无人艇运动数学模型,为文献综述分析提供框架基础。然后,根据任务复杂度,依次综述单目视觉伺服在无人艇航向控制、镇定控制、轨迹跟踪控制和集群控制等典型应用场景下的研究进展和挑战。最后,系统性地总结无人艇单目视觉伺服自主控制的潜在研究方向和发展趋势。

基于视觉与激光融合的井下灾后救援无人机自主位姿估计

无人机在灾后矿井的自主导航能力是其胜任抢险救灾任务的前提,而在未知三维空间的自主位姿估计技术是无人机自主导航的关键技术之一。目前基于视觉的位姿估计算法由于单目相机无法直接获取三维空间的深度信息且易受井下昏暗光线影响,导致位姿估计尺度模糊和定位性能较差,而基于激光的位姿估计算法由于激光雷达存在视角小、扫描图案不均匀及受限于矿井场景结构特征,导致位姿估计出现错误。针对上述问题,提出了一种基于视觉与激光融合的井下灾后救援无人机自主位姿估计算法。首先,通过井下无人机搭载的单目相机和激光雷达分别获取井下的图像数据和激光点云数据,对每帧矿井图像数据均匀提取ORB特征点,使用激光点云的深度信息对ORB特征点进行深度恢复,通过特征点的帧间匹配实现基于视觉的无人机位姿估计。其次,对每帧井下激光点云数据分别提取特征角点和特征平面点,通过特征点的帧间匹配实现基于激光的无人机位姿估计。然后,将视觉匹配误差函数和激光匹配误差函数置于同一位姿优化函数下,基于视觉与激光融合来估计井下无人机位姿。最后,通过视觉滑动窗口和激光局部地图引入历史帧数据,构建历史帧数据和最新估计位姿之间的误差函数,通过对误差函数的非线性优化完成在局部约束下的无人机位姿的优化和修正,避免估计位姿的误差累计导致无人机轨迹偏移。模拟矿井灾后复杂环境进行仿真实验,结果表明:基于视觉与激光融合的位姿估计算法的平均相对平移误差和相对旋转误差分别为0.0011 m和0.0008°,1帧数据的平均处理时间低于100 ms,且算法在井下长时间运行时不会出现轨迹漂移问题;相较于仅基于视觉或激光的位姿估计算法,该融合算法的准确性、稳定性均得到了提高,且实时性满足要求。

AUV终端对接的单目视觉导引方法研究

在水下复杂环境中高效回收自主潜水器(AUV)是一项重大的技术挑战。本文提出了一种利用单目视觉检测和跟踪对接站上光环标识进行终端对接的方法。为了提高光源特征识别,本文改进了Canny边缘检测算法,采用自适应阈值方法动态调整光源轮廓,并在阈值最优时采用最小外接圆方法确定光源中心。此外,利用几何位置和面积约束消除水面反射的干扰。在视觉定位和跟踪方面,采用张氏标定法获取相机的内部参数和畸变参数,通过比较光环中心和图像中心来估计AUV的航向和俯仰偏差,然后利用位置-姿态PID控制器进行快速跟踪。水池实验结果表明,该方法简单实用,鲁棒性强,能够为未来水下机器人的可靠回收提供一定技术参考。

一种在线更新的单目视觉里程计

现有的基于深度学习的视觉里程计(visual odometry,VO)训练样本与应用场景存在差异时,普遍存在难以适应新环境的问题,因此提出了一种在线更新单目视觉里程计算法OUMVO。其特点在于应用阶段利用实时采集到的图像序列在线优化位姿估计网络模型,提高网络的泛化能力和对新环境的适用能力。该方法使用了自监督学习方法,无须额外标注地面真值,并采用了Transformer对图像流进行序列建模,以充分利用局部窗口内的视觉信息,提高位姿估计精度,以避免传统方法只能利用相邻两帧图像来估计位姿的局限,还可以弥补采用RNN进行序列建模无法并行计算的缺点。此外,采用图像空间几何一致性约束,解决了传统单目视觉里程计算法存在的尺度漂移问题。在KITTI数据集上的定量和定性实验结果表明,OUMVO的位姿估计精度和对新环境的适应能力均优于现有的先进单目视觉里程计方法。

视觉发育关键期单眼剥夺对小鼠大脑不同区域星形胶质细胞的影响

目的研究视觉发育关键期单眼剥夺(MD)对小鼠上丘、海马、视皮层星形胶质细胞表达的影响。方法选取C57BL/6J小鼠18只,随机分为正常对照组(CON组)和MD组,每组各9只,在12 h/12 h昼夜循环环境中饲养。CON组不做处理,MD组小鼠在生后27 d右眼行MD,7 d后取材。逆转录实时荧光定量聚合酶链式反应(RT-qPCR)检测两组小鼠上丘、海马、视皮层胶质纤维酸性蛋白(GFAP)mRNA表达情况。Western blot检测两组小鼠上丘、海马、视皮层GFAP蛋白表达的变化。免疫荧光染色检测两组小鼠上丘、海马、视皮层GFAP和中枢神经特异性蛋白(S100β)共定位标记的星形胶质细胞数量。结果RT\|qPCR和Western blot检测结果显示,与CON组相比,MD组小鼠上丘、海马(锥体细胞1区、锥体细胞3区、齿状回区)、视皮层GFAP mRNA和蛋白表达均减少,差异均有统计学意义(均为P<0.05)。免疫荧光染色结果显示,与CON组相比,MD组小鼠上丘、海马(锥体细胞1区、锥体细胞3区、齿状回区)、视皮层GFAP和S100β共定位标记的星形胶质细胞数量均减少,差异均有统计学意义(均为P<0.05)。结论视觉发育关键期对小鼠进行MD导致其上丘、海马、视皮层星形胶质细胞数量减少。

小鼠上丘Rap1B表达的发育性变化及关键期单眼形觉剥夺对其的影响

目的探讨小鼠上丘Rap1B表达的发育性变化以及视觉发育关键期单眼形觉剥夺(MD)对其表达的影响。方法SPF级健康雄性C57BL/6J小鼠45只为实验对象。实验一:小鼠上丘Rap1B表达的发育性变化。该实验小鼠分为3组,即关键期前组[生后18 d(P18)小鼠]、关键期组(P32小鼠)和成年组(P60小鼠),采用qPCR和Western blot检测各组小鼠左右两侧上丘中Rap1B基因及蛋白的表达量。实验二:MD对上丘Rap1B基因及蛋白表达的影响。该实验小鼠分为2组:右眼MD组(小鼠P28时右眼建立MD模型,至P32时行检测)与NC组(正常对照组,P32小鼠),采用qPCR和Western blot检测各组小鼠左右两侧上丘中Rap1B基因及蛋白的表达。结果关键期前组小鼠左右两侧上丘中Rap1B基因和蛋白表达量均较低;与关键期前组相比,关键期组小鼠Rap1B基因及蛋白表达量均显著增高(均为P<0.05);与关键期组相比,成年组小鼠Rap1B基因和蛋白表达量均显著降低(均为P<0.05)。与NC组相比,右眼MD组小鼠左侧上丘Rap1B基因和蛋白表达量均显著增高(均为P<0.05),而右侧上丘Rap1B基因和蛋白表达量均显著降低(均为P<0.05)。结论小鼠上丘中Rap1B基因表达的发育性变化趋势与视觉发育关键期相一致,Rap1B可能参与了关键期MD对上丘发育的调控。

基于改进ArUco标签的位姿估计方法

针对使用ArUco标签在远距离测量时精度不高的问题,提出改进ArUco标签的位姿估计方法。该方法在ArUco标签的基础上设计了阵列特征圆环,提取圆环的中心点并对其进行排序,根据交比不变特性判断出圆心检测的准确性;利用加权最小二乘法建立ArUco标签角点与阵列圆环圆心之间的直线拟合关系,实现角点修正;采用EPnP算法解算出改进ArUco标签的位姿参数,并利用LM算法对位姿进行优化,提高位姿测量精度。设计了机械臂视觉定位系统验证所提出方法的可行性和有效性。实验结果表明,此改进方法在2.5~4 m范围内测量的最大平移误差为8.72 mm,最大旋转误差为1.78°。所提出的方法在4 m处纵向距离测量的相对误差降低0.69%,提高了测量精度。

融合点线特征的视觉惯性同时定位及建图

为了解决移动机器人在低纹理场景中的定位精度较低且容易跟踪丢失问题,本文设计了一种点线特征提取和匹配策略,并以此构建了视觉惯性同时定位和建图系统。提出线特征提取和匹配算法,通过改良线特征提取算法的隐藏参数,提高了线特征提取的精度。利用点线特征不同匹配筛选框架减少误匹配的数目,得出了可以应用于视觉惯性同时定位和建图系统的线特征提取匹配算法。在现有视觉惯性框架中引入提出的线特征约束,搭建了能在未知低纹理环境下鲁棒运行的视觉惯性同时定位及建图系统。研究表明:在真实环境中的移动机器人定位实验证明了该系统的精度和鲁棒性优于现有的视觉惯性框架,其室内定位精度提高了24.2%,走廊定位精度提高了8%,对于移动机器人在低纹理场景下的高精度定位具有较高价值。

单目视觉里程计/惯性组合导航算法(英文)

提出一种单目视觉里程计/捷联惯性组合导航定位算法。与视觉里程计估计相机姿态不同,惯导系统连续提供相机拍摄时刻对应的三维姿态,克服了单纯由视觉估计相机姿态精度低造成的长距离导航误差大的问题。通过配准和时间同步,用惯导系统解算的速度和视觉里程计计算的速度之差作为组合导航的观测量,采用Kalman滤波修正组合导航系统的误差,同时估计视觉里程计标度因数误差。分别在室内外不同环境下进行了22 m的推车实验和1412 m的跑车实验,定位误差分别为3.2%和4.0%。与Clark采用姿态传感器定期更新相机姿态估计结果的方法相比,单目视觉里程计/惯性组合导航定位精度更高,定位误差随距离增长率低,适合步行机器人或轮式移动机器人在复杂地形环境下车轮严重打滑时的自主定位导航。