Depth-Guided Vision Transformer With Normalizing Flows for Monocular 3D Object Detection

Monocular 3D object detection is challenging due to the lack of accurate depth information.Some methods estimate the pixel-wise depth maps from off-the-shelf depth estimators and then use them as an additional input to augment the RGB images.Depth-based methods attempt to convert estimated depth maps to pseudo-LiDAR and then use LiDAR-based object detectors or focus on the perspective of image and depth fusion learning.However,they demonstrate limited performance and efficiency as a result of depth inaccuracy and complex fusion mode with convolutions.Different from these approaches,our proposed depth-guided vision transformer with a normalizing flows(NF-DVT)network uses normalizing flows to build priors in depth maps to achieve more accurate depth information.Then we develop a novel Swin-Transformer-based backbone with a fusion module to process RGB image patches and depth map patches with two separate branches and fuse them using cross-attention to exchange information with each other.Furthermore,with the help of pixel-wise relative depth values in depth maps,we develop new relative position embeddings in the cross-attention mechanism to capture more accurate sequence ordering of input tokens.Our method is the first Swin-Transformer-based backbone architecture for monocular 3D object detection.The experimental results on the KITTI and the challenging Waymo Open datasets show the effectiveness of our proposed method and superior performance over previous counterparts.

Modeling monocular form deprivation in rabbits using a simulated-cataract intraocular lens

AIM:To establish an animal model of form deprivation amblyopia based on a simulated cataract intraocular lens(IOLs).METHODS:Poly(dimethyl siloxane)-SiO_(2) thin films(PSF)with different degrees of opacity as IOL materials were prepared.The light transmission of the PSF-IOL was measured,and its in vitro biosafety was determined by cell counting kit(CCK)-8 assay using the HLEC-B3 cell line and ARPE-19 cell line.Subsequently,the in vivo safety was determined by implanting the PSF-IOL with 10%wt SiO_(2) into the right eyes of New Zealand white rabbits(PSF-IOL group),and compared with two control groups:contralateral comparison group and normal control(NC)group(Contralateral comparison group:the fellow eye;NC group:a group of binocular normal rabbits without intervention).The flash visual-evoked potentials(F-VEPs)were measured to verify amblyopia.RESULTS:PSFs containing 0,2%,and 10%wt SiO_(2) were successfully constructed.The 0 SiO_(2) PSF was transparent,while the 10%wt SiO_(2) PSF was completely opaque.It was found that PSF did not induce unwanted cytotoxicity in HLECs and ARPE19 cells in vitro.In vitro,PSF-IOL with 10%wt SiO_(2) was also non-toxic,and no significant inflammation or structural changes occurred after four weeks of PSF-IOL implantation.Finally,our IOL-simulated congenital cataract rabbit detected by F-VEPs suggested tentative amblyopia.CONCLUSION:A PSF-IOL that mimics cataracts is created.A novel form deprivation model is created by the IOL-simulated congenital cataract rabbit.It can be developed fast and stable and holds great potential for future study.

Design of a road vehicle detection system based on monocular vision

In order to decrease vehicle crashes, a new rear view vehicle detection system based on monocular vision is designed. First, a small and flexible hardware platform based on a DM642 digtal signal processor （DSP） micro-controller is built. Then, a two-step vehicle detection algorithm is proposed. In the first step, a fast vehicle edge and symmetry fusion algorithm is used and a low threshold is set so that all the possible vehicles have a nearly 100% detection rate （TP） and the non-vehicles have a high false detection rate （FP）, i. e., all the possible vehicles can be obtained. In the second step, a classifier using a probabilistic neural network （PNN） which is based on multiple scales and an orientation Gabor feature is trained to classify the possible vehicles and eliminate the false detected vehicles from the candidate vehicles generated in the first step. Experimental results demonstrate that the proposed system maintains a high detection rate and a low false detection rate under different road, weather and lighting conditions.

基于上下文信息增强和深度引导的单目3D目标检测

为了充分利用单目图像提供的特征信息,提出上下文信息增强和深度引导的单目3D目标检测方法.设计高效的上下文信息增强模块,使用多个大核卷积自适应地增强多尺度目标的上下文信息,利用深度可分离卷积和条形卷积操作有效减少大核卷积的参数量和计算复杂度.统计分析3D目标框各个属性的预测误差,发现3D目标框的长度和深度属性预测不准确是导致预测框偏差大的主要原因.设计深度误差加权损失函数,在训练过程中进行目标的长度和深度预测监督,提高长度和深度属性的预测精度,进而提升3D预测框的准确性.在KITTI数据集上开展实验,结果表明,所提方法在数据集的多个级别上的平均准确度高于现有的单目3D目标检测方法.

基于图像特征识别的隧道三维重构方法研究及应用

隧道三维模型能直观的显示隧道围岩的裂隙节理信息,准确实现隧道的三维重构有助于获取隧道地质情况。本文拟采用Python语言,借助于开源数据库提供的相关函数,首先对拍摄的多幅隧道图像进行特征点识别,采用随机抽样一致性算法实现了相邻图像的正确匹配。其次采用张氏标定法求解出相机内参数,并通过单目视觉计算求解出外参矩阵,计算出由图像像素到三维点的变换,提取全部像素点得到三维稠密点云模型后,用德劳内三角划分对导出点云做网格生成处理形成隧道表面,再对三角网格面片做颜色平均值插入处理重现彩色隧道模型,并集成编译了三维重构相关算法。最后以湖南邵阳某隧道为例,完整地实现了隧道的重构,验证了基于图像特征识别实现三维重构的可行性。

Relationship between monocularly deprivation and amblyopia rats and visual system development

Objective:To explore the changes of lateral geniculate body and visual cortex in monocular strabismus and form deprived amblyopic rat,and visual development plastic stage and visual plasticity in adult rats.Methods:A total of 60 SD rats ages 13 d were randomly divided into A,B,C three groups with 20 in each group,group A was set as the normal control group without any processing,group B was strabismus amblyopic group,using the unilateral extraocular rectus resection to establish the strabismus amblyopia model,group C was monocular form deprivation amblyopia group using unilateral eyelid edge resection+lid suture.At visual developmental early phase(P2S),meta phase(P3S),late phase(P45)and adult phase(P120),the lateral geniculate body and visual cortex area 17 of five rats in each group were exacted for C-fos Immunocytochemistry.Neuron morphological changes in lateral geniculate body and visual cortex was observed,the positive neurons differences of C-fos expression induced by light stimulation was measured in each group,and the condition of radiation development of P120 amblyopic adult rats was observed.Results:In groups B and C,C-fos positive cells were significantly lower than the control group at P25(P<0.05),there was no statistical difference of C-fos protein positive cells between group B and group A(P>0.05),C-fos protein positive cells level of group B was significantly lower than that of group A(P<0.05).The binoculus C-fos protein positive cells level of groups B and C were significantly higher than that of control group at P35,P4S and P120 with statistically significant differences(P<0.05).Conclusions:The increasing of C-fos expression in geniculate body and visual cortex neurons of adult amblyopia suggests the visual cortex neurons exist a certain degree of visual plasticity.

Influences of levodopa on expression of N-methyl-D-aspartate receptor-1-subunit in the visual cortex of monocular deprivation rats

AIM: Many studies have demonstrated N-methyl-D-aspartate receptor-1-subunit (NMDAR1) is associated with amblyopia. The effectiveness of levodopa in improving the visual function of the children with amblyopia has also been proved. But the mechanism is undefined. Our study was to explore the possible mechanism. METHODS: Sixty 14-day-old healthy SD rats were randomly divided into 4 groups, including normal group, monocular deprivation group, levodopa group and normal saline group, 15 rats each. We sutured all the rats' unilateral eyelids except normal group to establish the monocular deprivation animal model and raise them in normal sunlight till 45-day-old. NMDAR1 was detected in the visual cortex with immunohistochemistry methods, Western Blot and Real time PCR. LD and NS groups were gavaged with levodopa (40mg/kg) and normal saline for 28 days respectively. NMDAR1 was also detected with the methods above. RESULTS: NMDAR1 in the visual cortex of MD group was less than that of normal group. NMDAR1 in the visual cortex of LD group was more than that of NS group. CONCLUSION: NMDAR1 is associated with the plasticity of visual development. Levodopa may influence the expression of NMDAR1 and improve visual function, and its target may lie in the visual cortex.

A Dynamic and Quantitative Study of Pattern Visual Evoked Potentials and Gamma-Aminobutyric Acid Neurones in the Lateral Geniculate Nucleus and the Visual Cortex of Monocular Deprivation Cats

Purpose: To assess the effects of monocular lid closure during critical period on cortical activity.Method: Pattern visual evoked potentials (PVEP) of the normal and the monocular deprivation (MD) cats were dynamically measured and the number of gam-maaminobutyric acid immunopositive (GABA-IP) neurones of the area 17 of the visual cortex and the lateral geniculate nucleus (LGN) was quantitatively compared by using immunohistochemical method (ABC).Result: The amplitude of the N1-P1 attenuated in deprived eyes (DE) , NE/DE at postnatal week (PNW) 7-8 (P【0. 05), NE/DE at PNW 15-16 (P【0. 01); while P1 latency delayed, NE/DE at PNW 7-8 (P】0. 05), NE/DE at PNW 15 - 16(P【0. 05). The numbers of GABA-IP neurones in layer A1 of the ipsilateral LGN and in layer A of the contralateral LGN, compared to those in the corresponding normal laminae, were not significant at PNW 7 - 8 and PNW 11 - 12 (P 】0. 05), while in the same cats a reduction in the number of GABA-IP neurcnes was found in layer IV of area 17 at PNW

Mobile Robot Hierarchical Simultaneous Localization and Mapping Using Monocular Vision

A hierarchical mobile robot simultaneous localization and mapping (SLAM) method that allows us to obtain accurate maps was presented. The local map level is composed of a set of local metric feature maps that are guaranteed to be statistically independent. The global level is a topological graph whose arcs are labeled with the relative location between local maps. An estimation of these relative locations is maintained with local map alignment algorithm, and more accurate estimation is calculated through a global minimization procedure using the loop closure constraint. The local map is built with Rao-Blackwellised particle filter (RBPF), where the particle filter is used to extending the path posterior by sampling new poses. The landmark position estimation and update is implemented through extended Kalman filter (EKF). Monocular vision mounted on the robot tracks the 3D natural point landmarks, which are structured with matching scale invariant feature transform (SIFT) feature pairs. The matching for multi-dimension SIFT features is implemented with a KD-tree in the time cost of O(lbN). Experiment results on Pioneer mobile robot in a real indoor environment show the superior performance of our proposed method.

Mobile Robot Localization and Navigation System Based on Monocular Vision

A system for mobile robot localization and navigation was presented.With the proposed system,the robot can be located and navigated by a single landmark in a single image.And the navigation mode may be following-track,teaching and playback,or programming.The basic idea is that the system computes the differences between the expected and the recognized position at each time and then controls the robot in a direction to reduce those differences.To minimize the robot sensor equipment,only one omnidirectional camera was used.Experiments in disturbing environments show that the presented algorithm is robust and easy to implement,without camera rectification.The rootmean-square error(RMSE) of localization is 1.4,cm,and the navigation error in teaching and playback is within 10,cm.

Calibration of laser beam direction based on monocular vision

In the laser displacement sensors measurement system,the laser beam direction is an important parameter.Particularly,the azimuth and pitch angles are the most important parameters to a laser beam.In this paper,based on monocular vision,a laser beam direction measurement method is proposed.First,place the charge coupled device(CCD)camera above the base plane,and adjust and fix the camera position so that the optical axis is nearly perpendicular to the base plane.The monocular vision localization model is established by using circular aperture calibration board.Then the laser beam generating device is placed and maintained on the base plane at fixed position.At the same time a special target block is placed on the base plane so that the laser beam can project to the special target and form a laser spot.The CCD camera placed above the base plane can acquire the laser spot and the image of the target block clearly,so the two-dimensional(2D)image coordinate of the centroid of the laser spot can be extracted by correlation algorithm.The target is moved at an equal distance along the laser beam direction,and the spots and target images of each moving under the current position are collected by the CCD camera.By using the relevant transformation formula and combining the intrinsic parameters of the target block,the2D coordinates of the gravity center of the spot are converted to the three-dimensional(3D)coordinate in the base plane.Because of the moving of the target,the3D coordinates of the gravity center of the laser spot at different positions are obtained,and these3D coordinates are synthesized into a space straight line to represent the laser beam to be measured.In the experiment,the target parameters are measured by high-precision instruments,and the calibration parameters of the camera are calibrated by a high-precision calibration board to establish the corresponding positioning model.The measurement accuracy is mainly guaranteed by the monocular vision positioning accuracy and the gravity center extraction accuracy.The experimental results show the maximum error of the angle between laser beams reaches to0.04°and the maximum error of beam pitch angle reaches to0.02°.

Autonomous Landing of Small Unmanned Aerial Rotorcraft Based on Monocular Vision in GPS-denied Area

Focusing on the low-precision attitude of a current small unmanned aerial rotorcraft at the landing stage, the present paper proposes a new attitude control method for the GPS-denied scenario based on the monocular vision. Primarily, a robust landmark detection technique is developed which leverages the well-documented merits of supporting vector machines (SVMs) to enable landmark detection. Then an algorithm of nonlinear optimization based on Newton iteration method for the attitude and position of camera is put forward to reduce the projection error and get an optimized solution. By introducing the wavelet analysis into the adaptive Kalman filter, the high frequency noise of vision is filtered out successfully. At last, automatic landing tests are performed to verify the method's feasibility and effectiveness. © 2014 Chinese Association of Automation.

Robust and Accurate Monocular Visual Navigation Combining IMU for a Quadrotor

In this paper, we present a multi-sensor fusion based monocular visual navigation system for a quadrotor with limited payload, power and computational resources. Our system is equipped with an inertial measurement unit (IMU), a sonar and a monocular down-looking camera. It is able to work well in GPS-denied and markerless environments. Different from most of the keyframe-based visual navigation systems, our system uses the information from both keyframes and keypoints in each frame. The GPU-based speeded up robust feature (SURF) is employed for feature detection and feature matching. Based on the flight characteristics of quadrotor, we propose a refined preliminary motion estimation algorithm combining IMU data. A multi-level judgment rule is then presented which is beneficial to hovering conditions and reduces the error accumulation effectively. By using the sonar sensor, the metric scale estimation problem has been solved. We also present the novel IMU+3P (IMU with three point correspondences) algorithm for accurate pose estimation. This algorithm transforms the 6-DOF pose estimation problem into a 4-DOF problem and can obtain more accurate results with less computation time. We perform the experiments of monocular visual navigation system in real indoor and outdoor environments. The results demonstrate that the monocular visual navigation system performing in real-time has robust and accurate navigation results of the quadrotor. © 2014 Chinese Association of Automation.

Research on Vehicle Anti-collision Technique Based on Monocular Vision

Vehicle anti-collision technique is a hot topic in the research area of Intelligent Transport System. The research on preceding vehicles detection and the distance measurement, which are the key techniques, makes great contributions to safe-driving. This paper presents a method which can be used to detect preceding vehicles and get the distance between own car and the car ahead. Firstly, an adaptive threshold method is used to get shadow feature, and a shadow!area merging approach is used to deal with the distortion of the shadow border. Region of interest(ROI) is obtained using shadow feature. Then in the ROI, symmetry feature is analyzed to verify whether there are vehicles and to locate the vehicles. Finally, using monocular vision distance measurement based on camera interior parameters and geometrical reasoning, we get the distance between own car and the preceding one. Experimental results show that the proposed method can detect the preceding vehicle effectively and get the distance between vehicles accurately.

Simultaneous Localization and Mapping Solutions Using Monocular and Stereo Visual Sensors with Baseline Scaling System

In this paper, SLAM systems are introduced using monocular and stereo visual sensors. The SLAM solutions are implemented in both indoor and outdoor. The SLAM samples have been taken in different modes, such as a straight line that enables us to measure the drift, in addition to the loop sample that is used to test the loop closure and its corresponding trajectory deformation. In order to verify the trajectory scale, a baseline method has been used. In addition, a ground truth has been captured for both indoor and outdoor samples to measure the biases and drifts caused by the SLAM solution. Both monocular and stereo SLAM data have been captured with the same visual sensors which in the stereo situation had a baseline of 20.00 cm. It has been shown that, the stereo SLAM localization results are 75% higher precision than the monocular SLAM solution. In addition, the indoor results of the monocular SLAM are more precise than the outdoor. However, the outdoor results of the stereo SLAM are more precise than the indoor results by 30%, which is a result of the small stereo baseline cameras. In the vertical SLAM localization component, the stereo SLAM generally shows 60% higher precision than the monocular SLAM results.

Multi-Scale Human Pose Tracking in 2D Monocular Images

In this paper we address the problem of tracking human poses in multiple perspective scales in 2D monocular images/videos. In most state-of-the-art 2D tracking approaches, the issue of scale variation is rarely discussed. However in reality, videos often contain human motion with dynamically changed scales. In this paper we propose a tracking framework that can deal with this problem. A scale checking and adjusting algorithm is proposed to automatically adjust the perspective scales during the tracking process. Two metrics are proposed for detecting and adjusting the scale change. One metric is from the height value of the tracked target, which is suitable for some sequences where the tracked target is upright and with no limbs stretching. The other metric employed in this algorithm is more generic, which is invariant to motion types. It is the ratio between the pixel counts of the target silhouette and the detected bounding boxes of the target body. The proposed algorithm is tested on the publicly available datasets (HumanEva). The experimental results show that our method demonstrated higher accuracy and efficiency compared to state-of-the-art approaches.

Central Retinal Microperimetry by Scanning Laser Ophthalmoscopy in Chiasmal Patients with Visual Inattention Temporally at Monocular Visual Acuity Testing

Background: In a previous study, a specific visual behaviour was described in four chiasmal patients. It manifested as inattention/ignorance within the temporal visual space (“visual blocking”) at monocular visual acuity testing. Moreover, in 3 out of the 4, the process of reading a text appeared cognitively impaired. Methods: As a supplement to conventional visual field testing, the present analysis focus was on microperimetry by Scanning Laser Ophthalmocopy (SLO). Our aim was to identify the perceptual retinal counterparts to the temporally located visual field defects as caused by the visual pathway lesion on a chiasmal level, possibly also to indicate the apparently ineffective saccadic movements underlying that only part of the line on the chart could be given. Results and Conclusions: The central retinal areas with lacking recognition by SLO testing were given by black rectangles expressing scotomatous points;they clustered in a zone extending nasally from the fixation area, as expected. The methodology further depicted an orderly fixation in two cases, and only minor deviation in two. Fixation saccades thus were considered by and large within normal and with only exceptional outlier points recorded. All considered, the shortcomings of the perceptual mechanisms underlying the lateralised visual inattention as observed in the four patients have remained without a satisfactory explanation.

Relative Navigation of Long-Range Non-Cooperative Targets Based on Monocular Sequence Images

Along with the increase of the number of failed satellites,plus space debris,year by year,it will take considerable manpower and resources if we rely just on ground surveillance and early warning.An alternative effective way would be to use autonomous long-range non-cooperative target relative navigation to solve this problem.For longrange non-cooperative targets,the stereo cameras or lidars that are commonly used would not be applicable.This paper studies a relative navigation method for long-range relative motion estimation of non-cooperative targets using only a monocular camera.Firstly,the paper provides the nonlinear relative orbit dynamics equations and then derives the discrete recursive form of the dynamics equations.An EKF filter is then designed to implement the relative navigation estimation.After that,the relative"locally weakly observability"theory for nonlinear systems is used to analyze the observability of monocular sequence images.The analysis results show that by relying only on monocular sequence images it has the possibility of deducing the relative navigation for long-range non-cooperative targets.Finally,numerical simulations show that the method given in this paper can achieve a complete estimation of the relative motion of longrange non-cooperative targets without conducting orbital maneuvers.

Monocular Vision Based Boundary Avoidance for Non-Invasive Stray Control System for Cattle: A Conceptual Approach

Building fences to manage the cattle grazing can be very expensive;cost inefficient. These do not provide dynamic control over the area in which the cattle are grazing. Existing virtual fencing techniques for the control of herds of cattle, based on polygon coordinate definition of boundaries is limited in the area of land mass coverage and dynamism. This work seeks to develop a more robust and an improved monocular vision based boundary avoidance for non-invasive stray control system for cattle, with a view to increase land mass coverage in virtual fencing techniques and dynamism. The monocular vision based depth estimation will be modeled using concept of global Fourier Transform (FT) and local Wavelet Transform (WT) of image structure of scenes (boundaries). The magnitude of the global Fourier Transform gives the dominant orientations and textual patterns of the image;while the local Wavelet Transform gives the dominant spectral features of the image and their spatial distribution. Each scene picture or image is defined by features v, which contain the set of global (FT) and local (WT) statistics of the image. Scenes or boundaries distances are given by estimating the depth D by means of the image features v. Sound cues of intensity equivalent to the magnitude of the depth D are applied to the animal ears as stimuli. This brings about the desired control as animals tend to move away from uncomfortable sounds.

Human skeleton proportions from monocular data

This paper introduces a novel method for estimating the skeleton proportions of a human figure from monocular data. The proposed system will first automatically extract the key frames and recover the perspective camera model from the 2D data. The human skeleton proportions are then estimated from the key frames using the recovered camera model without posture re- construction. The proposed method is tested to be simple, fast and produce satisfactory results for the input data. The human model with estimated proportions can be used in future research involving human body modeling or human motion reconstruction.