Clustering Reference Images Based on Covisibility for Visual Localization

In feature-based visual localization for small-scale scenes,local descriptors are used to estimate the camera pose of a query image.For large and ambiguous environments,learning-based hierarchical networks that employ local as well as global descriptors to reduce the search space of database images into a smaller set of reference views have been introduced.However,since global descriptors are generated using visual features,reference images with some of these features may be erroneously selected.In order to address this limitation,this paper proposes two clustering methods based on how often features appear as well as their covisibility.For both approaches,the scene is represented by voxels whose size and number are computed according to the size of the scene and the number of available 3Dpoints.In the first approach,a voxel-based histogram representing highly reoccurring scene regions is generated from reference images.A meanshift is then employed to group the most highly reoccurring voxels into place clusters based on their spatial proximity.In the second approach,a graph representing the covisibility-based relationship of voxels is built.Local matching is performed within the reference image clusters,and a perspective-n-point is employed to estimate the camera pose.The experimental results showed that camera pose estimation using the proposed approaches was more accurate than that of previous methods.

Domain-Invariant Similarity Activation Map Contrastive Learning for Retrieval-Based Long-Term Visual Localization

Visual localization is a crucial component in the application of mobile robot and autonomous driving.Image retrieval is an efficient and effective technique in image-based localization methods.Due to the drastic variability of environmental conditions,e.g.,illumination changes,retrievalbased visual localization is severely affected and becomes a challenging problem.In this work,a general architecture is first formulated probabilistically to extract domain-invariant features through multi-domain image translation.Then,a novel gradientweighted similarity activation mapping loss(Grad-SAM)is incorporated for finer localization with high accuracy.We also propose a new adaptive triplet loss to boost the contrastive learning of the embedding in a self-supervised manner.The final coarse-to-fine image retrieval pipeline is implemented as the sequential combination of models with and without Grad-SAM loss.Extensive experiments have been conducted to validate the effectiveness of the proposed approach on the CMU-Seasons dataset.The strong generalization ability of our approach is verified with the RobotCar dataset using models pre-trained on urban parts of the CMU-Seasons dataset.Our performance is on par with or even outperforms the state-of-the-art image-based localization baselines in medium or high precision,especially under challenging environments with illumination variance,vegetation,and night-time images.Moreover,real-site experiments have been conducted to validate the efficiency and effectiveness of the coarse-to-fine strategy for localization.

Autonomous map query:robust visual localization in urban environments using Multilayer Feature Graph

When a vehicle travels in urban areas, onboard global positioning system （GPS） signals may be obstructed by high-rise buildings and thereby cannot provide accurate positions. It is proposed to perform localization by registering ground images to a 2D building boundary map which is generated from aerial images. Multilayer feature graphs （MFG） is employed to model building facades from the ground images. MFG was reported in the previous work to facilitate the robot scene understand- ing in urhan areas. By constructing MFG, the 2D/3D positions of features can be obtained, inclu- cling line segments, ideal lines, and all primary vertical planes. Finally, a voting-based feature weighted localization method is developed based on MFGs and the 2D building boundary map. The proposed method has been implemented and validated in physical experiments. In the proposed ex- periments, the algorithm has achieved an overall localization accuracy of 2.2m, which is better than commercial GPS working in open environments.

A high precision visual localization sensor and its working methodology for an indoor mobile robot

To overcome the shortcomings of existing robot localization sensors, such as low accuracy and poor robustness, a high precision visual localization system based on infrared-reflective artificial markers is designed and illustrated in detail in this paper. First, the hardware system of the localization sensor is developed. Secondly, we design a novel kind of infrared-reflective artificial marker whose characteristics can be extracted by the acquisition and processing of the infrared image. In addition, a confidence calculation method for marker identification is proposed to obtain the probabilistic localization results. Finally, the autonomous localization of the robot is achieved by calculating the relative pose relation between the robot and the artificial marker based on the perspective-3-point(P3P) visual localization algorithm. Numerous experiments and practical applications show that the designed localization sensor system is immune to the interferences of the illumination and observation angle changes. The precision of the sensor is ±1.94 cm for position localization and ±1.64° for angle localization. Therefore, it satisfies perfectly the requirements of localization precision for an indoor mobile robot.

ReLoc:Indoor Visual Localization with Hierarchical Sitemap and View Synthesis

Indoor visual localization,i.e.,6 Degree-of-Freedom camera pose estimation for a query image with respect to a known scene,is gaining increased attention driven by rapid progress of applications such as robotics and augmented reality.However,drastic visual discrepancies between an onsite query image and prerecorded indoor images cast a significant challenge for visual localization.In this paper,based on the key observation of the constant existence of planar surfaces such as floors or walls in indoor scenes,we propose a novel system incorporating geometric information to address issues using only pixelated images.Through the system implementation,we contribute a hierarchical structure consisting of pre-scanned images and point cloud,as well as a distilled representation of the planar-element layout extracted from the original dataset.A view synthesis procedure is designed to generate synthetic images as complementary to that of a sparsely sampled dataset.Moreover,a global image descriptor based on the image statistic modality,called block mean,variance,and color(BMVC),was employed to speed up the candidate pose identification incorporated with a traditional convolutional neural network(CNN)descriptor.Experimental results on a popular benchmark demonstrate that the proposed method outperforms the state-of-the-art approaches in terms of visual localization validity and accuracy.

Method for Visual Localization of Oil and Gas Wellhead Based on Distance Function of Projected Features

A localization method based on distance function of projected features is presented to solve the accuracy reduction or failure problem due to occlusion and blurring caused by smog, when dealing with vision based localization for target oil and gas wellhead （OGWH）. Firstly, the target OGWH is modeled as a cylinder with marker, and a vector with redundant parameter is used to describe its pose. Secondly, the explicit mapping relationship between the pose vector with redundant parameter and projected features is derived. Then, a 2D-point-to-feature distance function is proposed, as well as its derivative. Finally, based on this distance function and its derivative, an algorithm is proposed to estimate the pose of target OGWH directly according to the 2D image information, and the validity of the method is verified by both synthetic data and real image experiments. The results show that this method is able to accomplish the localization in the case of occlusion and blurring, and its anti-noise ability is good especially with noise ratio of less than 70%.

Scene Visual Perception and AR Navigation Applications

With the rapid popularization of mobile devices and the wide application of various sensors,scene perception methods applied to mobile devices occupy an important position in location-based services such as navigation and augmented reality(AR).The development of deep learning technologies has greatly improved the visual perception ability of machines to scenes.The basic framework of scene visual perception,related technologies and the specific process applied to AR navigation are introduced,and future technology development is proposed.An application(APP)is designed to improve the application effect of AR navigation.The APP includes three modules:navigation map generation,cloud navigation algorithm,and client design.The navigation map generation tool works offline.The cloud saves the navigation map and provides navigation algorithms for the terminal.The terminal realizes local real-time positioning and AR path rendering.

High dimension feature extraction based visualized SOM fault diagnosis method and its application in p-xylene oxidation process

Purified terephthalic acid(PTA) is an important chemical raw material. P-xylene(PX) is transformed to terephthalic acid(TA) through oxidation process and TA is refined to produce PTA. The PX oxidation reaction is a complex process involving three-phase reaction of gas, liquid and solid. To monitor the process and to improve the product quality, as well as to visualize the fault type clearly, a fault diagnosis method based on selforganizing map(SOM) and high dimensional feature extraction method, local tangent space alignment(LTSA),is proposed. In this method, LTSA can reduce the dimension and keep the topology information simultaneously,and SOM distinguishes various states on the output map. Monitoring results of PX oxidation reaction process indicate that the LTSA–SOM can well detect and visualize the fault type.

RB-SLAM:visual SLAM based on rotated BEBLID feature point description

The extraction and description of image features are very important for visual simultaneous localization and mapping(V-SLAM).A rotated boosted efficient binary local image descriptor(BEBLID)SLAM(RB-SLAM)algorithm based on improved oriented fast and rotated brief(ORB)feature description is proposed in this paper,which can solve the problems of low localization accuracy and time efficiency of the current ORB-SLAM3 algorithm.Firstly,it uses the BEBLID to replace the feature point description algorithm of the original ORB to enhance the expressiveness and description efficiency of the image.Secondly,it adds rotational invariance to the BEBLID using the orientation information of the feature points.It also selects the rotationally stable bits in the BEBLID to further enhance the rotational invariance of the BEBLID.Finally,it retrains the binary visual dictionary based on the BEBLID to reduce the cumulative error of V-SLAM and improve the loading speed of the visual dictionary.Experiments show that the dictionary loading efficiency is improved by more than 10 times.The RB-SLAM algorithm improves the trajectory accuracy by 24.75%on the TUM dataset and 26.25%on the EuRoC dataset compared to the ORB-SLAM3 algorithm.

基于深度学习的室内动态场景下视觉SLAM技术研究

视觉同步定位与建图(visual simultaneous localization and mapping,VSLAM)技术是近年来机器人和计算机视觉领域的重点研究方向之一,但当前的主流算法主要面向静态环境,当场景中存在运动的物体时,算法的定位精度和稳定性会受到很大影响。为了解决上述问题,提出了一种惯性测量单元(inertial measurement unit,IMU)积分与YOLOv4语义分割结合的VSLAM前端动态特征点剔除算法,通过YOLOv4网络对图像进行语义分割,识别图像中有运动可能的物体;再将IMU积分与语义分割结合,对目标检测框内有运动可能的特征点进行重投影误差的解算,识别并剔除环境中运动的特征点。在TUM Visual-Inertial Dataset上验证该算法,结果表明,在包含运动物体的室内场景下,该算法可以有效剔除环境中的运动物体,显著提升SLAM系统的定位精度和稳定性。

A monocular visual SLAM system augmented by lightweight deep local feature extractor using in-house and low-cost LIDAR-camera integrated device

Simultaneous Localization and Mapping(SLAM)has been widely used in emergency response,self-driving and city-scale 3D mapping and navigation.Recent deep-learning based feature point extractors have demonstrated superior performance in dealing with the complex environmental challenges(e.g.extreme lighting)while the traditional extractors are struggling.In this paper,we have successfully improved the robustness and accuracy of a monocular visual SLAM system under various complex scenes by adding a deep learning based visual localization thread as an augmentation to the visual SLAM framework.In this thread,our feature extractor with an efficient lightweight deep neural network is used for absolute pose and scale estimation in real time using the highly accurate georeferenced prior map database at 20cm geometric accuracy created by our in-house and low-cost LiDAR and camera integrated device.The closed-loop error provided by our SLAM system with and without this enhancement is 1.03m and 18.28m respectively.The scale estimation of the monocular visual SLAM is also significantly improved(0.01 versus 0.98).In addition,a novel camera-LiDAR calibration workflow is also provided for large-scale 3D mapping.This paper demonstrates the application and research potential of deep-learning based vision SLAM with image and LiDAR sensors.

Research on Visual Autonomous Navigation Indoor for Unmanned Aerial Vehicle

The aim of this paper is to study visual autonomous navigation of unmanned aerial vehicle (UAV) in indoor global positioning system (GPS) denied environment. The UAV platform of the autonomous navigation flight control system is designed and built. The principle of visual localization and mapping algorithm is studied. According to the characteristics of UAV platform, the visual localization is designed and improved. Experimental results demonstrate that the UAV platform can realize the tasks of autonomous localization, navigation and mapping based on visual in unknown environments. © 2017, Shanghai Jiaotong University and Springer-Verlag Berlin Heidelberg.

Bearing-only Visual SLAM for Small Unmanned Aerial Vehicles in GPS-denied Environments

This paper presents a hierarchical simultaneous localization and mapping（SLAM） system for a small unmanned aerial vehicle（UAV） using the output of an inertial measurement unit（IMU） and the bearing-only observations from an onboard monocular camera.A homography based approach is used to calculate the motion of the vehicle in 6 degrees of freedom by image feature match.This visual measurement is fused with the inertial outputs by an indirect extended Kalman filter（EKF） for attitude and velocity estimation.Then,another EKF is employed to estimate the position of the vehicle and the locations of the features in the map.Both simulations and experiments are carried out to test the performance of the proposed system.The result of the comparison with the referential global positioning system/inertial navigation system（GPS/INS） navigation indicates that the proposed SLAM can provide reliable and stable state estimation for small UAVs in GPS-denied environments.

Real-time Visual Odometry Estimation Based on Principal Direction Detection on Ceiling Vision

In this paper,we present a novel algorithm for odometry estimation based on ceiling vision.The main contribution of this algorithm is the introduction of principal direction detection that can greatly reduce error accumulation problem in most visual odometry estimation approaches.The principal direction is defned based on the fact that our ceiling is flled with artifcial vertical and horizontal lines which can be used as reference for the current robot s heading direction.The proposed approach can be operated in real-time and it performs well even with camera s disturbance.A moving low-cost RGB-D camera（Kinect）,mounted on a robot,is used to continuously acquire point clouds.Iterative closest point（ICP） is the common way to estimate the current camera position by registering the currently captured point cloud to the previous one.However,its performance sufers from data association problem or it requires pre-alignment information.The performance of the proposed principal direction detection approach does not rely on data association knowledge.Using this method,two point clouds are properly pre-aligned.Hence,we can use ICP to fne-tune the transformation parameters and minimize registration error.Experimental results demonstrate the performance and stability of the proposed system under disturbance in real-time.Several indoor tests are carried out to show that the proposed visual odometry estimation method can help to signifcantly improve the accuracy of simultaneous localization and mapping（SLAM）.

Evaluation of a Pointwise Local Visual Pattern Exploration Method

Sensitivity analysis is a powerful method for discovering the significant factors that contribute to understanding the interaction between variables in multivariate datasets. A number of sensitivity analysis methods fall into the class of local analysis, in which the sensitivity is defined as the partial derivatives of a target variable with respect to a group of independent variables. In a recent paper, we presented a novel pointwise local pattern exploration system for visual sensitivity analysis. Using this system, analysts are able to explore local patterns and the sensitivity at individual data points, which reveals the relationships between a focal point and its neighbors this paper we present several evaluations of the system, including case studies with real datasets, user studies the effectiveness of the visualizations and interactions, and a detailed description of the experience of a user In on

A Protocol for Digitalized Collection of Traditional Chinese Medicine(TCM)Pulse Information Using Bionic Pulse Diagnosis Equipment

Pulse diagnosis equipment used in Traditional Chinese Medicine(TCM)has long been developed for collecting pulse information and in TCM research.However,it is still difficult to implement pulse taking automatically or efficiently in clinical practice.Here,we present a digital protocol for TCM pulse information collection based on bionic pulse diagnosis equipment,which ensures high efficiency,reliability and data integrity of pulse diagnosis information.A four-degree-of-freedom pulse taking platform together with a wrist bracket can satisfy the spatial positioning and angle requirements for individually adaptive pulse acquisition.Three-dimensional reconstruction of a wrist surface and an image localization model are combined to provide coordinates of the acquisition position and detection direction automatically.Three series elastic joints can not only simulate the TCM pulse taking method that“Three fingers in a straight line,the middle finger determining the‘Guan’location and finger pulp pressing on the radial artery,”but also simultaneously carry out the force-controlled multi-gradient pressing process.In terms of pulse information integrity,this proposed protocol can generate rich pulse information,including basic individual information,pulse localization distribution,multi-gradient dynamic pulse force time series,and objective pulse parameters,which can help establish the fundamental data sets that are required as the pulse phenotype for subsequent comprehensive analysis of pulse diagnosis.The implementation of this scheme is beneficial to promote the standardization of the digitalized collection of pulse information,the effectiveness of detecting abnormal health status,and the promotion of the fundamental and clinical research of TCM,such as TCM pulse phenomics.

Approach for improved development of advanced driver assistance systems for future smart mobility concepts

To use the benefits of Advanced Driver Assistance Systems(ADAS)-Tests in simulation and reality a new approach for using Augmented Reality(AR)in an automotive vehicle for testing ADAS is presented in this paper.Our procedure provides a link between simulation and reality and should enable a faster development process for future increasingly complex ADAS tests and future mobility solutions.Test fields for ADAS offer a small number of orientation points.Furthermore,these must be detected and processed at high vehicle speeds.That requires high computational power both for developing our method and its subsequent use in testing.Using image segmentation(IS),artificial intelligence(AI)for object recognition,and visual simultaneous localization and mapping(vSLAM),we aim to create a three-dimensional model with accurate information about the test site.It is expected that using AI and IS will significantly improve performance as computational speed and accuracy for AR applications in automobiles.

Illumination robust image transformations for feature-based SLAM using photometric and feature matches loss

Simultaneous localization and mapping(SLAM) technology becomes more and more important in robot localization. The purpose of this paper is to improve the robustness of visual features to lighting changes and increase the recall rate of map re-localization under different lighting environments by optimizing the image transformation model. An image transformation method based on matches and photometric error(name the method as MPT) is proposed in this paper, and it is seamlessly integrated into the pre-processing stage of the feature-based visual SLAM framework. The results of the experiment show that the MPT method has a better matching effect on different visual features. In addition, the image transformation module encapsulated by a robot operating system(ROS) can be used with multiple visual SLAM systems and improve its re-localization effect under different lighting environments.

A New Monocular Vision Measurement Method to Estimate 3D Positions of Objects on Floor

A new visual measurement method is proposed to estimate three-dimensional （3D） position of the object on the floor based on a single camera. The camera fixed on a robot is in an inclined position with respect to the floor. A measurement model with the camera＇s extrinsic parameters such as the height and pitch angle is described. Single image of a chessboard pattern placed on the floor is enough to calibrate the camera＇s extrinsic parameters after the camera＇s intrinsic parameters are calibrated. Then the position of object on the floor can be computed with the measurement model. Furthermore, the height of object can be calculated with the paired-points in the vertical line sharing the same position on the floor. Compared to the conventional method used to estimate the positions on the plane, this method can obtain the 3D positions. The indoor experiment testifies the accuracy and validity of the proposed method.

传统数据中心网络向AD-DC平滑演进技术分析

本文阐述了传统数据中心网络如何向AD-DC平滑演进的三种场景,每种场景对应的迁移方案,以及通过一个示例描述整个迁移过程。随着云计算、大数据、移动互联网的兴起,数据中心流量与日俱增,业务上线节奏加快,要求数据中心网络做出快速响应,传统数据中心网络向AD-DC演进将是必然趋势,AD-DC解决方案满足数字化转型中不断扩张的网络需求,不断迭代创新为客户提供智能融合、安全可靠、先进成熟、开放包容的全栈数据中心网络解决方案,助力更多行业客户的数字化转型实践。