Clustering Reference Images Based on Covisibility for Visual Localization

In feature-based visual localization for small-scale scenes,local descriptors are used to estimate the camera pose of a query image.For large and ambiguous environments,learning-based hierarchical networks that employ local as well as global descriptors to reduce the search space of database images into a smaller set of reference views have been introduced.However,since global descriptors are generated using visual features,reference images with some of these features may be erroneously selected.In order to address this limitation,this paper proposes two clustering methods based on how often features appear as well as their covisibility.For both approaches,the scene is represented by voxels whose size and number are computed according to the size of the scene and the number of available 3Dpoints.In the first approach,a voxel-based histogram representing highly reoccurring scene regions is generated from reference images.A meanshift is then employed to group the most highly reoccurring voxels into place clusters based on their spatial proximity.In the second approach,a graph representing the covisibility-based relationship of voxels is built.Local matching is performed within the reference image clusters,and a perspective-n-point is employed to estimate the camera pose.The experimental results showed that camera pose estimation using the proposed approaches was more accurate than that of previous methods.

Study of real-time LWD data visual interpretation and geo-steering technology

LWD（logging while drilling） data has been used to explore complex subtle reservoirs by realtime visual interpretation and geo-steering.The method comprises of computer communication,well log data processing,formation recognition,reservoir modeling and model updating in real time.We studied the key technologies related to real-time LWD data visual interpretation and geo-steering and developed computer software with Chinese intellectual property rights covering the following important aspects： 1） real-time computer communication of well site LWD data;2） visualization of geological model and borehole information;3） real-time interpretation of LWD data;4） real-time geological model updating and geo-steering technology.We use field application examples to demonstrate the feasibility and validity of the proposed technologies.

Current trends in three-dimensional visualization and real-time navigation as well as robot-assisted technologies in hepatobiliary surgery

With the continuous development of digital medicine,minimally invasive precision and safety have become the primary development trends in hepatobiliary surgery.Due to the specificity and complexity of hepatobiliary surgery,traditional preoperative imaging techniques such as computed tomography and magnetic resonance imaging cannot meet the need for identification of fine anatomical regions.Imaging-based three-dimensional(3D)reconstruction,virtual simulation of surgery and 3D printing optimize the surgical plan through preoperative assessment,improving the controllability and safety of intraoperative operations,and in difficult-to-reach areas of the posterior and superior liver,assistive robots reproduce the surgeon’s natural movements with stable cameras,reducing natural vibrations.Electromagnetic navigation in abdominal surgery solves the problem of conventional surgery still relying on direct visual observation or preoperative image assessment.We summarize and compare these recent trends in digital medical solutions for the future development and refinement of digital medicine in hepatobiliary surgery.

Domain-Invariant Similarity Activation Map Contrastive Learning for Retrieval-Based Long-Term Visual Localization

Visual localization is a crucial component in the application of mobile robot and autonomous driving.Image retrieval is an efficient and effective technique in image-based localization methods.Due to the drastic variability of environmental conditions,e.g.,illumination changes,retrievalbased visual localization is severely affected and becomes a challenging problem.In this work,a general architecture is first formulated probabilistically to extract domain-invariant features through multi-domain image translation.Then,a novel gradientweighted similarity activation mapping loss(Grad-SAM)is incorporated for finer localization with high accuracy.We also propose a new adaptive triplet loss to boost the contrastive learning of the embedding in a self-supervised manner.The final coarse-to-fine image retrieval pipeline is implemented as the sequential combination of models with and without Grad-SAM loss.Extensive experiments have been conducted to validate the effectiveness of the proposed approach on the CMU-Seasons dataset.The strong generalization ability of our approach is verified with the RobotCar dataset using models pre-trained on urban parts of the CMU-Seasons dataset.Our performance is on par with or even outperforms the state-of-the-art image-based localization baselines in medium or high precision,especially under challenging environments with illumination variance,vegetation,and night-time images.Moreover,real-site experiments have been conducted to validate the efficiency and effectiveness of the coarse-to-fine strategy for localization.

RSSI Based Localization of Bluetooth Devices for Visually Impaired

Localization for visually impaired people in dynamically changing environments with unexpected hazards and obstacles is a current need. Many techniques have been discussed in the literature with respect to location-based services and techniques used for the positioning of devices. Time difference of arrival (TDOA), time of arrival (TOA) and received signal strength (RSS) have been widely used for the positioning but narrow band signals such as Bluetooth cannot efficiently utilize TDOA or TOA. Received signal strength indicator (RSSI) to measure RSS, has been found to be more reliable. RSSI measurement estimations depend heavily on the environmental interference. RSSI measurement estimations of Bluetooth systems can be improved either by improving the existing methodologies used to implement them or by using fusion techniques that employ Kalman filters to combine more than one RSSI method to improve the results significantly. This paper focuses on improving the existing methodology of measuring RSSI by proposing a new method using trilateration for localization of Bluetooth devices for visually impaired people. To validate the new method, class 2 Bluetooth devices (Blue Giga WT-12) were used with an evaluation board. The software required was developed in National Instruments LabView. The PCB was designed and manufactured as well. Experiments were then conducted, and surface plots of Bluetooth modules were obtained to show the signal interference and other environmental effects. Lastly, the results were discussed, and relevant conclusions were drawn.

Real-Time Data and Visualization Monitoring of Computer Numerical Control Machine Tools Based on Hyper Text Markup Language 5

In order to ensure the safety,quality and efficiency of computer numerical control(CNC)machine tool processing,a real-time monitoring and visible solution for CNC machine tools based on hyper text markup language(HTML)5 is proposed.The characteristics of the real-time monitoring technology of CNC machine tools under the traditional Client/Server(C/S)structure are compared and analyzed,and the technical drawbacks are proposed.Web real-time communication technology and browser drawing technology are deeply studied.A real-time monitoring and visible system for CNC machine tool data is developed based on Metro platform,combining WebSocket real-time communication technology and Canvas drawing technology.The system architecture is given,and the functions and implementation methods of the system are described in detail.The practical application results show that the WebSocket real-time communication technology can effectively reduce the bandwidth and network delay and save server resources.The numerical control machine data monitoring system can intuitively reflect the machine data,and the visible effect is good.It realizes timely monitoring of equipment alarms and prompts maintenance and management personnel.

A New Method for Visual Real-Time Monitoring of Low Frequency Oscillation

Visual real-time monitoring is the premise of low frequency oscillation control in power grids. This paper showed a visual method for the control center of power grids to monitor low frequency oscillation. It processed the PMU real-time data with incomplete S-transform, and converted the waveforms to two-dimensional time-frequency figures which showed the initial time, frequency and amplitude of each low frequency oscillation mode directly. GPU was used to show figures and calculate FFT with the purpose of improving calculation efficiency. The results of practical cases show that the real-time characters of low frequency oscillation can be identified availably by this visualization real-time monitoring method which is helpful and suitable for practical application.

Real-time Visualization of Urban with Gigantic Amount of Detailed Buildings

Complex urban scenery is generally composed of gigantic amount of detailed buildings, efficient representation and rendering are essential for its visualization. We present an accelerating method for urban visualization. Our approach can optimize the organization of models in accordance with the quadtree based terrain, which makes the parallelization easier. Through minimizing the draw call within one rendering process, our approach can reduce the time cost of each frame and improve the framerate greatly. Hereby, our system can handle large-scale detailed models with high diversity and also can afford the ability to adjust the rendering strategy automatically according to the state of the hardware.

Autonomous map query:robust visual localization in urban environments using Multilayer Feature Graph

When a vehicle travels in urban areas,onboard global positioning system(GPS) signals may be obstructed by high-rise buildings and thereby cannot provide accurate positions.It is proposed to perform localization by registering ground images to a 2D building boundary map which is generated from aerial images.Multilayer feature graphs(MFG) is employed to model building facades from the ground images.MFG was reported in the previous work to facilitate the robot scene understanding in urban areas.By constructing MFG,the 2D/3D positions of features can be obtained,including line segments,ideal lines,and all primary vertical planes.Finally,a voting-based feature weighted localization method is developed based on MFGs and the 2D building boundary map.The proposed method has been implemented and validated in physical experiments.In the proposed experiments,the algorithm has achieved an overall localization accuracy of 2.2m,which is better than commercial GPS working in open environments.

Analyzing the Impact of Scene Transitions on Indoor Camera Localization through Scene Change Detection in Real-Time

Real-time indoor camera localization is a significant problem in indoor robot navigation and surveillance systems.The scene can change during the image sequence and plays a vital role in the localization performance of robotic applications in terms of accuracy and speed.This research proposed a real-time indoor camera localization system based on a recurrent neural network that detects scene change during the image sequence.An annotated image dataset trains the proposed system and predicts the camera pose in real-time.The system mainly improved the localization performance of indoor cameras by more accurately predicting the camera pose.It also recognizes the scene changes during the sequence and evaluates the effects of these changes.This system achieved high accuracy and real-time performance.The scene change detection process was performed using visual rhythm and the proposed recurrent deep architecture,which performed camera pose prediction and scene change impact evaluation.Overall,this study proposed a novel real-time localization system for indoor cameras that detects scene changes and shows how they affect localization performance.

Experimental Study on Local Mass Transfer of Structured Packing with the Method of Flow Visualization

chromochemical 反应集体转移技术被采用了到收拾行李组织的学习本地人团转移特征。实验采用的这种技术是由与颜色化学药品反应的结果在一份过滤器报纸上分析颜色分发产出转移质量的表面分发的一个氨吸附方法(AAM ) 。处理技术的一幅数字图象被申请数据可视化。三座山峰在收拾行李组织的一个联合起来的房间的不同位置上珍视的在那里存在的本地集体转移系数表演的三维的阴谋。为了改进质量，转移组织收拾行李的效率，一个困惑在包装表之间被增加。作为结果，平均集体转移系数增加由(10-20 )% 和压力落下减少由(15-55 )% 。

Place recognition based on saliency for topological localization

Based on salient visual regions for mobile robot navigation in unknown environments, a new place recognition system was presented. The system uses monocular camera to acquire omni-directional images of the environment where the robot locates. Salient local regions are detected from these images using center-surround difference method, which computes opponencies of color and texture among multi-scale image spaces. And then they are organized using hidden Markov model (HMM) to form the vertex of topological map. So localization, that is place recognition in our system, can be converted to evaluation of HMM. Experimental results show that the saliency detection is immune to the changes of scale, 2D rotation and viewpoint etc. The created topological map has smaller size and a higher ratio of recognition is obtained.

Local Electroretinogram and Pattern Visual Evoked Potential in Maculopathies

Purposes: To observe the characteristics of local electroretinogram (LERG) in normal subjects and patients with maculopathies, and to evaluate the applied worth of LERG and pattern visual evoked potential (PVEP) in maculopathies. Methods: LERGs at 5° and 15° macular regions were recorded from 27 normal subjects (54 eyes). The factors of age, different eyes and stimulate areas for LERG influence were observed. Meanwhile, the LERG and PVEP were recorded from 25 patients (35 eyes) with maculopathies for making contrast study. Results: In normal subjects, there was no significant influence of age to LERG. As the stimulated areas increased, the a- and b-wave amplitudes of LERG increased. In the patients with maculopathies, the a- and b-wave amplitudes of LERG at 5°, 10°and 15° macular regions were significantly lowered and the mean values of P1 latency were prolonged and N1-P1 amplitudes of VEP were lowered, comparing with the control group. In the nearing stimulated area (5°LERG and 14. 9×19°PVEP.),

A Cooperative Distributed System for Real-Time Route Guidance

This paper describes a cooperative decentralized architecture for reactive real-time route guidance. The architecture is cooperative in the sense that it allows adjacent local controllers to exchange information regarding the traffic conditions in their territories. A set of local decision rules and associated heuristic functions to support the cooperative architecture are specified. A protocol governing the knowledge exchange among local adjacent controllers is developed. A simulation-assignment modeling framework is used for assessing the effectiveness of this cooperative architecture under various levels of controller knowledge and network traffic congestion. The cooperative decentralized system is tested under various scenarios of knowledge and cooperation and network traffic demand levels. The cooperative system is compared against the shortest path algorithm as a benchmark.

Scene Visual Perception and AR Navigation Applications

With the rapid popularization of mobile devices and the wide application of various sensors,scene perception methods applied to mobile devices occupy an important position in location-based services such as navigation and augmented reality(AR).The development of deep learning technologies has greatly improved the visual perception ability of machines to scenes.The basic framework of scene visual perception,related technologies and the specific process applied to AR navigation are introduced,and future technology development is proposed.An application(APP)is designed to improve the application effect of AR navigation.The APP includes three modules:navigation map generation,cloud navigation algorithm,and client design.The navigation map generation tool works offline.The cloud saves the navigation map and provides navigation algorithms for the terminal.The terminal realizes local real-time positioning and AR path rendering.

The research on visual global and local cognitive functions of patients with Parkinson's disease

Objective To study visual global and local cognitive functions in the patients with Parkinson’s disease(PD).Methods The visual cognitive software,which was devised by the Cognition Laboratory of National Science and Technology University,was used.30 PD patients and 31 normal controls(NC) were enrolled in the study.Result No significant difference on reaction time(RT) of visual global and local processing was found between PD patients and NC.Furthermore,the PD patients were divided into two groups:left side onset(LPD) and right side onset(RPD).The RT was longer in PD than in NC and RT was shorter in RPD than in NC.But there were no significant differences between PD patients and NC except global processing(big and small arrows at the same direction) in LPD.The RT was significantly longer in LPD than in RPD except local processing(big and small arrows at different direction)(P< 0.05).Conclusion The present study indicates that right basal ganglion might play a role in global and local processing and left basal ganglia might modulate(inhibit) the processing activity of right side.

基于LIO-SAM建图和激光视觉融合定位的温室自主行走系统

为解决传统导航方案在温室内无法应对光照变化大、作物行间距窄、接收GPS信号差等问题,该研究提出了基于即时定位与地图构建技术的激光视觉融合式自主导航算法。该系统利用三维激光雷达VLP-16(Velodyne LiDAR,VLP-16)和惯性测量单元获取温室环境信息,采用基于紧耦合的雷达惯导定位建图(tightly-coupled lidar inertial odometry via smoothing and mapping,LIO-SAM)算法构建导航地图,基于轮式里程计和视觉里程计采用扩展卡尔曼滤波器算法实现局部定位,融合激光点云配准算法和自适应蒙特卡洛定位算法实现全局定位。同时,在自主行走系统应用A*算法规划全局路径和动态窗口算法规划局部路径,从而实现自主导航。试验结果表明,LIO-SAM算法构建的温室导航地图最大相对误差、最大绝对误差和均方根误差分别为9.9%、0.081和0.063 m,在温室内改进后的定位算法横向偏差小于0.020 m,纵向偏差小于0.090 m;当自主行走系统以0.15、0.30和0.50 m/s的速度运行时,横向偏差、纵向偏差和航向偏角的平均值分别小于0.120 m、0.10 m和8.5°,标准差分别小于0.070 m、0.140 m和6.6°。该导航方案满足自主行走系统在温室内高精度建图、定位和导航的需求,可为自主移动平台提供理论与技术支撑。

视觉SLAM方法综述

实时定位与建图(SLAM)技术搭载特定传感器,使移动机器人在无任何环境先验条件下,在运动过程中自主建立环境模型来计算自身位姿,大幅提高其自主导航能力,以及对不同应用环境的适应性。视觉SLAM方法以相机作为外部传感器,通过采集周围环境信息来创建地图并实时估计机器人自身位姿。为此,介绍了具有代表性的经典视觉SLAM方法及与深度学习相结合的视觉SLAM方法,分析了视觉SLAM方法中采用的不同特征检测方法、后端优化、闭环检测,以及动态环境下视觉SLAM方法的应用,总结了视觉SLAM方法的问题,并探讨了视觉SLAM方法在未来的热点研究方向和发展前景。

图像语义信息在视觉SLAM中的应用研究进展

视觉同步定位与建图(visual simultaneous localization and mapping,VSLAM)技术以相机为主要传感器采集图像数据,基于多视几何、状态估计等算法原理获取载体的位置和姿态,同时构建一张用于导航定位的地图。视觉SLAM是自动驾驶、AR(augmented reality)、VR(virtual reality)、MR(mix reality)、智能机器人、无人机飞控中的关键技术。近年来,随着各个产业对智能导航定位的需求日渐增多,原本以几何测量为主的视觉SLAM逐渐融入对环境的语义理解。语义信息是指能够被人类直观感受和理解的概念,而图像语义信息是指图像中物体的轮廓、类别、显著性等信息。相比于图像中的几何特征,语义信息更具时空一致性,且更贴近人类感知的结果。将图像语义信息引入视觉SLAM,既能促进系统各个模块的性能,还能够提升视觉SLAM的智能感知能力,形成集几何测量、定位定姿、环境理解等多种功能的视觉语义SLAM。本文根据图像语义信息的应用方式,对视觉语义SLAM经典方案和最新研究进展进行归纳梳理。在此基础上,本文总结了视觉语义SLAM的现存问题与挑战,指出该领域未来的研究方向,以推动其面向智能导航定位进一步发展。

面向动态环境的视觉惯性定位方法

针对传统的视觉惯性里程计在动态环境下定位精度低和系统鲁棒性差等问题,提出了面向动态环境的视觉惯性定位方法。首先,利用语义分割提取环境中的语义信息,借助环境先验信息识别出动态物体。同时,采用深度生成网络对动态物体区域进行背景修复,生成只包含静态场景的图像,并将生成的图像用于后续的特征提取和跟踪,以减弱动态物体的影响。后端构建了紧耦合的图优化模型,将视觉数据与IMU数据相互融合,在滑动窗口中以非线性优化的方式估计位姿。实验结果表明,方法可以有效降低动态物体对定位的影响,提高系统的定位精度和鲁棒性。