Automatic Miscalibration Detection and Correction of LiDAR and Camera Using Motion Cues

This paper aims to develop an automatic miscalibration detection and correction framework to maintain accurate calibration of LiDAR and camera for autonomous vehicle after the sensor drift.First,a monitoring algorithm that can continuously detect the miscalibration in each frame is designed,leveraging the rotational motion each individual sensor observes.Then,as sensor drift occurs,the projection constraints between visual feature points and LiDAR 3-D points are used to compute the scaled camera motion,which is further utilized to align the drifted LiDAR scan with the camera image.Finally,the proposed method is sufficiently compared with two representative approaches in the online experiments with varying levels of random drift,then the method is further extended to the offline calibration experiment and is demonstrated by a comparison with two existing benchmark methods.

Analyzing the Impact of Scene Transitions on Indoor Camera Localization through Scene Change Detection in Real-Time

Real-time indoor camera localization is a significant problem in indoor robot navigation and surveillance systems.The scene can change during the image sequence and plays a vital role in the localization performance of robotic applications in terms of accuracy and speed.This research proposed a real-time indoor camera localization system based on a recurrent neural network that detects scene change during the image sequence.An annotated image dataset trains the proposed system and predicts the camera pose in real-time.The system mainly improved the localization performance of indoor cameras by more accurately predicting the camera pose.It also recognizes the scene changes during the sequence and evaluates the effects of these changes.This system achieved high accuracy and real-time performance.The scene change detection process was performed using visual rhythm and the proposed recurrent deep architecture,which performed camera pose prediction and scene change impact evaluation.Overall,this study proposed a novel real-time localization system for indoor cameras that detects scene changes and shows how they affect localization performance.

Improved plasma position detection method in EAST Tokamak using fast CCD camera

To control the steady-state operation of Tokamak plasma, it is crucial to accurately obtain its shape and position. This paper presents a method for use in rapidly detecting plasma configuration during discharge of the Experimental Advanced Superconducting Tokamak device. First, a visible/infrared integrated endoscopy diagnostic system with a large field of view is introduced,and the PCO.edge5.5 camera in this system is used to acquire a plasma discharge image. Based on the analysis of various traditional edge detection algorithms, an improved wavelet edge detection algorithm is then introduced to identify the edge of the plasma. In this method, the local maximum of the modulus of wavelet transform is searched along four gradient directions, and the adaptive threshold is adopted. Finally, the detected boundary is fitted using the least square iterative method to accurately obtain the position of the plasma. Experimental results obtained using the EAST device show that the method presented in this paper can realize expected goals and produce ideal effects;this method thus has significant potential for application in further feedback control of plasma.

Two cameras detect more lesions in the small-bowel than one

AIM:To explore the feasibility of dual camera capsule (DCC)small-bowel(SB)imaging and to examine if two cameras complement each other to detect more SB lesions.METHODS:Forty-one eligible,consecutive patients underwent DCC SB imaging.Two experienced investigators examined the videos and compared the total number of detected lesions to the number of lesions detected by each camera separately.Examination tolerability was assessed using a questionnaire.RESULTS:One patient was excluded.DCC cameras detected 68 positive findings(POS)in 20(50%)cases.Fifty of them were detected by the"yellow"camera,48 by the"green"and 28 by both cameras;44%(n=22)of the"yellow"camera’s POS were not detected by the"green"camera and 42%(n=20)of the"green" camera’s POS were not detected by the"yellow"camera.In two cases,only one camera detected significant findings.All participants had 216 findings of unknown significance(FUS).The"yellow","green"and both cameras detected 171,161,and 116 FUS,respectively;32%(n=55)of the"yellow"camera’s FUS were not detected by the"green"camera and 28%(n=45)of the"green"camera’s FUS were not detected by the "yellow"camera.There were no complications related to the examination,and 97.6%of the patients would repeat the examination,if necessary.CONCLUSION:DCC SB examination is feasible and well tolerated.The two cameras complement each other to detect more SB lesions.

EDGE DETECTION METHOD OF REMOTE SENSING IMAGES BASED ON MATHEMATICAL MORPHOLOGY OF MULTI-STRUCTURE ELEMENTS

This paper puts forward an effective, specific algorithm for edge detection. Based on multi-structure elements of gray mathematics morphology, in the light of difference between noise and edge shape of RS images, the paper establishes multi-structure elements to detect edge by utilizing the grey form transformation principle. Compared with some classical edge detection operators, such as Sobel Edge Detection Operator, LOG Edge Detection Operator, and Canny Edge Detection Operator, the experiment indicates that this new algorithm possesses very good edge detection ability, which can detect edges more effectively, but its noise-resisting ability is relatively low. Because of the bigger noise & remote sensing image, the authors probe into putting forward other edge detection method based on combination of wavelet directivity checkout technology and small-scale Mathematical Morphology finally. So, position at the edge can be accurately located, the noise can be inhibited to a certain extent and the effect of edge detection is obvious.

USING COVARIANCE INTERSECTION FOR CHANGE DETECTION IN REMOTE SENSING IMAGES

In this paper,an unsupervised change detection technique for remote sensing images acquired on the same geographical area but at different time instances is proposed by conducting Covariance Intersection(CI) to perform unsupervised fusion of the final fuzzy partition matrices from the Fuzzy C-Means(FCM) clustering for the feature space by applying compressed sampling to the given remote sensing images.The proposed approach exploits a CI-based data fusion of the membership function matrices,which are obtained by taking the Fuzzy C-Means(FCM) clustering of the frequency-domain feature vectors and spatial-domain feature vectors,aimed at enhancing the unsupervised change detection performance.Compressed sampling is performed to realize the image local feature sampling,which is a signal acquisition framework based on the revelation that a small collection of linear projections of a sparse signal contains enough information for stable recovery.The experimental results demonstrate that the proposed algorithm has a good change detection results and also performs quite well on denoising purpose.

Position detection of component aperture based on integral imaging technology

Aiming at the problem that it is difficult to locate all the aperture positions of the large size component using Houghcircle detection method,this article presents a non-contact measurement method combining the integral imaging technology withHough circle detection algorithm.Firstly,a set of integral imaging information acquisition algorithms were proposed accordingto the classical imaging theory.Secondly,the camera array experiment device was built by using two-dimensional translationstage and charge coupled device(CCD)camera.When the system is operating,element image array captured with the camera isused to achieve the positioning of the component aperture using Hough circle detection and coordinate acquisition algorithm.Based on the above theory,a verification experiment was carried out.The results show that the detection error of the componentaperture position is within0.3mm,which provides effective theoretical support for the application of integral imagingtechnology in high precision detection

A robust system for real-time pedestrian detection and tracking

A real-time pedestrian detection and tracking system using a single video camera was developed to monitor pedestrians. This system contained six modules: video flow capture, pre-processing, movement detection, shadow removal, tracking, and object classification. The Gaussian mixture model was utilized to extract the moving object from an image sequence segmented by the mean-shift technique in the pre-processing module. Shadow removal was used to alleviate the negative impact of the shadow to the detected objects. A model-free method was adopted to identify pedestrians. The maximum and minimum integration methods were developed to integrate multiple cues into the mean-shift algorithm and the initial tracking iteration with the competent integrated probability distribution map for object tracking. A simple but effective algorithm was proposed to handle full occlusion cases. The system was tested using real traffic videos from different sites. The results of the test confirm that the system is reliable and has an overall accuracy of over 85%.

Using UAVs for detection of trees from digital surface models

A difficult problem in forestry is tree inventory.In this study, a GoProHero attached to a small unmanned aerial vehicle was used to capture images of a small area covered by pinus pinea trees. Then, a digital surface model was generated with image matching. The elevation model representing the terrain surface, a ‘digital terrain model’,was extracted from the digital surface model using morphological filtering. Individual trees were extracted by analyzing elevation flow on the digital elevation model because the elevation reached the highest value on the tree peaks compared to the neighborhood elevation pixels. The quality of the results was assessed by comparison with reference data for correctness of the estimated number of trees. The tree heights were calculated and evaluated with ground truth dataset. The results showed 80% correctness and 90% completeness.

Embedded System Based Raspberry Pi 4 for Text Detection and Recognition

Detecting and recognizing text from natural scene images presents a challenge because the image quality depends on the conditions in which the image is captured,such as viewing angles,blurring,sensor noise,etc.However,in this paper,a prototype for text detection and recognition from natural scene images is proposed.This prototype is based on the Raspberry Pi 4 and the Universal Serial Bus(USB)camera and embedded our text detection and recognition model,which was developed using the Python language.Our model is based on the deep learning text detector model through the Efficient and Accurate Scene Text Detec-tor(EAST)model for text localization and detection and the Tesseract-OCR,which is used as an Optical Character Recognition(OCR)engine for text recog-nition.Our prototype is controlled by the Virtual Network Computing(VNC)tool through a computer via a wireless connection.The experiment results show that the recognition rate for the captured image through the camera by our prototype can reach 99.75%with low computational complexity.Furthermore,our proto-type is more performant than the Tesseract software in terms of the recognition rate.Besides,it provides the same performance in terms of the recognition rate with a huge decrease in the execution time by an average of 89%compared to the EasyOCR software on the Raspberry Pi 4 board.

Vision-based fatigue crack detection using global motion compensation and video feature tracking

Fatigue cracks that develop in civil infrastructure such as steel bridges due to repetitive loads pose a major threat to structural integrity.Despite being the most common practice for fatigue crack detection,human visual inspection is known to be labor intensive,time-consuming,and prone to error.In this study,a computer vision-based fatigue crack detection approach using a short video recorded under live loads by a moving consumer-grade camera is presented.The method detects fatigue crack by tracking surface motion and identifies the differential motion pattern caused by opening and closing of the fatigue crack.However,the global motion introduced by a moving camera in the recorded video is typically far greater than the actual motion associated with fatigue crack opening/closing,leading to false detection results.To overcome the challenge,global motion compensation(GMC)techniques are introduced to compensate for camera-induced movement.In particular,hierarchical model-based motion estimation is adopted for 2D videos with simple geometry and a new method is developed by extending the bundled camera paths approach for 3D videos with complex geometry.The proposed methodology is validated using two laboratory test setups for both in-plane and out-of-plane fatigue cracks.The results confirm the importance of motion compensation for both 2D and 3D videos and demonstrate the effectiveness of the proposed GMC methods as well as the subsequent crack detection algorithm.

基于Camera Link标准的图像采集处理系统及其应用

针对非接触在线高速实时检测的要求,提出了一种基于线阵CCD和DSP的图像采集处理系统.系统采用Camera Link接收器DS90CR288A将CCD输出的速度高达80MHz的低压差分信号(LVDS)转换成低压TTL(LVTTL)信号,然后将数据送入高速FIFO以协调CCD和DSP的时钟差异,最后采用EDMA方式将FIFO数据转存到SDRAM中,一幅图像传完之后中断DSP进行图像处理,MCU负责尺寸显示.实验结果表明,该系统达到了较高的测量精度.

高精度Camera Link相机视频采集处理系统的设计

在电晕探测系统中,为满足高精度Camera Link接口相机的高实时视频采集、处理需要,同时降低系统成本及复杂度,提出了一种通过DM642的视频端口扩展Camera Link接口的方法。与一般采用FPGA作为处理前端不同,该方法基于视频端口的原始捕获模式,实现了在DM642上的Camera Link的直接扩展。设计了硬件电路及VP口采集驱动程序,并采用高精度(16bit)紫外相机进行了实时采集。结果表明,该接口可以满足采用Camera Link接口的紫外相机实时采集需求。基于该接口的高速数字图像处理系统已用于紫外探测系统信息处理中。

A Sensor-Service Collaboration Approach for Target Tracking in Wireless Camera Networks

Mobile target tracking is a necessary function of some emerging application domains, such as virtual reality, smart home and intelligent healthcare. However, existing portable devices for target tracking are resource intensive and high-cost. Camera tracking is an effective location tracking way for those emerging applications which can reuse the existing ubiquitous video monitoring system. This paper proposes a dynamic community-based camera collaboration(D3C) framework for target location and tracking. The contributions of D3C mainly include that(1) nonlinear perspective projection model is selected as the camera sensing model and sequential Monte Carlo is employed to predict the target location;(2) a dynamic collaboration scheme is proposed, it is based on the local community-detection theory deriving from social network analysis. The performance of proposed approach is validated by both synthetic datasets and real-world application. The experiment results show that D3C meets the versatility, real-time and fault tolerance requirements of target tracking applications.

Temporal patterns of three sympatric pheasant species in the Nanling Mountains: N-mixture modeling applied to detect abundance

Background:The reliability of long-term population estimates is crucial for conservation and management purposes.Most previous studies assume that count indices are proportionally related to abundance;however,this assumption may not hold when detection varies spatially and temporally.We examined seasonal variations in abundance of three bird species(Cabot’s Tragopan Tragopan caboti,Silver Pheasant Lophura nycthemera,and Whitenecklaced Partridge Arborophila gingica) along an elevational gradient,using N-mixture models that take into account imperfect detection in our bird data.Methods:Camera-trapping was used to monitor temporal activity patterns of these species at Guangdong Nanling National Nature Reserve from December 2013 to November 2017(4 seasons per year).For abundance analysis(N-mixture modeling),we divided a year into 4 seasons,i.e.3 months per season,and performed the analysis by season.Elevation was incorporated into the N-mixture model as a covariate that may affect abundance.We compared the N-mixture model with a null model(no covariate model) and selected the better model based on AIC values to make an inference.Results:From 24 sampling sites,we obtained 6786 photographs of 8482 individuals of 44 bird species and 26 mammal species.Silver Pheasant was photographed much more frequently and showed higher temporal activity frequency than White-necklaced Partridge or Cabot’s Tragopan.Silver Pheasant was camera-captured most frequently in summer,and other two species in winters.All three species had two daytime activity peaks:between 6:00 a.m.and 10:00 a.m.,and between 5:00 p.m.and 7:00 p.m.,respectively.Our estimated abundance and detection probability from the N-mixture model were variable by season.In particular,all three species showed greater abundance in summer than in winter,and estimated abundance patterns of all three species were more similar with observed cameratrapping counts in summers.Moreover,in winter,elevation had a positive impact on abundance of Silver Pheasant and Cabot’s Tragopan,but not on White-necklaced Partridge.Conclusions:Our results demonstrate that the N-mixture model performed well in the estimation of temporal popu lation abundance at local fixed permanent plots in mountain habitat in southern China,based on the modeling of repeated camera-trapping counts.The seasonal differences in abundance of the three endemic bird species and the strong effect of elevation on abundance of two species in winter were only indicative of variations in spatio-tempora distribution within species and between species.In identifying suitable habitat for endemic pheasants,the positive elevational effect also suggests that more attention should be paid to conservation of areas with higher elevation in the Nanling Mountains.

Lane Keeping Algorithm for Off-The-Shelf Wide-Angle Camera

Automobile accidents cost over a trillion-do llar every year and this figure will continue increasing without employing new technological solutions.Among these solutions,the automated lane-keeping system is one of the promising ones and such a system consists of two essential technologies:road detection and steering control.In this paper,novel lane keeping algorithms are proposed and are implemented using only a single off-the-shelf wide-angle camera as input.The implemented system is verified,through both simulation and experiments,and is found providing satisfactory performance for an automated lane-keeping system.When compared to the state-of-the-art lane-keeping systems,the implemented system can perform consistently across various ambient light conditions including the most challenging ones.

基于动态视觉传感器的无人机目标检测与避障

针对无人机在动态环境中感知动态目标与躲避高速动态障碍物,提出了基于动态视觉传感器的目标检测与避障算法。设计了滤波方法和运动补偿算法,滤除事件流中背景噪声、热点噪声及由相机自身运动产生的冗余事件;设计了一种融合事件图像和RGB图像的动态目标融合检测算法,保证检测的可靠性。根据检测结果对目标运动轨迹进行估计,结合障碍物运动特点和无人机动力学约束改进速度障碍法躲避动态障碍物。大量仿真试验、手持试验及飞行试验验证了所提算法的可行性。

基于红外相机和毫米波雷达融合的烟雾遮挡无人驾驶车辆目标检测与跟踪

战场环境下无人驾驶车辆的感知系统易受烟雾、扬尘等天气的影响,对关键目标的检测与跟踪能力大大下降,造成目标漏检、目标误检、目标丢失等严重后果。针对该问题,开发毫米波雷达和红外相机融合系统,采用目标级融合方式建立简洁有效的融合规则,提炼和组合各传感器的优势信息,最终输出稳定的目标感知结果。对毫米波雷达的目标进行有效性检验和提取,并提出改进的基于密度的含噪声空间聚类应用算法,以减少毫米波雷达噪音干扰。以YOLOv4网络为基础,引入MobileNetv2主干网络,在网络训练过程中运用迁移学习方法,同时对红外数据样本进行扩充,解决了红外图像训练样本少的问题。试验结果表明,相较于仅基于红外相机的算法,融合检测算法在烟雾环境下的精度显著提升,且算法实时性高,实现了烟雾环境下毫米波雷达与红外相机融合的目标检测与跟踪,提高了无人驾驶车辆目标检测与跟踪系统的抗烟雾干扰能力。

Optimized Method for Real-time Texture Reconstruction with RGB-D Camera

With the appearance of RGB-D camera, the field of three-dimensional (3D) reconstruction receives more and more attention. In this paper, we present an optimization approach to produce high-quality textured 3D models based on the real-time 3D reconstruction system. The resulting models of real-time texture reconstruction often suffer from blurring, ghosting, and other artifacts. Our approach addresses this texture quality problem using blur detection and an optimized weight function. Experimental results demonstrate that our approach can improve the quality of textured 3D models by reducing the blur and ghosts on the model surface. © 2017, Tianjin University and Springer-Verlag GmbH Germany.

航拍电力杆塔图像中销钉唯一性识别研究

近年来,基于深度学习的缺陷销钉检测技术在电力杆塔巡检中得到广泛应用。但是该技术只能在图像空间对缺陷销钉进行定位,无法准确给出缺陷销钉在实际电力杆塔中对应销钉的唯一编号。针对该问题,该文提出一种融合深度学习与三维重投影的缺陷销钉唯一性识别方法。首先通过构建巡检电力杆塔系统的三维模型,进行电力杆塔系统中部件模型的编号获取,以及部件模型三维空间包围盒顶点和中心点坐标的计算;其次,利用YOLOv5模型在图像空间对销钉及上下文相关目标进行检测;再次,通过该文提出的融合结构约束的相机位姿估计算法对航拍图像的相机位姿进行估计;最后,将三维电力杆塔系统中的销钉包围盒按相机位姿进行重投影与销钉检测框进行匹配度计算,在图像空间中输出带有具体编号的销钉检测框。在YOLOv5模型输出的销钉检测框基础上进行销钉唯一性识别,利用仿真数据进行实验得到的唯一性识别正确率为100%,利用实拍数据进行实验得到的唯一性识别正确率为93.3%,验证了该文提出的销钉唯一性识别方法的有效性。通过对缺陷销钉进行唯一性识别,可以减少人工对故障图像的二次审核工作量,并准确地统计缺陷销钉的数量及具体位置信息,为后续故障维修及故障相关性分析等精细化管理提供支持。