Intelligent extraction of road cracks based on vehicle laser point cloud and panoramic sequence images

In light of the limited efficacy of conventional methods for identifying pavement cracks and the absence of comprehensive depth and location data in two-dimensional photographs,this study presents an intelligent strategy for extracting road cracks.This methodology involves the integration of laser point cloud data obtained from a vehicle-mounted system and a panoramic sequence of images.The study employs a vehicle-mounted LiDAR measurement system to acquire laser point cloud and panoramic sequence image data simultaneously.A convolutional neural network is utilized to extract cracks from the panoramic sequence image.The extracted sequence image is then aligned with the laser point cloud,enabling the assignment of RGB information to the vehicle-mounted three dimensional(3D)point cloud and location information to the two dimensional(2D)panoramic image.Additionally,a threshold value is set based on the crack elevation change to extract the aligned roadway point cloud.The three-dimensional data pertaining to the cracks can be acquired.The experimental findings demonstrate that the use of convolutional neural networks has yielded noteworthy outcomes in the extraction of road cracks.The utilization of point cloud and image alignment techniques enables the extraction of precise location data pertaining to road cracks.This approach exhibits superior accuracy when compared to conventional methods.Moreover,it facilitates rapid and accurate identification and localization of road cracks,thereby playing a crucial role in ensuring road maintenance and traffic safety.Consequently,this technique finds extensive application in the domains of intelligent transportation and urbanization development.The technology exhibits significant promise for use in the domains of intelligent transportation and city development.

Bionic Mosaic Method of Panoramic Image Based on Compound Eye of Fly

To satisfy the requirements of real-time and high quality mosaics, a bionic compound eye visual system was designed by simulating the visual mechanism of a fly compound eye. Several CCD cameras were used in this system to imitate the small eyes of a compound eye. Based on the optical analysis of this system, a direct panoramic image mosaic algorithm was proposed. Several sub-images were collected by the bionic compound eye visual system, and then the system obtained the overlapping proportions of these sub-images and cut the overlap sections of the neighboring images. Thus, a panoramic image with a large field of view was directly mosaicked, which expanded the field and guaranteed the high resolution. The experimental results show that the time consumed by the direct mosaic algorithm is only 2.2% of that by the traditional image mosaic algorithm while guaranteeing mosaic quality. Furthermore, the proposed method effectively solved the problem of misalignment of the mosaic image and eliminated mosaic cracks as a result of the illumination factor and other factors. This method has better real-time properties compared to other methods.

Research and Implementation of Algorithm for Image Enhancement and Unwrapped Distortion Correction for SLVF Panoramic Night Vision Image

Abstract： Based on digital signal processor（DSP） and field programmable gate array（FPGA） techniques, the architecture of super large view field（SLVF） panoramic night vision image processing hardware platform was established. The panoramic unwrapping and correcting algorithm, up to a full 360°, based on coordinate rotation digital computer （CORDIC） and night vision image enhancement algorithm, based on histogram equalization theory and edge detection theory, was presented in this paper, with the purpose of processing night vision dynamic panoramic annular image. The annular image can be unwrapped and corrected to conventional rectangular panorama by the panoramic image processing algorithm, which uses the pipelined CORDIC configuration to realize a trigonometric function generator with high speed and high precision. Histogram equalization algorithm can perfectly enhance the contrast of the night vision image. Edge detection algorithm can be propitious to find and detect small dim dynamic targets in night vision circumstances. After abundant experiment, the al- gorithm for panoramic image processing and night vision image enhancement is successfully implemented in FPGA and DSP. The panoramic night vision image system is a compact device, with no external rotating parts. And the system can reliably and dynamically detect 360＊ SLVF panoramic night vision image.

A 360-Degree Panoramic Image Inpainting Network Using a Cube Map

Inpainting has been continuously studied in the field of computer vision.As artificial intelligence technology developed,deep learning technology was introduced in inpainting research,helping to improve performance.Currently,the input target of an inpainting algorithm using deep learning has been studied from a single image to a video.However,deep learning-based inpainting technology for panoramic images has not been actively studied.We propose a 360-degree panoramic image inpainting method using generative adversarial networks(GANs).The proposed network inputs a 360-degree equirectangular format panoramic image converts it into a cube map format,which has relatively little distortion and uses it as a training network.Since the cube map format is used,the correlation of the six sides of the cube map should be considered.Therefore,all faces of the cube map are used as input for the whole discriminative network,and each face of the cube map is used as input for the slice discriminative network to determine the authenticity of the generated image.The proposed network performed qualitatively better than existing single-image inpainting algorithms and baseline algorithms.

Electronic stabilization of catadioptric panoramic image

An effective algorithm of electronic image stabilization （EIS） of catadioptric panoramic imaging system for track robots is presented. The key techniques of this algorithm are as follows：① A model of electronic image stabilization is built by analyzing the imaging theory and the principle of EIS, and the image shift function of unwrapped panoramic image is deduced;② The relationship equation between motion estimation parameters of annular panoramic image and motion estimation parameters of unwrapped panoramic image is developed according to the constrained aspect ratio of real objects, motion parameters of annular panoramic image are firstly estimated, and then motion parameters among the image shift function are carried out according to the relationship equation;③ An excessive stabilization threshold is presented to prevent the phenomena of excessive stabilization, and the Kalman filtering is adopted to smooth the image sequences. Numerical experimental results show that this algorithm can effectively smooth out the unwanted motion and follow the intentional camera movement under certain resolutions.

Real-time parameters monitoring system for underground equipment based on panoramic images

Complex terrain and working equipment in coal mine underground need a way to ensure coal mine safety. In this paper, the way to monitor the real-time status of underground equipment was put forward, and it was proved to be effective as commanding and dispatching system. Monitoring system for underground equipment based on panoramic images was effectively combined with real-time sensor data and static panoramic images of underground surrounding, which not only realizes real-time status monitoring for underground equipment, but also gets a direct scene for underground surrounding. B/S mode was applied in the monitoring system and this is convenient for users to monitor the equipment. Meantime, it can reduce the waste of the data resource.

Panoramic Imaging System Inspired by Insect Compound Eyes

Inspired by the unique structure of insect compound eyes,a multi-channel image acquisition system is designed to photograph a cylindrical panorama of its surroundings with one shot. The hardware structure consists of an embedded ARM system and one array of 16 micro-image sensors. The system achieves the synchronization of captured photos in 10 ms,as well as 10 f /s video capture. The software architecture includes the TCP /IP protocol,video capture procedures in"Poll/Read"or"video streaming"modes,thread pool monitoring in multi-threading mutex,synchronization control with the"event""mutex signal"and"critical region"functions,and a synthetic image algorithm characterized by its portability,modularity,and remote transmission. The panoramic imaging system is expected to be a vision sensor for mobile robotics.

Fully Convolutional Networks for Street Furniture Identification in Panorama Images

Panoramic images are widely used in many scenes,especially in virtual reality and street view capture.However,they are new for street furniture identification which is usually based on mobile laser scanning point cloud data or conventional 2D images.This study proposes to perform semantic segmentation on panoramic images and transformed images to separate light poles and traffic signs from background implemented by pre-trained Fully Convolutional Networks(FCN).FCN is the most important model for deep learning applied on semantic segmentation for its end to end training process and pixel-wise prediction.In this study,we use FCN-8s model that pre-trained on cityscape dataset and finetune it by our own data.Then replace cross entropy loss function with focal loss function in the FCN model and train it again to produce the predictions.The results show that in all results from pre-trained model,fine-tuning,and FCN model with focal loss,the light poles and traffic signs are detected well and the transformed images have better performance than panoramic images in the prediction according to the Recall and IoU evaluation.

Image Stitching Method of Non-Ferrous Smelting Scene Based on SIFT Algorithm and Color Constancy

In non-ferrous smelting scenes, complex environments, uneven lighting conditions, and inaccurate automatic exposure parameters result in the panoramic image having obvious seams and significant brightness differences in different regions. To address these issues, a novel image stitching method based on Scale-Invariant Feature Transform (SIFT) and color constancy theory was proposed. Initially, the input image was preprocessed followed by feature extraction and matching. Subsequently, color constancy processing based on matching points was performed to acquire an image with consistent brightness to be spliced. This process was complemented by the integration of multi-band fusion to enhance the original fusion procedure. Eventually, a spliced image with seamless blending and even luminosity was generated. The method proposed in this paper can obtain panoramic images that were more suitable for human eyes to observe, and greatly improve the subjective and objective performance of panoramic images. In the non-ferrous smelting scene, the PSNR and SSIM scores were improved by 3.555 dB and 0.16 respectively.

Integration and application of panorama field geological images and Google maps

Panorama field geological image is an important form of displaying field geology information with high resolution.Taking an example for Xingcheng,the authors studied the use of panorama field geological images based on the Flex for rich client display technology,application of Google maps API for Flash,and discussed construction method for field geological panoramic information system which created the rich client Internet application program.The results indicate that using Google maps API to mark position and route of panoramic image collection point can be fully integrated field geological information and map information.It can provide visual field geological information for users.This program is characterized by quick execution speed,strong interaction and rich function,which promotes the users'experience effectively.

CIRCUMFERENTIAL EQUISPACED CURVES'S IMAGE DETECTION TECHNIQUE AND ITS APPLAICATION ON RIFLING ANGLE MEASUREMENT

Aiming at the parameter detection of the circumferential equispaced curves,and on the background of the rifling parameter detection, a panoramic image based detectiontechnique is proposed, which employs the panoramic image photographed by a tamper reflector and aCCD camera to detect the parameter of the circumferential equispaced curves. The effect of thesystem's off-center error, deflection error and pixel error on the parameter detection is analyzed,which shows that this technique make a good use of the property of the circumferential equispaced,and has a high power of anti off-center error and anti deflection error, resulting a high precisionat the parameter measurement on circumferential equispaced curves.

Terrain reconstruction from Chang'e-3 PCAM images

The existing terrain models that describe the local lunar surface have limited resolution and accuracy, which can hardly meet the needs of rover navigation,positioning and geological analysis. China launched the lunar probe Chang＇e-3 in December, 2013. Chang＇e-3 encompassed a lander and a lunar rover called ＂Yutu＂（Jade Rabbit）. A set of panoramic cameras were installed on the rover mast. After acquiring panoramic images of four sites that were explored, the terrain models of the local lunar surface with resolution of 0.02 m were reconstructed. Compared with other data sources, the models derived from Chang＇e-3 data were clear and accurate enough that they could be used to plan the route of Yutu.

A survey on image and video stitching

Image/video stitching is a technology for solving the field of view(FOV)limitation of images/videos.It stitches multiple overlapping images/videos to generate a wide-FOV image/video,and has been used in various fields such as sports broadcasting,video surveillance,street view,and entertainment.This survey reviews image/video stitching algorithms,with a particular focus on those developed in recent years.Image stitching first calculates the corresponding relationships between multiple overlapping images,deforms and aligns the matched images,and then blends the aligned images to generate a wide-FOV image.A seamless method is always adopted to eliminate such potential flaws as ghosting and blurring caused by parallax or objects moving across the overlapping regions.Video stitching is the further extension of image stitching.It usually stitches selected frames of original videos to generate a stitching template by performing image stitching algorithms,and the subsequent frames can then be stitched according to the template.Video stitching is more complicated with moving objects or violent camera movement,because these factors introduce jitter,shakiness,ghosting,and blurring.Foreground detection technique is usually combined into stitching to eliminate ghosting and blurring,while video stabilization algorithms are adopted to solve the jitter and shakiness.This paper further discusses panoramic stitching as a special-extension of image/video stitching.Panoramic stitching is currently the most widely used application in stitching.This survey reviews the latest image/video stitching methods,and introduces the fundamental principles/advantages/weaknesses of image/video stitching algorithms.Image/video stitching faces long-term challenges such as wide baseline,large parallax,and low-texture problem in the overlapping region.New technologies may present new opportunities to address these issues,such as deep learning-based semantic correspondence,and 3D image stitching.Finally,this survey discusses the challenges of image/video stitching and proposes potential solutions.

Rapid digitalization and panoramic evaluation of weld X-ray film

The computer evaluation of weld X-ray film is an attractive technique for weld seam NDT （ nondestructive testing）. To achieve this target, digitalization of film is the first step and automatic defect identification is another key technique. In this paper, a weld X-ray film digitalizing system has been established with linear array CCD and highlight LED light source. Its space resolution can reach 0. 04 mm/pixel and scanning speed can reach 100 mm/s for an industrial film. The transfer function curves of the system have been measured and the results indicate that its image gray resolution can reach 88 G/D at 4. 5D, and its dynamic range can be wider than 2. OD. In order to facilitate the evaluation of large welded structure, a panoramic evaluation algorithm is developed also. The algorithm includes image matching, image fusion and panoramic evaluation of the long linked film image.

Rheumatoid Arthritis of the Temporomandibular Joint;Comparison of Digital Panoramic Radiographs Taken Using the Joint Limitation Program [JLA View] and CT Scans

Objective: The aim of this study was to evaluate the efficacy of digital panoramic radiographs using the JLA view pro-gram in cases of rheumatoid arthritis and compare them to CT scans of the patients. Methods: 40 patients with known condition of RA and clinical symptoms in the TMJ were selected for the study. Radiological evaluation included a panoramic radiograph of the TMJs that was taken and a computer tomography of the joints. In the panoramic radio-graphs taken, isolation of the TMJs was done using the JLA view program, while in the CT scans of the patients, all scans were taken with closed mouth, with a distance of 0.5 mm per slice. The parameters examined were: 1) Bony changes of the condyle;2) The position of the condyle in the mandibular fossa;3) The joint space;4) Bony changes of mandibular fossa. Results: There were no statistically significant differences found between the two observers or be-tween the two joints of the same patient [right and left] on the panoramic radiographs. For the case of CT scans there were significant differences between the joint space of right and left joints, while in the ANOVA performed differences were found for the evaluation of the bony changes of the condyle. Conclusion: There were no significant differences between the two radiographic methods selected and therefore when a proper simple radiograph is taken and well evalu-ated, the conclusions drawn from it are well based and there is no need for further

Registration of Two Dental Panoramic Radiographs

In dental panoramic images, the information on physical changes of alveolar bone or jaw bone is very important to diagnose several diseases. To detect such change, it is useful to compare two panoramic x-ray images acquired at different times. These two images are usually acquired with different conditions in terms of the positioning of the dental arch, and thus these images can be impaired from some geometrical changes related to the scale of the panoramic images and deformation of the teeth and jaw bones. As a result of this, it is very hard to make an accurate registration. To cope with this issue, we developed a dedicated image registration method to match these two images by a newly introduced non-rigid transformation method and registration method using the cross-correlation of localized regions. We evaluated our proposed method with several sets of two images acquired with different geometrical conditions. The material evaluated in this study was a skull phantom. The results of these experiments showed the validity and intrinsic ability of our proposed method in clinical examinations.

Perceptual quality assessment of panoramic stitched contents for immersive applications:a prospective survey

The recent advancements in the field of Virtual Reality(VR)and Augmented Reality(AR)have a substantial impact on modern day technology by digitizing each and everything related to human life and open the doors to the next generation Software Technology(Soft Tech).VR and AR technology provide astonishing immersive contents with the help of high quality stitched panoramic contents and 360°imagery that widely used in the education,gaming,entertainment,and production sector.The immersive quality of VR and AR contents are greatly dependent on the perceptual quality of panoramic or 360°images,in fact a minor visual distortion can significantly degrade the overall quality.Thus,to ensure the quality of constructed panoramic contents for VR and AR applications,numerous Stitched Image Quality Assessment(SIQA)methods have been proposed to assess the quality of panoramic contents before using in VR and AR.In this survey,we provide a detailed overview of the SIQA literature and exclusively focus on objective SIQA methods presented till date.For better understanding,the objective SIQA methods are classified into two classes namely Full-Reference SIQA and No-Reference SIQA approaches.Each class is further categorized into traditional and deep learning-based methods and examined their performance for SIQA task.Further,we shortlist the publicly available benchmark SIQA datasets and evaluation metrices used for quality assessment of panoramic contents.In last,we highlight the current challenges in this area based on the existing SIQA methods and suggest future research directions that need to be target for further improvement in SIQA domain.

全景影像快速加载与精确量测算法研究

针对全景影像加载速度慢及量测精度低的问题,提出一种全景球金字塔切片模型动态约束加载算法。首先,基于格网球体投影模型构建三维全景场景;其次,对全景影像进行压缩与切片,并设计全景切片金字塔模型;然后,采用逆向墨卡托投影将全景切片作为纹理映射到球体内表面进行渲染,形成全景球金字塔切片模型;最后,按照可视区域约束对金字塔模型切片进行快速加载实现动态可视化,并结合深度图像实现全景影像的精确量测。实验结果表明,选用顶点数量为11904的全景球金字塔切片模型,量测中误差小于3 cm,同时提高了加载速度。

基于LM优化的车载全景影像绝对测量方法

【目的】为解决车载全景影像在智慧城市、智慧城管、智慧交通等领域中应用不足的问题,提出了一种基于LM优化的车载序列全景影像绝对测量的方法。【方法】首先,对待测目标点在全景影像上的成像过程进行分析,详细推导了目标点绝对坐标测量的理论模型。其次,为解决该理论模型的优化求解问题,引入了对初值不敏感且收敛速度较快的LM算法,并对LM算法中的初值获取、函数构建、参数设置等问题进行了设计。最后,结合实际车载全景影像和目标点位实测数据,对所提的方法进行验证。【结果】该方法获得的目标点绝对坐标平面中误差和高程中误差分别为0.123 m和0.118 m,符合《城市测量规范》(CJJ/T 8—2011)中的相关要求。【结论】该方法能够有效获取全景影像范围内待测目标点的绝对坐标,从而为全景影像在城市部件普查、交通基础设施测量等领域的应用提供了可能。