The State-of-the-Art Review on Applications of Intrusive Sensing,Image Processing Techniques,and Machine Learning Methods in Pavement Monitoring and Analysis

In modern transportation,pavement is one of the most important civil infrastructures for the movement of vehicles and pedestrians.Pavement service quality and service life are of great importance for civil engineers as they directly affect the regular service for the users.Therefore,monitoring the health status of pavement before irreversible damage occurs is essential for timely maintenance,which in turn ensures public transportation safety.Many pavement damages can be detected and analyzed by monitoring the structure dynamic responses and evaluating road surface conditions.Advanced technologies can be employed for the collection and analysis of such data,including various intrusive sensing techniques,image processing techniques,and machine learning methods.This review summarizes the state-ofthe-art of these three technologies in pavement engineering in recent years and suggests possible developments for future pavement monitoring and analysis based on these approaches.

Transverse Velocity Field Measurements in High-resolution Solar Images Based on Deep Learning

To address the problem of the low accuracy of transverse velocity field measurements for small targets in highresolution solar images,we proposed a novel velocity field measurement method for high-resolution solar images based on PWCNet.This method transforms the transverse velocity field measurements into an optical flow field prediction problem.We evaluated the performance of the proposed method using the Hαand TiO data sets obtained from New Vacuum Solar Telescope observations.The experimental results show that our method effectively predicts the optical flow of small targets in images compared with several typical machine-and deeplearning methods.On the Hαdata set,the proposed method improves the image structure similarity from 0.9182 to0.9587 and reduces the mean of residuals from 24.9931 to 15.2818;on the TiO data set,the proposed method improves the image structure similarity from 0.9289 to 0.9628 and reduces the mean of residuals from 25.9908 to17.0194.The optical flow predicted using the proposed method can provide accurate data for the atmospheric motion information of solar images.The code implementing the proposed method is available on https://github.com/lygmsy123/transverse-velocity-field-measurement.

Automatic removal of false image stars in disk-resolved images of the Cassini Imaging Science Subsystem

Taking a large number of images,the Cassini Imaging Science Subsystem(ISS)has been routinely used in astrometry.In ISS images,disk-resolved objects often lead to false detection of stars that disturb the camera pointing correction.The aim of this study was to develop an automated processing method to remove the false image stars in disk-resolved objects in ISS images.The method included the following steps:extracting edges,segmenting boundary arcs,fitting circles and excluding false image stars.The proposed method was tested using 200 ISS images.Preliminary experimental results show that it can remove the false image stars in more than 95%of ISS images with disk-resolved objects in a fully automatic manner,i.e.,outperforming the traditional circle detection based on Circular Hough Transform(CHT)by 17%.In addition,its speed is more than twice as fast as that of the CHT method.It is also more robust(no manual parameter tuning is needed)when compared with CHT.The proposed method was also applied to a set of ISS images of Rhea to eliminate the mismatch in pointing correction in automatic procedure.Experiment results showed that the precision of final astrometry results can be improve by roughly 2 times that of automatic procedure without the method.It proved that the proposed method is helpful in the astrometry of ISS images in a fully automatic manner.

High-resolution Solar Image Reconstruction Based on Non-rigid Alignment

Suppressing the interference of atmospheric turbulence and obtaining observation data with a high spatial resolution are an issue to be solved urgently for ground observations. One way to solve this problem is to perform a statistical reconstruction of short-exposure speckle images. Combining the rapidity of Shift-Add and the accuracy of speckle masking, this paper proposes a novel reconstruction algorithm-NASIR(Non-rigid Alignment based Solar Image Reconstruction). NASIR reconstructs the phase of the object image at each frequency by building a computational model between geometric distortion and intensity distribution and reconstructs the modulus of the object image on the aligned speckle images by speckle interferometry. We analyzed the performance of NASIR by using the correlation coefficient, power spectrum, and coefficient of variation of intensity profile in processing data obtained by the NVST(1 m New Vacuum Solar Telescope). The reconstruction experiments and analysis results show that the quality of images reconstructed by NASIR is close to speckle masking when the seeing is good, while NASIR has excellent robustness when the seeing condition becomes worse. Furthermore, NASIR reconstructs the entire field of view in parallel in one go, without phase recursion and block-by-block reconstruction, so its computation time is less than half that of speckle masking. Therefore, we consider NASIR is a robust and highquality fast reconstruction method that can serve as an effective tool for data filtering and quick look.

Wavelet Analysis of Space Solar Telescope Images

The scientific satellite SST (Space Solar Telescope) is an important research project strongly supported by the Chinese Academy of Sciences. Every day, SST acquires 50 GB of data (after processing) but only 10GB can be transmitted to the ground because of limited time of satellite passage and limited channel volume. Therefore, the data must be compressed before transmission. Wavelets analysis is a new technique developed over the last 10 years, with great potential of application. We start with a brief introduction to the essential principles of wavelet analysis, and then describe the main idea of embedded zerotree wavelet coding, used for compressing the SST images. The results show that this coding is adequate for the job.

Reconstructing the landing trajectory of the CE-3 lunar probe by using images from the landing camera

An accurate determination of the landing trajectory of Chang＇e-3 （CE-3） is significant for verifying orbital control strategy, optimizing orbital planning, accu- rately determining the landing site of CE-3 and analyzing the geological background of the landing site. Due to complexities involved in the landing process, there are some differences between the planned trajectory and the actual trajectory of CE-3. The land- ing camera on CE-3 recorded a sequence of the landing process with a frequency of 10 frames per second. These images recorded by the landing camera and high-resolution images of the lunar surface are utilized to calculate the position of the probe, so as to reconstruct its precise trajectory. This paper proposes using the method of trajectory reconstruction by Single Image Space Resection to make a detailed study of the hov- ering stage at a height of 100 m above the lunar surface. Analysis of the data shows that the closer CE-3 came to the lunar surface, the higher the spatial resolution of im- ages that were acquired became, and the more accurately the horizontal and vertical position of CE-3 could be determined. The horizontal and vertical accuracies were 7.09 m and 4.27 m respectively during the hovering stage at a height of 100.02 m. The reconstructed trajectory can reflect the change in CE-3＇s position during the powered descent process. A slight movement in CE-3 during the hovering stage is also clearly demonstrated. These results will provide a basis for analysis of orbit control strategy, and it will be conducive to adjustment and optimization of orbit control strategy in follow-up missions.

A new lunar absolute control point: established by images from the landing camera on Chang'e-3

The establishment of a lunar control network is one of the core tasks in selenodesy, in which defining an absolute control point on the Moon is the most im- portant step. However, up to now, the number of absolute control points has been very sparse. These absolute control points have mainly been lunar laser ranging retrore- flectors, whose geographical location can be observed by observations on Earth and also identified in high resolution lunar satellite images. The Chang＇e-3 （CE-3） probe successfully landed on the Moon, and its geographical location has been monitored by an observing station on Earth. Since its positional accuracy is expected to reach the meter level, the CE-3 landing site can become a new high precision absolute control point. We use a sequence of images taken from the landing camera, as well as satellite images taken by CE-1 and CE-2, to identify the location of the CE-3 lander. With its geographical location known, the CE-3 landing site can be established as a new abso- lute control point, which will effectively expand the current area of the lunar absolute control network by 22%, and can greatly facilitate future research in the field of lunar surveying and mapping, as well as selenodesy.

Machine Learning-based Identification of Contaminated Images in Light Curve Data Preprocessing

Attitude is one of the crucial parameters for space objects and plays a vital role in collision prediction and debris removal.Analyzing light curves to determine attitude is the most commonly used method.In photometric observations,outliers may exist in the obtained light curves due to various reasons.Therefore,preprocessing is required to remove these outliers to obtain high quality light curves.Through statistical analysis,the reasons leading to outliers can be categorized into two main types:first,the brightness of the object significantly increases due to the passage of a star nearby,referred to as“stellar contamination,”and second,the brightness markedly decreases due to cloudy cover,referred to as“cloudy contamination.”The traditional approach of manually inspecting images for contamination is time-consuming and labor-intensive.However,we propose the utilization of machine learning methods as a substitute.Convolutional Neural Networks and SVMs are employed to identify cases of stellar contamination and cloudy contamination,achieving F1 scores of 1.00 and 0.98 on a test set,respectively.We also explore other machine learning methods such as ResNet-18 and Light Gradient Boosting Machine,then conduct comparative analyses of the results.

Terrain reconstruction from Chang'e-3 PCAM images

The existing terrain models that describe the local lunar surface have limited resolution and accuracy, which can hardly meet the needs of rover navigation,positioning and geological analysis. China launched the lunar probe Chang＇e-3 in December, 2013. Chang＇e-3 encompassed a lander and a lunar rover called ＂Yutu＂（Jade Rabbit）. A set of panoramic cameras were installed on the rover mast. After acquiring panoramic images of four sites that were explored, the terrain models of the local lunar surface with resolution of 0.02 m were reconstructed. Compared with other data sources, the models derived from Chang＇e-3 data were clear and accurate enough that they could be used to plan the route of Yutu.

A coarse-to-fine strategy for the registration of the multi-wavelength high-resolution solar images

The registration of multi-wavelength high-resolution solar images is an important task in the research of solar physics. This paper proposed a coarse-to-fine strategy to realize the accurate registration of high-resolution photospheric images and chromospheric images observed by the New Vacuum Solar Telescope(NVST) whose field-of-view is about 2′~ 3′, and the spatial resolution can reach 0.1′′after image reconstruction. In this strategy, the full-disk solar images with relatively lower resolution taken by other space-or ground-based telescopes are taken as transition images, and the Fourier-Merlin transform,Template matching and a local statistical information based algorithm are used in combination. After registration, the geometric transformation between multi-wavelength images of NVST are corrected at the level of sub-arcseconds, including the rotation, scaling and translation relations. Two sets of data observed in active regions(i.e., the NOAA 11982 and the NOAA 12673) are used to illustrate our method step by step.The result shows that the registration accuracy can reach less than 1′′. Moreover, this work also has facilitated the combination of high-resolution observations of NVST with the continuum, ultraviolet passbands and magnetic field observations of the Solar Dynamic Observation(SDO), which is highly beneficial to the multi-instrument joint measurement of solar activities.

Obtaining space-based SDO/AIA solar UV and EUV images from ground-based Hαobservations by deep learning

In this work,we explore the mappings from solar images taken in Hα(6563 A)by the Global Oscillation Network Group(GONG)on the ground to those observed in eight different wavelengths(94,131,171,193,211,304,335 and 1600 A)by SDO/AIA in space.Eight mappings are built by training the conditional Generative Adversarial Networks(cGANs)on datasets with 500 paired images,which are[Hα,AIA94],[Hα,AIA131],[Hα,AIA171],[Hα,AIA193],[Hα,AIA211],[Hα,AIA304],[Hα,AIA335]and[Hα,AIA1600].We evaluate the eight trained cGANs models on validation and test datasets with 154-pair images and 327-pair images,respectively.The model generated fake AIA images match the corresponding observed AIA images well on large-scale structures such as large active regions and prominences.But the small-scale flare loops and filament threads are difficult to reconstruct.Four quantitative comparisons are carried out on the validation and test datasets to score the mappings.We find that the model-generated images in 304 and 1600 A match the corresponding observed images best.This exploration suggests that the cGANs are promising methods for mappings between ground-based Ha and space-based EUV/UV images,while some improvements are necessary.

Contour detection in Cassini ISS images based on Hierarchical Extreme Learning Machine and Dense Conditional Random Field

In Cassini ISS(Imaging Science Subsystem)images,contour detection is often performed on disk-resolved objects to accurately locate their center.Thus,contour detection is a key problem.Traditional edge detection methods,such as Canny and Roberts,often extract the contour with too much interior details and noise.Although the deep convolutional neural network has been applied successfully in many image tasks,such as classification and object detection,it needs more time and computer resources.In this paper,a contour detection algorithm based on H-ELM(Hierarchical Extreme Learning Machine)and Dense CRF(Dense Conditional Random Field)is proposed for Cassini ISS images.The experimental results show that this algorithm’s performance is better than both traditional machine learning methods,such as Support Vector Machine,Extreme Learning Machine and even deep Convolutional Neural Network.The extracted contour is closer to the actual contour.Moreover,it can be trained and tested quickly on the general configuration of PC,and thus can be applied to contour detection for Cassini ISS images.

Example-based super-resolution for single-image analysis from the Chang'e-1 Mission

Due to the low spatial resolution of images taken from the Chang＇e-1 （CE-I） orbiter, the details of the lunar surface are blurred and lost. Considering the limited spatial resolution of image data obtained by a CCD camera on CE-1, an example-based super-resolution （SR） algorithm is employed to obtain high- resolution （HR） images. SR reconstruction is important for the application of image data to increase the resolution of images. In this article, a novel example-based algorithm is proposed to implement SR reconstruction by single-image analysis, and the computational cost is reduced compared to other example-based SR methods. The results show that this method can enhance the resolution of images using SR and recover detailed information about the lunar surface. Thus it can be used for surveying HR terrain and geological features. Moreover, the algorithm is significant for the HR processing of remotely sensed images obtained by other imaging systems.

A new segmentation algorithm for lunar surface terrain based on CCD images

Terrain classification is one of the critical steps used in lunar geomorphologic analysis and landing site selection. Most of the published works have focused on a Digital Elevation Model （DEM） to distinguish different regions of lunar terrain. This paper presents an algorithm that can be applied to lunar CCD images by blocking and clustering according to image features, which can accurately distinguish between lunar highland and lunar mare. The new algorithm, compared with the traditional algo- rithm, can improve classification accuracy. The new algorithm incorporates two new features and one Tamura texture feature. The new features are generating an enhanced image histogram and modeling the properties of light reflection, which can represent the geological characteristics based on CCD gray level images. These features are ap- plied to identify texture in order to perform image clustering and segmentation by a weighted Euclidean distance to distinguish between lunar mare and lunar highlands. The new algorithm has been tested on Chang＇e-1 CCD data and the testing result has been compared with geological data published by the U.S. Geological Survey. The result has shown that the algorithm can effectively distinguish the lunar mare from highlands in CCD images. The overall accuracy of the proposed algorithm is satisfactory, and the Kappa coefficient is 0.802, which is higher than the result of combining the DEM with CCD images.

Fast compression and reconstruction of astronomical images based on compressed sensing

With the fast increase in the resolution of astronomical images, the question of how to process and transfer such large images has become a key issue in astronomy. We propose a new real-time compression and fast reconstruction algorithm for astronomical images based on compressive sensing techniques. We first reconstruct tile Original signal with fewer measurements, according to its compressibility. Then, based on the characteristics of astronomical images, we apply Daubechies orthogonal wavelets to obtain a sparse representation. A matrix representing a random Fourier ensembleis used to obtain a sparse representation in a lower dimensional space. For reconstructing the image, we propose a novel minimum total variation with block addptive sensing to balance the accuracy and eomputation time. Our experimental results show that the proposed algorithm can efficiently reconstruct colorful astronomicai images with high resolution and improve the applicability of compressed sensing.

Automated removal of stripe interference in full-disk solar images

The quality of full-disk solar Hα images is significantly degraded by stripe interference. In this paper, to improve the analysis of morphological evolution, a robust solution for stripe interference removal in a partial full-disk solar Hα image is proposed. The full-disk solar image is decomposed into a set of support value images on different scales by convolving the image with a sequence of multiscale support value filters, which are calculated from the mapped least-squares support vector machines （LS-SVMs）. To match the resolution of the support value images, a scale-adaptive LS-SVM regression model is used to remove stripe interference from the support value images. We have demonstrated the advantages of our method on solar Hα images taken in 2001-2002 at the Huairou Solar Observing Station. Our experimental results show that our method can remove the stripe interference well in solar Hα images and the restored image can be used in morphology researches.

How to co-add images? I. A new iterative method for image reconstruction of dithered observations

By employing the previous Voronoi approach and replacing its nearest neighbor approx- imation with Drizzle in iterative signal extraction, we develop a fast iterative Drizzle algorithm, namedfiDrizzle, to reconstruct the underlying band-limited image from undersampled dithered frames. Compared with the existing iDrizzle, the new algorithm improves rate of convergence and accelerates the computational speed. Moreover, under the same conditions （e.g. the same number of dithers and iterations）, fiDrizzle can make a better quality reconstruction than iDrizzle, due to the newly discov- ered High Sampling caused Decelerating Convergence （HSDC） effect in the iterative signal extraction process.fiDrizzle demonstrates its powerful ability to perform image deconvolution from undersampled dithers.

Identifying Host Galaxies of Extragalactic Radio Emission Structures using Machine Learning

This paper presents an automatic multi-band source cross-identification method based on deep learning to identify the hosts of extragalactic radio emission structures.The aim is to satisfy the increased demand for automatic radio source identification and analysis of large-scale survey data from next-generation radio facilities such as the Square Kilometre Array and the Next Generation Very Large Array.We demonstrate a 97%overall accuracy in distinguishing quasi-stellar objects,galaxies and stars using their optical morphologies plus their corresponding mid-infrared information by training and testing a convolutional neural network on Pan-STARRS imaging and WISE photometry.Compared with an expert-evaluated sample,we show that our approach has 95%accuracy at identifying the hosts of extended radio components.We also find that improving radio core localization,for instance by locating its geodesic center,could further increase the accuracy of locating the hosts of systems with a complex radio structure,such as C-shaped radio galaxies.The framework developed in this work can be used for analyzing data from future large-scale radio surveys.

Detecting HI Galaxies with Deep Neural Networks in the Presence of Radio Frequency Interference

In the neutral hydrogen(H I)galaxy survey,a significant challenge is to identify and extract the H I galaxy signal from the observational data contaminated by radio frequency interference(RFI).For a drift-scan survey,or more generally a survey of a spatially continuous region,in the time-ordered spectral data,the H I galaxies and RFI all appear as regions that extend an area in the time-frequency waterfall plot,so the extraction of the H I galaxies and RFI from such data can be regarded as an image segmentation problem,and machine-learning methods can be applied to solve such problems.In this study,we develop a method to effectively detect and extract signals of H I galaxies based on a Mask R-CNN network combined with the PointRend method.By simulating FAST-observed galaxy signals and potential RFI impact,we created a realistic data set for the training and testing of our neural network.We compared five different architectures and selected the best-performing one.This architecture successfully performs instance segmentation of H I galaxy signals in the RFI-contaminated time-ordered data,achieving a precision of 98.64%and a recall of 93.59%.

Pulsar Candidate Classification Using a Computer Vision Method from a Combination of Convolution and Attention

Artificial intelligence methods are indispensable to identifying pulsars from large amounts of candidates.We develop a new pulsar identification system that utilizes the CoAtNet to score two-dimensional features of candidates,implements a multilayer perceptron to score one-dimensional features,and relies on logistic regression to judge the corresponding scores.In the data preprocessing stage,we perform two feature fusions separately,one for one-dimensional features and the other for two-dimensional features,which are used as inputs for the multilayer perceptron and the CoAtNet respectively.The newly developed system achieves 98.77%recall,1.07%false positive rate(FPR)and 98.85%accuracy in our GPPS test set.