Machine Learning-based Identification of Contaminated Images in Light Curve Data Preprocessing

Attitude is one of the crucial parameters for space objects and plays a vital role in collision prediction and debris removal.Analyzing light curves to determine attitude is the most commonly used method.In photometric observations,outliers may exist in the obtained light curves due to various reasons.Therefore,preprocessing is required to remove these outliers to obtain high quality light curves.Through statistical analysis,the reasons leading to outliers can be categorized into two main types:first,the brightness of the object significantly increases due to the passage of a star nearby,referred to as“stellar contamination,”and second,the brightness markedly decreases due to cloudy cover,referred to as“cloudy contamination.”The traditional approach of manually inspecting images for contamination is time-consuming and labor-intensive.However,we propose the utilization of machine learning methods as a substitute.Convolutional Neural Networks and SVMs are employed to identify cases of stellar contamination and cloudy contamination,achieving F1 scores of 1.00 and 0.98 on a test set,respectively.We also explore other machine learning methods such as ResNet-18 and Light Gradient Boosting Machine,then conduct comparative analyses of the results.

Numerical simulation of direct shear tests on mechanical properties of talus deposits based on self-adaptive PCNN digital image processing

The macro mechanical properties of materials with characteristics of large scale and complicated structural composition can be analyzed through its reconstructed meso-structures.In this work,the meso-structures of talus deposits that widely exist in the hydro-power engineering in the southwest of China were first reconstructed by small particles according to the in-situ photographs based on the self-adaptive PCNN digital image processing,and then numerical direct shear tests were carried out for studying the mechanical properties of talus deposits.Results indicate that the reconstructed meso-structures of talus deposits are more consistent with the actual situation because the self-adaptive PCNN digital image processing has a higher discrimination in the details of soil-rock segmentation.The existence and random distribution of rock blocks make the initial shear stiffness,the peak strength and the residual strength higher than those of the "pure soil" with particle size less than 1.25 cm apparently,but reduce the displacements required for the talus deposits reaching its peak shear strength.The increase of rock proportion causes a significant improvement in the internal friction angle of talus deposit,which to a certain degree leads to the characteristics of shear stress-displacement curves having a changing trend from the plastic strain softening deformation to the nonlinear strain hardening deformation,while an unconspicuous increase in cohesion.The uncertainty and heterogeneity of rock distributions cause the differences of rock proportion within shear zone,leading to a relatively strong fluctuation in peak strengths during the shear process,while movement features of rock blocks,such as translation,rotation and crossing,expand the scope of shear zone,increase the required shear force,and also directly lead to the misjudgment that the lower shear strength is obtained from the samples with high rock proportion.That,however,just explains the reason why the shear strength gained from a small amount of indoor test data is not consistent with engineering practice.

An Example of Machine Vision Applied in Printing Quality Checking——Research on the Checking of Printing Quality by Image Processing

The traditional printing checking method always uses printing control strips,but the results are not very well in repeatability and stability. In this paper,the checking methods for printing quality basing on image are taken as research objects. On the base of the traditional checking methods of printing quality,combining the method and theory of digital image processing with printing theory in the new domain of image quality checking,it constitute the checking system of printing quality by image processing,and expound the theory design and the model of this system. This is an application of machine vision. It uses the high resolution industrial CCD(Charge Coupled Device) colorful camera. It can display the real-time photographs on the monitor,and input the video signal to the image gathering card,and then the image data transmits through the computer PCI bus to the memory. At the same time,the system carries on processing and data analysis. This method is proved by experiments. The experiments are mainly about the data conversion of image and ink limit show of printing.

Image Post-Processing Method for Visual Data Mining

Visual data mining is one of important approach of data mining techniques. Most of them are based on computer graphic techniques but few of them exploit image-processing techniques. This paper proposes an image processing method, named RNAM （resemble neighborhood averaging method）, to facilitate visual data mining, which is used to post-process the data mining result-image and help users to discover significant features and useful patterns effectively. The experiments show that the method is intuitive, easily-understanding and effectiveness. It provides a new approach for visual data mining.

Image Processing on Geological Data in Vector Format and Multi-Source Spatial Data Fusion

The geological data are constructed in vector format in geographical information system (GIS) while other data such as remote sensing images, geographical data and geochemical data are saved in raster ones. This paper converts the vector data into 8 bit images according to their importance to mineralization each by programming. We can communicate the geological meaning with the raster images by this method. The paper also fuses geographical data and geochemical data with the programmed strata data. The result shows that image fusion can express different intensities effectively and visualize the structure characters in 2 dimensions. Furthermore, it also can produce optimized information from multi-source data and express them more directly.

Evaluation of Two Absolute Radiometric Normalization Algorithms for Pre-processing of Landsat Imagery

In order to evaluate radiometric normalization techniques, two image normalization algorithms for absolute radiometric correction of Landsat imagery were quantitatively compared in this paper, which are the Illumination Correction Model proposed by Markham and Irish and the Illumination and Atmospheric Correction Model developed by the Remote Sensing and GIS Laboratory of the Utah State University. Relative noise, correlation coefficient and slope value were used as the criteria for the evaluation and comparison, which were derived from pseudo-invarlant features identified from multitemporal Landsat image pairs of Xiamen （厦门） and Fuzhou （福州） areas, both located in the eastern Fujian （福建） Province of China. Compared with the unnormalized image, the radiometric differences between the normalized multitemporal images were significantly reduced when the seasons of multitemporal images were different. However, there was no significant difference between the normalized and unnorrealized images with a similar seasonal condition. Furthermore, the correction results of two algorithms are similar when the images are relatively clear with a uniform atmospheric condition. Therefore, the radiometric normalization procedures should be carried out if the multitemporal images have a significant seasonal difference.

Structural Interpretation of Lineaments Uses Satellite Images Processing: A Case Study in North-Eastern Tunisia

This paper presents a lineament detection method using multi-band remote sensing images. The main objective of this work is to design an automatic image processing tool for lineament mapping from Landsat-7 ETM + satellite data. Five procedures were involved: 1) The Principal Component Analysis;2) image enhancement using histogram equalization technique 3) directional Sobel filters of the original data;4) histogram segmentation and 5) binary image generation. The applied methodology was contributed in identifying several known large-scale faults in the Northeast of Tunisia. The statistical and spatial analyses of lineament map indicate a difference of morphological appearance of lineaments in the satellite image. Indeed, all the lineaments present a specific organization. Five groups were classified based on three orientations: NE-SW, E-W and NW-SE. The overlapping of lineament map with the geologic map confirms that these lineaments of diverse directions can be identified and recognized on the field as a fault. The identified lineaments were linked to a deep faults caused by tectonic movements in Tunisia. This study shows the performance of the satellite image processing in the analysis and mapping of the accidents in the northern Atlas.

Hadoop and Its Role in Modern Image Processing

This paper introduces MapReduce as a distributed data processing model using open source Hadoop framework for manipulating large volume of data. The huge volume of data in the modern world, particularly multimedia data, creates new requirements for processing and storage. As an open source distributed computational framework, Hadoop allows for processing large amounts of images on an infinite set of computing nodes by providing necessary infrastructures. This paper introduces this framework, current works and its advantages and disadvantages.

How does the Behaviour of Dairy Cows during Recording Affect an Image Processing Based Calculation of the Udder Depth?

Precision Livestock Farming studies are based on data that was measured from animals via technical devices. In the means of automation, it is usually not accounted for the animals’ reaction towards the devices or individual animal behaviour during the gathering of sensor data. In this study, 14 Holstein-Friesian cows were recorded with a 2D video camera while walking through a scanning passage comprising six Microsoft Kinect 3D cameras. Elementary behavioural traits like how long the cows avoided the passage, the time they needed to walk through or the number of times they stopped walking were assessed from the video footage and analysed with respect to the target variable “udder depth” that was calculated from the recorded 3D data using an automated procedure. Ten repeated passages were recorded of each cow. During the repetitions, the cows adjusted individually (p < 0.001) to the recording situations. The averaged total time to complete a passage (p = 0.05) and the averaged number of stops (p = 0.07) depended on the lactation numbers of the cows. The measurement precision of target variable “udder depth” was affected by the time the cows avoided the recording (p = 0.06) and by the time it took them to walk through the scanning passage (p = 0.03). Effects of animal behaviour during the collection of sensor data can alter the results and should, thus, be considered in the development of sensor based devices.

An Advanced Image Processing Technique for Backscatter-Electron Data by Scanning Electron Microscopy for Microscale Rock Exploration

Backscatter electron analysis from scanning electron microscopes(BSE-SEM)produces high-resolution image data of both rock samples and thin-sections,showing detailed structural and geochemical(mineralogical)information.This allows an in-depth exploration of the rock microstructures and the coupled chemical characteristics in the BSE-SEM image to be made using image processing techniques.Although image processing is a powerful tool for revealing the more subtle data“hidden”in a picture,it is not a commonly employed method in geoscientific microstructural analysis.Here,we briefly introduce the general principles of image processing,and further discuss its application in studying rock microstructures using BSE-SEM image data.

Quantification by Signal to Noise Ratio of Active Infrared Thermography Data Processing Techniques

In this paper, the use of a signal to noise ratio (SNR) is proposed for the quantification of the goodness of some selected processing techniques of thermographic images, such as differentiated absolute contrast, skewness and kurtosis based algorithms, pulsed phase transform, principal component analysis and thermographic signal reconstruction. A new hybrid technique is also applied (PhAC—Phase absolute contrast), it combines three different processing techniques: phase absolute contrast, pulsed phase thermography and thermographic signal reconstruction. The quality of the results is established on the basis of the values of the parameter SNR, assessed for the present defects in the analyzed specimen, which enabled to quantify and compare their identification and the quality of the results of the employed technique.

DATA PROCESSING ON LINEARIZATION OF HOT-WIRE ANEMOMETER

A method of data processing to determine the coefficients of linearization equations for 1050 anemometer (produced by Thermo-Systems Inc. -TSI, USA) with the sensors made of domestic hot wire using the program preferred in this Paper is described. By calculation and test, it is indicated that the error resulting from this method is about 0. 5% of the full scale and less than TSl's. By using this method we can set up the calibration curve according to the measurement range and the diameter of the hot wire at a certain accuracy.

Fast distributed and parallel pre-processing on massive satellite data using grid computing

Distributed/parallel-processing system like sun grid engine(SGE) that utilizes multiple nodes/cores is proposed for the faster processing of large sized satellite image data. After verification, distributed process environment for pre-processing performance can be improved by up to 560.65% from single processing system. Through this, analysis performance in various fields can be improved, and moreover, near-real time service can be achieved in near future.

Lossless Compression Method for the Magnetic and Helioseismic Imager(MHI)Payload

The Solar Polar-orbit Observatory(SPO),proposed by Chinese scientists,is designed to observe the solar polar regions in an unprecedented way with a spacecraft traveling in a large solar inclination angle and a small ellipticity.However,one of the most significant challenges lies in ultra-long-distance data transmission,particularly for the Magnetic and Helioseismic Imager(MHI),which is the most important payload and generates the largest volume of data in SPO.In this paper,we propose a tailored lossless data compression method based on the measurement mode and characteristics of MHI data.The background out of the solar disk is removed to decrease the pixel number of an image under compression.Multiple predictive coding methods are combined to eliminate the redundancy utilizing the correlation(space,spectrum,and polarization)in data set,improving the compression ratio.Experimental results demonstrate that our method achieves an average compression ratio of 3.67.The compression time is also less than the general observation period.The method exhibits strong feasibility and can be easily adapted to MHI.

Big Data Testing Techniques:Taxonomy,Challenges and Future Trends

Big Data is reforming many industrial domains by providing decision support through analyzing large data volumes.Big Data testing aims to ensure that Big Data systems run smoothly and error-free while maintaining the performance and quality of data.However,because of the diversity and complexity of data,testing Big Data is challenging.Though numerous research efforts deal with Big Data testing,a comprehensive review to address testing techniques and challenges of BigData is not available as yet.Therefore,we have systematically reviewed the Big Data testing techniques’evidence occurring in the period 2010–2021.This paper discusses testing data processing by highlighting the techniques used in every processing phase.Furthermore,we discuss the challenges and future directions.Our findings show that diverse functional,non-functional and combined(functional and non-functional)testing techniques have been used to solve specific problems related to Big Data.At the same time,most of the testing challenges have been faced during the MapReduce validation phase.In addition,the combinatorial testing technique is one of the most applied techniques in combination with other techniques(i.e.,random testing,mutation testing,input space partitioning and equivalence testing)to find various functional faults through Big Data testing.

ACQUIREMENT AND COMPUTER PROCESSING OF 3-D SURFACE SHAPE DATA

The laser scanning and CCD image-transmitting measurement method and principle on acquiring 3-D curved surface shape data are discussed. Computer processing technique of 3-D curved surface shape(be called“ 3 - D surface shape”for short) data is analysed. This technique in- cludes these concrete methods and principles such as data smoothing, fitting, reconstructing ,elimi- nating and so on. The example and result about computer processing of 3- D surface shape data are given .

Processing and analysis of transient pressure measurements from permanent down-hole gauges

With permanent down-hole gauges （PDGs） widely installed in oilfields around the world in recent years, a continuous stream of transient pressure data in real time is now available, which motivates a new round of research interests in further developing pressure transient processing and analysis techniques. Transient pressure measurements from PDG are characterized by long term and high volume data. These data are recorded under unconstrained circumstances, so effects due to noise, rate fluctuation and interference from other wells cannot be avoided. These effects make the measured pressure trends decline or rise and then obscure or distort the actual flow behavior, which makes subsequent analysis difficult. In this paper, the problems encountered in analysis of PDG transient pressure are investigated. A newly developed workflow for processing and analyzing PDG transient pressure data is proposed. Numerical well testing synthetic studies are performed to demonstrate these procedures. The results prove that this new technique works well and the potential for practical application looks very promising.

MICROCOMPUTER PROCESSING THE PHOTOGRAPH OF TIMBER WITH HOLE KNOTS

A reliable method non-destructive X-ray test of wood defects with computer digital image processing technique was presented in the paper which made the knot-hole in wood clear and easily recognized. It was the first time to use maximtun entropy to determine the threshold of binary picture and thc result was satisfying. The improved Robert was used in the paper in the paper to outline the knot-hole so that the processing was almost on real time.

Detection of Alzheimer’s disease onset using MRI and PET neuroimaging:longitudinal data analysis and machine learning

The scientists are dedicated to studying the detection of Alzheimer’s disease onset to find a cure, or at the very least, medication that can slow the progression of the disease. This article explores the effectiveness of longitudinal data analysis, artificial intelligence, and machine learning approaches based on magnetic resonance imaging and positron emission tomography neuroimaging modalities for progression estimation and the detection of Alzheimer’s disease onset. The significance of feature extraction in highly complex neuroimaging data, identification of vulnerable brain regions, and the determination of the threshold values for plaques, tangles, and neurodegeneration of these regions will extensively be evaluated. Developing automated methods to improve the aforementioned research areas would enable specialists to determine the progression of the disease and find the link between the biomarkers and more accurate detection of Alzheimer’s disease onset.