Scientific data products and the data pre-processing subsystem of the Chang'e-3 mission

The Chang＇e-3 （CE-3） mission is China＇s first exploration mission on the surface of the Moon that uses a lander and a rover. Eight instruments that form the scientific payloads have the following objectives： （1） investigate the morphological features and geological structures at the landing site; （2） integrated in-situ analysis of minerals and chemical compositions; （3） integrated exploration of the structure of the lunar interior; （4） exploration of the lunar-terrestrial space environment, lunar sur- face environment and acquire Moon-based ultraviolet astronomical observations. The Ground Research and Application System （GRAS） is in charge of data acquisition and pre-processing, management of the payload in orbit, and managing the data products and their applications. The Data Pre-processing Subsystem （DPS） is a part of GRAS. The task of DPS is the pre-processing of raw data from the eight instruments that are part of CE-3, including channel processing, unpacking, package sorting, calibration and correction, identification of geographical location, calculation of probe azimuth angle, probe zenith angle, solar azimuth angle, and solar zenith angle and so on, and conducting quality checks. These processes produce Level 0, Level 1 and Level 2 data. The computing platform of this subsystem is comprised of a high-performance computing cluster, including a real-time subsystem used for processing Level 0 data and a post-time subsystem for generating Level 1 and Level 2 data. This paper de- scribes the CE-3 data pre-processing method, the data pre-processing subsystem, data classification, data validity and data products that are used for scientific studies.

High Speed Regular Expression Matching Engine with Fast Pre-Processing

Regular expression matching is playing an important role in deep inspection. The rapid development of SDN and NFV makes the network more dynamic, bringing serious challenges to traditional deep inspection matching engines. However, state-of-theart matching methods often require a significant amount of pre-processing time and hence are not suitable for this fast updating scenario. In this paper, a novel matching engine called BFA is proposed to achieve high-speed regular expression matching with fast pre-processing. Experiments demonstrate that BFA obtains 5 to 20 times more update abilities compared to existing regular expression matching methods, and scales well on multi-core platforms.

Pre-processing filter design at transmitters for IBI mitigation in an OFDM system

In order to meet the demands for high transmission rates and high service quality in broadband wireless communication systems, orthogonal frequency division multiplexing （OFDM） has been adopted in some standards. However, the inter-block interference （IBI） and inter-carrier interference （ICI） in an OFDM system affect the performance. To mitigate IBI and ICI, some pre-processing approaches have been proposed based on full channel state information （CSI）, which improved the system performance. A pre-processing filter based on partial CSI at the transmitter is designed and investigated. The filter coefficient is given by the optimization processing, the symbol error rate （SER） is tested, and the computation complexity of the proposed scheme is analyzed. Computer simulation results show that the proposed pre-processing filter can effectively mitigate IBI and ICI and the performance can be improved. Compared with pre-processing approaches at the transmitter based on full CSI, the proposed scheme has high spectral efficiency, limited CSI feedback and low computation complexity.

The Role of Combined OSR and SDF Method for Pre-Processing of Microarray Data that Accounts for Effective Denoising and Quantification

Microarray data is inherently noisy due to the noise contaminated from various sources during the preparation of microarray slide and thus it greatly affects the accuracy of the gene expression. How to eliminate the effect of the noise constitutes a challenging problem in microarray analysis. Efficient denoising is often a necessary and the first step to be taken before the image data is analyzed to compensate for data corruption and for effective utilization for these data. Hence preprocessing of microarray image is an essential to eliminate the background noise in order to enhance the image quality and effective quantification. Existing denoising techniques based on transformed domain have been utilized for microarray noise reduction with their own limitations. The objective of this paper is to introduce novel preprocessing techniques such as optimized spatial resolution (OSR) and spatial domain filtering (SDF) for reduction of noise from microarray data and reduction of error during quantification process for estimating the microarray spots accurately to determine expression level of genes. Besides combined optimized spatial resolution and spatial filtering is proposed and found improved denoising of microarray data with effective quantification of spots. The proposed method has been validated in microarray images of gene expression profiles of Myeloid Leukemia using Stanford Microarray Database with various quality measures such as signal to noise ratio, peak signal to noise ratio, image fidelity, structural content, absolute average difference and correlation quality. It was observed by quantitative analysis that the proposed technique is more efficient for denoising the microarray image which enables to make it suitable for effective quantification.

Pre-process algorithm for satellite laser ranging data based on curve recognition from points cloud

The satellite laser ranging （SLR） data quality from the COMPASS was analyzed, and the difference between curve recognition in computer vision and pre-process of SLR data finally proposed a new algorithm for SLR was discussed data based on curve recognition from points cloud is proposed. The results obtained by the new algorithm are 85 % （or even higher） consistent with that of the screen displaying method, furthermore, the new method can process SLR data automatically, which makes it possible to be used in the development of the COMPASS navigation system.

Pre-processing and post-processing method for geostress simulation using seismic interpretation results

In order to carry out numerical simulation using geologic structural data obtained from Landmark(seismic interpretation system), underground geological structures are abstracted into mechanical models which can reflect actual situations and facilitate their computation and analyses.Given the importance of model building, further processing methods about traditional seismic interpretation results from Landmark should be studied and the processed result can then be directly used in numerical simulation computations.Through this data conversion procedure, Landmark and FLAC(the international general stress software) are seamlessly connected.Thus, the format conversion between the two systems and the pre-and post-processing in simulation computation is realized.A practical application indicates that this method has many advantages such as simple operation, high accuracy of the element subdivision and high speed, which may definitely satisfy the actual needs of floor grid cutting.

Review of pre-processing technologies for ice cores

High-resolution ice core records covering long time spans enable reconstruction of the past climatic and environmental conditions allowing the investigation of the earth system's evolution. Preprocessing of ice cores has direct impacts on the data quality control for further analysis since the conventional ice core processing is time-consuming, produces qualitative data, leads to ice mass loss, and leads to risks of potential secondary pollution. However, over the past several decades, preprocessing of ice cores has received less attention than the improvement of ice drilling, the analytical methodology of various indices, and the researches on the climatic and environmental significance of ice core records. Therefore, this papers reviews the development of the processing for ice cores including framework, design as well as materials, analyzes the technical advantages and disadvantages of the different systems. In the past, continuous flowanalysis(CFA) has been successfully applied to process the polar ice cores. However, it is not suitable for ice cores outside polar region because of high level of particles, the memory effect between samples, and the filtration before injection. Ice core processing is a subtle and professional operation due to the fragility of the nonmetallic materials and the random distribution of particles and air bubbles in ice cores, which aggravates uncertainty in the measurements. The future developments of CFA are discussed in preprocessing, memory effect, challenge for brittle ice, coupling with real-time analysis and optimization of CFA in the field. Furthermore, non-polluting cutters with many different configurations could be designed to cut and scrape in multiple directions and to separate inner and outer portions of the core. This system also needs to be coupled with streamlined operation of packaging, coding, and stacking that can be implemented at high resolution and rate, avoiding manual intervention. At the same time, information of the longitudinal sections could be scanned andidentified, and then classified to obtain quantitative data. In addition, irregular ice volume and weight can also be obtained accurately. These improvements are recorded automatically via user-friendly interfaces. These innovations may be applied to other paleomedias with similar features and needs.

Intelligent Data Pre-processing Model in Integrated Ocean Observing Network System

There are a number of dirty data in observation data set derived from integrated ocean observing network system.Thus,the data must be carefully and reasonably processed before they are used for forecasting or analysis.This paper proposes a data pre-processing model based on intelligent algorithms.Firstly,we introduce the integrated network platform of ocean observation.Next,the pre-processing model of data is presented,and an intelligent cleaning model of data is proposed.Based on fuzzy clustering,the Kohonen clustering network is improved to fulfill the parallel calculation of fuzzy c-means clustering.The proposed dynamic algorithm can automatically find the new clustering center with the updated sample data.The rapid and dynamic performance of the model makes it suitable for real time calculation,and the efficiency and accuracy of the model is proved by test results through observation data analysis.

Image pre-processing research of coal level in underground coal pocket

Mathematical morphology is widely applicated in digital image procesing.Vari- ary morphology construction and algorithm being developed are used in deferent digital image processing.The basic idea of mathematical morphology is to use construction ele- ment measure image morphology for solving understand problem.The article presented advanced cellular neural network that forms mathematical morphological cellular neural network (MMCNN) equation to be suit for mathematical morphology filter.It gave the theo- ries of MMCNN dynamic extent and stable state.It is evidenced that arrived mathematical morphology filter through steady of dynamic process in definite condition.

Research of Image pre-processing Algorithm in Automation Fingerprint Identification

In the paper, a valid method of fingerprint Image pre- processing is introduced. Experiment results show that this kind of algorithm can availably wipe off yawp imported by the incom- plete leave fingerprint - marking of sensor surface when finger- print sensor record fingerprint. Meanwhile, it can extract the ef- fective and uneffective zone of fingerprint effectively, and also further enhance ridge line and vale line of fingerprint so that make the lines of fingerprint clear, continuum, lubricity and has better contrast, at the same time, has quite quick speed, this fingerprint Image pre- processing time can be shorten greatly.

Concurrent multi-task pre-processing method for LEO mega-constellation based on dynamic spatio-temporal grids

The Low Earth Orbit(LEO)remote sensing satellite mega-constellation has the characteristics of large quantity and various types which make it have unique superiority in the realization of concurrent multiple tasks.However,the complexity of resource allocation is increased because of the large number of tasks and satellites.Therefore,the primary problem of implementing concurrent multiple tasks via LEO mega-constellation is to pre-process tasks and observation re-sources.To address the challenge,we propose a pre-processing algorithm for the mega-constellation based on highly Dynamic Spatio-Temporal Grids(DSTG).In the first stage,this paper describes the management model of mega-constellation and the multiple tasks.Then,the coding method of DSTG is proposed,based on which the description of complex mega-constellation observation resources is realized.In the third part,the DSTG algorithm is used to realize the processing of concurrent multiple tasks at multiple levels,such as task space attribute,time attribute and grid task importance evaluation.Finally,the simulation result of the proposed method in the case of constellation has been given to verify the effectiveness of concurrent multi-task pre-processing based on DSTG.The autonomous processing process of task decomposition and task fusion and mapping to grids,and the convenient indexing process of time window are verified.

数值预报中气象卫星资料同化前处理技术进展

在数值天气预报变分同化中,利用同化前处理将卫星资料完成有效信息优选、资料拼接和稀疏化、初级通道选择、下边界参数耦合等处理,实现卫星资料同化对数值天气预报业务的正贡献,是决定海量卫星资料同化效率、质量和效果的重要环节。针对多种格式的卫星资料,中国气象局研发标准格式的高时效卫星资料拼接等技术,有效减小整轨卫星资料时间滞后对数值天气预报业务的负面影响。对于风云气象卫星资料,将云和降水检测、资料质量分析等处理置于同化前处理中,实现多光谱资料融合的同化预质量控制,保证了风云卫星微波温度探测资料和红外高光谱资料的同化正贡献。利用统一资料格式对预处理卫星资料进行再处理,拓展针对卫星成像和主动探测资料的处理,将卫星资料同化的部分质量控制功能置于卫星资料同化前处理中,是风云卫星资料同化前处理技术发展的重要趋势。

白酒中风味成分的检测分析技术研究进展

白酒历史悠久,是中国的国酒。白酒中的风味成分虽然含量较低,但其类别、组成及比例对产品口感、风味及香气极为重要。近年来,对风味成分的研究已成为白酒领域的热点,本文综述了白酒风味成分检测分析方面的研究进展,介绍了风味成分检测前处理技术与分析方法,总结了各大香型白酒中的风味挖掘及分析的最新成果,探讨了研究中存在的局限与不足,并展望了未来技术发展的方向和趋势。

采用自主研发的在线前处理系统同时分析脱硫废水中阴阳离子

电厂专用的全自动脱硫废水分析仪可以实现对超高浓度脱硫废水的自动化前处理以及分析,样品通过自动二步稀释,并与双系统离子色谱仪联用,对脱硫废水中的阴、阳离子同时进行检测。脱硫废水中阳离子主要有Na+、NH4+、K+、Mg2+、Ca2+,阴离子主要有F-、Cl-、NO3-、SO42-。本研究对仪器相关参数进行了验证,测试结果为,4种阴离子和5种阳离子的混合标准溶液在其线性关系内相关系数均大于0.999;4种阴离子的峰高和峰面积重复性RSD小于2.29%,5种阳离子的峰高和峰面积重复性RSD小于1.78%,实际样品中4种加标阴离子回收率在93.5%~100.2%之间;5种阳离子加标回收率在95.4%~103.8%之间。结果证明,该系统大大提高了分析效率,节省了人工成本,填补了市场空白。

Review of Recent Trends in the Hybridisation of Preprocessing-Based and Parameter Optimisation-Based Hybrid Models to Forecast Univariate Streamflow

Forecasting river flow is crucial for optimal planning,management,and sustainability using freshwater resources.Many machine learning(ML)approaches have been enhanced to improve streamflow prediction.Hybrid techniques have been viewed as a viable method for enhancing the accuracy of univariate streamflow estimation when compared to standalone approaches.Current researchers have also emphasised using hybrid models to improve forecast accuracy.Accordingly,this paper conducts an updated literature review of applications of hybrid models in estimating streamflow over the last five years,summarising data preprocessing,univariate machine learning modelling strategy,advantages and disadvantages of standalone ML techniques,hybrid models,and performance metrics.This study focuses on two types of hybrid models:parameter optimisation-based hybrid models(OBH)and hybridisation of parameter optimisation-based and preprocessing-based hybridmodels(HOPH).Overall,this research supports the idea thatmeta-heuristic approaches precisely improveML techniques.It’s also one of the first efforts to comprehensively examine the efficiency of various meta-heuristic approaches(classified into four primary classes)hybridised with ML techniques.This study revealed that previous research applied swarm,evolutionary,physics,and hybrid metaheuristics with 77%,61%,12%,and 12%,respectively.Finally,there is still room for improving OBH and HOPH models by examining different data pre-processing techniques and metaheuristic algorithms.

Product quality prediction based on RBF optimized by firefly algorithm

With the development of information technology,a large number of product quality data in the entire manufacturing process is accumulated,but it is not explored and used effectively.The traditional product quality prediction models have many disadvantages,such as high complexity and low accuracy.To overcome the above problems,we propose an optimized data equalization method to pre-process dataset and design a simple but effective product quality prediction model:radial basis function model optimized by the firefly algorithm with Levy flight mechanism(RBFFALM).First,the new data equalization method is introduced to pre-process the dataset,which reduces the dimension of the data,removes redundant features,and improves the data distribution.Then the RBFFALFM is used to predict product quality.Comprehensive expe riments conducted on real-world product quality datasets validate that the new model RBFFALFM combining with the new data pre-processing method outperforms other previous me thods on predicting product quality.

Machine-Learning Based Packet Switching Method for Providing Stable High-Quality Video Streaming in Multi-Stream Transmission

Broadcasting gateway equipment generally uses a method of simply switching to a spare input stream when a failure occurs in a main input stream.However,when the transmission environment is unstable,problems such as reduction in the lifespan of equipment due to frequent switching and interruption,delay,and stoppage of services may occur.Therefore,applying a machine learning(ML)method,which is possible to automatically judge and classify network-related service anomaly,and switch multi-input signals without dropping or changing signals by predicting or quickly determining the time of error occurrence for smooth stream switching when there are problems such as transmission errors,is required.In this paper,we propose an intelligent packet switching method based on the ML method of classification,which is one of the supervised learning methods,that presents the risk level of abnormal multi-stream occurring in broadcasting gateway equipment based on data.Furthermore,we subdivide the risk levels obtained from classification techniques into probabilities and then derive vectorized representative values for each attribute value of the collected input data and continuously update them.The obtained reference vector value is used for switching judgment through the cosine similarity value between input data obtained when a dangerous situation occurs.In the broadcasting gateway equipment to which the proposed method is applied,it is possible to perform more stable and smarter switching than before by solving problems of reliability and broadcasting accidents of the equipment and can maintain stable video streaming as well.

Design of a Multi-Stage Ensemble Model for Thyroid Prediction Using Learning Approaches

This research concentrates to model an efficient thyroid prediction approach,which is considered a baseline for significant problems faced by the women community.The major research problem is the lack of automated model to attain earlier prediction.Some existing model fails to give better prediction accuracy.Here,a novel clinical decision support system is framed to make the proper decision during a time of complexity.Multiple stages are followed in the proposed framework,which plays a substantial role in thyroid prediction.These steps include i)data acquisition,ii)outlier prediction,and iii)multi-stage weight-based ensemble learning process(MS-WEL).The weighted analysis of the base classifier and other classifier models helps bridge the gap encountered in one single classifier model.Various classifiers aremerged to handle the issues identified in others and intend to enhance the prediction rate.The proposed model provides superior outcomes and gives good quality prediction rate.The simulation is done in the MATLAB 2020a environment and establishes a better trade-off than various existing approaches.The model gives a prediction accuracy of 97.28%accuracy compared to other models and shows a better trade than others.