A Novel Efficient and Effective Preprocessing Algorithm for Text Classification

Text classification is an essential task of natural language processing. Preprocessing, which determines the representation of text features, is one of the key steps of text classification architecture. It proposed a novel efficient and effective preprocessing algorithm with three methods for text classification combining the Orthogonal Matching Pursuit algorithm to perform the classification. The main idea of the novel preprocessing strategy is that it combined stopword removal and/or regular filtering with tokenization and lowercase conversion, which can effectively reduce the feature dimension and improve the text feature matrix quality. Simulation tests on the 20 newsgroups dataset show that compared with the existing state-of-the-art method, the new method reduces the number of features by 19.85%, 34.35%, 26.25% and 38.67%, improves accuracy by 7.36%, 8.8%, 5.71% and 7.73%, and increases the speed of text classification by 17.38%, 25.64%, 23.76% and 33.38% on the four data, respectively.

Optimizing Facial Expression Recognition through Effective Preprocessing Techniques

Analyzing human facial expressions using machine vision systems is indeed a challenging yet fascinating problem in the field of computer vision and artificial intelligence. Facial expressions are a primary means through which humans convey emotions, making their automated recognition valuable for various applications including man-computer interaction, affective computing, and psychological research. Pre-processing techniques are applied to every image with the aim of standardizing the images. Frequently used techniques include scaling, blurring, rotating, altering the contour of the image, changing the color to grayscale and normalization. Followed by feature extraction and then the traditional classifiers are applied to infer facial expressions. Increasing the performance of the system is difficult in the typical machine learning approach because feature extraction and classification phases are separate. But in Deep Neural Networks (DNN), the two phases are combined into a single phase. Therefore, the Convolutional Neural Network (CNN) models give better accuracy in Facial Expression Recognition than the traditional classifiers. But still the performance of CNN is hampered by noisy and deviated images in the dataset. This work utilized the preprocessing methods such as resizing, gray-scale conversion and normalization. Also, this research work is motivated by these drawbacks to study the use of image pre-processing techniques to enhance the performance of deep learning methods to implement facial expression recognition. Also, this research aims to recognize emotions using deep learning and show the influences of data pre-processing for further processing of images. The accuracy of each pre-processing methods is compared, then combination between them is analysed and the appropriate preprocessing techniques are identified and implemented to see the variability of accuracies in predicting facial expressions. .

PREPROCESSING AND POSTPROCESSING SYSTEM FOR FINITE ELEMENT COMPUTATION

研究了有限元计算中的单元自动划分、曲线处理、节点疏密处理、载荷自动分布、与优化设计的衔接和等应力线绘制等方面的技术问题，阐述了有限元平面八节点单元前后置处理软件的设计过程。

Data preprocessing and preliminary results of the Moon-based Ultraviolet Telescope on the CE-3 lander

The Moon-based Ultraviolet Telescope （MUVT） is one of the payloads on the Chang＇e-3 （CE-3） lunar lander. Because of the advantages of having no at- mospheric disturbances and the slow rotation of the Moon, we can make long-term continuous observations of a series of important celestial objects in the near ultra- violet band （245-340 nm）, and perform a sky survey of selected areas, which can- not be completed on Earth. We can find characteristic changes in celestial brightness with time by analyzing image data from the MUVT, and deduce the radiation mech- anism and physical properties of these celestial objects after comparing with a phys- ical model. In order to explain the scientific purposes of MUVT, this article analyzes the preprocessing of MUVT image data and makes a preliminary evaluation of data quality. The results demonstrate that the methods used for data collection and prepro- cessing are effective, and the Level 2A and 2B image data satisfy the requirements of follow-up scientific researches.

Diabetes Type 2: Poincaré Data Preprocessing for Quantum Machine Learning

Quantum Machine Learning(QML)techniques have been recently attracting massive interest.However reported applications usually employ synthetic or well-known datasets.One of these techniques based on using a hybrid approach combining quantum and classic devices is the Variational Quantum Classifier(VQC),which development seems promising.Albeit being largely studied,VQC implementations for“real-world”datasets are still challenging on Noisy Intermediate Scale Quantum devices(NISQ).In this paper we propose a preprocessing pipeline based on Stokes parameters for data mapping.This pipeline enhances the prediction rates when applying VQC techniques,improving the feasibility of solving classification problems using NISQ devices.By including feature selection techniques and geometrical transformations,enhanced quantum state preparation is achieved.Also,a representation based on the Stokes parameters in the PoincaréSphere is possible for visualizing the data.Our results show that by using the proposed techniques we improve the classification score for the incidence of acute comorbid diseases in Type 2 Diabetes Mellitus patients.We used the implemented version of VQC available on IBM’s framework Qiskit,and obtained with two and three qubits an accuracy of 70%and 72%respectively.

Adaptive preprocessing algorithms of corneal topography in polar coordinate system

New adaptive preprocessing algorithms based on the polar coordinate system were put forward to get high-precision corneal topography calculation results. Adaptive locating algorithms of concentric circle center were created to accurately capture the circle center of original Placido-based image, expand the image into matrix centered around the circle center, and convert the matrix into the polar coordinate system with the circle center as pole. Adaptive image smoothing treatment was followed and the characteristics of useful circles were extracted via horizontal edge detection, based on useful circles presenting approximate horizontal lines while noise signals presenting vertical lines or different angles. Effective combination of different operators of morphology were designed to remedy data loss caused by noise disturbances, get complete image about circle edge detection to satisfy the requests of precise calculation on follow-up parameters. The experimental data show that the algorithms meet the requirements of practical detection with characteristics of less data loss, higher data accuracy and easier availability.

AN LBP-BASED MULTI-SCALE ILLUMINATION PREPROCESSING METHOD FOR FACE RECOGNITION

It is one of the major challenges for face recognition to minimize the disadvantage of il- lumination variations of face images in different scenarios. Local Binary Pattern (LBP) has been proved to be successful for face recognition. However, it is still very rare to take LBP as an illumination preprocessing approach. In this paper, we propose a new LBP-based multi-scale illumination pre- processing method. This method mainly includes three aspects: threshold adjustment, multi-scale addition and symmetry restoration/neighborhood replacement. Our experiment results show that the proposed method performs better than the existing LBP-based methods at the point of illumination preprocessing. Moreover, compared with some face image preprocessing methods, such as histogram equalization, Gamma transformation, Retinex, and simplified LBP operator, our method can effectively improve the robustness for face recognition against illumination variation, and achieve higher recog- nition rate.

DATA PREPROCESSING AND RE KERNEL CLUSTERING FOR LETTER

Many classifiers and methods are proposed to deal with letter recognition problem. Among them, clustering is a widely used method. But only one time for clustering is not adequately. Here, we adopt data preprocessing and a re kernel clustering method to tackle the letter recognition problem. In order to validate effectiveness and efficiency of proposed method, we introduce re kernel clustering into Kernel Nearest Neighbor classification(KNN), Radial Basis Function Neural Network(RBFNN), and Support Vector Machine(SVM). Furthermore, we compare the difference between re kernel clustering and one time kernel clustering which is denoted as kernel clustering for short. Experimental results validate that re kernel clustering forms fewer and more feasible kernels and attain higher classification accuracy.

D-IMPACT: A Data Preprocessing Algorithm to Improve the Performance of Clustering

In this study, we propose a data preprocessing algorithm called D-IMPACT inspired by the IMPACT clustering algorithm. D-IMPACT iteratively moves data points based on attraction and density to detect and remove noise and outliers, and separate clusters. Our experimental results on two-dimensional datasets and practical datasets show that this algorithm can produce new datasets such that the performance of the clustering algorithm is improved.

Adaptive Gradient-Based and Anisotropic Diffusion Equation Filtering Algorithm for Microscopic Image Preprocessing

In image acquisition process, the quality of microscopic images will be degraded by electrical noise, quantizing noise, light illumination etc. Hence, image preprocessing is necessary and important to improve the quality. The background noise and pulse noise are two common types of noise existing in microscopic images. In this paper, a gradient-based anisotropic filtering algorithm was proposed, which can filter out the background noise while preserve object boundary effectively. The filtering performance was evaluated by comparing that with some other filtering algorithms.

Preprocessing Model of Manuscripts in Javanese Characters

Manuscript preprocessing is the earliest stage in transliteration process of manuscripts in Javanese scripts. Manuscript preprocessing stage is aimed to produce images of letters which form the manuscripts to be processed further in manuscript transliteration system. There are four main steps in manuscript preprocessing, which are manuscript binarization, noise reduction, line segmentation, and character segmentation for every line image produced by line segmentation. The result of the test on parts of PB.A57 manuscript which contains 291 character images, with 95% level of confidence concluded that the success percentage of preprocessing in producing Javanese character images ranged 85.9% - 94.82%.

Power Data Preprocessing Method of Mountain Wind Farm Based on POT-DBSCAN

Due to the frequent changes of wind speed and wind direction,the accuracy of wind turbine(WT)power prediction using traditional data preprocessing method is low.This paper proposes a data preprocessing method which combines POT with DBSCAN(POT-DBSCAN)to improve the prediction efficiency of wind power prediction model.Firstly,according to the data of WT in the normal operation condition,the power prediction model ofWT is established based on the Particle Swarm Optimization(PSO)Arithmetic which is combined with the BP Neural Network(PSO-BP).Secondly,the wind-power data obtained from the supervisory control and data acquisition(SCADA)system is preprocessed by the POT-DBSCAN method.Then,the power prediction of the preprocessed data is carried out by PSO-BP model.Finally,the necessity of preprocessing is verified by the indexes.This case analysis shows that the prediction result of POT-DBSCAN preprocessing is better than that of the Quartile method.Therefore,the accuracy of data and prediction model can be improved by using this method.

Image Preprocessing Methods to Identify Micro-cracks of Road Pavement

Standards of highway conservation and maintenance are improved gradually following the improvement of requirements of road service. Before obvious damage such as obvious cracking (block,transverse, longitudinal ) and rutting emerge, inconspicuous distress (micro-cracks, polishing, pockmarked) is generated previously. These inconspicuous distresses may provide basis and criteria for pavement preventive maintenance. Currently most of preventive conservation measures are determined by experienced experts in maintenance and repair of road after site visits. Thus method is difficult in operation, and has a certain amount of instability as it is based on experience and personal knowledge. In this paper, camera and laser were used for automated high-speed acquisition images. Methods to preprocess pavement image are compared. The pretreatment method suitable for analyze micro-cracks picture is elected, an effective way to remove shadow is also proposed.

An adaptive preprocessing algorithm for low bitrate video coding

At low bitrate, all block discrete cosine transform (BDCT) based video coding algorithms suffer from visible blocking and ringing artifacts in the reconstructed images because the quantization is too coarse and high frequency DCT coefficients are inclined to be quantized to zeros. Preprocessing algorithms can enhance coding efficiency and thus reduce the likelihood of blocking artifacts and ringing artifacts generated in the video coding process by applying a low-pass filter before video encoding to remove some relatively insignificant high frequent components. In this paper, we introduce a new adaptive preprocessing algo- rithm, which employs an improved bilateral filter to provide adaptive edge-preserving low-pass filtering which is adjusted ac- cording to the quantization parameters. Whether at low or high bit rate, the preprocessing can provide proper filtering to make the video encoder more efficient and have better reconstructed image quality. Experimental results demonstrate that our proposed preprocessing algorithm can significantly improve both subjective and objective quality.

Parallelized LEDAPS method for Remote Sensing Preprocessing Based on MPI

Based on Landsat image,the Landsat Ecosystem Disturbance Adaptive Processing System(LEDAPS)uses radiation change detection method for image processing and offers the surface reflectivity products for ecosystem carbon sequestration and carbon reserves.As the accumulation of massive remote sensing data especially for the Landsat image,the traditional serial LEDAPS for image processing has a long cycle that make a lot of difficulties in practical application.For this problem,this paper design a high performance parallel LEDAPS processing method based on MPI.The results not only aimed to improve the calculation speed and save computing time,but also considered the load balance between the flexibly extended computing nodes.Results show that the highest speed ratio of parallelized LEDAPS reached 7.37 when the number of MPI process is 8.It effectively improves the ability of LEDAPS to handle massive remote sensing data and reduces the forest carbon stocks calculation cycle by using the remote sensing images.

基于轻量化PointNet网络的林果园喷雾作业靶标实时识别方法

为了进一步提高喷雾机器人靶标检测的精准性、实时性和应用部署的实用性,该研究提出一种基于轻量化PointNet网络的林果园喷雾作业靶标实时识别方法。首先通过区域提取降采样、地面分割和改进DBSCAN聚类等点云预处理方法提取原始点云中的靶标;然后通过移动最小二乘上采样将靶标点云转化为满足点云识别网络输入要求的点云数据;最终通过在PointNet网络中引入残差模块和改进循环剪枝算法轻量化PointNet网络,完成林果树靶标的实时识别。试验结果表明,在ModelNet40数据集上,轻量化PointNet网络可达89.7%的准确率;在实际苗圃环境的试验中,该研究方法对靶标的识别准确率可达92.49%,同时误识率与拒识率分别为13.4%和6.47%,相较PointNet网络识别准确率提升了4.38个百分点,误识率和拒识率分别降低了7.2和4.07个百分点;轻量化PointNet网络识别准确率仅比PointNet++网络低1.14个百分点,误识率和拒识率分别高了0.9和1.12个百分点。但是轻量化PointNet网络的模型参数量较PointNet网络和PointNet++网络的模型参数量显著减少,仅为PointNet网络的11.5%,PointNet++网络的27.02%;运算量相较PointNet网络、PointNet++网络分别减少13.3和76.79个百分点。该研究提出的轻量化PointNet网络具有较高的实时性、精确性和鲁棒性,能够满足林果园喷雾作业的靶标识别需求,可为林果园喷雾作业靶标实时识别提供参考。

工业机械臂辅助机器视觉的工件识别与定位

传统的工业机器人系统由于缺乏感知判断能力,很难适应目标种类和位置的变化,而现代工业需要生产不同种类和规格的产品。为此,以多品种小批量生产的零件为背景提出了一种机械臂辅助机器视觉系统,该系统能够实现对工件的图像采集并进行预处理,通过改进的Canny轮廓的提取算法与基于Hausdorff距离的模版匹配完成对不同种类工件进行识别和定位。最后经实验的验证该系统的定位精度在0.35 mm内,基本满足了在实际生产中对小批量生产、不同形状且没有固定夹具的目标工件的抓取、搬运与检测等要求。

基于知识图谱的番茄种植管理可视化查询

为提高获取番茄种植管理知识的速度与准确率,研究了以图形式描述番茄在不同环境的种植管理,并基于知识图谱构建了番茄种植管理可视化查询系统。该方法利用“自顶向下”和“自底向上”的模块化CREATE解决了Neo4j的缓慢和准确率问题,并利用PyQt框架构建可视化查询界面,通过问题预处理和语义相似度计算输出最合适的番茄种植管理知识。试验结果表明:该方法的平均响应时间和平均准确率比Cypher查询语言分别提高88.33%及1.97%,可操性比Cypher语言友好。研究结果可以在不同环境下为番茄生产管理提供高质量的种植管理建议。

基于改进Res-UNet网络的织物瑕疵图像识别方法

复杂花色织物的纹理和色彩常常是非规则的,导致织物表面瑕疵识别难度较高。针对上述问题,研究一种基于改进Res-UNet网络的织物表面瑕疵图像识别方法。采集织物图像并对其实施灰度化、去噪以及直方图均衡化处理,利用蝙蝠算法求取最佳提取网络层数,通过增加特征提取网络层数改进Res-UNet网络,利用改进后的Res-UNet网络识别织物表面瑕疵,并且采用迁移学习算法进一步优化识别模型的参数,实现织物表面瑕疵准确识别。结果表明:本文方法应用下,无论是素色样本,还是花色样本,其识别系数均达到0.9以上,相比基于标签嵌入方法的织物瑕疵识别方法和双路高分辨率转换网络的布匹瑕疵检测方法,本文方法对复杂花色样本的轮廓系数识别更高,适用性更好,识别能力更强。

基于嵌入式设备的低功耗工业读码器系统设计

针对目前国内读码器产品种类稀少、成本过高,功耗和热量过高、识别率低等问题,提出了一种结构紧凑、性价比高、功耗低、稳定性强的智能读码器系统设计方案。该系统方案主要基于ARM平台和Linux系统开发,硬件方面采用NXP MX8Mini作为主控芯片,搭载XILINX的XC7A50T FPGA器件,配置2G容量的LDDR4内存用来做图像预处理工作,采用AR0144CS芯片作为图像传感器,并支持多种通讯协议和扩展。软件方面,主要介绍了系统FPGA图像预处理及高斯滤波具体实现、ARM与FPGA的通讯、上位机功能设计。经过测试,该读码器系统运行稳定、体积小、功耗低,一维码8mil Code128识别率达到99.99%,为当前国产读码器市场提供了一种新的解决方案。