Data processing of small samples based on grey distance information approach

Data processing of small samples is an important and valuable research problem in the electronic equipment test. Because it is difficult and complex to determine the probability distribution of small samples, it is difficult to use the traditional probability theory to process the samples and assess the degree of uncertainty. Using the grey relational theory and the norm theory, the grey distance information approach, which is based on the grey distance information quantity of a sample and the average grey distance information quantity of the samples, is proposed in this article. The definitions of the grey distance information quantity of a sample and the average grey distance information quantity of the samples, with their characteristics and algorithms, are introduced. The correlative problems, including the algorithm of estimated value, the standard deviation, and the acceptance and rejection criteria of the samples and estimated results, are also proposed. Moreover, the information whitening ratio is introduced to select the weight algorithm and to compare the different samples. Several examples are given to demonstrate the application of the proposed approach. The examples show that the proposed approach, which has no demand for the probability distribution of small samples, is feasible and effective.

Data partitioning based on sampling for power load streams

A novel data streams partitioning method is proposed to resolve problems of range-aggregation continuous queries over parallel streams for power industry.The first step of this method is to parallel sample the data,which is implemented as an extended reservoir-sampling algorithm.A skip factor based on the change ratio of data-values is introduced to describe the distribution characteristics of data-values adaptively.The second step of this method is to partition the fluxes of data streams averagely,which is implemented with two alternative equal-depth histogram generating algorithms that fit the different cases：one for incremental maintenance based on heuristics and the other for periodical updates to generate an approximate partition vector.The experimental results on actual data prove that the method is efficient,practical and suitable for time-varying data streams processing.

Characteristics analysis on high density spatial sampling seismic data

China＇s continental deposition basins are characterized by complex geological structures and various reservoir lithologies. Therefore, high precision exploration methods are needed. High density spatial sampling is a new technology to increase the accuracy of seismic exploration. We briefly discuss point source and receiver technology, analyze the high density spatial sampling in situ method, introduce the symmetric sampling principles presented by Gijs J. O. Vermeer, and discuss high density spatial sampling technology from the point of view of wave field continuity. We emphasize the analysis of the high density spatial sampling characteristics, including the high density first break advantages for investigation of near surface structure, improving static correction precision, the use of dense receiver spacing at short offsets to increase the effective coverage at shallow depth, and the accuracy of reflection imaging. Coherent noise is not aliased and the noise analysis precision and suppression increases as a result. High density spatial sampling enhances wave field continuity and the accuracy of various mathematical transforms, which benefits wave field separation. Finally, we point out that the difficult part of high density spatial sampling technology is the data processing. More research needs to be done on the methods of analyzing and processing huge amounts of seismic data.

A real-time AI-assisted seismic monitoring system based on new nodal stations with 4G telemetry and its application in the Yangbi M_(S) 6.4 aftershock monitoring in southwest China

A rapidly deployable dense seismic monitoring system which is capable of transmitting acquired data in real time and analyzing data automatically is crucial in seismic hazard mitigation after a major earthquake.However,it is rather difficult for current seismic nodal stations to transmit data in real time for an extended period of time,and it usually takes a great amount of time to process the acquired data manually.To monitor earthquakes in real time flexibly,we develop a mobile integrated seismic monitoring system consisting of newly developed nodal units with 4G telemetry and a real-time AI-assisted automatic data processing workflow.The integrated system is convenient for deployment and has been successfully applied in monitoring the aftershocks of the Yangbi M_(S) 6.4 earthquake occurred on May 21,2021 in Yangbi County,Dali,Yunnan in southwest China.The acquired seismic data are transmitted almost in real time through the 4G cellular network,and then processed automat-ically for event detection,positioning,magnitude calculation and source mechanism inversion.From tens of seconds to a couple of minutes at most,the final seismic attributes can be presented remotely to the end users through the integrated system.From May 27 to June 17,the real-time system has detected and located 7905 aftershocks in the Yangbi area before the internal batteries exhausted,far more than the catalog provided by China Earthquake Networks Center using the regional permanent stations.The initial application of this inte-grated real-time monitoring system is promising,and we anticipate the advent of a new era for Real-time Intelligent Array Seismology(RIAS),for better monitoring and understanding the subsurface dynamic pro-cesses caused by Earth's internal forces as well as anthropogenic activities.

Patient Centered Real-Time Mobile Health Monitoring System

In this paper, we introduce a system architecture for a patient centered mobile health monitoring (PCMHM) system that deploys different sensors to determine patients’ activities, medical conditions, and the cause of an emergency event. This system combines and analyzes sensor data to produce the patients’ detailed health information in real-time. A central computational node with data analyzing capability is used for sensor data integration and analysis. In addition to medical sensors, surrounding environmental sensors are also utilized to enhance the interpretation of the data and to improve medical diagnosis. The PCMHM system has the ability to provide on-demand health information of patients via the Internet, track real-time daily activities and patients’ health condition. This system also includes the capability for assessing patients’ posture and fall detection.

基于傅里叶变换与近端采样的序列推荐算法

传统推荐算法比较注重于模型本身对于推荐效果的提升,实际上数据质量对于算法的影响更为重要,但目前在推荐算法领域对于数据的科学处理方法比较零散,没有形成一个系统的框架。针对以上问题,基于傅里叶变换与近端序列采样的数据预处理,结合SASRec提出可以普遍应用的序列推荐框架FTRRec。首先通过傅里叶变换将序列数据在时域和频域中相互转换,并根据序列数据的特点,过滤序列中的无用信息,其次使用近端序列采样替换传统的滑动窗口采样法,加速样本采样的同时,提升模型对于序列的特征捕获能力。通过在5个公开数据集上的实验,将框架应用于三个不同的主流推荐算法时,每种模型均有3%-5%的提升。

THE DESIGN AND DATA PROCESSING OF THE SAMPLING SURVEY OF CHILDREN'S SITUATION IN CHINA, 1987

In July of 1987, the Sampling Survey of Children's Situation was conducted in 9 provincesautonomous regions of China. A stratified two--stage cluster sampling plan was designed for thesurvey. The paper presents the methods of stratification, selecting n=2 PSU's (cities/counties) withunequal probabilities without replacement in each stratum and selecting residents/village committeein each sampled city/county. All formulae of estimating population characteristics (especiallypopulation totals and the ratios of two totals), and estimating variances of those estimators aregiven. Finally, we analyse the precision of the survey preliminarily from the result of dataprocessing.

面向复杂工业过程的虚拟样本生成综述

用于复杂工业过程难测运行指标和异常故障建模的样本具有量少稀缺、分布不平衡以及内涵机理知识匮乏等特性.虚拟样本生成(Virtual sample generation,VSG)作为扩充建模样本数量及其涵盖空间的技术,已成为解决上述问题的主要手段之一,但已有研究还存在缺乏理论支撑、分类准则与应用边界模糊等问题.本文在描述复杂工业过程难测运行指标和异常故障建模所存在问题的基础上,梳理虚拟样本定义及其内涵,给出面向工业过程回归与分类问题的VSG实现流程;接着,从样本覆盖区域、实现流程与推广应用等方向进行综述;然后,分析讨论VSG的下一步研究方向;最后,对全文进行总结并给出未来挑战.

健壮且自适应的学习型近似查询处理方法研究

由于在大规模数据集上执行精确查询耗时较长,因此近似查询处理(AQP)技术常被用于在线分析处理,目的是以较短的交互延迟返回查询结果,并尽可能地降低查询误差。现有的学习型AQP方法与底层数据解耦,将I/O密集型计算转化为CPU密集型计算,但是由于计算资源的限制,该类方法通常基于随机的数据样本进行模型训练,此类训练数据会引起稀有群组缺失问题,导致模型预测准确性不高。针对上述问题,提出一种基于分层样本学习的混合型和积网络模型,并基于该模型设计一种AQP框架。分层样本能够有效避免稀有群组缺失现象,基于该样本训练的模型预测准确性大幅提升。此外,针对数据动态更新的情况,提出一种模型自适应更新策略,使得模型能够及时检测数据偏移现象并自适应地执行更新。实验结果表明,与基于抽样和基于机器学习的AQP方法相比,该模型在真实数据集和合成数据集上的平均相对误差分别约降低18.3%和2.2%,在数据动态更新的场景下,其准确性和查询时延均呈现出良好的稳定性。

基于层次分析法的工控网络态势评估研究

现有的网络安全态势评估方法没有考虑到工业控制系统(industrial control system,ICS)网络安全需求的特殊性,无法实现准确的评估。此外,ICS传输大量异构数据,容易受到网络攻击,现有的分类方法无法有效处理多类别不平衡数据。针对该问题,本文首先分析了工控系统的特点,提出了基于层次分析法的工控系统安全态势量化评估方法,该方法可以更准确地反映ICS网络安全状况;然后针对多攻击类型数据不平衡问题,提出了平均欠过采样方法,以平衡数据并且不会导致数据量过大;最后基于极限梯度提升(extreme gradient boosting,XGBoost)算法构建了ICS网络态势评估分类器,实验表明,本文设计的分类模型相较于传统分类算法支持向量机、K近邻以及随机森林可以实现更好的精度。

基于稀疏自编码的智能电网海量运行数据清洗

为解决电网数据样本混合的问题并提升智能电网的运行与响应速率,提出基于稀疏自编码的智能电网海量运行数据清洗方法。按照稀疏自编码原则,建立自编码网络模型,联合数据样本稀疏矩阵,确定深度降维参数的实际取值范围,实现对智能电网运行数据的降维处理。根据数据仓库定义形式,估算数据样本运行状态,计算聚类清洗参量的具体数值,完成基于稀疏自编码的智能电网海量运行数据清洗算法的设计。实验结果表明,在稀疏自编码原则作用下,电网主机能够准确区分两类数据样本,从而解决电网数据样本混合的问题,达到提升智能电网运行与响应速率的实际应用需求。

新时代生态环境监测质量控制分析

生态环境监测在新时代中具有重要作用,但监测工作的难度相对较大,很容易出现质量问题,只有加强质量控制,才能够增强监测结果的准确性。因此,利用分析法等方法对新时代生态环境监测质量控制进行了探究。在研究过程中,对生态环境监测过程中的质量控制要点与对策进行了分析。结果表明:在进行质量控制时,需做好准备环节、现场采样、样品运输、分析测试等各个环节的控制工作,同时应针对人员素养低等问题采取针对性的策略,继而提升监测水平。

基于GMM-KNN-LSTM的烧结矿化学指标预测

针对烧结矿化学指标检测频率低导致无标签样本无法被机器学习利用的问题,提出了一种充分利用样本中有用信息的烧结矿化学指标预测模型.首先,结合高斯混合模型(GMM)和K-近邻(KNN)算法,将无标签样本转化为有标签样本,然后与长短期记忆(LSTM)单元相结合,用于预测烧结矿的总铁质量分数、FeO质量分数和碱度3个化学指标.通过与反向传播神经网络(BPNN)、循环神经网络(RNN)和LSTM三种模型对比,结果表明所建模型具有较低的预测误差.总铁质量分数和FeO质量分数的预测命中率在允许误差±0.5%内时分别达到98.73%和95.33%,碱度的预测命中率在允许误差±0.05内为98.13%,展现了较高的预测精度.

基于特征聚类的近似查询分块选择方法

大数据集群环境中,随机访问的低效性使得基于行级别抽样的近似查询处理方法在构建样本时效率低下。该文将利用集群环境中数据分块存储的特性,以分块级别来进行抽样。在基准测试数据集和真实数据集上的实验,显示此方法在降低数据读取率,提高查询响应速度的同时,保持较高的查询精度。实验中,仅需要读取少于20%的数据就可以获得低于5%的查询误差,且为数据集每个分块的预计算的特征数据所需要的存储空间小于数据集所占空间的0.04%。

海量大数据定向采样有差别挖掘算法仿真

在大数据中,不同类别之间可能存在数据分布不均衡的情况,即某些类别的数据样本数量远远少于其他类别。这种情况下,传统的采样方法无法正确反映所有类别的特征和差异。为提升大数据信息的应用性,文中研究海量大数据定向采样有差别挖掘算法。以网站统一资源定位器(URL)初始化为基础,在网络上抓取网页,采集网页的超文本标记语言(HTML)数据,提取定向数据的相关链接,并将其导入URL队列。根据网络搜索策略,实施相关的数据搜索和处理。完成数据搜索后,将自动进行下一网页的URL,继续进行海量大数据定向采样。结合模糊特征匹配与检测滤波方法实现大数据定向采样过程中的抗干扰处理。采用粗糙集算法实施挖掘,利用扩展差别矩阵对大数据决策表内的值实施约简,实现海量大数据的模式分类。实验结果显示,该算法数据采集过程中的丢包率基本控制在0.2%以下,具有较高的鲁棒性。

一种建立在GPT-2模型上的数据增强方法

针对句子分类任务常面临着训练数据不足,而且文本语言具有离散性,在语义保留的条件下进行数据增强具有一定困难,语义一致性和多样性难以平衡的问题,本文提出一种惩罚生成式预训练语言模型的数据增强方法(punishing generative pre-trained transformer for data augmentation,PunishGPT-DA)。设计了惩罚项和超参数α,与负对数似然损失函数共同作用微调GPT-2(generative pre-training 2.0),鼓励模型关注那些预测概率较小但仍然合理的输出;使用基于双向编码器表征模型(bidirectional encoder representation from transformers,BERT)的过滤器过滤语义偏差较大的生成样本。本文方法实现了对训练集16倍扩充,与GPT-2相比,在意图识别、问题分类以及情感分析3个任务上的准确率分别提升了1.1%、4.9%和8.7%。实验结果表明,本文提出的方法能够同时有效地控制一致性和多样性需求,提升下游任务模型的训练性能。

基于朴素贝叶斯的有效样本提取技术

在数控加工过程中会产生海量数据,如数控系统电控数据、工艺任务数据、切削响应数据等,其中存在数控加工中工况信息和响应信息自动标记困难,而且存在大量冗余、无效信息,当前缺乏有效样本信息提取算法。针对上述问题,文中研究了基于指令域的铣削加工工艺数据的标记技术,然后提出一种从标记数据中提取有效样本的算法,最后通过有效样本的提取与验证试验证实了该提取算法的有效性,为后续智能模型的训练奠定了坚实的基础。

生态环境监测过程中的质量控制研究

本文旨在研究生态环境监测过程中的质量控制策略,通过对生态环境监测的重要价值进行论述,引出了一系列质量控制策略。在强化样品采集与保存方面,应加强采样点选择、采样时间与方法的科学性,以保证样品在采集过程中的真实可靠。在仪器校准与维护方面,应注意选择合适的校准方法和维护措施,以确保仪器的准确和稳定。在数据处理与分析方面,要注重数据清洗和校验,正确选择统计方法和分析指标,以提高数据的准确性和可信度。此外,还应加强人才队伍建设并完善监督审查体系,以提高监测工作质量和数据的可靠性,为生态环境保护和治理提供科学依据。

建筑施工检试验样品管理系统研究

针对建筑施工现场检试验样品传统管理方式存在的样品管理复杂、管理工作量大、样品易丢失等问题,提出一种基于RFID技术构建的施工现场样品管理信息系统。从施工现场检试验样品的管理现状入手,对样品管理进行需求分析,提出技术解决思路,进而形成包括系统拓扑、系统架构、系统功能在内的整体解决方案。

面向流形数据的共享近邻密度峰值聚类算法

流形数据由一些弧线状或环状的类簇组成,其特点是同一类簇的样本间距离差距较大。密度峰值聚类算法不能有效识别流形类簇的类簇中心且分配剩余样本时易引发样本的连续误分配问题。为此,本文提出面向流形数据的共享近邻密度峰值聚类(density peaks clustering based on shared nearest neighbor for manifold datasets,DPC-SNN)算法。提出了一种基于共享近邻的样本相似度定义方式,使得同一流形类簇样本间的相似度尽可能高;基于上述相似度定义局部密度,不忽略距类簇中心较远样本的密度贡献,能更好地区分出流形类簇的类簇中心与其他样本;根据样本的相似度分配剩余样本,避免了样本的连续误分配。DPC-SNN算法与DPC、FKNNDPC、FNDPC、DPCSA及IDPC-FA算法的对比实验结果表明,DPC-SNN算法能够有效发现流形数据的类簇中心并准确完成聚类,对真实以及人脸数据集也有不错的聚类效果。