Anomaly IoT Node Detection Based on Local Outlier Factor and Time Series

The heterogeneous nodes in the Internet of Things(IoT)are relatively weak in the computing power and storage capacity.Therefore,traditional algorithms of network security are not suitable for the IoT.Once these nodes alternate between normal behavior and anomaly behavior,it is difficult to identify and isolate them by the network system in a short time,thus the data transmission accuracy and the integrity of the network function will be affected negatively.Based on the characteristics of IoT,a lightweight local outlier factor detection method is used for node detection.In order to further determine whether the nodes are an anomaly or not,the varying behavior of those nodes in terms of time is considered in this research,and a time series method is used to make the system respond to the randomness and selectiveness of anomaly behavior nodes effectively in a short period of time.Simulation results show that the proposed method can improve the accuracy of the data transmitted by the network and achieve better performance.

Anomalous Cell Detection with Kernel Density-Based Local Outlier Factor

Since data services are penetrating into our daily life rapidly, the mobile network becomes more complicated, and the amount of data transmission is more and more increasing. In this case, the traditional statistical methods for anomalous cell detection cannot adapt to the evolution of networks, and data mining becomes the mainstream. In this paper, we propose a novel kernel density-based local outlier factor(KLOF) to assign a degree of being an outlier to each object. Firstly, the notion of KLOF is introduced, which captures exactly the relative degree of isolation. Then, by analyzing its properties, including the tightness of upper and lower bounds, sensitivity of density perturbation, we find that KLOF is much greater than 1 for outliers. Lastly, KLOFis applied on a real-world dataset to detect anomalous cells with abnormal key performance indicators(KPIs) to verify its reliability. The experiment shows that KLOF can find outliers efficiently. It can be a guideline for the operators to perform faster and more efficient trouble shooting.

A Two-Level Approach based on Integration of Bagging and Voting for Outlier Detection

Purpose:The main aim of this study is to build a robust novel approach that is able to detect outliers in the datasets accurately.To serve this purpose,a novel approach is introduced to determine the likelihood of an object to be extremely different from the general behavior of the entire dataset.Design/methodology/approach:This paper proposes a novel two-level approach based on the integration of bagging and voting techniques for anomaly detection problems.The proposed approach,named Bagged and Voted Local Outlier Detection(BV-LOF),benefits from the Local Outlier Factor(LOF)as the base algorithm and improves its detection rate by using ensemble methods.Findings:Several experiments have been performed on ten benchmark outlier detection datasets to demonstrate the effectiveness of the BV-LOF method.According to the results,the BV-LOF approach significantly outperformed LOF on 9 datasets of 10 ones on average.Research limitations:In the BV-LOF approach,the base algorithm is applied to each subset data multiple times with different neighborhood sizes(k)in each case and with different ensemble sizes(T).In our study,we have chosen k and T value ranges as[1-100];however,these ranges can be changed according to the dataset handled and to the problem addressed.Practical implications:The proposed method can be applied to the datasets from different domains(i.e.health,finance,manufacturing,etc.)without requiring any prior information.Since the BV-LOF method includes two-level ensemble operations,it may lead to more computational time than single-level ensemble methods;however,this drawback can be overcome by parallelization and by using a proper data structure such as R*-tree or KD-tree.Originality/value:The proposed approach(BV-LOF)investigates multiple neighborhood sizes(k),which provides findings of instances with different local densities,and in this way,it provides more likelihood of outlier detection that LOF may neglect.It also brings many benefits such as easy implementation,improved capability,higher applicability,and interpretability.

An Intelligent Early Warning Method of Press-Assembly Quality Based on Outlier Data Detection and Linear Regression

Focusing on controlling the press-assembly quality of high-precision servo mechanism,an intelligent early warning method based on outlier data detection and linear regression is proposed.Linear regression is used to deal with the relationship between assembly quality and press-assembly process,then the mathematical model of displacement-force in press-assembly process is established and a qualified press-assembly force range is defined for assembly quality control.To preprocess the raw dataset of displacement-force in the press-assembly process,an improved local outlier factor based on area density and P weight(LAOPW)is designed to eliminate the outliers which will result in inaccuracy of the mathematical model.A weighted distance based on information entropy is used to measure distance,and the reachable distance is replaced with P weight.Experiments show that the detection efficiency of the algorithm is improved by 5.6 ms compared with the traditional local outlier factor(LOF)algorithm,and the detection accuracy is improved by about 2%compared with the local outlier factor based on area density(LAOF)algorithm.The application of LAOPW algorithm and the linear regression model shows that it can effectively carry out intelligent early warning of press-assembly quality of high precision servo mechanism.

基于时间序列压缩分割的监测数据异常识别算法研究

为有效识别桥梁健康监测数据的异常,减少误预警、漏预警现象,保障桥梁监测数据的质量和有效性,针对大跨度斜拉桥长期监测数据的缺失、离群和漂移3类异常数据,提出基于时间序列压缩分割的监测数据异常识别算法。该算法将原始监测数据时间序列通过基于序列重要点(Series Importance Point, SIP)的时间序列线性分段(Piecewise Linear Represent, PLR)算法(PLR_SIP)得到数条时间子序列;然后采用欧氏距离进行时间子序列的相似性分析,并基于改进的局部离群因子(Local Outlier Factor, LOF)算法计算每条时间子序列的局部离群因子;最后将其与设定的阈值相比较,从而识别出监测数据的异常。为验证该算法的准确性与工程实用性,对某公路大跨度斜拉桥健康监测数据进行异常识别。结果表明:采用PLR_SIP算法对原始时间序列压缩分割得到的时间子序列能够准确地反映原序列的变化趋势和范围;改进的LOF算法突破了传统LOF算法仅能识别离群值这类无持续时间异常的局限性,能够排除噪声的干扰,实现对离群、缺失和漂移3种异常的识别。该算法无需定义训练集,直接以原始监测数据作为算法的输入,同时能够自适应调整阈值参数,具有良好的可扩展性、实时性、准确性和高效性,适用于处理实时、大量的桥梁健康监测数据。

高校高价值专利技术机会识别研究——以“生成式人工智能”领域为例

提出一种高校高价值专利技术机会识别方法,使用主题建模、突变级数法、机器学习与离群值检测算法,在评估出高校高价值专利的基础上,进一步识别出具有潜在技术机会的技术主题与专利技术。以“生成式人工智能”领域为例进行实证,研究结果表明:“生成式人工智能”领域的潜在技术主题集中在深度学习、神经网络与机器学习等前沿领域,AI影像、AI诊疗等技术为该领域的潜在技术机会,且上述技术均有国家相关政策大力支撑。本研究方法突破了单一技术机会识别方法识别结果针对性不强、识别专利价值不大、识别结果形式较为单一等核心问题,相关识别结果可以为高校技术转移、技术研发与技术创新提供决策支撑。

基于动态阈值和差异性检验的自训练算法

针对自训练算法在迭代训练分类器的过程中存在难以有效选取高置信度样本以及误标记样本错误累积的问题,本文提出了基于动态阈值和差异性检验的自训练算法。引入样本的局部离群因子,据此剔除有标签样本中的离群点以及分类标注无标签样本,依据标注分批次处理无标签样本,以使模型更易选取到高置信度的无标签样本;根据新增伪标签样本的数量和对比隶属度的变化,设计一种动态隶属度阈值函数,提升高置信度样本的质量;定义密集距离度量样本间的差异性,分别计算伪标签样本与同类和不同类样本之间的密集距离之和,从而找出不确定度高的伪标签样本,并将此类样本并入下轮训练的无标签样本集中,缓解误标记样本错误累积的问题。实验结果表明,该算法在12个UCI基准数据集上均取得理想效果。

基于局部均值分解与局部离群因子动力电池故障诊断

动力电池故障诊断是保证电动汽车正常运行的关键。提出一种基于局部均值分解和局部离群因子的动力电池故障诊断方法,用于电池组故障识别与定位。通过局部均值分解对电压信号预处理,并根据相关系数高低重构电压信号。进一步提取重构信号的峭度因子作为故障特征输入到局部离群因子算法中,根据局部离群因子算法自适应阈值输出故障电池。采用实车数据验证了所提方法能有效、准确地检测出故障,具有较好的可靠性与鲁棒性。

启发式k-means聚类算法的改进研究

启发式k-means聚类算法通过在k-means第一次迭代后查看附近的集群来预测每个数据点可能会被划分到的集群子集,有效地加快了算法的运行速度。但由于启发式算法存在随机选择初始聚类中心以及无法有效识别数据集中离群点的缺陷,导致聚类结果的误差平方和较大并且轮廓系数偏小。针对这一问题,提出了CHk-means算法,该算法引入仔细播种方法,克服了启发式k-means算法随机选择初始聚类中心带来的局部最优解问题;该算法引入局部异常因子LOF算法对离群点进行检测,降低了离群点数据对聚类结果的影响。在多个数据集上对3种算法进行对比试验,结果表明CHk-means算法可有效降低聚类结果的误差平方和,增强聚类的轮廓系数,使聚类质量得到明显改善。

Online Capacitor Voltage Transformer Measurement Error State Evaluation Method Based on In-Phase Relationship and Abnormal Point Detection

The assessment of the measurement error status of online Capacitor Voltage Transformers (CVT) within the power grid is of profound significance to the equitable trade of electric energy and the secure operation of the power grid. This paper advances an online CVT error state evaluation method, anchored in the in-phase relationship and outlier detection. Initially, this method leverages the in-phase relationship to obviate the influence of primary side fluctuations in the grid on assessment accuracy. Subsequently, Principal Component Analysis (PCA) is employed to meticulously disentangle the error change information inherent in the CVT from the measured values and to compute statistics that delineate the error state. Finally, the Local Outlier Factor (LOF) is deployed to discern outliers in the statistics, with thresholds serving to appraise the CVT error state. Experimental results incontrovertibly demonstrate the efficacy of this method, showcasing its prowess in effecting online tracking of CVT error changes and conducting error state assessments. The discernible enhancements in reliability, accuracy, and sensitivity are manifest, with the assessment accuracy reaching an exemplary 0.01%.

耦合DWT-IFLOF的大坝监测数据异常检测算法

基于数据驱动的模型已广泛应用于大坝运行状态建模与短期变形预测等领域中,而监测数据中存在的异常值会削弱甚至破坏数据驱动模型的可信度和鲁棒性。针对上述问题,提出一种耦合DWT-IFLOF的大坝监测数据异常检测算法。首先,采用小波降噪技术减少监测数据集中采集噪声对模型建模的负面干扰。其次设计一种耦合孤立森林(iForest)异常检测算法与归一化局部离群因子值(LOF_(nor))的全新监测数据异常度量化表达式。最后通过引入拉伊达准则,依据计算得到的异常分数,对异常值进行定性筛查。试验结果表明,相较于热门异常检测算法,在查全率层面提升18.32%以上;查准率提升20.14%以上;准确率提升0.71%以上。可针对大坝安全监测数据中的异常值进行有效检测。

基于IKNN和LOF的变压器回复电压数据清洗方法研究

基于回复电压极化谱提取特征参量是目前广泛应用的变压器油纸绝缘状态评估方法,但极化谱易受工况干扰、人工失误等因素影响而出现特征数据异常的情况,严重降低评估准确性。针对上述问题,该文提出了一种基于局部离群因子(LOF)和改进K最近邻(IKNN)的回复电压数据清洗方法。首先,选取回复电压极化谱的回复电压极大值Urmax、初始斜率Sr与主时间常数tcdom作为老化特征参量,并基于LOF算法对非标准极化谱中的异常特征量数据进行识别与筛除。其次,利用模糊C均值(FCM)聚类算法减小噪声点对KNN算法的干扰,并通过加权欧氏距离标度突出各特征量间的关联性,进而构建出基于IKNN的数据填补模型架构以实现特征缺失数据的填补。最后,代入多组实测数据验证所提数据清洗方法的实效性。结果表明,数据清洗后的状态评估准确率相较于原有数据上升了50%左右,有效提高了变压器回复电压数据质量,为准确感知变压器运行状况奠定坚实的基础。

基于CART决策树的分布式数据离群点检测算法

在分布式计算环境中,离群点通常表示数据中的异常情况,例如故障、欺诈、攻击等。通过检测分布式数据的离群点,可以对这些异常数据进行集中处理,保护系统和数据的安全。而进行离群点检测时,不仅要考虑数据的规模和复杂性,还要在分布式环境下高效地发现离群点。因此,提出一种基于CART决策树的分布式数据离群点检测算法。在构建CART决策树时,使用类间中心距离作为分裂准则,根据分离类别对训练数据进行分类,从而确定数据的类型。在上述基础上,考虑到离群点的分布模式与其周围数据对象不同,使用空间局部偏离因子(SLDF)对空间内各个数据对象之间的离群程度展开度量,同时在高维空间内展开网格划分,引入SLDF算法检测剩余离群点集,最终实现分布式数据离群点检测。实验结果表明,所提方法的离散点检测错误率在0.010以内,可以更加精准地实现分布式数据离群点检测,具有良好的检测性能。

基于多子空间加权移动窗主成分分析的全厂流程早期故障检测

早期故障的特征不明显,在全厂流程中比常规故障难检测.为了提高全厂流程中早期故障的检测率和灵敏度,将检测视角由全局转移至局部,提出基于多子空间加权移动窗主成分分析(PCA)的早期故障检测方法.使用结合过程知识和数据驱动的双层子空间划分方法,将过程变量划分到不同子空间中.使用加权的移动窗口增大早期故障的偏移量,将局部离群因子(LOF)算法引入PCA,以便进一步关注数据的局部特征,在每个子空间中建立故障检测模型.通过贝叶斯推理融合法对各子空间的监测结果进行信息融合,获得分布式监测结果.通过工业实例验证所提方法的性能.结果表明,所提方法在全厂流程中有效提升了早期故障检测的准确率和灵敏度.

基于集成学习的脱硫剂加入量预测方法

为解决铁水预脱硫过程中脱硫剂加入量控制问题,提出一种基于集成学习的脱硫剂加入量预测方法。首先,对原始数据进行预处理,将空值、重复值、0值以及不符合工艺规范的数据行删除,并使用LOF算法结合专家经验剔除异常值;其次,基于GBDT算法计算每个输入特征的重要性占比,进行特征筛选;最后,采用Optuna超参数自动寻优框架对预测模型调优,寻找最佳超参数组合,预测脱硫剂加入量。利用某钢厂铁水预处理过程中的实际生产数据,分别采用XGBoost、RF、GBDT以及LightGBM等方法构建预测模型并进行对比试验。其中XGBoost模型的拟合精度(R^(2))、均方根误差(RMSE)、平均绝对误差(MAE)以及平均绝对百分比误差(MAPE)分别为0.8962、198.245、119.726以及7.897%,相较于其它模型均是最优。

基于CNN-LSTM-LOF的过程故障预测模型

在现代工业过程中,故障预测可以及时预测设备的潜在故障,减少事故的发生以及降低经济损失,因此故障预测对于过程系统至关重要。由于过程系统的复杂性以及运行数据集分布不均,使用正常数据集离线预测运行状态的方法没有较好的泛用性,且不太准确。针对以上问题,将卷积神经网络(CNN)与长短期记忆网络(LSTM)相结合,用于提取设备运行数据的特征,在线预测之后的运行状态;随后将预测结果送入在离线状态下训练好的局部异常因子(LOF)模型中,计算预测出时间序列的异常值;最后将异常值与离线状态下训练出的故障阈值进行比较,大于阈值则认为存在潜在故障。将模型用于田纳西-伊斯曼(TE)时间序列进行验证,并与传统的故障预测方法进行比较,实验结果表明:本文所提模型对于多故障以及单故障预测相比传统故障预测方法均具有更好的效果,可以提前1个采样窗口检测到数据异常,有应用于工业故障预测的潜力。

面向高维流数据的离群值检测算法

累计局部离群因子(cumulative local outlier factor,C_LOF)算法能有效解决数据流中的概念漂移问题和克服离群点检测中的伪装问题,但在处理高维数据时,时间复杂度较高。为有效解决时间复杂度高的问题,提出一种基于投影索引近邻的累计局部离群因子(cumulative local outlier factor based projection indexed nearest neighbor,PINN_C_LOF)算法。使用滑动窗口维护活跃数据点,在新数据到达和旧数据过期时,引入投影索引近邻(projection indexed nearest neighbor,PINN)方法,增量更新窗口中受影响数据点的近邻。实验结果表明,PINN_C_LOF算法在检测高维流数据离群值时,在保持检测精确度的前提下,其时间复杂度较C_LOF算法明显降低。

基于LOF算法的核辐射自动监测系统设计与实现

为适应对高精度、全覆盖的核辐射监测要求,本文基于局部离群因子算法(Local Outlier Factor,LOF)设计了一套完善的核辐射自动监测系统。首先,结合具体情况,进行核辐射自动监测站布点,实时获取核辐射监测数据并通过无线通信网络传至服务器,采用传输加密技术保障数据传输过程的安全性与可靠性;其次,通过引入LOF算法有效识别监测数据中存在的传感器故障或者设备缺陷导致的无效监测异常值,无效值将不列入数据统计;最后,将有效的监测实时数据传送至监测指挥中心。系统试运行测试结果表明:该系统能有效针对各种硬件故障导致的无效数据进行高可靠性的实时监测和识别,无效数据判断准确性超过95%,提高了核辐射环境自动监测的稳定性和可靠性,有效防范核辐射造成的危害,为促进生态系统的可持续发展提供参考。

导航卫星载荷分系统健康评估方法设计

面向下一代导航系统结合高中低轨构建导航星座的设想,随着遥测参数数量和种类的激增,针对传统健康评估方法面临的过往专家知识难以适用、故障机理储备难以覆盖全面的问题,提出了基于局部异常因子检测-贝叶斯网络结构学习的导航卫星载荷分系统健康评估方法;通过采集某卫星系统实际故障时间点前后数据,设计实验验证了局部异常因子检测方法能够以粗粒度正确输出单机级的健康状况;分析比较了3种评分函数下,贝叶斯结构学习的效率和模型的准确度;实验结果表明,当评分函数分别选为BDeuScore、K2Score以及BicScore时,学习到的模型对系统的健康评估准确度分别为87.4%、80.5%和85.2%;总结了局部异常因子检测-贝叶斯网络结构学习方法各自的不足,为导航卫星分系统健康评估方法提供了新方向和思路。

轨式输送机轮轨噪声预测技术研究

针对轨式输送机运行中产生的噪声污染问题,基于通过现场试验采集的噪声数据,提出一种基于参数优化多项式曲面拟合的轨式输送机轮轨噪声建模方法。根据均值和最大值筛选出异常数组,采用局部离群因子算法检测异常数组中的异常数据并替换。选取预处理后数据的最大值,以残差平方和最小值为目标函数,利用秃鹰搜索算法建立不同位置处轨式输送机轮轨噪声预测模型,揭示距离和速度对噪声的影响规律。分析结果表明:在轨式输送机机头、中间、机尾3个位置处模型拟合度达到0.9以上,模型预测值与实测值的误差小于0.7dB(A),模型吻合度较好;在距离和速度的影响下,机头处噪声高于机尾处噪声且变化趋势基本一致;中间处噪声低于机头与机尾处噪声且随速度与距离的增加呈二次曲线型变化。该研究成果可为轨式输送机噪声控制提供理论依据。