Hybrid model based on K-means++ algorithm, optimal similar day approach, and long short-term memory neural network for short-term photovoltaic power prediction

Photovoltaic(PV) power generation is characterized by randomness and intermittency due to weather changes.Consequently, large-scale PV power connections to the grid can threaten the stable operation of the power system. An effective method to resolve this problem is to accurately predict PV power. In this study, an innovative short-term hybrid prediction model(i.e., HKSL) of PV power is established. The model combines K-means++, optimal similar day approach,and long short-term memory(LSTM) network. Historical power data and meteorological factors are utilized. This model searches for the best similar day based on the results of classifying weather types. Then, the data of similar day are inputted into the LSTM network to predict PV power. The validity of the hybrid model is verified based on the datasets from a PV power station in Shandong Province, China. Four evaluation indices, mean absolute error, root mean square error(RMSE),normalized RMSE, and mean absolute deviation, are employed to assess the performance of the HKSL model. The RMSE of the proposed model compared with those of Elman, LSTM, HSE(hybrid model combining similar day approach and Elman), HSL(hybrid model combining similar day approach and LSTM), and HKSE(hybrid model combining K-means++,similar day approach, and LSTM) decreases by 66.73%, 70.22%, 65.59%, 70.51%, and 18.40%, respectively. This proves the reliability and excellent performance of the proposed hybrid model in predicting power.

基于K-means算法的建筑群震害分析模型缩减方法

基于建筑群模型和弹塑性时程分析的精细化城市震害模拟技术能够为防震减灾及应急救援决策提供必要的依据和参考。为了减小城市建筑群震害模拟的计算量和计算时间,本文提出一种基于聚类算法的建筑群模型缩减方法。该方法采用K-means聚类算法,首先基于建筑结构属性向量对建筑群进行聚类,将相似的建筑结构聚为一组;然后从每组选取一个代表建筑组成建筑群缩减模型,通过减少需要分析的建筑结构数量来减少建筑群震害模拟的计算量。本文对传统的K-means算法进行改进,通过设定组内建筑结构的差异上限自动调整聚类分组数量;提出将具体地震动作用下结构地震损伤指数作为结构属性向量进行聚类,并通过算例对比分别采用两种缩减模型,即基于损伤指数聚类的缩减模型与基于结构力学模型参数聚类的缩减模型,计算结构损伤状态准确程度。对比结果表明:在聚类分组数量相同的情况下,基于损伤指数的分组明显优于基于模型参数的分组,采用模型缩减方法能够在保证足够计算精度前提下显著减少建筑群震害模拟计算量和计算时间。

A Study of EM Algorithm as an Imputation Method: A Model-Based Simulation Study with Application to a Synthetic Compositional Data

Compositional data, such as relative information, is a crucial aspect of machine learning and other related fields. It is typically recorded as closed data or sums to a constant, like 100%. The statistical linear model is the most used technique for identifying hidden relationships between underlying random variables of interest. However, data quality is a significant challenge in machine learning, especially when missing data is present. The linear regression model is a commonly used statistical modeling technique used in various applications to find relationships between variables of interest. When estimating linear regression parameters which are useful for things like future prediction and partial effects analysis of independent variables, maximum likelihood estimation (MLE) is the method of choice. However, many datasets contain missing observations, which can lead to costly and time-consuming data recovery. To address this issue, the expectation-maximization (EM) algorithm has been suggested as a solution for situations including missing data. The EM algorithm repeatedly finds the best estimates of parameters in statistical models that depend on variables or data that have not been observed. This is called maximum likelihood or maximum a posteriori (MAP). Using the present estimate as input, the expectation (E) step constructs a log-likelihood function. Finding the parameters that maximize the anticipated log-likelihood, as determined in the E step, is the job of the maximization (M) phase. This study looked at how well the EM algorithm worked on a made-up compositional dataset with missing observations. It used both the robust least square version and ordinary least square regression techniques. The efficacy of the EM algorithm was compared with two alternative imputation techniques, k-Nearest Neighbor (k-NN) and mean imputation (), in terms of Aitchison distances and covariance.

基于狄利克雷多项式过程模型与K-means结合的菌群分析

群体分型是一种有助于更好的理解人类身心健康等复杂生物学问题的有效方法,聚类是一种为了对样本分组来降低复杂性的定义肠型的方法,而传统K-means聚类算法的K值选取无法确定,本文在传统K-means聚类算法的基础上进行了改进,并公开数据集上进行了验证,实验表明改进算法能够解决K值选取无法确定的问题,且聚类结果的稳定性、准确性和聚类质量都得到显著提高。将改进后的模型运用于肠道菌群OTUs数据,发现不仅能够有效地区分2-型糖尿病患者样本间的相似性,而且能鉴定出影响菌群结构异质性最大的OTUs菌,为临床解决2-型糖尿病问题提供了一种新的思路。

Plant Leaf Diseases Classification Using Improved K-Means Clustering and SVM Algorithm for Segmentation

Several pests feed on leaves,stems,bases,and the entire plant,causing plant illnesses.As a result,it is vital to identify and eliminate the disease before causing any damage to plants.Manually detecting plant disease and treating it is pretty challenging in this period.Image processing is employed to detect plant disease since it requires much effort and an extended processing period.The main goal of this study is to discover the disease that affects the plants by creating an image processing system that can recognize and classify four different forms of plant diseases,including Phytophthora infestans,Fusarium graminearum,Puccinia graminis,tomato yellow leaf curl.Therefore,this work uses the Support vector machine(SVM)classifier to detect and classify the plant disease using various steps like image acquisition,Pre-processing,Segmentation,feature extraction,and classification.The gray level co-occurrence matrix(GLCM)and the local binary pattern features(LBP)are used to identify the disease-affected portion of the plant leaf.According to experimental data,the proposed technology can correctly detect and diagnose plant sickness with a 97.2 percent accuracy.

Genetic Algorithm Combined with the K-Means Algorithm:A Hybrid Technique for Unsupervised Feature Selection

The dimensionality of data is increasing very rapidly,which creates challenges for most of the current mining and learning algorithms,such as large memory requirements and high computational costs.The literature includes much research on feature selection for supervised learning.However,feature selection for unsupervised learning has only recently been studied.Finding the subset of features in unsupervised learning that enhances the performance is challenging since the clusters are indeterminate.This work proposes a hybrid technique for unsupervised feature selection called GAk-MEANS,which combines the genetic algorithm(GA)approach with the classical k-Means algorithm.In the proposed algorithm,a new fitness func-tion is designed in addition to new smart crossover and mutation operators.The effectiveness of this algorithm is demonstrated on various datasets.Fur-thermore,the performance of GAk-MEANS has been compared with other genetic algorithms,such as the genetic algorithm using the Sammon Error Function and the genetic algorithm using the Sum of Squared Error Function.Additionally,the performance of GAk-MEANS is compared with the state-of-the-art statistical unsupervised feature selection techniques.Experimental results show that GAk-MEANS consistently selects subsets of features that result in better classification accuracy compared to others.In particular,GAk-MEANS is able to significantly reduce the size of the subset of selected features by an average of 86.35%(72%–96.14%),which leads to an increase of the accuracy by an average of 3.78%(1.05%–6.32%)compared to using all features.When compared with the genetic algorithm using the Sammon Error Function,GAk-MEANS is able to reduce the size of the subset of selected features by 41.29%on average,improve the accuracy by 5.37%,and reduce the time by 70.71%.When compared with the genetic algorithm using the Sum of Squared Error Function,GAk-MEANS on average is able to reduce the size of the subset of selected features by 15.91%,and improve the accuracy by 9.81%,but the time is increased by a factor of 3.When compared with the machine-learning based methods,we observed that GAk-MEANS is able to increase the accuracy by 13.67%on average with an 88.76%average increase in time.

Customer Segment Prediction on Retail Transactional Data Using K-Means and Markov Model

Retailing is a dynamic business domain where commodities and goods are sold in small quantities directly to the customers.It deals with the end user customers of a supply-chain network and therefore has to accommodate the needs and desires of a large group of customers over varied utilities.The volume and volatility of the business makes it one of the prospectivefields for analytical study and data modeling.This is also why customer segmentation drives a key role in multiple retail business decisions such as marketing budgeting,customer targeting,customized offers,value proposition etc.The segmentation could be on various aspects such as demographics,historic behavior or preferences based on the use cases.In this paper,historic retail transactional data is used to segment the custo-mers using K-Means clustering and the results are utilized to arrive at a transition matrix which is used to predict the cluster movements over the time period using Markov Model algorithm.This helps in calculating the futuristic value a segment or a customer brings to the business.Strategic marketing designs and budgeting can be implemented using these results.The study is specifically useful for large scale marketing in domains such as e-commerce,insurance or retailers to segment,profile and measure the customer lifecycle value over a short period of time.

基于K-means算法的跨国零售商客户细分研究

随着经济全球化及大数据技术的蓬勃发展,跨国零售商之间的竞争日益激烈,根据客户特征进行客户细分,协助客户进行个性化的服务体验,有利于跨国零售商实现精准营销和高效的客户关系管理。为了提高客户细分的精度,本文提出一种基于RFM模型的K-means聚类算法,使用簇内误方差(SSE)和轮廓系数(Silhouette Coefficient)计算聚类个数,优化K值选取。本文选取一家跨国零售商的数据进行实证检验,对细分后的结果进行特征分析,将客户划分为核心型客户、维护型客户和风险型客户三种类别,并为不同客户群体提供差异化营销策略,仅供参考。

电网需求侧资源动态分布式k-means聚类算法

为有效聚合电网需求侧资源,合理、高效利用电网资源,提出基于分布式k-means的电网需求侧资源动态聚类算法。通过基于置信半径的分布式k-means算法聚类采集到的电网需求侧资源数据,在模糊C均值进化神经网络中,以聚类得到的电网需求侧资源数据为输入向量,输出电网需求侧资源场景,依据场景存在概率,以电网侧资源日均峰谷差最小、DG消纳程度最高与日均负荷波动率最小为目标函数,以电网需求侧资源曲线波动率与负荷互补为约束条件,构建电网需求侧资源多场景聚类模型,经动态改变惯性因子(DCW)粒子群算法求解模型后,实现电网需求侧资源多场景聚类。试验结果表明:该方法可实现电网需求侧资源动态聚类,应用该方法聚类不同场景电网需求侧资源时的日负荷率较低,聚类效果较好,可满足实际电力需求侧资源动态聚类工作的需要。

A Hybrid Method Combining Improved K-means Algorithm with BADA Model for Generating Nominal Flight Profiles

A high-precision nominal flight profile,involving controllers′intentions is critical for 4Dtrajectory estimation in modern automatic air traffic control systems.We proposed a novel method to effectively improve the accuracy of the nominal flight profile,including the nominal altitude profile and the speed profile.First,considering the characteristics of trajectory data,we developed an improved K-means algorithm.The approach was to measure the similarity between different altitude profiles by integrating the space warp edit distance algorithm,thereby to acquire several fitted nominal flight altitude profiles.This approach breaks the constraints of traditional K-means algorithms.Second,to eliminate the influence of meteorological factors,we introduced historical gridded binary data to determine the en-route wind speed and temperature via inverse distance weighted interpolation.Finally,we facilitated the true airspeed determined by speed triangle relationships and the calibrated airspeed determined by aircraft data model to extract a more accurate nominal speed profile from each cluster,therefore we could describe the airspeed profiles above and below the airspeed transition altitude,respectively.Our experimental results showed that the proposed method could obtain a highly accurate nominal flight profile,which reflects the actual aircraft flight status.

Genetic algorithm-optimized backpropagation neural network establishes a diagnostic prediction model for diabetic nephropathy:Combined machine learning and experimental validation in mice

Background:Diabetic nephropathy(DN)is the most common complication of type 2 diabetes mellitus and the main cause of end-stage renal disease worldwide.Diagnostic biomarkers may allow early diagnosis and treatment of DN to reduce the prevalence and delay the development of DN.Kidney biopsy is the gold standard for diagnosing DN;however,its invasive character is its primary limitation.The machine learning approach provides a non-invasive and specific criterion for diagnosing DN,although traditional machine learning algorithms need to be improved to enhance diagnostic performance.Methods:We applied high-throughput RNA sequencing to obtain the genes related to DN tubular tissues and normal tubular tissues of mice.Then machine learning algorithms,random forest,LASSO logistic regression,and principal component analysis were used to identify key genes(CES1G,CYP4A14,NDUFA4,ABCC4,ACE).Then,the genetic algorithm-optimized backpropagation neural network(GA-BPNN)was used to improve the DN diagnostic model.Results:The AUC value of the GA-BPNN model in the training dataset was 0.83,and the AUC value of the model in the validation dataset was 0.81,while the AUC values of the SVM model in the training dataset and external validation dataset were 0.756 and 0.650,respectively.Thus,this GA-BPNN gave better values than the traditional SVM model.This diagnosis model may aim for personalized diagnosis and treatment of patients with DN.Immunohistochemical staining further confirmed that the tissue and cell expression of NADH dehydrogenase(ubiquinone)1 alpha subcomplex,4-like 2(NDUFA4L2)in tubular tissue in DN mice were decreased.Conclusion:The GA-BPNN model has better accuracy than the traditional SVM model and may provide an effective tool for diagnosing DN.

基于自组织神经网络和K-means算法的地下空间地质环境质量三维分类及评价

针对地下空间地质环境质量,前人运用三维地质信息化技术已开展了大量三维综合评价研究,但其评价结果对于规划和施工建议略显不足。其原因主要是评价过程主观性较强,综合评价结果难以充分表达地质环境的真实类别,难以关注更需受到重视的不良地质环境条件等。针对上述问题,利用自组织神经网络(SOM)和K-means算法对地下空间地质环境质量三维评价信息进行分类研究;以福建省厦门市马銮湾新城南岸片区为实例,基于三维空间分析方法提取三维评价指标因子,开展基于自组织神经网络和K-means算法的地下空间地质环境质量三维评价,最后利用评价获得的地质环境类别与主导因子进一步提出规划和施工建议。结果表明:基于自组织神经网络和K-means算法的评价方法能够有效挖掘多维多源地质数据中的隐含信息,识别出关键区分因子,为地下空间地质环境质量评价提供了新的思路和方法。

Estimating wheat fractional vegetation cover using a density peak k-means algorithm based on hyperspectral image data

Fractional vegetation cover(FVC)is an important parameter to measure crop growth.In studies of crop growth monitoring,it is very important to extract FVC quickly and accurately.As the most widely used FVC extraction method,the photographic method has the advantages of simple operation and high extraction accuracy.However,when soil moisture and acquisition times vary,the extraction results are less accurate.To accommodate various conditions of FVC extraction,this study proposes a new FVC extraction method that extracts FVC from a normalized difference vegetation index(NDVI)greyscale image of wheat by using a density peak k-means(DPK-means)algorithm.In this study,Yangfumai 4(YF4)planted in pots and Yangmai 16(Y16)planted in the field were used as the research materials.With a hyperspectral imaging camera mounted on a tripod,ground hyperspectral images of winter wheat under different soil conditions(dry and wet)were collected at 1 m above the potted wheat canopy.Unmanned aerial vehicle(UAV)hyperspectral images of winter wheat at various stages were collected at 50 m above the field wheat canopy by a UAV equipped with a hyperspectral camera.The pixel dichotomy method and DPK-means algorithm were used to classify vegetation pixels and non-vegetation pixels in NDVI greyscale images of wheat,and the extraction effects of the two methods were compared and analysed.The results showed that extraction by pixel dichotomy was influenced by the acquisition conditions and its error distribution was relatively scattered,while the extraction effect of the DPK-means algorithm was less affected by the acquisition conditions and its error distribution was concentrated.The absolute values of error were 0.042 and 0.044,the root mean square errors(RMSE)were 0.028 and 0.030,and the fitting accuracy R2 of the FVC was 0.87 and 0.93,under dry and wet soil conditions and under various time conditions,respectively.This study found that the DPK-means algorithm was capable of achieving more accurate results than the pixel dichotomy method in various soil and time conditions and was an accurate and robust method for FVC extraction.

Application of Wavelength Selection Combined with DS Algorithm for Model Transfer between NIR Instruments

This study aims to realize the sharing of near-infrared analysis models of lignin and holocellulose content in pulp wood on two different batches of spectrometers and proposes a combined algorithm of SPA-DS,MCUVE-DS and SiPLS-DS.The Successive Projection Algorithm(SPA),the Monte-Carlo of Uninformative Variable Elimination(MCUVE)and the Synergy Interval Partial Least Squares(SiPLS)algorithms are respectively used to reduce the adverse effects of redundant information in the transmission process of the full spectrum DS algorithm model.These three algorithms can improve model transfer accuracy and efficiency and reduce the manpower and material consumption required for modeling.These results show that the modeling effects of the characteristic wavelengths screened by the SPA,MCUVE and SiPLS algorithms are all greatly improved compared with the full-spectrum modeling,in which the SPA-PLS result in the best prediction with RPDs above 6.5 for both components.The three wavelength selection methods combined with the DS algorithm are used to transfer the models of the two instruments.Among them,the MCUVE combined with the DS algorithm has the best transfer effect.After the model transfer,the RMSEP of lignin is 0.701,and the RMSEP of holocellulose is 0.839,which was improved significantly than the full-spectrum model transfer of 0.759 and 0.918.

Runoff Modeling in Ungauged Catchments Using Machine Learning Algorithm-Based Model Parameters Regionalization Methodology

Model parameters estimation is a pivotal issue for runoff modeling in ungauged catchments.The nonlinear relationship between model parameters and catchment descriptors is a major obstacle for parameter regionalization,which is the most widely used approach.Runoff modeling was studied in 38 catchments located in the Yellow–Huai–Hai River Basin(YHHRB).The values of the Nash–Sutcliffe efficiency coefficient(NSE),coefficient of determination(R2),and percent bias(PBIAS)indicated the acceptable performance of the soil and water assessment tool(SWAT)model in the YHHRB.Nine descriptors belonging to the categories of climate,soil,vegetation,and topography were used to express the catchment characteristics related to the hydrological processes.The quantitative relationships between the parameters of the SWAT model and the catchment descriptors were analyzed by six regression-based models,including linear regression(LR)equations,support vector regression(SVR),random forest(RF),k-nearest neighbor(kNN),decision tree(DT),and radial basis function(RBF).Each of the 38 catchments was assumed to be an ungauged catchment in turn.Then,the parameters in each target catchment were estimated by the constructed regression models based on the remaining 37 donor catchments.Furthermore,the similaritybased regionalization scheme was used for comparison with the regression-based approach.The results indicated that the runoff with the highest accuracy was modeled by the SVR-based scheme in ungauged catchments.Compared with the traditional LR-based approach,the accuracy of the runoff modeling in ungauged catchments was improved by the machine learning algorithms because of the outstanding capability to deal with nonlinear relationships.The performances of different approaches were similar in humid regions,while the advantages of the machine learning techniques were more evident in arid regions.When the study area contained nested catchments,the best result was calculated with the similarity-based parameter regionalization scheme because of the high catchment density and short spatial distance.The new findings could improve flood forecasting and water resources planning in regions that lack observed data.

Employment Quality EvaluationModel Based on Hybrid Intelligent Algorithm

In order to solve the defect of large error in current employment quality evaluation,an employment quality evaluation model based on grey correlation degree method and fuzzy C-means(FCM)is proposed.Firstly,it analyzes the related research work of employment quality evaluation,establishes the employment quality evaluation index system,collects the index data,and normalizes the index data;Then,the weight value of employment quality evaluation index is determined by Grey relational analysis method,and some unimportant indexes are removed;Finally,the employment quality evaluation model is established by using fuzzy cluster analysis algorithm,and compared with other employment quality evaluation models.The test results show that the employment quality evaluation accuracy of the design model exceeds 93%,the employment quality evaluation error can meet the requirements of practical application,and the employment quality evaluation effect is much better than the comparison model.The comparison test verifies the superiority of the model.

基于K-means聚类算法的百货商场用户价值分析

使用真实百货商场用户会员信息数据对传统百货商场的用户进行价值分析,对商场会员用户结合销售流水表进行识别,分析并建立模型,将传统的RFM模型进行改进,结合入会时长生成更贴合模型的LRFM模型.运用K-means聚类算法对用户价值进行分类并细分,生成雷达图,使用户价值可视化,识别出不同价值的会员.该研究能够帮助传统线下商场更好地管理用户,进行价值分析,从而提高会员服务,具有一定的实用价值.

Parameters of new three-water model based on nuclear magnetic experiment and optimization algorithm

Clastic rock reservoir is the main reservoir type in the oil and gas field.Archie formula or various conductive models developed on the basis of Archie’s formula are usually used to interpret this kind of reservoir,and the three-water model is widely used as well.However,there are many parameters in the threewater model,and some of them are difficult to determine.Most of the determination methods are based on the statistics of large amount of experimental data.In this study,the authors determine the value of the parameters of the new three-water model based on the nuclear magnetic data and the genetic optimization algorithm.The relative error between the resistivity calculated based on these parameters and the resistivity measured experimentally at 100%water content is 0.9024.The method studied in this paper can be easily applied without much experimental data.It can provide reference for other regions to determine the parameters of the new three-water model.

Employee Attrition Classification Model Based on Stacking Algorithm

This paper aims to build an employee attrition classification model based on the Stacking algorithm.Oversampling algorithm is applied to address the issue of data imbalance and the Randomforest feature importance ranking method is used to resolve the overfitting problem after data cleaning and preprocessing.Then,different algorithms are used to establish classification models as control experiments,and R-squared indicators are used to compare.Finally,the Stacking algorithm is used to establish the final classification model.This model has practical and significant implications for both human resource management and employee attrition analysis.

Polarimetric Meteorological Satellite Data Processing Software Classification Based on Principal Component Analysis and Improved K-Means Algorithm

With the increasing variety of application software of meteorological satellite ground system, how to provide reasonable hardware resources and improve the efficiency of software is paid more and more attention. In this paper, a set of software classification method based on software operating characteristics is proposed. The method uses software run-time resource consumption to describe the software running characteristics. Firstly, principal component analysis (PCA) is used to reduce the dimension of software running feature data and to interpret software characteristic information. Then the modified K-means algorithm was used to classify the meteorological data processing software. Finally, it combined with the results of principal component analysis to explain the significance of various types of integrated software operating characteristics. And it is used as the basis for optimizing the allocation of software hardware resources and improving the efficiency of software operation.