Distance function selection in several clustering algorithms

Most clustering algorithms need to describe the similarity of objects by a predefined distance function. Three distance functions which are widely used in two traditional clustering algorithms k-means and hierarchical clustering were investigated. Both theoretical analysis and detailed experimental results were given. It is shown that a distance function greatly affects clustering results and can be used to detect the outlier of a cluster by the comparison of such different results and give the shape information of clusters. In practice situation, it is suggested to use different distance function separately, compare the clustering results and pick out the 搒wing points? And such points may leak out more information for data analysts.

基于可拓距的改进k-means聚类算法

针对现有聚类算法在初始聚类中心优化过程中存在首个初始聚类中心点落于边界非密集区域的不足,导致出现算法聚类效果不均衡问题,提出一种基于可拓距优选初始聚类中心的改进k-means算法。将样本经典距离向可拓区间映射,并通过可拓侧距计算方法得到可拓左侧距及可拓右侧距;引入平均可拓侧距概念,将平均可拓左侧距和平均可拓右侧距分别作为样本密集度和聚类中心疏远度的量化指标;在此基础上,给出初始聚类中心选取准则。通过与传统k-means聚类算法进行对比,结果表明改进后的k-means聚类算法选取的初始聚类中心分布更加均匀,聚类效果更好,尤其在对高维数据聚类时具有更高的聚类准确率和更好的均衡性。

一种基于扩展的K-means聚类算法

K-means算法是聚类方法中常用的一种划分方法.基于扩展划分的思想,提出了一种基于扩展的K-means聚类算法(EK-means),在一定程度上避免了聚类结果陷入局部解的现象,减少了原始K-means算法因采用误差平方和准则函数而出现将大的聚类簇分割开的情况.该算法使用了基于距离的技术来处理孤立点,引进了一种基于扩展的方法进行聚类.实验表明该算法可扩展性好,能够很好的识别出孤立点或噪声,并且有很好的精度.

Sonar Image Detection Algorithm Based on Two-Phase Manifold Partner Clustering

According to the characteristics of sonar image data with manifold feature,the sonar image detection method based on two-phase manifold partner clustering algorithm is proposed. Firstly,K-means block clustering based on euclidean distance is proposed to reduce the data set. Mean value,standard deviation,and gray minimum value are considered as three features based on the relatinship between clustering model and data structure. Then K-means clustering algorithm based on manifold distance is utilized clustering again on the reduced data set to improve the detection efficiency. In K-means clustering algorithm based on manifold distance,line segment length on the manifold is analyzed,and a new power function line segment length is proposed to decrease the computational complexity. In order to quickly calculate the manifold distance,new allsource shortest path as the pretreatment of efficient algorithm is proposed. Based on this,the spatial feature of the image block is added in the three features to get the final precise partner clustering algorithm. The comparison with the other typical clustering algorithms demonstrates that the proposed algorithm gets good detection result. And it has better adaptability by experiments of the different real sonar images.

Initial Value Filtering Optimizes Fast Global K-Means

K-means clustering algorithm is an important algorithm in unsupervised learning and plays an important role in big data processing, computer vision and other research fields. However, due to its sensitivity to initial partition, outliers, noise and other factors, the clustering results in data analysis, image segmentation and other fields are unstable and weak in robustness. Based on the fast global K-means clustering algorithm, this paper proposed an improved K-means clustering algorithm. Through the neighborhood filtering mechanism, the points in the neighborhood of the selected initial clustering center have not participated in the selection of the next initial clustering center, which can effectively reduce the randomness of initial partition and improve the efficiency of initial partition. Mahalanobis distance was used in the clustering process to better consider the global nature of data. Compared with the traditional clustering algorithm and other optimization algorithms, the results of real data set testing are significantly improved.

扩展空间对象聚类问题的研究

扩展空间对象的聚类分析是空间聚类研究的焦点问题。扩展空间对象的聚类分析要求在聚类中不仅要考虑对象的位置,而且要考虑对象所占据的范围。空间近似是研究扩展空间对象聚类的基本思想,但是点近似的方法忽视了扩展对象覆盖范围的影响,经常造成聚类质量不高,同时点近似方法的算法代价比较高,影响聚类的性能和可伸缩性;最小外接矩形(MBR)近似的方法一定程度地保留了扩展空间对象的位置和范围信息,配合MBR距离或者扩展距离,使得扩展对象的聚类分析无论从质量还是性能都可以满足要求。文章最后比较了几种扩展对象聚类方法的聚类性能和效果。

一种基于可拓距的特征变换方法及其在网络入侵检测中的应用

作为识别攻击或异常行为以保护网络安全的重要步骤之一,网络入侵检测常常与数据挖掘或机器学习技术结合应用.如今,随着网络数据的爆炸性增长,传统的入侵检测技术面临着海量数据检测处理的问题,现有入侵检测系统往往难以同时满足实时性和有效性的需求.本文尝试将可拓学中的可拓距概念引入网络入侵检测研究中,提出了一种基于可拓距的特征变换方法,将数据点的原特征映射为簇外中心距和簇内可拓距这两大部分,根据原始数据多维特征生成新的特征,以达到特征降维的目的,旨在同时满足网络入侵检测系统的实时性和有效性的需求.本文使用KDD CUP 99作为仿真数据集测试所提出的基于可拓距的方法在网络入侵检测特征变换中的应用效果.实验结果表明,较之传统的KNN算法,基于可拓距的方法明显地减少了检测时间,而同时其检测率的下降可以控制在1%之内,具有较好的时效性优势.

可拓K近邻算法在数据聚类分析中的应用

针对区间值数据的数据聚类问题,根据可拓学关联函数的定义,提出可拓距离的概念来度量数据之间的距离,利用K近邻的思想,根据可拓距离的大小对数据集的目标属性进行投票选择进行分类,设计了可拓K近邻算法(Extension K Nearest Neighbor,EKNN)。最后利用UCI的两个基准数据集Iris植物样本数据和糖尿病数据库PIDD进行验证,首先通过免疫网络约简算法对条件属性进行最小属性约简,然后利用EKNN算法分析和比较不同最小约简属性下的分类准确率。

中韩野生软枣猕猴桃种质资源遗传多样性分析

【目的】利用RAPD分子标记技术研究国内长白山地区和韩国由来野生软枣猕猴桃种质资源的遗传多样性。【方法】以长白山地区和韩国由来的28个野生软枣猕猴桃的叶片为材料,利用RAPD分子标记技术进行了遗传多样性分析,并明确了它们之间的亲缘关系。【结果】利用131个随机引物进行PCR,从中筛选出了多态性高、重复性好且扩增条带清晰的24个引物。24个引物在28个野生软枣猕猴桃中共扩增出191条带,其中多态性条带为186条,占97.4%。平均每个引物产生7.75个多态性条带。应用NTSYSpc 2.10e软件进行遗传一致度和遗传距离分析后用UPGMA方法进行聚类分析。28个野生软枣猕猴桃种质资源之间的遗传距离为0.020 2~0.934 2。遗传距离0.58时可将28个野生种划分成2大类。在遗传距离最小的0.02处,有蛟河2号和3号2个野生种,其RAPD分子标记相似性为98%。【结论】来自不同地理区域的野生软枣猕猴桃之间存在较高的遗传多样性,而在同一地理区域内遗传多样性较低。韩国野生软枣猕猴桃之间的遗传多样性较低,且与二道白河、汪清、左家等地的野生软枣猕猴桃亲缘关系较近。即具有同一地理区域聚类趋势,且不同地理区域间存在较高的遗传多样性。

Classifying and clustering in negative databases

Recently, negative databases （NDBs） are proposed for privacy protection. Similar to the traditional databases, some basic operations could be conducted over the NDBs, such as select, intersection, update, delete and so on. However, both classifying and clustering in negative databases have not yet been studied. Therefore, two algorithms, i.e., a k nearest neighbor （kNN） classification algorithm and a k-means clustering algorithm in NDBs, are proposed in this paper, respectively. The core of these two algorithms is a novel method for estimating the Hamming distance between a binary string and an NDB. Experimental results demonstrate that classifying and clustering in NDBs are promising.

Investigating distance halo effect of fixed automated speed camera based on taxi GPS trajectory data

Background:The deterrence effect of automated speed camera(ASC)is still inconclusive.Moreover,it is pointed out that ASC may have varying deterrence effects on different types of road users(e.g.,taxis).Objective:This study intends to investigate the distance halo effect of fixed ASC(hereafter called ASC)on taxis.Method:More than 1.34 million taxis’GPS trajectory data were collected.A novel indicator,the delta speed(defined as the difference between the traveling speed and the speed limit),was proposed to continuously describe the variations in traveling speeds.The upstream and downstream critical delta speeds during each time period on weekdays and weekends were obtained by using K-means clustering method,respectively.Based on the critical delta speeds,the ranges of upstream and downstream distance halo effects of ASC during different time periods on weekdays and weekends were determined separately and compared.Results:The downstream critical delta speed is smaller than the upstream one.The upstream and downstream distance halo effects of ASC on taxis are within a range of 8-2180 m and an area of 10-580 m to the ASC location,respectively.There are no obvious difference in the ranges of upstream and downstream distance halo effects of ASC on taxis between different time periods or between weekdays and weekends.Conclusion:The present study confirms that the upstream and downstream distance halo effects of ASC on taxis have different ranges and the stabilities of time-of-day and day-of-week.Practical application:The findings of this study can provide a basic reference for reasonably deploying ASCs within a region.

基于可拓距K-均值聚类和正弦微分进化算法的风储联合系统优化配置

以最大化风储联合投资商的收益和风电就地消纳率为目标,考虑源网荷协同优化和需求响应构建了风储联合系统的多目标优化配置模型;采用可拓距K-均值聚类算法对分布式风电出力和负荷需求的不确定性进行多场景分析,以实现更为准确而均衡的场景缩减;通过引入多核并行运行环境与正弦函数的思想,提出基于并行多目标正弦微分进化算法对优化配置模型进行高效求解;以IEEE 33节点配电系统为算例进行风储联合系统的优化配置,仿真结果验证了所建模型的有效性和优越性。