基于网络社区结构的训练集非均衡程度度量方法

Approach to Evaluate the Imbalanced Degree of the Training DataSets Using Community Structure

在机器学习和数据挖掘实际应用中,针对分类训练集的选取,通常要求训练集中每一类所包含的数据在数量上要尽可能的"均衡".本文以非均衡训练集与分类学习效率关系研究为依据,给出了"均匀度"和"内聚度"两种类型的训练集非均衡程度因素的概念;"均匀度"是用来描述训练集类之间(between-class)的非均衡程度,其含义是指训练集不同类之间数据数量的非均衡程度;"内聚度"是用来描述训练集类内部(within-class)的非均衡程度,指训练集中不同类在空间分布上的线性相关程度,通过训练集数据之间的相关程度,构建出训练集的网络结构,运用一种能体现训练集内聚性的网络拓扑结构的指标-网络社区结构作为度量,提出了基于网络社区模块结构的非均衡训练集度量方法,并指出了高均匀度和高内聚度是选取"优良"分类训练集的关键因素.通过对UCI标准训练集的实验,结果验证本方法作为选取训练集标准的有效性.

In present application of machine learning and data mining , the criterion to choose the right training datasets is the assumption that the number of data in different class is the only fact of the degree of class imbalance. Based on empirical studying the relationship between class imbalance and learning algorithms, in this paper, equality and cohesion of an imbalanced dataset which are the two important facts of the degree of class imbalance are proposed, equality is the between-class imbalance which means the number of data in different class, cohesion is the within-class imbalances whicht means that the distribution of the data within each class is also relevant. A new approach using high equality and high Cohesion is proposed to evaluate the degree of class imbalance. A new approach using the class distribution is proposed to evaluate the degree of class imbalance, the main idea is based on the community structure of data set which is a very valuable and crucial to understand the class distribution structure. New approach can help us to choose training data in real-world situations. by experiment study on UCI datasets, the newly approach is proved reasonable and viable.