用Shannon熵度量两个数据集的一致性

Measuring Consistency of Two Datasets Using Shannon Entropy

粗糙集理论的基本思想是根据已知数据自身的不可分辨关系,通过一对近似算子,对某一给定概念进行近似表示。这种思想被应用在研究一个数据集对于另一个数据集的分类一致性上。提出了一种测量两个数据集一致性的新方法,并用Shannon熵定义了分类一致性。考虑到不同数据临近关系的影响,引入了模糊概念将测量对象由清晰分类转化为模糊分类,进而构造了一个广义的一致性度量,这种方法可以产生稳定的可判结果,有效地阻止建模技术中常出现的"黑箱"现象。

The basic idea of rough set theory is based on an indiscernibility relation,and through a pair of approximate operators,it can approximatively represent a given concept.It is used in the study of a data set for classification consistency to another data set.This paper presented a new approach to measure consistency degree of two datasets,and defined classification consistency by Shannon entropy.Taking the influence of neighborhood relations of different data into account,ageneral consistency measure was defined by introducing the expert knowledge into a fuzzy inference system,then we constructed a consistent generalized metric.Moreover,this method can prevent the＂black box ＂phenomenon encountered in many modeling techniques and produce robust and interpretable results.