基于信息论的连续属性离散化

Discretization of Continuous-valued Attributes Using Information Theory

使用信息论的方法进行连续属性的离散化。引入Ｈｅｌｌｉｎｇｅｒ偏差ＨＤ （Ｈｅｌｌｉｎｇｅｒ Ｄｉ－ｖｅｒｇｅｎｃｅ）作为每个区间对决策的信息量度量，从而定义切分点的信息熵，最终的离散化结果是使各区间的信息量尽可能平均。分析了ＨＤ度量在两种离散化方法中的作用，说明它在划分算法中运用比较理想，而在归并算法中则有局限。

We adopt the method of information theory in the discretization of continuous numerical va- lues. We introduce Hellinger divergence as the measure of amount of information that each potential interval gives to the decision attributes. Then the entropy of cutpoint is defined. Our aim is to discretize numeric values so that the information content of each interval is as equal as possible. We analyze the act of Hellinger divergence in both discretization algorithms of merging and splitting, and draw a conclusion that it is a fairly ideal measure in the latter and has some limitations in the former.