摘要
短文本信息流在传递公开信息时携带了丰富且具有极大价值的信息资源。根据短文本信息流特点,利用训练数据集中的信息熵来构建决策树检测模型进行热点话题检测,该方法先是计算出各话题类别的平均信息量和每个特征词对于短文本信息流进行划分的信息增益率,再通过选择具有最大信息增益率的特征词进行测试,完成自上而下的决策树建树过程,最后利用叶子结点的类型确定热点话题。在真实短信文本信息流上实验表明,该方法具有明显的检测稳定性和较高的数据处理效率。
Potential information with high value are carried by short text information flow in transmission.A model of decision tree for hot topic is established with the information entropy of training data set,according to the characteristics of short text information flow.The average amount of information of each topic categories and the information gain ratio of each characteristic word for distinguishing short text information flow are computed in the first step by the above algorithm of decision tree.Then,the characteristic word with maximum information gain ratio is selected for the job of test,while the top-down construction process of the decision tree is accomplished.Finally,the hot topic is determined according to the leaf node type.The experiment result on real short text information flow shows that the proposed algorithm is more stable and faster than others.
出处
《数据采集与处理》
CSCD
北大核心
2015年第2期464-468,共5页
Journal of Data Acquisition and Processing
基金
国家级星火计划"农村民生建设信息反馈平台建设"(2011GA690190)资助项目
关键词
短文本
信息流
热点话题
决策树
short text
information flow
hot topic
decision tree
