基于统计学习的自然语言对话系统的设计与实现

Design and Implementation of a Natural Language Chat System Based on Statistical Learning

通过对大规模自然语言语料库的学习和优化多阶Markov链实现了语料的存储和语言信息统计,基于此我们建立了一个语言统计模型。通过该模型可进行答案文本的自动组织,并通过关键词扩展及改进的TF-IDF算法实现了对对话的主题识别与跟踪;采用基于句子相似度的计算模型进行问答式对话的答案筛选。实现了桌面和在线版本的两个人机对话平台。

Asan experiment in natural language processing,via learning from huge corpus based on the optimized multi -order Markov models, we built a language statistical model. This model can automatically create the answer - text, and can achieve topic detection and track based on the extended keywords and an optimized TF - IDF algorithm. A chat system, with a desktop version and an on - line version, was respectively implemented based on a free text in an Open - Domain.