齐普夫定律在中文短语知识中的呈现

Zipf’s Law in Chinese Phrases

本文对大规模汉语短语树库中的短语直接成分进行了抽取，在统计各短语结构直接成分频次的基础上排序，并进行短语成分的齐普夫定律验证。实验由局部及整体两部分组成：在局部实验中，对单一短语直接成分进行齐普夫定律验证；在整体实验中，对全部短语直接成分进行齐普夫定律验证。实验结果表明，在汉语短语树库中，短语的直接成分分布符合齐普夫定律。这个现象表明，从计量角度而言，汉语短语具有其内核知识，并以内核为基础在最省力原则下对不同语境进行派生，这种派生首先要围绕语法内核知识进行构建，同时又要根据不同语境进行拓展。本研究一方面实现了面向短语层次的语法知识抽取，从计量角度揭示了汉语短语构成的分布规律；另一方面对于中文信息处理中的短语识别、自动句法分析等研究提供短语基础知识支持，这些知识最终可应用于术语短语识别、自动标引、知识抽取、语义分析等研究。

The paper extracts immediate constituents of Chinese phrases based on large-scale Chinese treebank, Tsinghua Chinese Treebank and Penn Chinese Treebank. After arranging immediate constituents sequences according to their ranks, we take local and global tests about Zipf＇s Law. In two Chinese treebanks, the phenomenon of Zipf＇s Law is apparently arrested. From the point of view of quantitative linguistics, the paper proves that each of Chinese phrases has a core concept of inter-relationship among various derivations. These derivations should reflect the core concept and then could be extended in different context according to Zipf~ Principle of Least Effort. On the one hand, the discovery in this paper reveals the distribution of Chinese phrase＇s syntactic constituents by extracting syntactical knowledge on phrase level. On the other hand, this discovery will have a strong influence on the researches in Chinese information processing, such as auto-identification of Chinese phrase, automatic syntactic analysis, and so on. The phrase knowledge could be applied toautomatic identification of term phrases, automatic indexing, knowledge extraction, semantic analysis and so on.