关于汉语语言处理的若干理论思考

On Chinese Language Processing

汉语语法检查是汉语语言处理发展中的瓶颈,涉及汉语语言处理的各个方面,与汉字拼写检查、文本切分、词性标注、汉字输入、汉字编码等问题紧密相联。语义理解是语法检查及自然语言处理的制高点。新的发展趋势是研究者越来越注重语义在语言结构和语言表达上的制约作用,试图用统计大规模语料为手段来攻克难关。这种发展趋势对自然语言处理的冲击不在于研究命题的转移,而在于研究方法和论证手段的量化。在汉语的各种制约关系中,可能有一种超越已知句法语义关系的认知心理语法,它与特定历史时期内人们对物质世界和客观社会文化关系的理解相一致,有可能是自然语言处理的最后一个堡垒。

No matter how Chinese language processing (CLP) is investigated, Chinese grammar checker has been the bottleneck of all the components and phases of CLP: from character coding, word indexing, text segmenting,... all the way to machine translating and interpreting. The more investigators study the Chinese language, the more they realize the importance of meaning-constraints in the language. The new trend seems to focus on finding rules and patterns through statistics drawn from large corpuses. Above all the recognized syntactic and semantic rules or patterns, as well as theories, there seem to be some cognitive and psychological constraints that go beyond the understanding of traditional Chinese grammar, and that may become the last fortress to be conquered in CLP.