Hidden Markov Model（HMM） is a main solution to ambiguities in Chinese segmentation anti POS （part-of-speech） tagging. While most previous works tot HMM-based Chinese segmentation anti POS tagging eonsult POS infor...Hidden Markov Model（HMM） is a main solution to ambiguities in Chinese segmentation anti POS （part-of-speech） tagging. While most previous works tot HMM-based Chinese segmentation anti POS tagging eonsult POS informatiou in contexts, they do not utilize lexieal information which is crucial for resoMng certain morphologieal ambiguity. This paper proposes a method which incorporates lexieal information and wider context information into HMM. Model induction anti related smoothing technique are presented in detail. Experiments indicate that this technique improves the segmentation and tagging accuracy by nearly 1%.展开更多
In this paper, we present a modular incremental statistical model for English full parsing. Unlike other full parsing approaches in which the analysis of the sentence is a uniform process, our model separates the full...In this paper, we present a modular incremental statistical model for English full parsing. Unlike other full parsing approaches in which the analysis of the sentence is a uniform process, our model separates the full parsing into shallow parsing and sentence skeleton parsing. In shallow parsing, we finish POS tagging, Base NP identification, prepositional phrase attachment and subordinate clause identification. In skeleton parsing, we use a layered feature-oriented statistical method. Modularity possesses the advantage of solving different problems in parsing with corresponding mechanisms. Feature-oriented rule is able to express the complex lingual phenomena at the key point if needed. Evaluated on Penn Treebank corpus, we obtained 89.2% precision and 89.8% recall.展开更多
基金国家高技术研究发展计划(863计划)，the National Natural Science Foundation of China
文摘Hidden Markov Model（HMM） is a main solution to ambiguities in Chinese segmentation anti POS （part-of-speech） tagging. While most previous works tot HMM-based Chinese segmentation anti POS tagging eonsult POS informatiou in contexts, they do not utilize lexieal information which is crucial for resoMng certain morphologieal ambiguity. This paper proposes a method which incorporates lexieal information and wider context information into HMM. Model induction anti related smoothing technique are presented in detail. Experiments indicate that this technique improves the segmentation and tagging accuracy by nearly 1%.
文摘In this paper, we present a modular incremental statistical model for English full parsing. Unlike other full parsing approaches in which the analysis of the sentence is a uniform process, our model separates the full parsing into shallow parsing and sentence skeleton parsing. In shallow parsing, we finish POS tagging, Base NP identification, prepositional phrase attachment and subordinate clause identification. In skeleton parsing, we use a layered feature-oriented statistical method. Modularity possesses the advantage of solving different problems in parsing with corresponding mechanisms. Feature-oriented rule is able to express the complex lingual phenomena at the key point if needed. Evaluated on Penn Treebank corpus, we obtained 89.2% precision and 89.8% recall.
