This paper applied Maximum Entropy (ME) model to Pinyin-To-Character (PTC) conversion in-stead of Hidden Markov Model (HMM) that could not include complicated and long-distance lexical informa-tion. Two ME models were...This paper applied Maximum Entropy (ME) model to Pinyin-To-Character (PTC) conversion in-stead of Hidden Markov Model (HMM) that could not include complicated and long-distance lexical informa-tion. Two ME models were built based on simple and complex templates respectively, and the complex one gave better conversion result. Furthermore, conversion trigger pair of y A → y B cBwas proposed to extract the long-distance constrain feature from the corpus; and then Average Mutual Information (AMI) was used to se-lect conversion trigger pair features which were added to the ME model. The experiment shows that conver-sion error of the ME with conversion trigger pairs is reduced by 4% on a small training corpus, comparing with HMM smoothed by absolute smoothing.展开更多
基金Supported by the National Natural Science Foundation of China as key program (No.60435020) and The HighTechnology Research and Development Programme of China (2002AA117010-09).
文摘This paper applied Maximum Entropy (ME) model to Pinyin-To-Character (PTC) conversion in-stead of Hidden Markov Model (HMM) that could not include complicated and long-distance lexical informa-tion. Two ME models were built based on simple and complex templates respectively, and the complex one gave better conversion result. Furthermore, conversion trigger pair of y A → y B cBwas proposed to extract the long-distance constrain feature from the corpus; and then Average Mutual Information (AMI) was used to se-lect conversion trigger pair features which were added to the ME model. The experiment shows that conver-sion error of the ME with conversion trigger pairs is reduced by 4% on a small training corpus, comparing with HMM smoothed by absolute smoothing.
