巧用黑盒法逆推百度中文分词算法

Inferring Baidu's Chinese Word Segmentation Algorithm by Supposing a Black Box

为了更好地利用中文搜索引擎系统,需要了解搜索引擎的一些关键性技术。笔者选择了国内知名度最高的搜索引擎系统百度作为研究对象。介绍了几种常用的中文分词算法以及中文分词在搜索引擎中的作用,并在此基础上采用黑盒的方法推导出百度的中文分词算法——正向最大匹配分词算法。该算法是最常见的一种中文分词算法,比较符合人的思维习惯。了解了搜索引擎的分词算法有益于搜索词设置的优化以及搜索引擎系统的研究和改进。

In order to make better use of Chinese search engine systems, key technologies about search engine are required. Chooses Baidu, the best - known search engine system in China, as the study object. Several commonly- used Chinese word segmentation algo- rithms and their functions in search engines are introduced. On this basis, the writers suppose a black box and infer Baidu＇ s Chinese word segmentation algorithm--Maximum Matching Method which is the most common method and similar to people＇s thinking mode. Understanding of segmentation algorithm in search engine algorithms is conducive to the setting of search terms as well as the further research and improvement on the search engine system.