基于语义理解力的我国省级政府网站智能问答服务质量评价研究

A Semantic Understanding Oriented Evaluation of the Intelligent Q&A Service on Chinese Provincial Government Websites

[目的/意义]智能问答系统已成为各类网站提供信息咨询服务的重要设施。政务咨询问题的复杂性,对政府网站智能问答系统的语义理解能力提出了更高要求。[方法/过程]文章采用南开大学网络社会治理研究中心开发的“基于语义理解力的政府网站智能客服评价体系”以及配套的测试题集,对我国30个省级政府网站智能问答系统的“问题解决质量”“服务交互质量”“基础建设质量”进行了评测。[结果/结论]上海、浙江、北京总分名列前三;当前政府网站智能问答系统在语义理解及场景化服务方面存在明显不足,仅有30%的系统得分达到及格线以上,部分问答系统在基础功能与服务交互方面存在较大提升空间。最后,文章提出扩展知识库、提升问题匹配精度、增加人文关怀等对策建议。

[Purpose/significance]Intelligent question answering(Q&A)system has become an important facility for websites to provide consulting services.The complexity of government consultation issues poses higher requirements for the semantic understanding ability of intelligent Q&A systems on government websites.[Method/process]This study evaluates the Q&A systems of 30 Chinese provincial government websites from three aspects of problem solving quality,service interaction quality and basic construction quality by using the"Semantic Understanding based Evaluation Indicator System for Intelligent Q&A Service on Government Websites"developed by the Center for Network Society Governance of Nankai University and the supporting test sets.[Result/conclusion]The results show that Shanghai,Zhejiang and Beijing rank in the top three in terms of total scores.The Q&A systems on government websites have significant shortcomings in semantic understanding and scenario based services,with only 30%scoring above the passing line.Some Q&A systems have significant room for improvement in basic functionality and service interaction.Finally,this study proposes a few of suggestions to enhance the semantic understanding ability of government Q&A systems,such as expanding knowledge base,improving the accuracy of problem matching,and increasing humanistic care.