基于AI分析的本地网情聚合系统设计与实现

Design and Implementation of Local Network Information Aggregation System Based on AI Analysis

随着移动互联网的深度普及,普通用户从信息的接收者成为网络内容的创造者和传播者。一条域内的网情信息通过大众的关注、评论、转发,可以快速地触达全网,从而产生广泛的社会影响。利用Spark on YARN对爬虫采集的多平台信息进行大数据清洗,结合AI自然语言处理技术,对情感、行业分类、热词、热门话题等指标进行分析,通过ECharts大屏展示技术对各类指标进行直观展示,并基于用户配置的地域行业兴趣点给予消息推送,从而为地方政府提供直观的网情监督手段,进而为其智慧政务的建设贡献力量。

With the deep popularization of mobile Internet, the general user has turned into the creator and disseminator of network content from the receiver of information. The network information in a domain can quickly reach the whole network through the public’s noticing, comments and forwarding, causing a wide social impact. This paper uses Spark on YARN to clean the big data of the multi platform information collected by the crawler, and combined with AI natural language processing technology to analyze the indicators such as emotion, industry classification, hot words and hot topics. Then, the analysis indicators are visually displayed through ECharts technology, and messages are pushed based on the regional industry interest points configured by users, so as to provide intuitive means of online situation supervision for local governments, and to contribute to intelligent governance.