基于大数据技术的新闻采集和事件分析系统的设计与实现

Design and Implementation of News Acquisition and Event Analysis System Based on Big Data Technology

为了解决获取互联网新闻时查找慢、数据杂、阅读效率低的问题,本文提出了一种基于大数据技术的新闻采集和事件分析系统。该系统首先采集互联网新闻数据,运用数据清洗、分布式存储等技术存到数据库;接着对数据进行分析,运用机器学习等技术,进而得到新闻主主题;然后新闻进行聚合,获取到事件主题;最后根据事件主体对所有时间进行展示,实现了对于互联网新闻的采集与事件分析的全过程。该系统提高了获取互联网新闻的效率,并解决了数据杂乱和阅读效率低的问题。

In order to solve the problems of slow search,miscellaneous data and low reading efficiency in obtaining Internet news,this paper proposes a news acquisition and event analysis system based on big data technology.The system first collects Internet news data and stores it in the database by using data cleaning and distributed storage technology;then it analyses the data and uses machine learning technology to get the main theme of news;then it aggregates the news to get the event theme;finally,it exhibits all the time according to the subject of the event.It implements the whole process of collecting and analyzing Internet news.The system improves the efficiency of obtaining Internet news,and solves the problems of data clutter and low reading efficiency.