新浪微博地震舆情数据库的设计与实现

Design and Implementation of Sina Weibo Earthquake Public Opinion Database

社交媒体地震舆情数据可实时反映灾区现状,为震后应急提供较好的地震灾情舆情研判服务。选取新浪微博作为数据源,对接EQIM,采用新浪微博开放平台API与分布式网络爬虫相结合的方式获取地震舆情数据。在数据清洗和基于fastText地震相关性分类模型进行数据预处理的基础上,应用MySQL关系型数据库,设计并实现我国内地地区准实时新浪微博地震舆情数据库及管理平台。现阶段,该数据库收集了2021年1月1日至2022年3月31日我国内地地区3.0级及以上地震447次、地震舆情数据732949条,实现数据下载、快速查询及精确查询,方便数据查询与管理,有助于震后舆情监控与灾情快速获取。

Social media earthquake public opinion data reflect the current situation of the disaster area in real time,and provide a better service of earthquake disaster public opinion evaluation for post-earthquake emergency response.In this paper,selecting Sina Weibo as the data source and connecting EQIM,the combination of Weibo API interface and distributed web crawler is used to obtain earthquake public opinion data.On the basis of data cleaning and data preprocessing based on fastText seismic correlation classification model,using MySQL relational database,we design and achieve the quasi real-time Sina Weibo earthquake public opinion database and database management platform in China's Mainland area.At present,the database has collected 447 earthquakes with MS3.0 or above in China's Mainland from January 1,2021 to March 31,2022,and 732949 earthquake public opinion data.It lays a foundation for social media earthquake public opinion data mining and analysis research.