基于大数据技术精细化社会治理系统研究

Research on Refined Social Governance System Based on Big Data Technology

为解决常用系统管理大数据时存储器容量较小,不能满足大数据存储需求,导致系统运行的并发性和可扩展性较差的问题,设计了基于大数据技术的精细化社会治理系统。硬件设计方面,采取分块模式设计系统总体结构,通过接口板和存储板组成大数据存储器,选取Virtex-4 FPGA芯片作为存储板内部芯片,优化存储板PCB布局,扩增存储器容量,满足大数据大容量存储需求;软件设计方面,通过网络爬虫采集社会治理相关大数据,预处理大数据并对其进行分类,得到精细化数据项,为社会治理工作提供数据支持。设置对比实验,结果表明,设计系统执行大数据写入/读取操作时,提高了并行加速度,扩展比更接近1,并发性和可扩展性优于常用系统。

In order to solve the problem that the memory capacity of common systems managing big data is small and can not meet the storage requirements of big data,resulting in poor concurrency and scalability of system operation,a refined social governance system based on big data technology is designed.In terms of hardware design,the overall structure of the system is designed in block mode.The big data memory is composed of interface board and memory board.Virtex-4 FPGA chip is selected as the internal chip of the memory board to optimize the PCB layout of the memory board and expand the memory capacity to meet the demand of big data storage;in terms of software design,through the web crawler to collect social governance related big data,preprocess big data and classify it,get fine data items,provide data support for social governance work.A comparative experiment was set up,and the results showed that when the design system performs big data write/read operations,the parallel acceleration is improved,the expansion ratio is closer to 1,and the concurrency and scalability are better than common systems.