基于Hadoop的蛋鸡设施养殖智能监测管理系统研究

Design of Intelligent Monitoring and Management System Based on Hadoop for Large-scale Layer House

为实现对蛋鸡生产过程中长期积累的海量数据进行高效存储和实时查询,利用Hadoop生态系统,设计了规模化蛋鸡设施养殖智能监测管理系统。针对环境数据的实时监测及大规模数据查询,用My SQL数据库存储近期数据、HBase存储历史数据,有效提升了检索速度;针对海量异构视频数据的统一管理,设计实现了基于MapReduce并行处理框架的分布式转码模块,将1.5 GB的视频分割为多个128 MB分段后进行转码,转码效率提高了50%。该系统实现了规模化蛋鸡场生产养殖中对实时信息、历史信息、基础设施信息、生产过程信息的统一管理,并提供了统计分析模块对采集获取的数据进行整合分析,开发了Web端网页版本及移动端APP版本的智能监测管理系统,便于用户进行实时访问,提高了生产养殖的工作效率。

With the appearance and continuous development of the Internet of things,the monitoring data grows explosively. Accordingly,traditional data storage and processing can not meet the requirements. In order to store data effectively and query data in real time,intelligent monitoring and management system based on Hadoop for large-scale layer house was developed. The HDFS file system and HBase database in the Hadoop ecosystem can store massive data distributed. The environmental monitoring data had the characteristics of once writing and multiple queries. In order to realize the real-time monitoring and largescale data query for environmental data,My SQL database was used to store recent data and HBase database was used to store historical data. Experiments indicated that the query speed was improved effectively. For the unified management of massive heterogeneous video data,the distributed transcoding of video was designed and implemented. Experimental results showed that the proposed scheme can increase about 50% of the transcoding efficiency when the video size was 1. 5 GB and the segment size was 128 MB. The system realized real-time information display, historical information query,infrastructure management,production process management and statistical data analysis,environmental alerts and system management in production and breeding of large-scale layer house,which can be accessed through web pages and mobile APP by users in real time. The actual application showed that the system helped managers to control the production process on all aspects and improve the efficiency of the production personnel.