基于分布式流式计算的蛋鸡养殖实时监测与预警系统

Real-time Monitoring and Early Warning System Based on Stream Computing for Laying Hens Raise

为实现蛋鸡养殖生产过程参数实时监测与预警,研发了基于分布式流式计算框架Data-Canal的蛋鸡养殖实时监测与预警系统。Data-Canal是面向数据流的分布式计算框架,使用控制流集中、数据流分散的模型,以分布式文件系统为中间结果的存储,支持异地多数据源的实时采集和处理。系统以Data-Canal为基础设施,在具有一定扩展性的情况下,保证实时性。系统采用Brower/Server模式,用户通过浏览器即可访问,提升了信息共享的便捷性。系统实现了规模化蛋鸡生产过程实时数据采集与展示、生产信息管理、实时预警、决策分析和系统管理功能,对蛋鸡养殖全生命周期进行了全方位的管理。运行效果表明,该系统可以解决规模化蛋鸡生产过程中产生海量数据信息化和实时处理问题,在部署8台机器的情况下,Data-Canal集群的处理能力峰值达到160 MB/s,延迟在分钟级别,在线上实验环境中,Data-Canal集群每天处理约25 GB的数据,而且系统后期维护和升级都极为便利。

With the rapid development of computer technology,it＇s possible to process multi-type and mass data real-timely. In order to achieve real-time monitoring and early warning in laying hen raise,a system based on a distributed streaming computing framework ＂Data-Canal＂was developed. Data-Canal is a data flow oriented distributed computing framework with the control flow and data flow dispersion model,which using the distributed file system as the storage of intermediate result,supporting real-time acquisition and processing multiple remote data sources. Data-Canal is the basic facility of the system,which ensures the extend and real-time processing of system. The system was developed in Browser /Server mode. The users can access the system through the browser,which improves the convenience of informatization sharing. The system realizes real-time data acquisition and display,production information management,early warning,decision analysis,and system management functions. The result shows that the system solves the problem of information and real-time processing of mass data in laying hens raise. In the case of eight machines,the highest throughput of Data-Canal cluster reaches 160 MB / s,and the delay is at the minute level. In the online experiment,the Data-Canal cluster processeed about 25 GB of data every day. The maintenance and upgrade of system are convenient. Further research will be done on the mobile client.