Spark集群下基于关联规则的产品加工质量分析

Product Processing Quality Analysis Using Association Rules in Spark Cluster

工业大数据的有效应用成为支撑企业转型升级的强力引擎。利用关联规则可以从海量产品加工过程质量数据中发掘加工工序质量参数之间的隐含相关性,如果能对具有相关性的上游质量指标进行及时有效的监控和调整,将有效地减少质量指标之间的误差传播、耦合与积累对各工序质量监控点配置决策的影响。针对传统的关联规则挖掘时空复杂度和I/O代价高,难以适应大数据处理的问题,以及传统Hadoop分布式平台的计算和数据管理方式导致的高数据频繁读写代价,提出了一种基于Spark计算平台的并行频繁项集挖掘HBPFP(High Balanced parallel Fp-Growth)算法,该算法利用新定义的节点计算量预估模型,实现了各计算节点间计算任务的均衡分组,从而有效地提高了集群资源利用率;并在详细分析产品工艺流程和质量管理特点的基础上,构建了基于关联规则的加工质量控制模型;最后以冷轧辊生产加工过程为应用案例,设计与实现了基于关联规则的冷轧辊加工质量分析原型系统,为制造企业在大数据时代实现全面质量控制和管理,提供了新的解决途径和决策支持手段。

The effective application of industrial big data has become a powerful engine to support the transformation and upgrading of enterprises.Association rules can be deployed to discover implied rules among the quality parameters from the accumulated product processing quality data.If the relevant upstream quality indicators can be monitored and adjusted in a timely and effective manner,the impact of error propagation,coupling,and accumulation among quality indicators on the configuration decisions for process quality monitoring points will be effectively reduced.The time and space complexity and I/O cost for traditional association rules mining are difficult to adapt to the processing of big data,and the calculation and data management methods of Hadoop distributed platform will lead to frequent reading and writing overhead.Addressing to these issues a parallel frequent itemsets mining algorithm HBPFP(High Balanced parallel FP-Growth)based on Spark computing platform is proposed.The algorithm uses the newly defined node calculation estimation model to achieve the balanced grouping of computing tasks among computing nodes,Therefore,the resource utilization rate is improved.Next,a process quality control model with a detailed analysis of product process flow and quality management characteristic is constructed.Finally,taking the production of cold roll products from a cold rolling mill branch of an iron and steel company as a case study,a cold forged roll quality management prototype system is developed.The study offers scientific and technical underpinnings to optimize quality control and management for manufacturing enterprises,and provides crucial decision support for end users.