构建自动演进的天文大数据负载模型

Towards the Automatic Evolution of Workload Models in Large-scale Astronomical Data Management

基准测试在数据管理系统的选型和优化中发挥指导作用的前提是其采用的负载模型能够:运行在目标场景中的各类系统上(移植性);反映目标场景中典型任务的特点和数据访问偏好(代表性)。当前天文大数据管理领域的新系统和新任务层出不穷,导致现有方法构建的负载模型容易失去移植性和代表性。提出了自动演进的负载建模方法:采用抽象操作保持对新型系统的移植性,通过分析负载日志保持对新型任务的代表性。通过一个系统优化案例展示了该方法的可行性。

The benchmark＇s guiding role in system selection/optimization requires its workload model has the ability to： Run on various systems of the target application scenario（be portable）; Reflect the typical tasks＇ characteristics and data access patterns（be representative）. The emerging systems and tasks in large-scale astronomical data management field have led workload models constructed by existing methods to be prone to lose portability and representativeness. An automatic evolutionary workload modeling method has been proposed： Abstract operations are used to keep the workload model＇s portability; Automatic workload log analytics are used to keep the workload model＇s representativeness. The feasibility of this method is verified by a cluster optimization case.