Hypergraph-Based Data Reduced Scheduling Policy for Data-Intensive Workflow in Clouds

Data-intensive computing is expected to be the next-generation IT computing paradigm. Data-intensive workflows in clouds are becoming more and more popular. How to schedule data-intensive workflow efficiently has become the key issue. In this paper, first, we build a directed hypergraph model for data-intensive workflow, since Hypergraphs can more accurately model communication volume and better represent asymmetric problems, and the cut metric of hypergraphs is well suited for minimizing the total volume of communication.Second, we propose a concept data supportive ability to help the presentation of data-intensive workflow application and provide the merge operation details considering the data supportive ability. Third, we present an optimized hypergraph multi-level partitioning algorithm. Finally we bring a data reduced scheduling policy HEFT-P for data-intensive workflow. Through simulation,we compare HEFT-P with three typical workflow scheduling policies.The results indicate that HEFT-P could obtain reduced data scheduling and reduce the makespan of executing data-intensive

An Energy-Aware Heuristic Scheduling for Data-Intensive Workflows in Virtualized Datacenters

With the development of cloud computing, more and more data-intensive workflows have been deployed on virtualized datacenters. As a result, the energy spent on massive data accessing grows rapidly. In this paper, an energy-aware scheduling algorithm is proposed, which introduces a novel heuristic called Minimal Data-Accessing Energy Path for scheduling data-intensive workflows aiming to reduce the energy consumption of intensive data accessing. Extensive experiments based on both synthetical and real workloads are conducted to investigate the effectiveness and performance of the proposed scheduling approach. The experimental results show that the proposed heuristic scheduling can significantly reduce the energy consumption of storing/retrieving intermediate data generated during the execution of data-intensive workflow. In addition, it exhibits better robustness than existing algorithms when cloud systems are in presence of I/O- intensive workloads.

Modeling and Analysis of Data Dependencies in Business Process for Data-Intensive Services

With the growing popularity of data-intensive services on the Internet, the traditional process-centric model for business process meets challenges due to the lack of abilities to describe data semantics and dependencies, resulting in the inflexibility of the design and implement for the processes. This paper proposes a novel data-aware business process model which is able to describe both explicit control flow and implicit data flow. Data model with dependencies which are formulated by Linear-time Temporal Logic(LTL) is presented, and their satisfiability is validated by an automaton-based model checking algorithm. Data dependencies are fully considered in modeling phase, which helps to improve the efficiency and reliability of programming during developing phase. Finally, a prototype system based on j BPM for data-aware workflow is designed using such model, and has been deployed to Beijing Kingfore heating management system to validate the flexibility, efficacy and convenience of our approach for massive coding and large-scale system management in reality.

Adaptive resource allocation for workflow containerization on Kubernetes

In a cloud-native era,the Kubernetes-based workflow engine enables workflow containerized execution through the inherent abilities of Kubernetes.However,when encountering continuous workflow requests and unexpected resource request spikes,the engine is limited to the current workflow load information for resource allocation,which lacks the agility and predictability of resource allocation,resulting in over and underprovisioning resources.This mechanism seriously hinders workflow execution efficiency and leads to high resource waste.To overcome these drawbacks,we propose an adaptive resource allocation scheme named adaptive resource allocation scheme(ARAS)for the Kubernetes-based workflow engines.Considering potential future workflow task requests within the current task pod’s lifecycle,the ARAS uses a resource scaling strategy to allocate resources in response to high-concurrency workflow scenarios.The ARAS offers resource discovery,resource evaluation,and allocation functionalities and serves as a key component for our tailored workflow engine(KubeAdaptor).By integrating the ARAS into KubeAdaptor for workflow containerized execution,we demonstrate the practical abilities of KubeAdaptor and the advantages of our ARAS.Compared with the baseline algorithm,experimental evaluation under three distinct workflow arrival patterns shows that ARAS gains time-saving of 9.8% to 40.92% in the average total duration of all workflows,time-saving of 26.4% to 79.86% in the average duration of individual workflow,and an increase of 1% to 16% in centrol processing unit(CPU)and memory resource usage rate.

Splitting and placement of data-intensive applications with machine learning for power system in cloud computing

Aiming to meet the growing demand for observation and analysis in power systems that based on Internet of Things(IoT),machine learning technology has been adopted to deal with the data-intensive power electronics applications in IoT.By feeding previous power electronic data into the learning model,accurate information is drawn,and the quality of IoT-based power services is improved.Generally,the data-intensive electronic applications with machine learning are split into numerous data/control constrained tasks by workflow technology.The efficient execution of this data-intensive Power Workflow(PW)needs massive computing resources,which are available in the cloud infrastructure.Nevertheless,the execution efficiency of PW decreases due to inappropriate sub-task and data placement.In addition,the power consumption explodes due to massive data acquisition.To address these challenges,a PW placement method named PWP is devised.Specifically,the Non-dominated Sorting Differential Evolution(NSDE)is used to generate placement strategies.The simulation experiments show that PWP achieves the best trade-off among data acquisition time,power consumption,load distribution and privacy preservation,confirming that PWP is effective for the placement problem.

FMS-Workflow Modeling Based on P-Timed Stochastic Petri Net

In this paper, we propose astochastic Petri net model P-timed Workflow (WPTSPN) to specify, verify, and analyze a business process (BP) of a Flexible Manufacturing System (FMS). After formalizing the semantics of our model, we illustrate how to verifysome of its properties (reachability, safety, boundedness, liveness, correctness, alive tokens, and security) in the P-Timed context. Next, we validate the relevance of the proposed model with MATLAB simulation through a specific FMS case study. Finally, we use a generalized truncated density function to predict the duration of a token’s sojourn (residence) in a timed place with respect to the sequence states of the global FMS workflow.

基于工作流的无服务边缘计算框架设计与实现

边缘计算具有高性能、体积小、计算力超强等优点。但云设计的边缘计算函数链存在调用时延较长的问题,导致大规模部署和存储能力以及云端和设备端协同困难、无缝对接效果较差。为解决上述问题,基于开源无服务平台OpenFaaS设计了一个无服务边缘计算框架(SECP),为边缘侧提供基于无状态函数的计算服务。对计算服务调用过程采用建模分析,采用了工作流模型对为云设计的无服务计算在边缘侧存在的函数链式调用时延放大的问题改进。性能测试结果表明,SECP在边缘计算场景下有良好的性能表现,降低了边缘计算平台中服务请求的响应时间,解决了无服务计算函数功能单一问题,提高了边缘侧资源的利用率,并改善了边缘侧计算服务部署、维护繁杂的问题。

高校图书馆手稿数字化与在线揭示的实践研究——以华东师范大学图书馆手稿数据库建设为例

国内高校图书馆收藏了大量的纸质手稿,但由于缺少数字化、长期保存以及在线揭示的整体解决方案,无法有效发挥其价值。本研究在手稿数据库需求分析的基础上,以华东师范大学手稿数据库建设为例,利用开源软件Goobi实现分布式手稿加工与发布,借助人工智能技术对手稿进行深度挖掘,基于IIIF的底层架构使特藏库可以整合国际知名文化遗产机构发布的手稿资源到本地。根据手稿资源的特点,构建手稿在线阅读图像工具、注释转录、手写识别、特征识别、自动标签等功能,提升用户体验。未来还将不断探索名人角色扮演参考咨询、多光谱三维手稿浏览以及手稿鉴定等功能,推动国内高校图书馆手稿数据库的建设。

时间约束下的云工作流动态混合资源调度

随着云计算的出现和云基础设施的快速部署,越来越多的大型工作流应用正在积极向云迁移.同时,如何在满足任务时间约束的前提下优化执行成本,提高资源利用率成为新的挑战.本文提出一种云工作流动态混合资源调度算法DHRS,不仅满足任务的时间约束而且在混合资源租用时取得较低的成本.首先,根据任务的优先级关系对任务进行预处理,基于概率升序对任务进行排序,并为子任务分配子截止日期;然后,依据顺序为工作流选择满足截止日期且成本较低的服务;最后,对每个服务动态选择预留资源和按需资源,基于预留资源的空闲时间段调度,进一步降低成本.在随机生成的不同的科学工作流上进行实验,并通过与现有算法对比,DHRS在满足时间约束并且降低执行成本方面具有一定的优势.

古建筑保护中色彩采集方法及流程设计的应用研究——以岳麓书院为例

建筑色彩数据库的建立,是古建筑保护工作深化的重要技术节点,其技术流程可包括色彩体系的选择、取色仪器的匹配及色彩样本的误差处理。当前古建筑保护存在着色彩原真意识淡薄、数据资料缺乏以及采集规范不全等问题,古建筑色彩保护严谨性与科学性亟待提高。通过梳理色彩数据采集流程以及方法,初步建立了一个系统、明确、可执行的工作模式,为古建筑色彩采集工作提供参考;并规范色彩数据采集工作的流程设计以及数据定量纠偏方法,为古建筑色彩定量研究提供支持。实践结合岳麓书院古建筑群色谱研究,致力于解决建筑古迹色彩采集所涉及的样本选择、高度障碍、材质不均、光强条件、湿度差异、色温影响等重难点问题,为岳麓书院古建筑群后续色彩研究奠定数据基础,是古建筑色彩保护中色彩采集技术路线的有效支撑。

面向工业互联网数据分析的机器学习工作流推荐方法

工业大数据具有多模态和强关联等特性,这给数据分析与应用带来了新的挑战。如何根据工业应用需求的特点实施有效的数据分析过程通常是一项非常复杂、耗时耗力的任务。针对该问题,提出一种面向工业互联网数据分析的机器学习工作流推荐方法。该方法以已有解决方案为起点,将其所使用的数据集和机器学习工作流作为推荐参考,基于Doc2vec模型与最大平均差异方法计算文本描述相似度与数据分布特征相似度,可根据当前数据分析任务需求,推荐合适的已有解决方案中的机器学习工作流。仿真实验说明了该方法的有效性。

多源遥感卫星地面调度系统设计与实现

针对卫星地面调度处理系统进行研究与设计,着重关注工作流、线程池和Redis缓存等关键技术。首先,对工作流的分析和建模以提高系统任务的执行效率和资源利用率。其次,引入线程池技术来管理和调度系统中的线程资源,实现任务的并发执行和负载均衡,提高系统的吞吐量和响应速度。最后,利用Redis缓存作为系统的缓存和Kafka消息队列,优化数据存取和通信效率,提供更高的性能和可靠性。整体通过SpringBoot框架进行实验和性能评估,验证了该系统在任务处理能力、资源利用率和数据访问效率等方面的优势。本研究对于提高卫星地面调度处理系统的效率和性能具有一定的实践意义和推广价值。

执行时间预测驱动的工作流作业调度

针对工作流作业调度问题,提出使用关键路径法进行工作流的执行时间预测和资源分配。工作流执行时间预测算法使用并行应用有向无环图描述工作流中子作业的执行顺序。基于此顺序,为子作业进行系统资源的逻辑分配。根据子作业的特征和资源分配信息,使用梯度提升决策树进行子作业执行时间预测,并计算工作流的关键路径。关键路径上所有子作业的完成时间之和即为工作流的执行时间。若预测的工作流执行时间满足用户要求,则根据子作业执行顺序和资源分配方案进行作业调度,执行工作流。对比实验表明,两个工作流的执行时间预测误差分别为5.72%和1.57%。与Spark默认调度算法相比,工作流调度算法将两个工作流的完成时间分别缩短了15.71%和15.44%。

基于Kubernetes的工业AI项目开发平台

随着人工智能(AI)成为改变世界的前沿技术,企业也迫切需要系统地开发和实现人工智能来升级新一代工业系统的基础。考虑到将人工智能应用开发集成到工业开发中存在开发流程环节多、AI项目开发人员能力背景不一等问题,以企业实际需求为背景设计实现了一个AI项目开发平台。探讨了人工智能项目开发平台的概念和实现,并且基于企业实际需求对其进行定制化。在传统开发流程基础上完善更多开发工作流环节。在企业内部成功进行部署和测试。结果表明:项目开发平台的功能合理、运行正常、管理便捷对AI项目开发效率、体验、规范化有积极影响,提高了计算加速部件利用率。

高性能CAE软件性能评测平台的设计与应用

高性能计算机辅助工程(CAE)软件在复杂多物理过程仿真、大规模高逼真度数值模拟中发挥重要作用。高性能CAE软件的性能评测是衡量它性能指标的重要手段。传统手工基于算例评测的方式难以适应批量软件大量算例的并发评测,数据采集、性能分析等各阶段的数据管理、资源调度、性能指标计算以及评测报告撰写等活动脱节,制约高性能CAE软件的研发效率和评估效率。从高性能CAE软件研发模式和性能评测的需求出发,提出性能评测的数据模型和执行模型,基于科学工作流技术实现多软件、多算例、多规模的三层嵌套并发评测,设计并实现了高性能CAE软件性能评测平台,平台采用松耦合的可执行组件封装技术,提供软件和算例等信息管理、超算资源无缝对接、评测流程自动化执行和评测报告自动生成功能,在实际项目中采用该平台完成了数十个软件上百算例的并发评测。

代建工程项目管理冲突及化解对策研究——以高校实验楼代建项目为例

长期以来代建制在实验楼等复杂个性项目上施行时,存在“冲突频发并需政府干预”的突出问题,代建制要求和现有流程之间的矛盾,使得项目建设中的工作冲突容易演变为利益冲突,妨碍了代建项目有效推进。文章在解析管理冲突及其原因的基础上,尝试对相关工作流程进行合理修正,从而摆脱“冲突解决依赖政府干预”的境况,以期为代建制的深入推广提供参考。

数字化颌位关系记录与转移方法的研究进展

近年来,口腔数字化技术的日益成熟开启了全口种植修复的新时代。颌位关系的正确记录和转移是全口义齿修复成功的先决条件之一,相较于传统技术,数字化技术的应用可缩短患者的诊疗周期,提高诊疗效率。本文阐述了数字化技术在记录和转移无牙颌患者颌位关系时的应用和进展,深入探讨口腔数字化为临床医生带来的机遇与挑战,以期为临床医生在全口数字化技术的应用方面提供参考。

基于KNIME工作流机器学习预测室分天线外打故障

室内分布系统天线外打故障因难以快速准确定位,对用户体验和网络运维带来了严重挑战。基于KNIME工作流的机器学习算法预测模型,提出了一种新方法。这种方法能够及时发现并提前解决室内分布系统天线外打故障,实现了从“事后发现”到“事先预测”以及从“全面排查”到“精准定位”的转变,确保室外分布系统天线故障的全面精准定位。

前列腺癌患者放射性肠炎预防护理流程改进的行动研究

目的基于行动研究法改进前列腺癌患者放射性肠炎预防护理流程,并探讨其有效性。方法便利选取112例前列腺癌拟行根治性放疗患者,按时间段分为对照组和行动研究组。对照组采用前列腺癌放射性肠炎优化护理;行动研究组采用行动研究法,通过计划-行动-观察-反思螺旋式循环,继续改进护理流程,共开展2个循环。比较对照组与行动研究组放射性肠炎发生率及严重程度、心理困扰程度以及生活质量。结果行动研究组放射性肠炎发生率、心理困扰程度显著低于对照组(均P<0.05),生活质量总分及维度得分(除性行为维度外)显著高于对照组(均P<0.05)。结论应用行动研究法优化前列腺癌患者放射性肠炎预防护理流程,可降低放射性肠炎发生率,改善患者心理困扰程度,提高生活质量。

高职跨境电商专业英语课程开发改革——基于德国工作流程分析的方法

高职院校的专业英语课程需要凸显岗位适应性,课程开发应当注重实操实践,以职业行动为导向,能力训练融合专业知识和英语素养,突破传统英语“听说读写译”式课程框架。基于德国工作流程分析的方法,重构高职院校跨境电商专业英语课程,具体包括:以就业为根本,准确定位专业培养目标;以基于工作过程为导向,强化课程开发的整体性和岗位适应性;以能力本位为主导,构建开放性的课程体系;以教学实施为根本,推行课程内容的综合化或模块化。项目化构建专业英语课程,课程教学内容与岗位职业要求相匹配,为我国高职院校基于职业能力开发专业英语课程提供参考。