Cloud computing provides a scalable and flexible platform for executing scientific workflows. Workflow ensembles, which consist of workflows with similar structures but varying input data and sizes, require efficient scheduling to optimize resource utilization, reduce costs, and improve workflow completion rates. Traditional scheduling methods often leave idle time slots between tasks, leading to resource wastage and decreased system efficiency. This paper presents Workflow Ensemble Scheduling with Gap analysis (WESG), a novel approach designed to overcome these challenges while adhering to budget and deadline constraints. WESG employs a five step algorithm to analyze and reduce resource idleness, optimizing task allocation and resource provisioning. By effectively utilizing these gaps, WESG enhances resource efficiency, prioritizes high-priority workflows, and increases the overall number of completed workflows. Experimental results show that WESG outperforms traditional scheduling techniques, achieving better resource utilization, reducing idle gaps, and significantly improving workflow completion rates.

Keywords