Using schedflow for performance evaluation of workflow applications

Gustavo Martinez*, Elisa Heymann, Miguel Angel Senar, Emilio Luque, Barton P. Miller

*Corresponding author for this work

Research output: Chapter in BookChapterResearchpeer-review

1 Citation (Scopus)

Abstract

Computational science increasingly relies on the execution of workflows in distributed networks to solve complex applications. However, the heterogeneity of resources in these environments complicates resource management and the scheduling of such applications. Sophisticated scheduling policies are being developed for workflows, but they have had little impact in practice because their integration into existing workflow engines is complex and time consuming as each policy has to be individually ported to a particular workflow engine. In addition, choosing a particular scheduling policy is difficult, as factors like machine availability, workload, and communication volume between tasks are difficult to predict. In this paper, we describe SchedFlow, a tool that integrates scheduling policies into workflow engines such as Taverna, DAGMan or Karajan. We show how SchedFlow was used to take advantage of different scheduling policies at different times, depending on the dynamic workload of the workflows. Our experiments included two real workflow applications and four different scheduling policies. We show that no single scheduling policy is the best for all scenarios, so tools like SchedFlow can improve performance by providing flexibility when scheduling workflows.

Original languageAmerican English
Title of host publication2010 5th Workshop on Workflows in Support of Large-Scale Science, WORKS 2010
DOIs
Publication statusPublished - 2010

Publication series

Name2010 5th Workshop on Workflows in Support of Large-Scale Science, WORKS 2010

Keywords

  • Distributed environments
  • Scheduling policies
  • Workflow management

Fingerprint Dive into the research topics of 'Using schedflow for performance evaluation of workflow applications'. Together they form a unique fingerprint.

Cite this