Adapting scientific workflow structures using multi-objective optimization strategies

Hdl Handle:
http://hdl.handle.net/10545/617829
Title:
Adapting scientific workflow structures using multi-objective optimization strategies
Authors:
Habib, Irfan; Anjum, Ashiq; Mcclatchey, Richard; Rana, Omer
Abstract:
Scientific workflows have become the primary mechanism for conducting analyses on distributed computing infrastructures such as grids and clouds. In recent years, the focus of optimization within scientific workflows has primarily been on computational tasks and workflow makespan. However, as workflow-based analysis becomes ever more data intensive, data optimization is becoming a prime concern. Moreover, scientific workflows can scale along several dimensions: (i) number of computational tasks, (ii) heterogeneity of computational resources, and the (iii) size and type (static versus streamed) of data involved. Adapting workflow structure in response to these scalability challenges remains an important research objective. Understanding how a workflow graph can be restructured in an automated manner (through task merge, for instance), to address constraints of a particular execution environment is explored in this work, using a multi-objective evolutionary approach. Our approach attempts to adapt the workflow structure to achieve both compute and data optimization. The question of when to terminate the evolutionary search in order to conserve computations is tackled with a novel termination criterion. The results presented in this article demonstrate the feasibility of the termination criterion and demonstrate that significant optimization can be achieved with a multi-objective approach.
Affiliation:
University of Derby, UK
Citation:
Habib, I. et al., 'Adapting scientific workflow structures using multi-objective optimization strategies' 2013, 8 (1):1 ACM Transactions on Autonomous and Adaptive Systems
Publisher:
Association for Computing Machinery
Journal:
ACM Transactions on Autonomous and Adaptive Systems
Issue Date:
1-Apr-2013
URI:
http://hdl.handle.net/10545/617829
DOI:
10.1145/2451248.2451252
Additional Links:
http://dl.acm.org/citation.cfm?doid=2451248.2451252
Type:
Article
Language:
en
ISSN:
15564665
Appears in Collections:
Department of Electronics, Computing & Maths

Full metadata record

DC FieldValue Language
dc.contributor.authorHabib, Irfanen
dc.contributor.authorAnjum, Ashiqen
dc.contributor.authorMcclatchey, Richarden
dc.contributor.authorRana, Omeren
dc.date.accessioned2016-08-02T13:57:33Z-
dc.date.available2016-08-02T13:57:33Z-
dc.date.issued2013-04-01-
dc.identifier.citationHabib, I. et al., 'Adapting scientific workflow structures using multi-objective optimization strategies' 2013, 8 (1):1 ACM Transactions on Autonomous and Adaptive Systemsen
dc.identifier.issn15564665-
dc.identifier.doi10.1145/2451248.2451252-
dc.identifier.urihttp://hdl.handle.net/10545/617829-
dc.description.abstractScientific workflows have become the primary mechanism for conducting analyses on distributed computing infrastructures such as grids and clouds. In recent years, the focus of optimization within scientific workflows has primarily been on computational tasks and workflow makespan. However, as workflow-based analysis becomes ever more data intensive, data optimization is becoming a prime concern. Moreover, scientific workflows can scale along several dimensions: (i) number of computational tasks, (ii) heterogeneity of computational resources, and the (iii) size and type (static versus streamed) of data involved. Adapting workflow structure in response to these scalability challenges remains an important research objective. Understanding how a workflow graph can be restructured in an automated manner (through task merge, for instance), to address constraints of a particular execution environment is explored in this work, using a multi-objective evolutionary approach. Our approach attempts to adapt the workflow structure to achieve both compute and data optimization. The question of when to terminate the evolutionary search in order to conserve computations is tackled with a novel termination criterion. The results presented in this article demonstrate the feasibility of the termination criterion and demonstrate that significant optimization can be achieved with a multi-objective approach.en
dc.language.isoenen
dc.publisherAssociation for Computing Machineryen
dc.relation.urlhttp://dl.acm.org/citation.cfm?doid=2451248.2451252en
dc.rightsArchived with thanks to ACM Transactions on Autonomous and Adaptive Systemsen
dc.subjectMulti-objective optimizationen
dc.subjectEvolutionary computingen
dc.subjectScientific workflowsen
dc.subjectHypervolumeen
dc.subjectWorkflow planningen
dc.subjectTermination Criteriaen
dc.titleAdapting scientific workflow structures using multi-objective optimization strategiesen
dc.typeArticleen
dc.contributor.departmentUniversity of Derby, UKen
dc.identifier.journalACM Transactions on Autonomous and Adaptive Systemsen
All Items in UDORA are protected by copyright, with all rights reserved, unless otherwise indicated.