Show simple item record

dc.contributor.authorHasham, Khawar
dc.contributor.authorDelgado Peris, Antonio
dc.contributor.authorAnjum, Ashiq
dc.contributor.authorEvans, Dave
dc.contributor.authorGowdy, Stephen
dc.contributor.authorHernandez, José M.
dc.contributor.authorHuedo, Eduardo
dc.contributor.authorHufnagel, Dirk
dc.contributor.authorvan Lingen, Frank
dc.contributor.authorMcClatchey, Richard
dc.contributor.authorMetson, Simon
dc.date.accessioned2018-10-12T14:26:00Z
dc.date.available2018-10-12T14:26:00Z
dc.date.issued2011-06
dc.identifier.citationHasham, K. et al. (2011) 'CMS workflow execution using intelligent job scheduling and data access strategies', IEEE Transactions on Nuclear Science. 58(3), pp.1221 - 1232. DOI:10.1109/TNS.2011.2146276en
dc.identifier.issn0018-9499
dc.identifier.doi10.1109/TNS.2011.2146276
dc.identifier.urihttp://hdl.handle.net/10545/623036
dc.description.abstractComplex scientific workflows can process large amounts of data using thousands of tasks. The turnaround times of these workflows are often affected by various latencies such as the resource discovery, scheduling and data access latencies for the individual workflow processes or actors. Minimizing these latencies will improve the overall execution time of a workflow and thus lead to a more efficient and robust processing environment. In this paper, we propose a pilot job concept that has intelligent data reuse and job execution strategies to minimize the scheduling, queuing, execution and data access latencies. The results have shown that significant improvements in the overall turnaround time of a workflow can be achieved with this approach. The proposed approach has been evaluated, first using the CMS Tier0 data processing workflow, and then simulating the workflows to evaluate its effectiveness in a controlled environment.
dc.description.sponsorshipN/Aen
dc.language.isoenen
dc.publisherIEEEen
dc.relation.urlhttp://ieeexplore.ieee.org/lpdocs/epic03/wrapper.htm?arnumber=5773464en
dc.rightsArchived with thanks to IEEE Transactions on Nuclear Scienceen
dc.subjectWorkflowsen
dc.subjectLatencyen
dc.subjectPilot Jobsen
dc.subjectGrid and cloudsen
dc.subjectData cacheen
dc.titleCMS workflow execution using intelligent job scheduling and data access strategies.en
dc.typeArticleen
dc.identifier.eissn1558-1578
dc.contributor.departmentUniversity of the West of Englanden
dc.contributor.departmentEuropean Center for Nuclear Researchen
dc.contributor.departmentCentro de Investigaciones Energéticas, Medioambientales y Tecnológicasen
dc.contributor.departmentFermi National Accelerator Laboratoryen
dc.contributor.departmentUniversidad Complutense de Madriden
dc.contributor.departmentUniversity of Bristolen
dc.identifier.journalIEEE Transactions on Nuclear Scienceen
dc.internal.reviewer-noteNo University of Derby affiliation - does this matter? SR 6/10/18en
refterms.dateFOA2019-02-28T17:36:42Z
html.description.abstractComplex scientific workflows can process large amounts of data using thousands of tasks. The turnaround times of these workflows are often affected by various latencies such as the resource discovery, scheduling and data access latencies for the individual workflow processes or actors. Minimizing these latencies will improve the overall execution time of a workflow and thus lead to a more efficient and robust processing environment. In this paper, we propose a pilot job concept that has intelligent data reuse and job execution strategies to minimize the scheduling, queuing, execution and data access latencies. The results have shown that significant improvements in the overall turnaround time of a workflow can be achieved with this approach. The proposed approach has been evaluated, first using the CMS Tier0 data processing workflow, and then simulating the workflows to evaluate its effectiveness in a controlled environment.


Files in this item

Thumbnail
Name:
IEEE TNS CMS Workflow Scheduling ...
Size:
394.0Kb
Format:
PDF
Description:
pre-print

This item appears in the following Collection(s)

Show simple item record