Ashiq Anjum
Bulk scheduling with the DIANA scheduler
Anjum, Ashiq; McClatchey, Richard; Ali, Arshad; Willers, Ian
Authors
Abstract
Results from the research and development of a Data Intensive and Network Aware (DIANA) scheduling engine, to be used primarily for data intensive sciences such as physics analysis, are described. In Grid analyses, tasks can involve thousands of computing, data handling, and network resources. The central problem in the scheduling of these resources is the coordinated management of computation and data at multiple locations and not just data replication or movement. However, this can prove to be a rather costly operation and efficient scheduling can be a challenge if compute and data resources are mapped without considering network costs. We have implemented an adaptive algorithm within the so-called DIANA Scheduler which takes into account data location and size, network performance and computation capability in order to enable efficient global scheduling. DIANA is a performance-aware and economy-guided Meta Scheduler. It iteratively allocates each job to the site that is most likely to produce the best performance as well as optimizing the global queue for any remaining jobs. Therefore, it is equally suitable whether a single job is being submitted or bulk scheduling is being performed. Results indicate that considerable performance improvements can be gained by adopting the DIANA scheduling approach. © 2006 IEEE.
Citation
Anjum, A., McClatchey, R., Ali, A., & Willers, I. (2006). Bulk scheduling with the DIANA scheduler. IEEE Transactions on Nuclear Science, 53(6), 3818-3829. https://doi.org/10.1109/TNS.2006.886047
Journal Article Type | Conference Paper |
---|---|
Publication Date | Dec 1, 2006 |
Journal | IEEE Transactions on Nuclear Science |
Print ISSN | 0018-9499 |
Publisher | Institute of Electrical and Electronics Engineers |
Peer Reviewed | Peer Reviewed |
Volume | 53 |
Issue | 6 |
Pages | 3818-3829 |
DOI | https://doi.org/10.1109/TNS.2006.886047 |
Keywords | bulk scheduling, data-intensive and network-aware (DIANA) scheduler, network-aware scheduling decisions, priority-driven multiqueue feedback algorithm |
Public URL | https://uwe-repository.worktribe.com/output/1034947 |
Publisher URL | http://dx.doi.org/10.1109/TNS.2006.886047 |
You might also like
NeuroProv: Provenance data visualisation for neuroimaging analyses
(2019)
Journal Article
Data provenance tracking as the basis for a biomedical virtual research environment
(2017)
Journal Article
Cloud provider capacity augmentation through automated resource bartering
(2017)
Journal Article
Downloadable Citations
About UWE Bristol Research Repository
Administrator e-mail: repository@uwe.ac.uk
This application uses the following open-source libraries:
SheetJS Community Edition
Apache License Version 2.0 (http://www.apache.org/licenses/)
PDF.js
Apache License Version 2.0 (http://www.apache.org/licenses/)
Font Awesome
SIL OFL 1.1 (http://scripts.sil.org/OFL)
MIT License (http://opensource.org/licenses/mit-license.html)
CC BY 3.0 ( http://creativecommons.org/licenses/by/3.0/)
Powered by Worktribe © 2024
Advanced Search