Context Models Proposed solution Experimentation Conclusions and perspectives Finding allocations for Budget-constrained PTG workflow applications eric Desprez 1 , Eddy Caron 1 , Adrian Mures , an 1 , Fr´ Fr´ ed´ ed´ eric Suter 2 1 Ecole Normale Sup´ erieure de Lyon, France 2 IN2P3 Computing Center, CNRS, IN2P3 7th Scheduling for Large Scale Systems Workshop F. Desprez, E. Caron, A. Mures , an, F. Suter Budget-constrained workflow scheduling 1/21
Context Models Proposed solution Experimentation Conclusions and perspectives Outline Context 1 Models 2 Proposed solution 3 Experimentation 4 Conclusions and perspectives 5 F. Desprez, E. Caron, A. Mures , an, F. Suter Budget-constrained workflow scheduling 2/21
Context Models Proposed solution Experimentation Conclusions and perspectives Scientific workflow applications Montage (Astronomy) RAMSES (Astronomy) Epigenomics (Bioinformatics) Climate / ocean current / tectonic plate modeling . . . Characteristics some have sequential and parallel tasks some have non-deterministic transitions F. Desprez, E. Caron, A. Mures , an, F. Suter Budget-constrained workflow scheduling 3/21
Context Models Proposed solution Experimentation Conclusions and perspectives Our goal consider a most general model of the applications consider on-demand resources and a budget limit find a good allocation strategy Why on-demand resources? more efficient resource usage eliminate overbooking of resources F. Desprez, E. Caron, A. Mures , an, F. Suter Budget-constrained workflow scheduling 4/21
Context Models Proposed solution Experimentation Conclusions and perspectives Outline Context 1 Models 2 Proposed solution 3 Experimentation 4 Conclusions and perspectives 5 F. Desprez, E. Caron, A. Mures , an, F. Suter Budget-constrained workflow scheduling 5/21
Context Models Proposed solution Experimentation Conclusions and perspectives Application model Non-deterministic workflows An application is a graph G = ( V , E ), where V = { v i | i = 1 , . . . , | V |} is a set of vertexes E = { e i , j | ( i , j ) ∈ { 1 , . . . , | V |} × { 1 , . . . , | V |}} is a set of edges representing precedence and flow constraints Vertexes a computation [ parallel , moldable ] an OR-split vertex [transitions described by random variables] an OR-join vertex F. Desprez, E. Caron, A. Mures , an, F. Suter Budget-constrained workflow scheduling 6/21
Context Models Proposed solution Experimentation Conclusions and perspectives Example workflow Figure: Example workflow F. Desprez, E. Caron, A. Mures , an, F. Suter Budget-constrained workflow scheduling 7/21
Context Models Proposed solution Experimentation Conclusions and perspectives Platform model A provider of on-demand resources from a catalog: C = { vm i = ( nCPU i , cost i ) | i ≥ 1 } nCPU represents the number of equivalent virtual CPUs cost represents a monetary cost per running hour Makespan C = max i C ( v i ) is the global makespan where C ( v i ) is the finish time of task v i ∈ V Cost of a schedule S Cost = � ∀ vm i ∈S ⌈ T end i − T start i ⌉ × cost i T start i , T end i represent the start end end times of vm i cost i is the catalog cost of virtual resource vm i F. Desprez, E. Caron, A. Mures , an, F. Suter Budget-constrained workflow scheduling 8/21
Context Models Proposed solution Experimentation Conclusions and perspectives Problem statement Given G a workflow application C a provider of resources from the catalog B a budget find a schedule S such that Cost ≤ B budget limit is not passed C (makespan) is minimized with a predefined confidence. F. Desprez, E. Caron, A. Mures , an, F. Suter Budget-constrained workflow scheduling 9/21
Context Models Proposed solution Experimentation Conclusions and perspectives Outline Context 1 Models 2 Proposed solution 3 Experimentation 4 Conclusions and perspectives 5 F. Desprez, E. Caron, A. Mures , an, F. Suter Budget-constrained workflow scheduling 10/21
Context Models Proposed solution Experimentation Conclusions and perspectives Proposed approach 1 Decompose the non-DAG workflow into DAG sub-workflows 2 Distribute the budget to the sub-workflows 3 Determine allocations by adapting an existing allocation approach F. Desprez, E. Caron, A. Mures , an, F. Suter Budget-constrained workflow scheduling 11/21
Context Models Proposed solution Experimentation Conclusions and perspectives Step 1: Decomposing the workflow Figure: Decomposing a nontrivial workflow F. Desprez, E. Caron, A. Mures , an, F. Suter Budget-constrained workflow scheduling 12/21
Context Models Proposed solution Experimentation Conclusions and perspectives Step 2: Allocating budget Give each sub-workflow a ratio of the budget proportional to its work contribution . Work contribution of a sub-workflow G i as the sum of the average execution times of its tasks average execution time computed over the catalog C task speedup model is taken into consideration multiple executions of a sub-workflow also considered F. Desprez, E. Caron, A. Mures , an, F. Suter Budget-constrained workflow scheduling 13/21
Context Models Proposed solution Experimentation Conclusions and perspectives Step 3: Determining allocations Two algorithms based on the bi-CPA algorithm. Eager algorithm one allocation for each task good trade-off between makespan and average time-cost area fast algorithm considers allocation-time cost estimations only Deferred algorithm outputs multiple allocations for each task good trade-off between makespan and average time-cost area slower algorithm one allocation is chosen at scheduling time F. Desprez, E. Caron, A. Mures , an, F. Suter Budget-constrained workflow scheduling 14/21
Context Models Proposed solution Experimentation Conclusions and perspectives Outline Context 1 Models 2 Proposed solution 3 Experimentation 4 Conclusions and perspectives 5 F. Desprez, E. Caron, A. Mures , an, F. Suter Budget-constrained workflow scheduling 15/21
Context Models Proposed solution Experimentation Conclusions and perspectives Methodology Used synthetic workflows for three types of applications Fast Fourier Transform Strassen matrix multiplication Random workloads Used a virtual resource catalog inspired by Amazon EC2 Used a classic list-scheduler for task mapping Measured Cost and makespan after task mapping F. Desprez, E. Caron, A. Mures , an, F. Suter Budget-constrained workflow scheduling 16/21
Context Models Proposed solution Experimentation Conclusions and perspectives Equality 1.6 1.4 Relative makespan 1.2 1 0.8 0.6 0.4 0.2 0 1 5 10 15 20 25 30 35 40 45 50 Budget limit Eager Figure: Relative makespan ( Deferred ) for all workflow applications F. Desprez, E. Caron, A. Mures , an, F. Suter Budget-constrained workflow scheduling 17/21
Context Models Proposed solution Experimentation Conclusions and perspectives 5 Equality 4 Relative cost 3 2 1 0 1 5 10 15 20 25 30 35 40 45 Budget limit Figure: Relative cost ( Eager Deferred ) for all workflow applications F. Desprez, E. Caron, A. Mures , an, F. Suter Budget-constrained workflow scheduling 18/21
Context Models Proposed solution Experimentation Conclusions and perspectives Outline Context 1 Models 2 Proposed solution 3 Experimentation 4 Conclusions and perspectives 5 F. Desprez, E. Caron, A. Mures , an, F. Suter Budget-constrained workflow scheduling 19/21
Context Models Proposed solution Experimentation Conclusions and perspectives Conclusions allocations for non-DAG workflow apps that target Cloud platform proposed two algorithms – Eager and Deferred Eager is fast but cannot guarantee budget constraint after mapping Deferred is slower, but guarantees budget constraint After a certain budget they yield identical allocations Perspectives implement the two using an existing Cloud platform determine per application type which is the tipping point F. Desprez, E. Caron, A. Mures , an, F. Suter Budget-constrained workflow scheduling 20/21
Budget distribution algorithm 1: ω ∗ ← 0 2: for all G i = ( V i , E i ) ⊆ G do nExec i ← CDF − 1 ( D i , Confidence ) 3: 1 ω i ← � � × nExec i T ( v j , vm k ) 4: |C| v j ∈V i vm k ∈C ω ∗ ← ω ∗ + ω i 5: 6: end for 7: for all G i ⊆ G do B i ← B × ω i 1 ω ∗ × 8: nExec i 9: end for Algorithm 1 : Share Budget( B , G , Confidence ) F. Desprez, E. Caron, A. Mures , an, F. Suter Budget-constrained workflow scheduling 22/21
Algorithm parameters TA over |V i | = 1 � T over B ′ × ( T ( v j , Alloc ( v j )) × cost ( v j )) , A j =1 TA under |V i | = 1 � T under B ′ × ( T ( v j , Alloc ( v j )) × cost under ( v j )) A j =1 F. Desprez, E. Caron, A. Mures , an, F. Suter Budget-constrained workflow scheduling 23/21
Recommend
More recommend
