on the impact of platform models
play

On the Impact of Platform Models EPIT 2007 Arnaud Legrand - PowerPoint PPT Presentation

Sep 09, 2022 •420 likes •2.35k views

On the Impact of Platform Models EPIT 2007 Arnaud Legrand CNRS/INRIA, LIG laboratory June 6, 2007 A. Legrand (CNRS) INRIA-MESCAL 1 / 108 On the Impact of Platform Models Motivation Scientific computing : large needs in computation or

  1. Single-port (Full-Duplex) This model somehow makes sense when using networks like Myrinet that have few multiplexing units and with protocols without control flow [Mar07]. Even if it does not model well complex situations, such a model is not harmfull. A. Legrand (CNRS) INRIA-MESCAL Modeling Concurency 19 / 108 On the Impact of Platform Models

  2. Fluid Modeling When using TCP-based networks, it is generally reasonnable to use flows to model bandwidth sharing [MR99, Low03]. � Income Maximization maximize ρ r ∀ l ∈ L , � r ∈R ρ r � c l Max-Min Fairness maximize min r ∈R ρ r r ∈R s.t. l ∈ r � Proportional Fairness maximize log( ρ r ) r ∈R 1 � Potential Delay Minimization minimize ρ r r ∈R � Some weird function minimize arctan( ρ r ) r ∈R A. Legrand (CNRS) INRIA-MESCAL Modeling Concurency 20 / 108 On the Impact of Platform Models

  3. Fluid Modeling When using TCP-based networks, it is generally reasonnable to use flows to model bandwidth sharing [MR99, Low03]. � Income Maximization maximize ρ r ∀ l ∈ L , � r ∈R ρ r � c l Max-Min Fairness maximize min r ∈R ρ r ATM r ∈R s.t. l ∈ r � Proportional Fairness maximize log( ρ r ) r ∈R TCP Vegas 1 � Potential Delay Minimization minimize ρ r r ∈R � Some weird function minimize arctan( ρ r ) r ∈R TCP Reno A. Legrand (CNRS) INRIA-MESCAL Modeling Concurency 20 / 108 On the Impact of Platform Models

  4. Flows Extensions ◮ Note that this model is a multi-port model with capacity-constraints (like in the previous bounded multi-port). ◮ When latencies are large, using multiple connections enables to get more bandwidth. As a matter of fact, there is very few to loose in using multiple connections. . . ◮ Therefore many people enforce a sometimes artificial (but less intrusive) bound on the maximum number of connections per link [Wag05, MYCR06]. A. Legrand (CNRS) INRIA-MESCAL Modeling Concurency 21 / 108 On the Impact of Platform Models

  5. Outline Topology 1 Point to Point Communication Models 2 Modeling Concurency 3 Remind This is a Model, Hence Imperfect 4 A. Legrand (CNRS) INRIA-MESCAL Imperfection 22 / 108 On the Impact of Platform Models

  6. Remind This is a Model, Hence Imperfect ◮ The previous sharing models are nice but you generally do not know other flows. . . ◮ Communications use the memory bus and hence interfere with computations. Taking such interferences into account may be- come more and more important with multi-core architectures. ◮ Interference between communications are sometimes. . . surprising. Modeling is an art. You have to know your platform and your ap- plication to know what is negligeable and what is important. Even if your model is imperfect, you may still derive interesting results. A. Legrand (CNRS) INRIA-MESCAL Imperfection 23 / 108 On the Impact of Platform Models

  7. Part II Scheduling Case Study A. Legrand (CNRS) INRIA-MESCAL 24 / 108 On the Impact of Platform Models

  8. Outline Scheduling Divisible Workload 5 Star-like Network Under the Multi-port Model Bus-like Network Star-like Network Under the One-Port Model Multi-round algorithms Iterative Algorithms 6 Data Redistribution 7 A. Legrand (CNRS) INRIA-MESCAL Divisible Workload 25 / 108 On the Impact of Platform Models

  9. Context: Distributed Heterogeneous Platforms Scheduling divisible load on various architectures [Rob, BGMR96, Bea05, Yan07]. Sources of problems ◮ Point to point communication model (homogeneous/heterogeneous, with or without latency,. . . ) ◮ Concurrency impact. A. Legrand (CNRS) INRIA-MESCAL Divisible Workload 26 / 108 On the Impact of Platform Models

  10. Seismic Tomography of the Earth ◮ Model of the inner structure of the Earth ◮ The model is validated by comparing the propagation time of a seismic wave in the model to the actual propagation time. ◮ Set of all seismic events of the year 1999: 817101 ◮ Original program written for a parallel computer: if (rank = ROOT) raydata ← read n lines from data file; MPI Scatter(raydata, n/P , ..., rbuff, ..., ROOT, MPI COMM WORLD); compute work(rbuff); A. Legrand (CNRS) INRIA-MESCAL Divisible Workload 27 / 108 On the Impact of Platform Models

  11. Applications Covered by The Divisible Loads Model Applications made of a very (very) large number of fine grain com- putations. Computation time proportional to the size of the data to be pro- cessed. Independent computations: neither synchronizations nor communi- cations. A. Legrand (CNRS) INRIA-MESCAL Divisible Workload 28 / 108 On the Impact of Platform Models

  12. Outline Scheduling Divisible Workload 5 Star-like Network Under the Multi-port Model Bus-like Network Star-like Network Under the One-Port Model Multi-round algorithms Iterative Algorithms 6 Data Redistribution 7 A. Legrand (CNRS) INRIA-MESCAL Divisible Workload 29 / 108 On the Impact of Platform Models

  13. Star-like Network ����� ����� � � � � � � � � ����� ����� � � � � � � � � ����� ����� � � � � ����� ����� � � � � ����� ����� � � � � ����� ����� � � � � ����� ����� � � � � � � � � � � � � � � � � ����� ����� � � � � ����� ����� � � � � ����� ����� � � � � ����� ����� � � � � ����� ����� � � � � ����� ����� � � � � ����� ����� � � � � ����� ����� � � � � �� �� �� �� �� �� ���� ���� �� �� ���� ���� �� �� ���� ���� ���� ���� ���� ���� ���� ���� �� �� �� �� �� �� �� �� �� �� �� �� �� �� �� �� ��������������� ��������������� ������ ������ ������ ������ ��������������� ��������������� ��������������� ��������������� ������ ������ ◮ The links between the master and the workers have different characteristics. ◮ The workers have different computational power. ◮ Communications from the master to the workers can be done in parallel. A. Legrand (CNRS) INRIA-MESCAL Divisible Workload 30 / 108 On the Impact of Platform Models

  14. Notations ◮ A set P 1 , ..., P p of processors ◮ P 1 is the master processor: initially, it holds all the data. ◮ The overall amount of work: W total . ◮ Processor P i receives an amount of work α i W total with α i ∈ Q and � i α i = 1 . Length of a unit-size work on processor P i : w i . Computation time on P i : α i W total w i . ◮ Time needed to send a unit-message from P 1 to P i : c i . Communication time on P i : α i W total c i . Multi-port model: P 1 can send messages in parallel to all work- ers. A. Legrand (CNRS) INRIA-MESCAL Divisible Workload 31 / 108 On the Impact of Platform Models

  15. “Optimization” Problem If all communications start in parallel at time 0 , the completion time T i of processor P i is equal to: T i = α i W total c i + α i W total w i The makespan T of a load distribution is thus equal to: max α i W total ( c i + w i ) = T i Therefore this problem is really trivial as we just need to note that α i = T/ ( W total ( c i + w i )) and � i α j = 1 to get T . Hence, we minimize the makespan by setting: P 0 P 1 P 2 P 3 P 4 P 5 1 α i = c i + w i � j c j + w j A. Legrand (CNRS) INRIA-MESCAL Divisible Workload 32 / 108 On the Impact of Platform Models

  16. Latencies: just for fun Let’s assume that the time needed to send a message of size α i from P 1 to P i is now equal to: L i + c i × α i Therefore in the optimal solution: forall i such that α i > 0 , L i + α i W total × ( c i + w i ) = T . So just sort the processor by increasing latency and “fill” the W total units of fluid load (the “density” of one unit of load on P i being equal to c i + w i ). P 0 P 1 P 2 P 3 P 4 P 5 A. Legrand (CNRS) INRIA-MESCAL Divisible Workload 33 / 108 On the Impact of Platform Models

  17. Latencies: just for fun Let’s assume that the time needed to send a message of size α i from P 1 to P i is now equal to: L i + c i × α i Therefore in the optimal solution: forall i such that α i > 0 , L i + α i W total × ( c i + w i ) = T . So just sort the processor by increasing latency and “fill” the W total units of fluid load (the “density” of one unit of load on P i being equal to c i + w i ). P 0 P 1 P 2 P 3 P 4 P 5 A. Legrand (CNRS) INRIA-MESCAL Divisible Workload 33 / 108 On the Impact of Platform Models

  18. Latencies: just for fun Let’s assume that the time needed to send a message of size α i from P 1 to P i is now equal to: L i + c i × α i Therefore in the optimal solution: forall i such that α i > 0 , L i + α i W total × ( c i + w i ) = T . So just sort the processor by increasing latency and “fill” the W total units of fluid load (the “density” of one unit of load on P i being equal to c i + w i ). P 0 P 1 P 2 P 3 P 4 P 5 A. Legrand (CNRS) INRIA-MESCAL Divisible Workload 33 / 108 On the Impact of Platform Models

  19. Latencies: just for fun Let’s assume that the time needed to send a message of size α i from P 1 to P i is now equal to: L i + c i × α i Therefore in the optimal solution: forall i such that α i > 0 , L i + α i W total × ( c i + w i ) = T . So just sort the processor by increasing latency and “fill” the W total units of fluid load (the “density” of one unit of load on P i being equal to c i + w i ). P 0 P 1 P 2 P 3 P 4 P 5 A. Legrand (CNRS) INRIA-MESCAL Divisible Workload 33 / 108 On the Impact of Platform Models

  20. Latencies: just for fun Let’s assume that the time needed to send a message of size α i from P 1 to P i is now equal to: L i + c i × α i Therefore in the optimal solution: forall i such that α i > 0 , L i + α i W total × ( c i + w i ) = T . So just sort the processor by increasing latency and “fill” the W total units of fluid load (the “density” of one unit of load on P i being equal to c i + w i ). P 0 P 1 P 2 P 3 P 4 P 5 A. Legrand (CNRS) INRIA-MESCAL Divisible Workload 33 / 108 On the Impact of Platform Models

  21. Outline Scheduling Divisible Workload 5 Star-like Network Under the Multi-port Model Bus-like Network Star-like Network Under the One-Port Model Multi-round algorithms Iterative Algorithms 6 Data Redistribution 7 A. Legrand (CNRS) INRIA-MESCAL Divisible Workload 34 / 108 On the Impact of Platform Models

  22. Notations ◮ A set P 1 , ..., P p of processors ◮ P 1 is the master processor: initially, it holds all the data. ◮ The overall amount of work: W total . ◮ Processor P i receives an amount of work α i W total with α i W total ∈ Q and � i α i = 1 . Length of a unit-size work on processor P i : w i . Computation time on P i : n i w i . ◮ Time needed to send a unit-message from P 1 to P i : c . One-port model: P 1 sends a single message at a time, all pro- cessors communicate at the same speed with the master. A. Legrand (CNRS) INRIA-MESCAL Divisible Workload 35 / 108 On the Impact of Platform Models

  23. Equations For processor P i (with c 1 = 0 and c j = c otherwise): i � T i = α j W total .c j + α i W total .w i j =1   i � T = max α j W total .c j + α i W total .w i   1 � i � p j =1 We look for a data distribution α 1 , ..., α p which minimizes T . A. Legrand (CNRS) INRIA-MESCAL Divisible Workload 36 / 108 On the Impact of Platform Models

  24. Properties of Load-Balancing Lemma 1. In an optimal solution, all processors end their processing at the same time. A. Legrand (CNRS) INRIA-MESCAL Divisible Workload 37 / 108 On the Impact of Platform Models

  25. Demonstration of Lemma 1 Two workers i and i + 1 with T i < T i +1 . P 4 P 3 P 2 P 1 temps 0 fin We decrease α i +1 by ε . A. Legrand (CNRS) INRIA-MESCAL Divisible Workload 38 / 108 On the Impact of Platform Models

  26. Demonstration of Lemma 1 Two workers i and i + 1 with T i < T i +1 . P 4 P 3 P 2 P 1 temps 0 fin We decrease α i +1 by ε . A. Legrand (CNRS) INRIA-MESCAL Divisible Workload 38 / 108 On the Impact of Platform Models

  27. Demonstration of Lemma 1 Two workers i and i + 1 with T i < T i +1 . P 4 P 3 P 2 P 1 temps 0 fin We decrease α i +1 by ε . A. Legrand (CNRS) INRIA-MESCAL Divisible Workload 38 / 108 On the Impact of Platform Models

  28. Demonstration of Lemma 1 Two workers i and i + 1 with T i < T i +1 . P 4 P 3 P 2 P 1 temps 0 fin We increase α i by ε . A. Legrand (CNRS) INRIA-MESCAL Divisible Workload 38 / 108 On the Impact of Platform Models

  29. Demonstration of Lemma 1 Two workers i and i + 1 with T i < T i +1 . P 4 P 3 P 2 P 1 temps 0 fin We increase α i by ε . A. Legrand (CNRS) INRIA-MESCAL Divisible Workload 38 / 108 On the Impact of Platform Models

  30. Demonstration of Lemma 1 Two workers i and i + 1 with T i < T i +1 . P 4 P 3 P 2 P 1 temps 0 fin The communication time for the following processors is unchanged. A. Legrand (CNRS) INRIA-MESCAL Divisible Workload 38 / 108 On the Impact of Platform Models

  31. Demonstration of Lemma 1 Two workers i and i + 1 with T i < T i +1 . P 4 P 3 P 2 P 1 temps 0 fin We end up with a better solution ! A. Legrand (CNRS) INRIA-MESCAL Divisible Workload 38 / 108 On the Impact of Platform Models

  32. Property for the Resource Selection Lemma 2. In an optimal solution all processors work. A. Legrand (CNRS) INRIA-MESCAL Divisible Workload 39 / 108 On the Impact of Platform Models

  33. Property for the Resource Selection Lemma 2. In an optimal solution all processors work. Demonstration: this is just a corollary of lemma 1... A. Legrand (CNRS) INRIA-MESCAL Divisible Workload 39 / 108 On the Impact of Platform Models

  34. Resolution T = α 1 W total w 1 . A. Legrand (CNRS) INRIA-MESCAL Divisible Workload 40 / 108 On the Impact of Platform Models

  35. Resolution T = α 1 W total w 1 . w 1 T = α 2 ( c + w 2 ) W total . Therefore α 2 = c + w 2 α 1 . A. Legrand (CNRS) INRIA-MESCAL Divisible Workload 40 / 108 On the Impact of Platform Models

  36. Resolution T = α 1 W total w 1 . w 1 T = α 2 ( c + w 2 ) W total . Therefore α 2 = c + w 2 α 1 . w 2 T = ( α 2 c + α 3 ( c + w 3 )) W total . Therefore α 3 = c + w 3 α 2 . A. Legrand (CNRS) INRIA-MESCAL Divisible Workload 40 / 108 On the Impact of Platform Models

  37. Resolution T = α 1 W total w 1 . w 1 T = α 2 ( c + w 2 ) W total . Therefore α 2 = c + w 2 α 1 . w 2 T = ( α 2 c + α 3 ( c + w 3 )) W total . Therefore α 3 = c + w 3 α 2 . α i = w i − 1 c + w i α i − 1 for i � 2 . A. Legrand (CNRS) INRIA-MESCAL Divisible Workload 40 / 108 On the Impact of Platform Models

  38. Resolution T = α 1 W total w 1 . w 1 T = α 2 ( c + w 2 ) W total . Therefore α 2 = c + w 2 α 1 . w 2 T = ( α 2 c + α 3 ( c + w 3 )) W total . Therefore α 3 = c + w 3 α 2 . α i = w i − 1 c + w i α i − 1 for i � 2 . � n i =1 α i = 1 . A. Legrand (CNRS) INRIA-MESCAL Divisible Workload 40 / 108 On the Impact of Platform Models

  39. Resolution T = α 1 W total w 1 . w 1 T = α 2 ( c + w 2 ) W total . Therefore α 2 = c + w 2 α 1 . w 2 T = ( α 2 c + α 3 ( c + w 3 )) W total . Therefore α 3 = c + w 3 α 2 . α i = w i − 1 c + w i α i − 1 for i � 2 . � n i =1 α i = 1 . j � � w 1 w k − 1 � α 1 1 + + ... + + ... = 1 c + w 2 c + w k k =2 A. Legrand (CNRS) INRIA-MESCAL Divisible Workload 40 / 108 On the Impact of Platform Models

  40. Impact of the Order of Communications How important is the influence of the ordering of the processor on the solution ? ? A. Legrand (CNRS) INRIA-MESCAL Divisible Workload 41 / 108 On the Impact of Platform Models

  41. No Impact of The Order of the Communications Volume processed by processors P i and P i +1 during a time T . 1 T Processor P i : α i ( c + w i ) W total = T . Therefore α i = W total . c + w i Processor P i +1 : α i cW total + α i +1 ( c + w i +1 ) W total = T . w i 1 T T Thus α i +1 = c + w i +1 ( W total − α i c ) = W total . ( c + w i )( c + w i +1 ) Processors P i and P i +1 : c + w i + w i +1 α i + α i +1 = ( c + w i )( c + w i +1 ) A. Legrand (CNRS) INRIA-MESCAL Divisible Workload 42 / 108 On the Impact of Platform Models

  42. Choice of the Master Processor We compare processors P 1 and P 2 . A. Legrand (CNRS) INRIA-MESCAL Divisible Workload 43 / 108 On the Impact of Platform Models

  43. Choice of the Master Processor We compare processors P 1 and P 2 . 1 T Processor P 1 : α 1 w 1 W total = T . Then, α 1 = W total . w 1 A. Legrand (CNRS) INRIA-MESCAL Divisible Workload 43 / 108 On the Impact of Platform Models

  44. Choice of the Master Processor We compare processors P 1 and P 2 . 1 T Processor P 1 : α 1 w 1 W total = T . Then, α 1 = W total . w 1 1 T Processor P 2 : α 2 ( c + w 2 ) W total = T . Thus, α 2 = W total . c + w 2 A. Legrand (CNRS) INRIA-MESCAL Divisible Workload 43 / 108 On the Impact of Platform Models

  45. Choice of the Master Processor We compare processors P 1 and P 2 . 1 T Processor P 1 : α 1 w 1 W total = T . Then, α 1 = W total . w 1 1 T Processor P 2 : α 2 ( c + w 2 ) W total = T . Thus, α 2 = W total . c + w 2 Total volume processed: α 1 + α 2 = c + w 1 + w 2 w 1 ( c + w 2 ) = c + w 1 + w 2 cw 1 + w 1 w 2 A. Legrand (CNRS) INRIA-MESCAL Divisible Workload 43 / 108 On the Impact of Platform Models

  46. Choice of the Master Processor We compare processors P 1 and P 2 . 1 T Processor P 1 : α 1 w 1 W total = T . Then, α 1 = W total . w 1 1 T Processor P 2 : α 2 ( c + w 2 ) W total = T . Thus, α 2 = W total . c + w 2 Total volume processed: α 1 + α 2 = c + w 1 + w 2 w 1 ( c + w 2 ) = c + w 1 + w 2 cw 1 + w 1 w 2 Minimal when w 1 < w 2 . Master = the most powerfull processor (for computations). A. Legrand (CNRS) INRIA-MESCAL Divisible Workload 43 / 108 On the Impact of Platform Models

  47. Conclusion ◮ Closed-form expressions for the execution time and the distri- bution of data. ◮ Choice of the master. ◮ The ordering of the processors has no impact. ◮ All processors take part in the work. A. Legrand (CNRS) INRIA-MESCAL Divisible Workload 44 / 108 On the Impact of Platform Models

  48. Outline Scheduling Divisible Workload 5 Star-like Network Under the Multi-port Model Bus-like Network Star-like Network Under the One-Port Model Multi-round algorithms Iterative Algorithms 6 Data Redistribution 7 A. Legrand (CNRS) INRIA-MESCAL Divisible Workload 45 / 108 On the Impact of Platform Models

  49. Star-like Network � � � � ����� ����� � � � � ����� ����� � � � � � � � � ����� ����� � � � � ����� ����� � � � � ����� ����� � � � � ����� ����� � � � � ����� ����� � � � � � � � � � � � � � � � � ����� ����� � � � � ����� ����� � � � � ����� ����� � � � � ����� ����� � � � � ����� ����� � � � � ����� ����� � � � � ����� ����� � � � � ����� ����� � � � � �� �� �� �� �� �� ���� ���� �� �� ���� ���� �� �� ���� ���� ���� ���� ���� ���� ���� ���� �� �� �� �� �� �� �� �� �� �� �� �� �� �� �� �� ��������������� ��������������� ������ ������ ��������������� ��������������� ������ ������ ��������������� ��������������� ������ ������ ◮ The links between the master and the workers have different characteristics. ◮ The workers have different computational power. A. Legrand (CNRS) INRIA-MESCAL Divisible Workload 46 / 108 On the Impact of Platform Models

  50. Notations ◮ A set P 1 , ..., P p of processors ◮ P 1 is the master processor: initially, it holds all the data. ◮ The overall amount of work: W total . ◮ Processor P i receives an amount of work α i W total with � i n i = W total with α i W total ∈ Q and � i α i = 1 . Length of a unit-size work on processor P i : w i . Computation time on P i : n i w i . ◮ Time needed to send a unit-message from P 1 to P i : c i . One-port model: P 1 sends a single message at a time. A. Legrand (CNRS) INRIA-MESCAL Divisible Workload 47 / 108 On the Impact of Platform Models

  51. Star Network and Linear Cost Model Goal : maximize the number of processed tasks within a time-bound T f : � α i . A. Legrand (CNRS) INRIA-MESCAL Divisible Workload 48 / 108 On the Impact of Platform Models

  52. Star Network and Linear Cost Model Goal : maximize the number of processed tasks within a time-bound T f : � α i . Lemma 3. In any optimal solution of the StarLinear problem, all workers participate in the computation, and all processors finish computing simultaneously. A. Legrand (CNRS) INRIA-MESCAL Divisible Workload 48 / 108 On the Impact of Platform Models

  53. Star Network and Linear Cost Model Goal : maximize the number of processed tasks within a time-bound T f : � α i . Lemma 3. In any optimal solution of the StarLinear problem, all workers participate in the computation, and all processors finish computing simultaneously. Lemma 4. An optimal ordering for the StarLinear problem is obtained by serving the workers in the ordering of non decreasing link capacities c i . A. Legrand (CNRS) INRIA-MESCAL Divisible Workload 48 / 108 On the Impact of Platform Models

  54. Sketch of the Proof of Lemma 3 Two steps : ◮ All workers participate in the computation. . . A. Legrand (CNRS) INRIA-MESCAL Divisible Workload 49 / 108 On the Impact of Platform Models

  55. Sketch of the Proof of Lemma 3 P p α p w p ... Two steps : P i ◮ All workers participate in ... the computation. . . otherwise P 2 α 2 w 2 it would not be optimal. P 1 α 1 w 1 Network α 1 c 1 α 2 c 2 α p c p T 1 T 2 T p T f A. Legrand (CNRS) INRIA-MESCAL Divisible Workload 49 / 108 On the Impact of Platform Models

  56. Sketch of the Proof of Lemma 3 P p α p w p ... Two steps : P i α i w i ◮ All workers participate in ... the computation. . . otherwise P 2 α 2 w 2 it would not be optimal. P 1 α 1 w 1 ◮ All Network α 1 c 1 α 2 c 2 α p c p α i c i processors finish their T 1 T 2 T p T f work at the same time. A. Legrand (CNRS) INRIA-MESCAL Divisible Workload 49 / 108 On the Impact of Platform Models

  57. Sketch of the Proof of Lemma 3 P p α p w p ... Two steps : P i α i w i ◮ All workers participate in ... the computation. . . otherwise P 2 α 2 w 2 it would not be optimal. P 1 α 1 w 1 ◮ All Network α 1 c 1 α 2 c 2 ... α i c i ... α p c p processors finish their T 1 T 2 T p T f work at the same time. Maximize � β i , subject to � LB( i ) ∀ i, β i � 0 � i UB( i ) ∀ i, k =1 β k c k + β i w i � T f A. Legrand (CNRS) INRIA-MESCAL Divisible Workload 49 / 108 On the Impact of Platform Models

  58. Sketch of the Proof of Lemma 3 β 2 Two steps : ◮ All workers participate in the computation. . . otherwise it would not be optimal. ◮ All processors finish their work at the same time. β 1 Maximize � β i , subject to � LB( i ) ∀ i, β i � 0 � i UB( i ) ∀ i, k =1 β k c k + β i w i � T f A. Legrand (CNRS) INRIA-MESCAL Divisible Workload 49 / 108 On the Impact of Platform Models

  59. Sketch of the Proof of Lemma 3 β 2 Two steps : ◮ All workers participate in the computation. . . otherwise it would not be optimal. ◮ All processors finish their work at the same time. β 1 Maximize � β i , subject to � LB( i ) ∀ i, β i � 0 � i UB( i ) ∀ i, k =1 β k c k + β i w i � T f A. Legrand (CNRS) INRIA-MESCAL Divisible Workload 49 / 108 On the Impact of Platform Models

  60. Sketch of the Proof of Lemma 3 β 2 Two steps : ◮ All workers participate in the computation. . . otherwise it would not be optimal. ◮ All processors finish their work at the same time. β 1 Maximize � β i , subject to � LB( i ) ∀ i, β i � 0 � i UB( i ) ∀ i, k =1 β k c k + β i w i � T f A. Legrand (CNRS) INRIA-MESCAL Divisible Workload 49 / 108 On the Impact of Platform Models

  61. Sketch of the Proof of Lemma 3 β 2 Two steps : ◮ All workers participate in ( α 1 , α 2 ) the computation. . . otherwise it would not be optimal. ◮ All processors finish their work at the same time. β 1 Maximize � β i , subject to � LB( i ) ∀ i, β i � 0 � i UB( i ) ∀ i, k =1 β k c k + β i w i � T f A. Legrand (CNRS) INRIA-MESCAL Divisible Workload 49 / 108 On the Impact of Platform Models

  62. Sketch of the Proof of Lemma 3 β 2 Two steps : ◮ All workers participate in ( α 1 , α 2 ) the computation. . . otherwise it would not be optimal. ◮ All processors finish their work at the same time. β 1 Maximize � β i , subject to � LB( i ) ∀ i, β i � 0 � i UB( i ) ∀ i, k =1 β k c k + β i w i � T f A. Legrand (CNRS) INRIA-MESCAL Divisible Workload 49 / 108 On the Impact of Platform Models

  63. Sketch of the Proof of Lemma 4 The proof is based on the comparison of the amount of work that is performed by the first two workers, and then proceeds by induction. A. Legrand (CNRS) INRIA-MESCAL Divisible Workload 50 / 108 On the Impact of Platform Models

  64. Sketch of the Proof of Lemma 4 The proof is based on the comparison of the amount of work that is performed by the first two workers, and then proceeds by induction. t ( A ) t ( B ) T T α ( A ) α ( A ) α ( B ) α ( B ) P 1 1 c 1 1 w 1 P 1 1 c 1 1 w 1 α ( A ) α ( A ) α ( B ) α ( B ) P 2 P 2 2 c 2 2 w 2 2 c 2 2 w 2 (A) P 1 starts before P 2 (B) P 2 starts before P 1 A. Legrand (CNRS) INRIA-MESCAL Divisible Workload 50 / 108 On the Impact of Platform Models

  65. Sketch of the Proof of Lemma 4 The proof is based on the comparison of the amount of work that is performed by the first two workers, and then proceeds by induction. t ( A ) t ( B ) T T α ( A ) α ( A ) α ( B ) α ( B ) P 1 1 c 1 1 w 1 P 1 1 c 1 1 w 1 α ( A ) α ( A ) α ( B ) α ( B ) P 2 P 2 2 c 2 2 w 2 2 c 2 2 w 2 (A) P 1 starts before P 2 (B) P 2 starts before P 1 t ( A ) = t ( B ) , (1) A. Legrand (CNRS) INRIA-MESCAL Divisible Workload 50 / 108 On the Impact of Platform Models

  66. Sketch of the Proof of Lemma 4 The proof is based on the comparison of the amount of work that is performed by the first two workers, and then proceeds by induction. t ( A ) t ( B ) T T α ( A ) α ( A ) α ( B ) α ( B ) P 1 1 c 1 1 w 1 P 1 1 c 1 1 w 1 α ( A ) α ( A ) α ( B ) α ( B ) P 2 P 2 2 c 2 2 w 2 2 c 2 2 w 2 (A) P 1 starts before P 2 (B) P 2 starts before P 1 t ( A ) = t ( B ) , (1) and T ( c 2 − c 1 ) ( α ( A ) + α ( A ) ) − ( α ( B ) + α ( B ) ) = ( c 1 + w 1 )( c 2 + w 2 ) . (2) 1 2 1 2 A. Legrand (CNRS) INRIA-MESCAL Divisible Workload 50 / 108 On the Impact of Platform Models

  67. Conclusion ◮ The processors must be ordered by decreasing bandwidths ◮ All processors are working ◮ All processors end their work at the same time ◮ Formulas for the execution time and the distribution of data A. Legrand (CNRS) INRIA-MESCAL Divisible Workload 51 / 108 On the Impact of Platform Models

  68. Outline Scheduling Divisible Workload 5 Star-like Network Under the Multi-port Model Bus-like Network Star-like Network Under the One-Port Model Multi-round algorithms Iterative Algorithms 6 Data Redistribution 7 A. Legrand (CNRS) INRIA-MESCAL Divisible Workload 52 / 108 On the Impact of Platform Models

  69. One-round vs. Multi-round P p α p w p P p ... P 2 α 2 w 2 P 2 P 1 α 1 w 1 P 1 Network α 1 g α 2 g α p g Network T 1 T 2 T p T f R 0 R 1 R k One round Multi-round A. Legrand (CNRS) INRIA-MESCAL Divisible Workload 53 / 108 On the Impact of Platform Models

  70. One-round vs. Multi-round P p α p w p P p ... P 2 α 2 w 2 P 2 P 1 α 1 w 1 P 1 Network α 1 g α 2 g α p g Network T 1 T 2 T p T f R 0 R 1 R k One round Multi-round � long idle-times Efficient when W total large Intuition: start with small rounds, then increase chunks. Problems: A. Legrand (CNRS) INRIA-MESCAL Divisible Workload 53 / 108 On the Impact of Platform Models

  71. One-round vs. Multi-round P p α p w p P p ... P 2 α 2 w 2 P 2 P 1 α 1 w 1 P 1 Network α 1 g α 2 g α p g Network T 1 T 2 T p T f R 0 R 1 R k One round Multi-round � long idle-times Efficient when W total large Intuition: start with small rounds, then increase chunks. Problems: ◮ linear communication model leads to absurd solution ◮ resource selection ◮ number of rounds ◮ size of each round A. Legrand (CNRS) INRIA-MESCAL Divisible Workload 53 / 108 On the Impact of Platform Models

  72. Notations ◮ A set P 1 , ..., P p of processors ◮ P 1 is the master processor: initially, it holds all the data. ◮ The overall amount of work: W total . ◮ Processor P i receives an amount of work α i W total with � i n i = W total with α i W total ∈ Q and � i α i = 1 . Length of a unit-size work on processor P i : w i . Computation time on P i : n i w i . ◮ Time needed to send a message of size α i P 1 to P i : L i + c i × α i . One-port model: P 1 sends and receives a single message at a time. A. Legrand (CNRS) INRIA-MESCAL Divisible Workload 54 / 108 On the Impact of Platform Models

  73. Complexity Definition: One round, ∀ i, c i = 0 . Given W total , p workers, ( P i ) 1 � i � p , ( L i ) 1 � i � p , and a rational number T � 0 , and assuming that bandwidths are infinite, is it possible to compute all W total load units within T time units? Theorem 1. The problem with one-round and infinite bandwidths is NP- complete. A. Legrand (CNRS) INRIA-MESCAL Divisible Workload 55 / 108 On the Impact of Platform Models

  74. Complexity Definition: One round, ∀ i, c i = 0 . Given W total , p workers, ( P i ) 1 � i � p , ( L i ) 1 � i � p , and a rational number T � 0 , and assuming that bandwidths are infinite, is it possible to compute all W total load units within T time units? Theorem 1. The problem with one-round and infinite bandwidths is NP- complete. What is the complexity of the general problem with finite bandwidths and several rounds ? A. Legrand (CNRS) INRIA-MESCAL Divisible Workload 55 / 108 On the Impact of Platform Models

  75. Complexity Definition: One round, ∀ i, c i = 0 . Given W total , p workers, ( P i ) 1 � i � p , ( L i ) 1 � i � p , and a rational number T � 0 , and assuming that bandwidths are infinite, is it possible to compute all W total load units within T time units? Theorem 1. The problem with one-round and infinite bandwidths is NP- complete. What is the complexity of the general problem with finite bandwidths and several rounds ? The general problem is NP-hard, but does not appear to be in NP (no polynomial bound on the number of activations). A. Legrand (CNRS) INRIA-MESCAL Divisible Workload 55 / 108 On the Impact of Platform Models

  76. Fixed activation sequence Hypotheses 1 Number of activations : N act ; 2 Whether P i is the processor used during activation j : χ ( j ) i Minimize T, under the constraints  p N act χ ( j ) i α ( j )  � � = W total   i    j =1 i =1      p k N act � � χ ( j ) i ( L i + α ( j ) � χ ( j ) l α ( j )  + ∀ k � N act , ∀ l : i c i ) l w l � T      j =1 i =1 j = k    ∀ i, j : α ( j )  � 0  i (3) Can be solved in polynomial time. A. Legrand (CNRS) INRIA-MESCAL Divisible Workload 56 / 108 On the Impact of Platform Models

  77. Fixed number of activations Minimize T, under the constraints p N act  χ ( j ) i α ( j ) � �  = W total   i    j =1 i =1      k p N act   χ ( j ) i ( L i + α ( j ) χ ( j ) l α ( j )  � �  + �  ∀ k � N act , ∀ l : i c i ) l w l � T     j =1 i =1 j = k p  � χ ( k )  ∀ k � N act : � 1   i    i =1   ∀ i, j : χ ( j )  ∈ { 0 , 1 }   i   ∀ i, j : α ( j )   � 0 i (4) Exact but exponential (branch-and-bound algorithms). A. Legrand (CNRS) INRIA-MESCAL Divisible Workload 57 / 108 On the Impact of Platform Models

  78. Uniform Multi-Round round j round j + 1 round j + 2 α ( j ) α ( j +1) 1 c i c i 1 Transfer Worker 1 In a round: all workers have same α ( j ) 1 w 1 Compute computation time Transfer Worker 2 Compute . . α ( j ) i c i α ( j +1) c i . i Geometrical increase of rounds Transfer Worker i α ( j ) i w i = α ( j ) 1 w 1 Compute size L i . . α ( j ) α ( j +1) p c p c p . p No idle time in communications: Transfer Worker p α ( j ) p w p = α ( j ) 1 w 1 Compute T A T C T B time p α ( j ) � ( L k + α ( j +1) i w i = c k ) . k k =1 Heuristic processor selection: by decreasing bandwidths A. Legrand (CNRS) INRIA-MESCAL Divisible Workload 58 / 108 On the Impact of Platform Models

  79. Uniform Multi-Round round j round j + 1 round j + 2 α ( j ) α ( j +1) 1 c i c i 1 Transfer Worker 1 In a round: all workers have same α ( j ) 1 w 1 Compute computation time Transfer Worker 2 Compute . . α ( j ) i c i α ( j +1) c i . i Geometrical increase of rounds Transfer Worker i α ( j ) i w i = α ( j ) 1 w 1 Compute size L i . . α ( j ) α ( j +1) p c p c p . p No idle time in communications: Transfer Worker p α ( j ) p w p = α ( j ) 1 w 1 Compute T A T C T B time p α ( j ) � ( L k + α ( j +1) i w i = c k ) . k k =1 Heuristic processor selection: by decreasing bandwidths No guarantee. . . A. Legrand (CNRS) INRIA-MESCAL Divisible Workload 58 / 108 On the Impact of Platform Models

  80. Periodic Schedule T p L 1 α 1 c 1 L 1 α 1 c 1 L 1 α 1 c 1 Transfer α 1 w 1 α 1 w 1 α 1 w 1 Compute L 2 α 2 c 2 L 2 α 2 c 2 L 2 α 2 c 2 Transfer α 2 w 2 α 2 w 2 α 2 w 2 Compute L 3 α 3 c 3 L 3 α 3 c 3 L 3 α 3 c 3 Transfer α 3 w 3 α 3 w 3 α 3 w 3 Compute . . . L n α n c n L n α n c n L n α n c n Transfer α n w n α n w n α n w n Compute How to choose T p ? Which resources to select? A. Legrand (CNRS) INRIA-MESCAL Divisible Workload 59 / 108 On the Impact of Platform Models

Download Presentation
Download Policy: The content available on the website is offered to you 'AS IS' for your personal information and use only. It cannot be commercialized, licensed, or distributed on other websites without prior consent from the author. To download a presentation, simply click this link. If you encounter any difficulties during the download process, it's possible that the publisher has removed the file from their server.

Recommend

Automated Generation of Platform-Variant Applications from Platform-Independent Models via

Automated Generation of Platform-Variant Applications from Platform-Independent Models via

Automated Generation of Platform-Variant Applications from Platform-Independent Models via Templates Nuno Amlio, Christian Glodt, Frederico Pinto, Pierre Kelsen University of Luxembourg Introduction Manual code development is expensive

243 views • 21 slides
Ride Sharing Platform Vs Taxi Platform: the Impact on the Revenue Benjamin Bordais 1 , Costas

Ride Sharing Platform Vs Taxi Platform: the Impact on the Revenue Benjamin Bordais 1 , Costas

Ride Sharing Platform Vs Taxi Platform: the Impact on the Revenue Benjamin Bordais 1 , Costas Courcoubetis 2 1 ENS Rennes 2 Singapore University of Technology and Design March 28, 2019 Introduction There is great need of mobility in major

629 views • 27 slides
Logic Models Defining Impact and Change 1 28/06/2013 One Tool, Many Names Logic Models or

Logic Models Defining Impact and Change 1 28/06/2013 One Tool, Many Names Logic Models or

28/06/2013 Logic Models Defining Impact and Change 1 28/06/2013 One Tool, Many Names Logic Models or intended impact models Simplification of a program, initiative, or intervention that is a response to a given situation. These tools have

338 views • 15 slides
The Impact of Artificial Intelligence on Naval Platform and System Design C.D.Burnside Rotterdam

The Impact of Artificial Intelligence on Naval Platform and System Design C.D.Burnside Rotterdam

The Impact of Artificial Intelligence on Naval Platform and System Design C.D.Burnside Rotterdam | May 2020 Premise of Research Assess onboard and offboard naval combatant platform functions and identify those which can be enhanced

424 views • 18 slides
Computer Models of Micrometeoroid Impact on Fused Silica Glass Mirrors Hypervelocity Impact

Computer Models of Micrometeoroid Impact on Fused Silica Glass Mirrors Hypervelocity Impact

Computer Models of Micrometeoroid Impact on Fused Silica Glass Mirrors Hypervelocity Impact Symposium December 2003 Presented by David Davison Shock Transients, Inc., PO Box 5357, Hopkins, MN 55343-2357 USA (952) 944-3539, x101/944-8170

419 views • 16 slides
Future of Cultural Heritage Impact of external developments European Foresight Platform Workshop

Future of Cultural Heritage Impact of external developments European Foresight Platform Workshop

Future of Cultural Heritage Impact of external developments European Foresight Platform Workshop 18 December 2012, Brussels Project funded under the Socio- economic Sciences and Humanities European Foresight Platform 1. Raising awareness of

500 views • 6 slides
Impact of Pharmacovigilance Systems 4 th industry stakeholder platform - operation of EU

Impact of Pharmacovigilance Systems 4 th industry stakeholder platform - operation of EU

Impact of Pharmacovigilance Systems 4 th industry stakeholder platform - operation of EU pharmacovigilance legislation 12 June 2015 Almath Spooner, PRAC Peter Arlett, EMA Xavier Kurz, EMA Jacoline Bouvy, EMA Marie-Helene Pinheiro, EMA An

299 views • 7 slides
A multi-platform system for understanding, monitoring and forecasting the impact of aerosol

A multi-platform system for understanding, monitoring and forecasting the impact of aerosol

A multi-platform system for understanding, monitoring and forecasting the impact of aerosol pollutants in South-East Asia Santo V. Salinas [a] Team members: Tan Li [a] and Daniel Kalbermater [a] Collaborators: Ellsworth J. Welton [b] , Brent N.

669 views • 27 slides
The Web of Mathematical Models: A Schema-based, Wiki-like, Interactive Platform Thomas

The Web of Mathematical Models: A Schema-based, Wiki-like, Interactive Platform Thomas

The Web of Mathematical Models: A Schema-based, Wiki-like, Interactive Platform Thomas Grundmann, Jean-Marie Gaillourdet, Karsten Schmidt, Arnd Poetzsch-Heffter, Stefan Deloch and Martin Memmel MathWikis, Nijmegen, 2011 Outline

528 views • 17 slides
Neutrino Flavour Models and Impact on LFV Luca Merlo 26.06.2012,

Neutrino Flavour Models and Impact on LFV Luca Merlo 26.06.2012,

Neutrino Flavour Models and Impact on LFV Luca Merlo 26.06.2012, WHAT IS ? INVISIBLES12 and Alexei Smirnov Fest Outline News on neutrino mixings Impact on

1.09k views • 79 slides
1 SIP History Societal impact of pain (SIP) is a social platform created in 2009 which

1 SIP History Societal impact of pain (SIP) is a social platform created in 2009 which

1 SIP History Societal impact of pain (SIP) is a social platform created in 2009 which supports national stakeholder in developing national platforms Joint initiative of the European Pain Federation (EFIC) and the pharmaceutical

368 views • 13 slides
UDT 2020 The Impact of Artificial Intelligence on Naval Platform and System Design

UDT 2020 The Impact of Artificial Intelligence on Naval Platform and System Design

UDT 2020 UDT Extended Abstract Template Presentation/Panel UDT 2020 The Impact of Artificial Intelligence on Naval Platform and System Design C.D.Burnside System Design &amp; Architecting Mission Systems, BMT Defence and Security, Bath,

444 views • 3 slides
NSML : A Machine Learning Platform That Enables You Focus on Your Models. ML-Sys WS 2017 @ NIPS

NSML : A Machine Learning Platform That Enables You Focus on Your Models. ML-Sys WS 2017 @ NIPS

NSML : A Machine Learning Platform That Enables You Focus on Your Models. ML-Sys WS 2017 @ NIPS Nako Sung , Minkyu Kim, Hyunwoo Jo, Youngil Yang, Jinwoong Kim, Leonard Lausen, Youngkwan Kim, Gayoung Lee, Donghyun Kwak, Jung-Woo Ha, and Sunghun

1.24k views • 64 slides
AutonoVi-Sim: Modular Autonomous Vehicle Simulation Platform Supporting Diverse Vehicle Models,

AutonoVi-Sim: Modular Autonomous Vehicle Simulation Platform Supporting Diverse Vehicle Models,

AutonoVi-Sim: Modular Autonomous Vehicle Simulation Platform Supporting Diverse Vehicle Models, Sensor Configuration, and Traffic Conditions Andrew Best , Sahil Narang, Lucas Pasqualin, Daniel Barber, Dinesh Manocha University of North Carolina

642 views • 48 slides
Platform-Independent Debugging of Physical Interaction and Signal Flow Models Mehdi Dadfarnia 1

Platform-Independent Debugging of Physical Interaction and Signal Flow Models Mehdi Dadfarnia 1

Intro Background Debugging Methods Conclusions &amp; Future Work References &amp; Backup Slides Platform-Independent Debugging of Physical Interaction and Signal Flow Models Mehdi Dadfarnia 1 Raphael Barbau 2 1 National Institute of Standards

590 views • 40 slides
Delineating the Impact of Weightlessness on Human Physiology Using Computational Models Mohammad

Delineating the Impact of Weightlessness on Human Physiology Using Computational Models Mohammad

https://ntrs.nasa.gov/search.jsp?R=20160010063 2018-08-09T20:56:33+00:00Z 31 st Annual Meeting of ASGSR, Alexandria, VA Presidents Plenary Symposium Delineating the Impact of Weightlessness on Human Physiology Using Computational Models

944 views • 25 slides
Eye Witness Account: The Role of IT and Data Management in Expansion and Change at a Remote

Eye Witness Account: The Role of IT and Data Management in Expansion and Change at a Remote

Eye Witness Account: The Role of IT and Data Management in Expansion and Change at a Remote Research Unit in Kenya CODATA Conference 2002 Montreal, Canada 29 th September 3 rd October, 2002 Tom Oluoch - KEMRI/Wellcome Trust, Kilifi,

746 views • 20 slides
Accelerating cosmologies in an integrable model with noncommutative minisuperspace variables

Accelerating cosmologies in an integrable model with noncommutative minisuperspace variables

1 Accelerating cosmologies in an integrable model with noncommutative minisuperspace variables arXiv:1903.07895 [gr-qc] N a h o mi K a n ( N I T , G i f u C o l l e g e ) M a s a s h i K u n i y a s u ,

488 views • 23 slides
Syntactic Theory Tree-Adjoining Grammar (TAG) Yi Zhang Department of Computational Linguistics

Syntactic Theory Tree-Adjoining Grammar (TAG) Yi Zhang Department of Computational Linguistics

Syntactic Theory Tree-Adjoining Grammar (TAG) Yi Zhang Department of Computational Linguistics Saarland University November 17th, 2009 Outline XTAG: A Lexicalized Tree Adjoining Grammar for English NLP with Tree-Adjoining Grammars The XTAG

479 views • 23 slides
Heaving buoys in axisymmetric shallow water and the return to equilibrium problem Edoardo BOCCHI

Heaving buoys in axisymmetric shallow water and the return to equilibrium problem Edoardo BOCCHI

Heaving buoys in axisymmetric shallow water and the return to equilibrium problem Edoardo BOCCHI Supervisors: D. LANNES and C. PRANGE Institut de Math ematiques de Bordeaux 6 August, 2019 CEMRACS 2019, CIRM, Luminy Edoardo Bocchi (IMB,

268 views • 25 slides
Mathematical diseases in climate models and how to cure them Ali Ramadhan Valentin Churavy 50

Mathematical diseases in climate models and how to cure them Ali Ramadhan Valentin Churavy 50

Mathematical diseases in climate models and how to cure them Ali Ramadhan Valentin Churavy 50 km Image credit: NASA/Goddard Space Flight Center Scientific Visualization Studio Image credit: MITgcm LLC4320 output (Chris Hill &amp; Dimitris

716 views • 59 slides
Towards Visualization Literacy Jean-Daniel Fekete, INRIA with the help of: Jeremy Boy Luana

Towards Visualization Literacy Jean-Daniel Fekete, INRIA with the help of: Jeremy Boy Luana

Towards Visualization Literacy Jean-Daniel Fekete, INRIA with the help of: Jeremy Boy Luana Micallef Pierre Dragicevic Samuel Huron Benjamin Bach www.aviz.fr _ 4 INRIA Researchers 5 Post-docs 6 PhD students Lots of

542 views • 31 slides
CSC 151 Fall 2019 Topic: An Introduction to Algorithms August 30, 2019 Day 01 Agenda for today

CSC 151 Fall 2019 Topic: An Introduction to Algorithms August 30, 2019 Day 01 Agenda for today

CSC 151 Fall 2019 Topic: An Introduction to Algorithms August 30, 2019 Day 01 Agenda for today Introduction What is Computer Science? An Everyday Algorithm Wrap-up Introduce yourself 1) Please make a Name Card Preferred name

330 views • 12 slides
Supporting RTI -B in used in a manner consistent with TBSP training and recommendations and

Supporting RTI -B in used in a manner consistent with TBSP training and recommendations and

11/17/20 All TBSP training materials are informed by peer-reviewed research and our own field testing across the state. We make our training materials freely available to districts and schools on the following conditions: (1) the materials are

327 views • 19 slides

More recommend