broadcast trees for heterogeneous platforms

play

Broadcast Trees for Heterogeneous Platforms Olivier Beaumont, Yves - PowerPoint PPT Presentation

Dec 19, 2022 •341 likes •1.28k views

Broadcast Trees for Heterogeneous Platforms Olivier Beaumont, Yves Robert and Loris Marchal Laboratoire de lInformatique du Parall elisme Ecole Normale Sup erieure de Lyon, France Yves.Robert@ens-lyon.fr http://graal.ens-lyon.fr/

Broadcast Trees for Heterogeneous Platforms Olivier Beaumont, Yves Robert and Loris Marchal Laboratoire de l’Informatique du Parall´ elisme ´ Ecole Normale Sup´ erieure de Lyon, France Yves.Robert@ens-lyon.fr http://graal.ens-lyon.fr/ ∼ yrobert December 2004 Yves Robert Broadcast Trees for Heterogeneous Platforms 1/ 34
Outline Introduction Models and Framework Platform-based Heuristics One port-model Multi-port LP-based heuristics Simulations Conclusion Yves Robert Broadcast Trees for Heterogeneous Platforms 2/ 34
Outline Introduction Models and Framework Platform-based Heuristics One port-model Multi-port LP-based heuristics Simulations Conclusion Yves Robert Broadcast Trees for Heterogeneous Platforms 2/ 34
Outline Introduction Models and Framework Platform-based Heuristics One port-model Multi-port LP-based heuristics Simulations Conclusion Yves Robert Broadcast Trees for Heterogeneous Platforms 2/ 34
Outline Introduction Models and Framework Platform-based Heuristics One port-model Multi-port LP-based heuristics Simulations Conclusion Yves Robert Broadcast Trees for Heterogeneous Platforms 2/ 34
Outline Introduction Models and Framework Platform-based Heuristics One port-model Multi-port LP-based heuristics Simulations Conclusion Yves Robert Broadcast Trees for Heterogeneous Platforms 2/ 34
Outline Introduction Models and Framework Platform-based Heuristics One port-model Multi-port LP-based heuristics Simulations Conclusion Yves Robert Broadcast Trees for Heterogeneous Platforms 2/ 34
Outline Introduction Models and Framework Platform-based Heuristics One port-model Multi-port LP-based heuristics Simulations Conclusion Yves Robert Broadcast Trees for Heterogeneous Platforms 3/ 34
Broadcasting data ◮ Key collective communication operation ◮ Start: one processor has the data ◮ End: all processors own a copy ◮ Vast literature about broadcast, MPI Bcast ◮ Standard approach: use a spanning tree ◮ Finding the best spanning tree: NP-Complete problem (even in the telephone model) Yves Robert Broadcast Trees for Heterogeneous Platforms 4/ 34
Broadcasting data ◮ Key collective communication operation ◮ Start: one processor has the data ◮ End: all processors own a copy ◮ Vast literature about broadcast, MPI Bcast ◮ Standard approach: use a spanning tree ◮ Finding the best spanning tree: NP-Complete problem (even in the telephone model) Yves Robert Broadcast Trees for Heterogeneous Platforms 4/ 34
Broadcasting data ◮ Key collective communication operation ◮ Start: one processor has the data ◮ End: all processors own a copy ◮ Vast literature about broadcast, MPI Bcast ◮ Standard approach: use a spanning tree ◮ Finding the best spanning tree: NP-Complete problem (even in the telephone model) Yves Robert Broadcast Trees for Heterogeneous Platforms 4/ 34
Broadcasting data ◮ Key collective communication operation ◮ Start: one processor has the data ◮ End: all processors own a copy ◮ Vast literature about broadcast, MPI Bcast ◮ Standard approach: use a spanning tree ◮ Finding the best spanning tree: NP-Complete problem (even in the telephone model) Yves Robert Broadcast Trees for Heterogeneous Platforms 4/ 34
Broadcasting data ◮ Key collective communication operation ◮ Start: one processor has the data ◮ End: all processors own a copy ◮ Vast literature about broadcast, MPI Bcast ◮ Standard approach: use a spanning tree ◮ Finding the best spanning tree: NP-Complete problem (even in the telephone model) Yves Robert Broadcast Trees for Heterogeneous Platforms 4/ 34
Different broadcast problems Broadcast large messages ⇒ pipelining strategies ◮ split the messages into slices (application level) ◮ route them concurrently, possibly using different spanning trees ◮ throughput optimization (relaxation of makespan minimization) STA Singe Tree, Atomic message heuristics to minimize makespan: FNF. . . STP Single Tree, Pipelined series of messages MTP Multiple Tree, Pipelined series of messages ◮ polynomial algorithm to find optimal solution (LP formulation) ◮ hard to implement ⇒ concentrate on STP Yves Robert Broadcast Trees for Heterogeneous Platforms 5/ 34
Different broadcast problems Broadcast large messages ⇒ pipelining strategies ◮ split the messages into slices (application level) ◮ route them concurrently, possibly using different spanning trees ◮ throughput optimization (relaxation of makespan minimization) STA Singe Tree, Atomic message heuristics to minimize makespan: FNF. . . STP Single Tree, Pipelined series of messages MTP Multiple Tree, Pipelined series of messages ◮ polynomial algorithm to find optimal solution (LP formulation) ◮ hard to implement ⇒ concentrate on STP Yves Robert Broadcast Trees for Heterogeneous Platforms 5/ 34
Different broadcast problems Broadcast large messages ⇒ pipelining strategies ◮ split the messages into slices (application level) ◮ route them concurrently, possibly using different spanning trees ◮ throughput optimization (relaxation of makespan minimization) STA Singe Tree, Atomic message heuristics to minimize makespan: FNF. . . STP Single Tree, Pipelined series of messages MTP Multiple Tree, Pipelined series of messages ◮ polynomial algorithm to find optimal solution (LP formulation) ◮ hard to implement ⇒ concentrate on STP Yves Robert Broadcast Trees for Heterogeneous Platforms 5/ 34
Different broadcast problems Broadcast large messages ⇒ pipelining strategies ◮ split the messages into slices (application level) ◮ route them concurrently, possibly using different spanning trees ◮ throughput optimization (relaxation of makespan minimization) STA Singe Tree, Atomic message heuristics to minimize makespan: FNF. . . STP Single Tree, Pipelined series of messages MTP Multiple Tree, Pipelined series of messages ◮ polynomial algorithm to find optimal solution (LP formulation) ◮ hard to implement ⇒ concentrate on STP Yves Robert Broadcast Trees for Heterogeneous Platforms 5/ 34
Different broadcast problems Broadcast large messages ⇒ pipelining strategies ◮ split the messages into slices (application level) ◮ route them concurrently, possibly using different spanning trees ◮ throughput optimization (relaxation of makespan minimization) STA Singe Tree, Atomic message heuristics to minimize makespan: FNF. . . STP Single Tree, Pipelined series of messages MTP Multiple Tree, Pipelined series of messages ◮ polynomial algorithm to find optimal solution (LP formulation) ◮ hard to implement ⇒ concentrate on STP Yves Robert Broadcast Trees for Heterogeneous Platforms 5/ 34
Different broadcast problems Broadcast large messages ⇒ pipelining strategies ◮ split the messages into slices (application level) ◮ route them concurrently, possibly using different spanning trees ◮ throughput optimization (relaxation of makespan minimization) STA Singe Tree, Atomic message heuristics to minimize makespan: FNF. . . STP Single Tree, Pipelined series of messages MTP Multiple Tree, Pipelined series of messages ◮ polynomial algorithm to find optimal solution (LP formulation) ◮ hard to implement ⇒ concentrate on STP Yves Robert Broadcast Trees for Heterogeneous Platforms 5/ 34
Different broadcast problems Broadcast large messages ⇒ pipelining strategies ◮ split the messages into slices (application level) ◮ route them concurrently, possibly using different spanning trees ◮ throughput optimization (relaxation of makespan minimization) STA Singe Tree, Atomic message heuristics to minimize makespan: FNF. . . STP Single Tree, Pipelined series of messages MTP Multiple Tree, Pipelined series of messages ◮ polynomial algorithm to find optimal solution (LP formulation) ◮ hard to implement ⇒ concentrate on STP Yves Robert Broadcast Trees for Heterogeneous Platforms 5/ 34
Outline Introduction Models and Framework Platform-based Heuristics One port-model Multi-port LP-based heuristics Simulations Conclusion Yves Robert Broadcast Trees for Heterogeneous Platforms 6/ 34
Models Network = directed graph P = ( V, E ) P 1 P 0 P 3 P 2 time ◮ General case: affine model (includes latencies) ◮ Common variant: sending and receiving processors busy during whole transfer Yves Robert Broadcast Trees for Heterogeneous Platforms 7/ 34
Models Network = directed graph P = ( V, E ) P 1 P 0 P 3 P 2 time ◮ General case: affine model (includes latencies) ◮ Common variant: sending and receiving processors busy during whole transfer Yves Robert Broadcast Trees for Heterogeneous Platforms 7/ 34
Models Network = directed graph P = ( V, E ) P 1 link e 2 , 3 T 2 , 3 ( L ) P 0 P 3 P 2 time ◮ General case: affine model (includes latencies) ◮ Common variant: sending and receiving processors busy during whole transfer Yves Robert Broadcast Trees for Heterogeneous Platforms 7/ 34
Models Network = directed graph P = ( V, E ) P 1 P 2 send 2 , 3 link e 2 , 3 T 2 , 3 ( L ) P 0 P 3 P 2 time ◮ General case: affine model (includes latencies) ◮ Common variant: sending and receiving processors busy during whole transfer Yves Robert Broadcast Trees for Heterogeneous Platforms 7/ 34

Download Presentation

Download Policy: The content available on the website is offered to you 'AS IS' for your personal information and use only. It cannot be commercialized, licensed, or distributed on other websites without prior consent from the author. To download a presentation, simply click this link. If you encounter any difficulties during the download process, it's possible that the publisher has removed the file from their server.

Recommend

More recommend