A Parallel, In-Place, Rectangular Matrix Transpose Algorithm - PowerPoint PPT Presentation

Stefan Amberger ICA & RISC amberger.stefan@gmail.com A Parallel, In-Place, Rectangular Matrix Transpose Algorithm Computational Complexity Analysis

Table of Contents 1. Introduction 2. Revision of TRIP 3. Analysis of Computational Complexity a. Work b. Span c. Parallelism d. Generalizations 2

Introduction 3

Introduction Computational Complexity of Parallel Algorithms Work “execution time on one processor” Introduction to Algorithms Third Edition, p777ff i.e.: all vertices of computation dag approximation: # of nodes of computation dag Parallelism Span ● average amount of work per step along critical path “execution time on infinitely many processors” ● maximum possible speedup i.e. length of critical path of computation dag ● limit on possibility of attaining perfect approximation: # of nodes on critical path speedup 4

Revision of TRIP 5

TRIP : If matrix is rectangular TRIP transposes sub-matrices, then combines the result with merge or split merge : first rotates the middle part of the array, then recursively merges the left and right parts of the array split : first recursively splits the left and right parts of the array, then rotates the middle part of the array 6

Analysis of Computational Complexity 7

Restriction to Powers of Two “power condition” Matrix dimensions M x N and N x M recursive calls are symmetric TRIP ’s recursive call are either all merge or all split 8

Work 1. Example: Basic Algorithms 2. TRIP Result & Proof Sketch 9

Work Example Work of Base Algorithms 10

Result Work of TRIP Show that under power condition, for M x N matrix METHOD don’t count vertices in computation dag count inner nodes in recursion tree and swaps ● function calls (nodes in recursive call trees) ● memory accesses (swaps) 11

Work of TRIP Proof Sketch Work of TRIP Work of merge Work of rol 12

Work of TRIP wide matrices TRIP recursion is analogous for tall and wide matrices only difference: ● in merge rol is called before the recursive merge call ● in splitt rol is called after the recursive split call This difference does not cause a change in the amount of work of TRIP . 13

Visualization Work as function of Matrix Dimensions 14

Span 1. Example: Basic Algorithms 2. TRIP Result 15

Span Example Span of Base Algorithms 16

Span of TRIP Result Calculate span of tall matrix transpose count levels and swaps on critical path, that includes span of ● creating the divide tree ● combining the nodes via merge / split (itself recursive procedures) ● square-transposing in the leaf nodes 17

Visualization Span as function of Matrix Dimensions 18

Parallelism 19

Result Rectangular Matrices Square Matrices calculation: ● divide work by span ● case distinction rectangular / square ● simplification using Landau symbols 20

Generalizations power condition unsatisfied 21

Example: 7 x 5 Matrix 22

Generalization Power Condition not Satisfied 23

Generalization Power Condition not Satisfied 24

Generalization Power Condition not Satisfied 25

Thank you

Revision of TRIP 27

TRIP Algorithm If matrix is rectangular TRIP transposes sub-matrices, then combines the result with merge or split 28

merge Algorithm merge combines the transposes of sub-matrices of tall matrices merge first rotates the middle part of the array, then recursively merges the left and right parts of the array rol(arr, k) … left rotation (circular shift) of array arr by k elements 29

split Algorithm split combines the transposes of sub-matrices of wide matrices split first recursively splits the left and right parts of the array, then rotates the middle part of the array split and merge are inverse to each other 30

Work Proof 31

Work of TRIP Overview Calculate work of tall matrix transpose ● spanning the divide tree ● combining the nodes via merge / split (itself recursive procedures) ● square-transposing in the leaf nodes 32

Work of TRIP Proof - TRIP Tree Combining Nodes via merge Spanning Divide Tree 33

Work of TRIP Proof - Merge Tree Combining via merge , rotate sub-arrays 34

Work of TRIP Proof - Merge Tree Integrate rol result into merge work 35

Work of TRIP Proof - TRIP Tree Integrate merge result into TRIP work 36

Work of TRIP Proof - Square Transpose Recap 37

Work of TRIP Proof - Square Transpose Lower Bound on # of inner nodes Upper Bound on # of inner nodes purely ternary tree purely quaternary tree 38

Work of TRIP Proof - TRIP Tree Integrate square transpose result into TRIP work work of square transpose (including swapping) 39

Conclusions 40

Conclusions Novel Algorithm TRIP transposes rectangular matrices ● correctly ● in-place ● in highly parallel manner 41

Roadmap 1. Work 2. Span 3. Parallelism 42