SLIDE 1
Automation in Dense Linear Algebra
Paper by Paolo Bientinesi and Robert van de Geijn
2
Automation in Dense Linear Algebra Paper by Paolo Bientinesi and - - PowerPoint PPT Presentation
Automation in Dense Linear Algebra Paper by Paolo Bientinesi and - - PowerPoint PPT Presentation
Automation in Dense Linear Algebra Paper by Paolo Bientinesi and Robert van de Geijn Presented by Smy Zehnder Content Motivation Building a new algorithm Prototype Conclusion 2 Motivation Best algorithm for a dense linear algebra
SLIDE 2
SLIDE 3
3
Motivation
- Best algorithm for a dense linear algebra problem?
- LU
- Cholesky
- Eigenvalues
- SVD
- ...
- Highly used for:
- Interpolation of functions
- Solving systems of equations
- Optimization
SLIDE 4
4
Idea: Automatically build it!
Problem X Y
http://www.aices.rwth-aachen.de:8080/~pauldj/pubs/CC-2009.pdf
SLIDE 5
5
Content
Motivation Building a new algorithm Prototype Conclusion
SLIDE 6
6
Loop Loop-invariant PME
Steps needed
Γ(X) X Y
Code
SLIDE 7
7
Layout of the algorithm
SLIDE 8
8
Example: Cholesky factorization
- How do we compute L, so that A = LL
T ?
Cholesky Γ(A) = L A L
Γ(X)
PME L-Inv. Loop Code
SLIDE 9
9
Problem to PME
A=L L
T
Γ( A)=L
Γ(X)
PME L-Inv. Loop Code
SLIDE 10
10
Problem to PME
A=L L
T
Γ( A)=L
Γ(X)
PME L-Inv. Loop Code
SLIDE 11
11
Problem to PME
A=L L
T
Γ( A)=L
Γ(X)
PME L-Inv. Loop Code
SLIDE 12
12
Problem to PME
A=L L
T
Γ( A)=L
Γ(X)
PME L-Inv. Loop Code
SLIDE 13
13
Problem to PME
Γ(X)
PME L-Inv. Loop Code
SLIDE 14
14
Loop-invariant
- Holds before, at the begin and after the loop
Γ(X)
PME L-Inv. Loop Code
SLIDE 15
15
Loop-invariant
- Holds before, at the begin and after the loop
Γ(X)
PME L-Inv. Loop Code
SLIDE 16
16
Choosing a Loop-invariant
- Any subset of the
PME
- Some blocks can be
0x0-matrices
- Has to respect the
dependencies
http://www.aices.rwth-aachen.de:8080/~pauldj/pubs/CC-2009.pdf
Γ(X)
PME L-Inv. Loop Code
SLIDE 17
17
Choosing a Loop-invariant
Γ(X)
PME L-Inv. Loop Code
SLIDE 18
18
Choosing a Loop-invariant
- At the beginning:
Γ(X)
PME L-Inv. Loop Code
SLIDE 19
19
Choosing a Loop-invariant
- At the beginning:
- At the end:
Γ(X)
PME L-Inv. Loop Code
SLIDE 20
20
Ensuring progress
Γ(X)
PME L-Inv. Loop Code
SLIDE 21
21
Construction of the loop
Γ(X)
PME L-Inv. Loop Code
SLIDE 22
22
Construction of the loop
Γ(X)
PME L-Inv. Loop Code
PME:
SLIDE 23
23
Construction of the loop
Γ(X)
PME L-Inv. Loop Code
PME:
SLIDE 24
24
- 1. Building the PME
- 2. Choosing the Loop-Invariant
- 3. Construction of the loop
- Repartioning
- Computation of one step
Recap
Γ(X)
PME L-Inv. Loop Code
SLIDE 25
25
- 1. Building the PME
- 2. Choosing the Loop-Invariant
- 3. Construction of the loop
- Repartioning
- Computation of one step
Recap
Γ(X)
PME L-Inv. Loop Code
SLIDE 26
26
- 1. Building the PME
- 2. Choosing the Loop-Invariant
- 3. Construction of the loop
- Repartioning
- Computation of one step
Recap
Γ(X)
PME L-Inv. Loop Code
SLIDE 27
27
- 1. Building the PME
- 2. Choosing the Loop-Invariant
- 3. Construction of the loop
- Repartioning
- Computation of one step
Recap
Γ(X)
PME L-Inv. Loop Code
SLIDE 28
28
Content
Motivation Building a new algorithm Prototype Conclusion
SLIDE 29
29
Prototype system
- Takes loop-invariant, returns loop-algorithm
- Generates worksheet, Matlab- or C-code
- More than 300 algorithms for the Level-3 BLAS
library
- Found 50 algorithms for the triangular coupled
Sylvester equation (3 previously known)
Γ(X)
PME L-Inv. Loop Code
SLIDE 30
30
Prototype system
http://www.aices.rwth-aachen.de:8080/~pauldj/pubs/CC-2009.pdf
function [A] = choleskyL1( A , nb ) [ ATL, ATR, ... ABL, ABR ] = FLA_Part_2x2( A,0,0,’FLA_TL’);
%% Loop Invariant %% ATL=choleskyL[ATL] %% ABL’=0 %% ABL=ABL %% ABR=ABR
while( size(ATL,1) ~= size(A,1) | size(ATL,2) ~= size(A,2) ) b = min( nb, min( size(ABR,1), size(ABR,2) )); [ A00, A01, A02, ...
A10, A11, A12, ... A20, A21, A22 ] = FLA_Repart_2x2_to_3x3(ATL, ATR,... ABL, ABR,... b, b, ’FLA_BR’);
%* *********************************************************** *% A10 = A10 . inv(A00)’;
A11 = choleskyL(A11 - A10 . A10’);
%* *********************************************************** *% [ ATL, ATR, ...
ABL, ABR ] = FLA_Cont_with_3x3_to_2x2(A00, A01, A02, ... A10, A11, A12, ... A20, A21, A22, ’FLA_TL’);
end; return ATL; Γ(X)
PME L-Inv. Loop Code
SLIDE 31
31
Content
Motivation Building a new algorithm Prototype Conclusion
SLIDE 32
32
Conclusion
+ Problem description (A = LLT) sufficient for
automatic algorithm generation
+ Possible to generate proof of correctness side
by side with generation of algorithm
+ Performance: Familiy of algorithms =>
autotune these
- Numerical stability is not ensured.
Proof for every algorithm needed.
http://kaichido.com/wp-content/uploads/2010/10/Fotolia_733842_XS-Balance-scale1.jpg
SLIDE 33
33