A F amily of Data P a rallel Derivations Maurice Clint - PDF document

A F amily of Data P a rallel Derivations Maurice Clint Stephen Fitzpatrick T erence J� Ha rmer P eter L� Kilpatrick Depa rtment of Computer Science The Queen�s Universit y of Belfast Belfast� No rthern Ireland� UK James M� Bo yle Mathematics and Computer Science Division Argonne National Lab o rato ry Argonne� USA This w o rk is supp o rted b y SERC Grant GR�G ������ b y a resea rch studentship from the Depa rtment of Education fo r No rthern Ireland and b y the O�ce of Scienti�c Computing� U�S� Depa rtment of Energy � under Contract W��������Eng���

Cla rit y V E�ciency High�level� a rchitecture�indep endent p rograms Easier to construct � Easier to understand � P o rtable � E�cient p rograms T ailo red to pa rticula r machine� � non�p o rtable Aw ash with details � Di�cult to construct � Di�cult to understand � �

Example� T ransp ose of a Matrix T De�nition� transp ose of matrix A m � n A is an matrix such that n � m T � � i� � � A � j� i � � i� j A j High�level implementation function transpose �A �m �n � � generate�� n� m� �f n� i� j� ��A �j �i �� E�cient sequential implementation fo r squa re matrix � m � n � SUBROUTINE transpose� A�n � DO i���n DO j�i���n t �� A�i�j� A�i�j� �� A�j�i� A�j�i� �� t END END �

Our resolution Programmer constructs sp eci�cation and implementation derived� automatically Sp eci�cation language F unctional p rogramming language Mathematically based � Simple semantics� easily understo o d � Useful mathematical p rop erties � Executable p rotot yp es � Implementation language Whatever required b y implementation environ� ment� usually version of F o rtran o r C� E�cient � Go o d vendo r supp o rt � Mo re convenient than machine language � Derivation b y p rogram transfo rmation �

Program T ransfo rmations Program rewrite rules� patter n � r epl acement All o ccurrences of in p rogram changed patter n to acement � r epl Achieves a small� lo cal change � Based on fo rmal p rop erties � Clea rly p reserves meaning of p rogram F o rmally de�ned in wide sp ectrum gram� � ma r F o rmal p ro of p ossible � Derivations Sequences of transfo rmations Complete metamo rphosis through many ap� � plications of many transfo rmations Automatically applied b y T AMPR system � �

F amily of Derivations Derivation p erfo rmed in steps � Sub�derivations b et w een sp eci�cation � Intermediate fo rms and implementation languages F o r example� SML � �calculus F o rtran�� � � Same intermediate fo rm fo r� other sp eci�cation languages � other a rchitectures�implementation languages � Combinations have included� � F o rtran � SML � CRA Y F o rtran � � Lisp � �calculus � � D AP F o rtran Miranda � � � C �

Other sub�derivations�intermediate fo rms fo r� Optimization e�g� � function unfolding common sub exp ression elimination T ailo ring fo r pa rticula r fo rms of data � e�g� spa rse matrices SML Lisp λ -calculus Unfolded Evaluated Sparse Sectioned Processes & CRAY Processes Communication Array Form Common Subexpression Shared Distributed Common Memory Memory Subexpression Fortran77 Fortran90 DAP Fortran �

Example Matrix�vecto r multiplication � � � � � � � � � � � � � � � � � � a � � � � � � � � � � � � � � � � a � � b � � c � � d � b � � � � � � � � � � � � � � � � � c � � � � � � � � � � � � � � � � � � d � � � � � � � � � � � � � � � � � fun times�U�ve ct or �V �ve ct or �� ve ct or � generate�s iz e� U� �fn �i �i nt �� �U �� i� �V ��i �� fun sum�U�vect or �� re al � reduce�U�� �� �� � fun innerprodu ct �U �v ect or �V �v ec to r� �r ea l � sum�times� U� V� � fun mvmult�A�m at ri x� V�v ec to r� �v ec to r � generate�s iz e� A� ��� fn�i�int� �� in ne rpr od uc t� ro w� A� i� �V �� SML sp eci�cation Data pa rallel functions de�nes vecto r�matrix � generate combines elements of vecto r�matrix � reduce into single value �

Optimize� generate�� n� � � i � reduce��n� � � j � real�times � element�A� �i �j ��� element�V� �j �� �� real�plus� �� �� � Sequential�CRA Y implementation� and implemented as lo ops generate reduce DO i���n�� AV�i����� DO j���n�� AV�i��AV�i� �A �i �j �� V�j � ENDDO ENDDO D AP implementation� whole�a rra y op erations AV�sumc�A �ma tr �V �n �� �

Assessment T echniques have b een applied to mo re com� plex algo rithms fo r sequential�vecto r� a rra y and sha red�memo ry a rchitectures� Compa ring with indep endent� manually con� structed implementations� Derived implementations simila r� � Execution p erfo rmance equal o r b etter� � T echniques a re b eing extended fo r y et mo re complex algo rithms� fo r distributed and sha red memo ry pa rallel a rchitectures and fo r further sp ecial data structures� �

With derivational app roach� p rogrammer develops implementation techniques � enco des techniques as derivations � Reusabilit y Multiple sp eci�cations Multiple implementations of each Algo rithm mo di�ed� mo dify sp eci�cation and re�apply derivation Extensibilit y New optimization technique o r new a rchitecture o r new data rep resentation� �slot in� new sub�derivation T ransferabilit y Sub�derivation requires no exp ertise to use One p rogrammer ma y use another�s w o rk Co rrectness Co rrectness of transfo rmations implies co rrectness of implementation ��