Overreaching with Outreach: Is HPC in Reach? Maggie Myers Robert - - PowerPoint PPT Presentation

Overreaching with Outreach: Is HPC in Reach?

Maggie Myers Robert van de Geijn

Are we crazy?

Three Massive Open Online Courses:

• Linear Algebra: FoundaEons to FronEers
• LAFF-On Programming for Correctness
• (LAFF-On) Programming for HPC

EducaEng the world about our world.

Insights for HPC we want to share

• When presenEng algorithms, abstract away from

details.

• When represenEng algorithms in code, use APIs.
• When opEmizing or parallelizing, don’t just start

with legacy code. Derive families of algorithms.

• When implemenEng soQware, layer.
• When sharing gained insights use the simplest

examples.

Where are we in our journey?

• LAFF

– Sixth offering on edX.

• LAFF-On Programming for Correctness

– First offered Spring 2017

• Programming for HPC

– Planning

What is LAFF?

• Linear Algebra: FoundaEons to FronEers (LAFF)
• A full semester class
• Connects the abstracEons in mathemaEcs with

algorithms and their instanEaEon in code.

– Layering of matrix-matrix/matrix-vector/vector-vector

• peraEons linked to layering with BLAS.
• Targets novices and those who want a refresher.
• Enriches fundamental linear algebra with HPC issues.
• Introduces FLAME notaEon and, in enrichments,

acquaints the world with our research.

Linear Algebra: Linear Algebra: Foundations to Fr Foundations to Frontiers

• ntiers

Slice and Dice

² Recognize, apply, verify, and make connecEons between properEes of vectors and their operaEons including geometric interpretaEons, orthogonality, length, vector algebra, and representaEons as linear combinaEons of unit basis vector. ² Develop and evaluate algorithms for matrix and vector operaEons. ² Make use of decomposiEon and blocking methods as well as recognize, develop and apply special characterisEcs of matrices including whether a matrix is triangular, symmetric, diagonal, and inverEble.

Transform and Solve

² Recognize, apply, and verify properEes of linear transformaEons, including their connecEon to matrices and systems

• f linear equaEons.

² Determine soluEons for systems of equaEons by applying a variety of methods including Gaussian eliminaEon as well as various LU and QR factorizaEon methods, and compare and contrast the different approaches. ² Find eigenvalues and eigenvectors

• f a matrix.

Abstract

² Develop and evaluate a variety of algorithms for determining the soluEons of systems. ² Understand and apply noEons such as span, column space, null space, rank, and linear independence in the context of vector spaces and subspaces. ² Recognize, develop, and apply various characterizaEons of linear independence. ² Create a small library of basic linear algebra subrouEnes.

You will learn to

How’s it goin’?

• Heartbeat

Spring 2014 Spring 2015 Summer 2015 Fall 2016 Spring 2016 Fall 2017

How’s it goin’?

Fall 2016 Spring 2016 Fall 2017

LAFF-On Programming for Correctness

• For many HPC applicaEons, including dense matrix

computaEons, implementaEons of key operaEons are loop-based.

• Classic techniques developed in the 1960s by Dijkstra

et al. use a loop invariant and the Principle of MathemaEcal InducEon to prove a loop correct.

• FLAME NotaEon hides indices and facilitates formal

derivaEon of families of algorithms.

• A priori determinaEon of loop invariants.
• Important for HPC: best algorithm for situaEon can be

chosen.

Annotate programs with asserEons about the states of variables. Prove programs correct with Hoare triples and weakest precondiEons. Apply goal oriented programming techniques to derive programs hand-in-hand with their proofs of correctness Avoid indexing errors by avoiding indexing. Determine loop invariants a priori. Develop families of algorithms for matrix and vector operaEons.

LAFF-On Programming for Correctness

How’s it goin’?

First offering First six weeks On campus course

The beat goes on…

And now (LAFF-On) Programming for HPC

• Illustrate issues in HPC with simple example:

matrix-matrix mulEplicaEon (gemm)

Envision three parts:

• OpEmizing gemm (for single core and

mulEcore)

• Data movement on distributed memory

architectures (MPI and collecEve communicaEon)

• Parallelizing gemm to distributed memory

Resources

• K. Goto and R. van de Geijn. Anatomy of High-Performance Matrix
• Mul7plica7on. ACM TOMS, 2008.
• F. Van Zee and R. van de Geijn. BLIS: A Framework for Rapidly

Instan7a7ng BLAS Func7onality. ACM TOMS, 2015.

• T. Smith et al. Anatomy of High-Performance Many-Threaded Matrix
• Mul7plica7on. IPDPS 2014.
• J. Huang et al. Strassen's Algorithm Reloaded. SC'16
• hlps://github.com/flame/how-to-opEmize-gemm/wiki
• hlps://github.com/flame/blislab
• E. Chan et al. Collec7ve communica7on: theory, prac7ce, and
• experience. C&C:P&E, 2007.
• M. Schatz et al. Parallel Matrix Mul7plica7on: A Systema7c Journey.

SISC, 2016.

Challenges

• Funding
• Access to interesEng architectures
• Keeping our sanity

Summary

• MOOCs are a way for us to reach a broad

audience.

• Benefits?

– Open source, open plaporm, open educaEon,

• pen HPC?

– Crowd source opEmizing DLA? – Demonstrate broad impact to funders!

• Can they democraEze HPC?

Are we Crazy?

Acknowledgments

Linear Algebra: FoundaEons to FronEers

• ParEally funded by

– First NSF SSI grant – University of Texas System

LAFF-On Programming for Correctness

• ParEally funded by

– Second NSF SSI grant – A giQ from Mathworks – Profit sharing with edX

(LAFF-On) Programming for HPC

• ParEally funded by

– NSF grant on Strassen’s algorithm – …