SLIDE 1
Legacy Fortran to Python: High Performance Transpilation
Mohamed Wahib
In collab. with Anshu Dubey [ANL] 26th OCTOBER 2017 Dagstuhl Seminar
2
Legacy Fortran to Python: High Performance Transpilation Mohamed - - PowerPoint PPT Presentation
Legacy Fortran to Python: High Performance Transpilation Mohamed - - PowerPoint PPT Presentation
Legacy Fortran to Python: High Performance Transpilation Mohamed Wahib In collab. with Anshu Dubey [ANL] 26 th OCTOBER 2017 Dagstuhl Seminar This code is my asset, supercomputers, architectures come and ago; my code will be used in
SLIDE 2
SLIDE 3
Legacy Code Options (not attractive …)
3
Do you need performance portability? You use different architectures ? I will wait for you guys to solve that, meanwhile let me do my science/engineering/math at my slow pace Lucky man! [next slide] NO NO YES YES Let me guess … you use low- performance high productivity tools (ex: Matlab)? Nevermind YES NO
SLIDE 4
4
Do you need performance portability? Your options Maintain different versions of the code Framework/DSL YES
Legacy Code Options (not attractive …)
- Has been done forever
(i.e. #if #def )
- Only few can afford cost
- Takes effort to learn and port
- Fear of commitment
(i.e. will the DSL be around in 5~10 years?)
- Hardly successful beyond
“home” application
- Debugging, dependencies, .. etc
Source-to-source
- Less effort for user
- Worst case, you keep
your original slow code
- Code generated
might not be readable
- A “hack”, and not a
systematic solution, as DSLs
SLIDE 5
§ Under the following assumptions:
§ Realistic goals à not on par with DSLs (in achiev. performance) § Focused à only subset of Fortran, specific parallel patterns § Incremental and best-effort approach § The generated code should be readable
5
Source-to-source
SLIDE 6
§ “Empower developers to navigate design choices in numerical methods”, Paul Kelly in Flash talks § Prototyping mode
6
“Progressive” Source-to-source
§ Provide the means to migrate away from the legacy code First goal Second goal § If putting in the effort for source-to-source solution
§ Two additional goals are possible
SLIDE 7
7
“Progressive” Source-to-source
§ To achieve the two extra goals, we propose: § Python is essentially an IR (Intermediate Representation) § Counter-intuitive; IR are typically lower level than source
Fortran (Application) Annotated Python Behavior /Performance Metadata Hot Kernels Fortran (v1) Hot Kernels Fortran (v2) Runtime Translate Extract Interpret (CPython) Compile (f2py) Syntax Generate
SLIDE 8
8
“Progressive” Source-to-source
§ The user now has:
§ Original Fortran Code § Target-optimized version(s) of hotspot kernels § Python version of code (that can use hotspot kernels)
§ So what to do?
§ Plug-in optimized hot spot kernels in original Fortran code § Use the Python version for prototyping (with Python goodies) § Drop the original Fortran version, migrate to the Python version
SLIDE 9
9
“Progressive” Source-to-source
Annotated Python Fortran (Opt. Target 1) Fortran (Opt. Target 2) Fortran Translate Generate
Prototyping Mode (Use Py goodies) First extra Goal (Prototype) Second extra Goal (Migrate)
SLIDE 10
10
Two-way Translation
SLIDE 11
11
Recipe for HPC-enabled Fortran à Python Migration
SLIDE 12
12
For it to Work
Defined subset (ex: adopting constraints enforced on FLASH kernel programmers)
SLIDE 13
13
Type Hints in Python
SLIDE 14
14
Selected Performance Issues in Python
There are many more …
SLIDE 15
15
How to Make Python Application Faster?
SLIDE 16
16
Another approach: delegation
SLIDE 17
17
Evaluation Environment
SLIDE 18
18
Matrix Multiplication
SLIDE 19
19
Matrix Multiplication – compilation/translation overhead
SLIDE 20
20
Matrix Multiplication – compiled binary reuse
SLIDE 21
21
Matrix Multiplication – Computational Results
SLIDE 22
22
Migrating Fortran: Miranda IO
SLIDE 23
23
Miranda IO – compilation/translation overhead
SLIDE 24
24
Miranda IO – Computational Results
SLIDE 25
25
Defining the Mapping - Types
SLIDE 26
26
Defining the Mapping – Example of Supported Syntax
SLIDE 27
27
Defining the Mapping – Example of Supported Syntax
SLIDE 28
28
Defining the Mapping – Example of Unsupported Syntax
SLIDE 29
§ A pragmatic, realistic, and focused source-to-source approach § Two-way transpilation: Fortran à Python à Fortran § Can also be us`eful for prototyping § Can provide a means to migrate from the legacy code § Missing: infrastructure for source-to-source translation
Conclusion
29
Code at GitHub: https://github.com/mbdevpl?tab=repositories
SLIDE 30
30
Boosting Python – existing approaches and
- ur approach