Legacy Fortran to Python: High Performance Transpilation Mohamed Wahib In collab. with Anshu Dubey [ANL] 26 th OCTOBER 2017 Dagstuhl Seminar
“This code is my asset, supercomputers, architectures come and ago; my code will be used in production for decades” A climate scientist in a heated discussion 2
Legacy Code Options ( not attractive … ) Do you need performance portability? YES NO You use different [next slide] architectures ? YES NO Let me guess … you use low- Lucky man! performance high productivity tools (ex: Matlab)? NO YES Nevermind I will wait for you guys to solve that, meanwhile let me do my science/engineering/math at my slow pace 3
Legacy Code Options ( not attractive … ) Do you need performance portability? YES Your options Source-to-source Maintain different Framework/DSL versions of the code - Only few can afford cost - Takes effort to learn and port - Code generated might not be readable - Fear of commitment - Has been done forever (i.e. will the DSL be around - A “hack”, and not a (i.e. #if #def ) systematic solution, as DSLs in 5~10 years?) - Less effort for user - Debugging, dependencies, .. etc - Worst case, you keep - Hardly successful beyond your original slow code “home” application 4
Source-to-source § Under the following assumptions: § Realistic goals à not on par with DSLs (in achiev. performance) § Focused à only subset of Fortran, specific parallel patterns § Incremental and best-effort approach § The generated code should be readable 5
“Progressive” Source-to-source § If putting in the effort for source-to-source solution § Two additional goals are possible First goal § “Empower developers to navigate design choices in numerical methods” , Paul Kelly in Flash talks § Prototyping mode Second goal § Provide the means to migrate away from the legacy code 6
“Progressive” Source-to-source § To achieve the two extra goals, we propose: Behavior Syntax Hot Kernels /Performance Fortran (v1) Metadata Generate Hot Kernels Fortran (v2) Translate Fortran Annotated (Application) Python Compile (f2py) Interpret Extract (CPython) Runtime § Python is essentially an IR (Intermediate Representation) § Counter-intuitive; IR are typically lower level than source 7
“Progressive” Source-to-source § The user now has: § Original Fortran Code § Target-optimized version(s) of hotspot kernels § Python version of code (that can use hotspot kernels) § So what to do? § Plug-in optimized hot spot kernels in original Fortran code § Use the Python version for prototyping (with Python goodies) § Drop the original Fortran version, migrate to the Python version 8
“Progressive” Source-to-source Fortran (Opt. Target 1) Translate Generate Fortran Annotated Second extra Goal Fortran (Opt. Target 2) Python (Migrate) First extra Goal (Prototype) Prototyping Mode (Use Py goodies) 9
Two-way Translation 10
Recipe for HPC-enabled Fortran à Python Migration 11
For it to Work Defined subset (ex: adopting constraints enforced on FLASH kernel programmers) 12
Type Hints in Python 13
Selected Performance Issues in Python There are many more … 14
How to Make Python Application Faster? 15
Another approach: delegation 16
Evaluation Environment 17
Matrix Multiplication 18
Matrix Multiplication – compilation/translation overhead 19
Matrix Multiplication – compiled binary reuse 20
Matrix Multiplication – Computational Results 21
Migrating Fortran: Miranda IO 22
Miranda IO – compilation/translation overhead 23
Miranda IO – Computational Results 24
Defining the Mapping - Types 25
Defining the Mapping – Example of Supported Syntax 26
Defining the Mapping – Example of Supported Syntax 27
Defining the Mapping – Example of Unsupported Syntax 28
Conclusion § A pragmatic, realistic, and focused source-to-source approach § Two-way transpilation: Fortran à Python à Fortran § Can also be us`eful for prototyping § Can provide a means to migrate from the legacy code § Missing: infrastructure for source-to-source translation Code at GitHub: https://github.com/mbdevpl?tab=repositories 29
Boosting Python – existing approaches and our approach 30
Recommend
More recommend
