Libabigail & ABI compatibility Taming the runtime linking - - PowerPoint PPT Presentation

Libabigail & ABI compatibility

Taming the runtime linking problem

Ben Woodard Consulting Engineer Dodji Seketeli Tools Engineer Aug 2016

INSERT DESIGNATOR, IF NEEDED

Problems due to ABI Compatibility

2

There are several problems cause by ABI compatibility problems either real or feared.

• Linking objects compiled by different compilers not recommended

○ Unforeseen problem at runtime ○ Need to compile libs for all compilers

• Library Version problems

○ Match library version with application ○ Match compiler

INSERT DESIGNATOR, IF NEEDED

Lots of compilers

3

We use a lot of compilers...

• GCC
• Intel
• PGI
• XL
• LLVM

INSERT DESIGNATOR, IF NEEDED

Lots of compilers

4

We use a lot of compilers...and a lot of different version of the compilers.

• GCC

○ 4.4.7, 4.7.2, 4.8.2, 4.9.2, 5.3, 6.1.0

• Intel

○ 9.1, 1[0-4].[01], 1[56]

• PGI

○ 7.[01], [89].0, 10.[2,9], 11.{1,10}, 12.[18], 13.{1,2,6,10}, 14.{3,7,10}, 15.

{1,5,7,10}, 16.[13]

• XL

○ ???

• LLVM

○ 3.[4-8]

INSERT DESIGNATOR, IF NEEDED

Runtime Linking Problems

5

Linking objects compiled by different compilers can sometimes cause unexplained problems. Why?

• ABI is supposed to make this work properly.

○ This does work to some extent with system libraries ○ Other libraries may hit corner cases especially C++

• We are talking about ABI not API

○ API directly affects this.

INSERT DESIGNATOR, IF NEEDED

Red Hat’s Problem

6

We make API and ABI guarantees to customers

• API and ABI should not change within a major release of RHEL
• There are some specific exceptions
• Manual verification very time consuming
• Occasionally mistakes have been made.
• Fedora doesn’t want to rebuild everything all the time
• Future projects in the devops world

INSERT DESIGNATOR, IF NEEDED

What is ABI?

7

• ABI is all the things that can affect inter-object linking
• It is not just what is written in the standard for Architecture and

language

• A first approximation is ELF and DWARF
• ??

INSERT DESIGNATOR, IF NEEDED

Libabigail

8

Libabigail reads ELF, DWARF, & XML to build an IR of the ABI

• ELF, DWARF -> IR of ABI -> can be saved as XML

○ Other formats theoretically possible e.g. Mach-O or Windows ○ Not just DWARF to DWARF comparison ■ Allows comparison between compilers ■

Saved ABI XML represents all ABI artifacts

○ Compare saved ABI to ELF, DWARF for new object ■ Used to maintain ABI stability ■ No need to keep old objects around.

INSERT DESIGNATOR, IF NEEDED

Higher level Libabigail utilities

9

Libabigail is a C++ library and is easily to write utilities using it

• abidw - dump the ABI to stdout

○ Used to save copies ABI ○ “abidw --abidiff elfobject.so” compares an object to itself. Used to test

libabigail

• abidiff - compare the ABI of one ELF or ABI XML file to another.

○ Compare two objects or saved ABI XML files. ○ “abidiff elffile1.so elffile2.so ○ “abidiff saved.abixml elffile.so

INSERT DESIGNATOR, IF NEEDED

Higher level Libabigail tools (cont)

10

Libabigail is a C++ library and is easily to write utilities using it

• abicompat - does a changed ABI matter to the program

○ If the only changed functions and variables are not used the program,

then it isn’t a problem.

○ Does not detect functional changes

• abipkgdiff - compares analogous objects from related packages or

directories

INSERT DESIGNATOR, IF NEEDED

Goals:

11

• Eliminate the need for versions of library for each and every

compiler version.

• e.g. GCC 4.4.7 4.7 4.9 5.2 …
• spack can assume rather rebuilding or discard duplicates
• Reduce the need for versions of library compiled with each

compiler

• e.g. GCC vs. Intel vs. Clang…
• This is still a ways off
• Detect library API and ABI changes
• Educate developers on library versioning
• Let runtime linker do most of the work

INSERT DESIGNATOR, IF NEEDED

Ultimate Goals:

12

• Improve the quality of ELF, DWARF generated by all compilers
• Identify and resolve areas where platform ABI is not specified or

incompletely specified.

• Fulfill the promise of ABI by providing safe inter-compiler cross

linking

• Even for C++
• Maybe even for Fortran

INSERT DESIGNATOR, IF NEEDED

Current Status

13

This past year was foundational

• Stabilized libabigail fixing many crashes and performance

pathologies.

• Now properly capturing and recording ABI artifacts.
• In the process of learning how to deal with compiler idioms
• type synonyms
• void *
• Working through issues with SPACK the build system
• Finding, reporting and/or resolving compiler bugs
• ELF bugs - improperly exported functions (PGI)
• Compiler specific internal functions in ELF (Intel)
• Type qualifiers omitted e.g. restrict (Intel)
• Double … aka DW_TAG_unspecified_parameters (GCC)
• _Bool vs. unsigned char (Intel)

INSERT DESIGNATOR, IF NEEDED

Upcoming work

14

Much is left to do

• Continue to work with SPACK developers to broaden range of

applications tested

• Currently mostly C
• C++ is the next big challenge
• Templates
• Very precise ABI requirements
• Fortran
• Not sure we’ve captured all ABI artifacts yet
• Performance trumps all ABI

THANK YOU

plus.google.com/+RedHat linkedin.com/company/red-hat youtube.com/user/RedHatVideos facebook.com/redhatinc twitter.com/RedHatNews

INSERT DESIGNATOR, IF NEEDED

Compare ABI

When referencing code snippets, use the template below. Resize the snippet box to the appropriate size for your text.

Type code snippet here 16