ABI Compatibility Through A Customizable Language Kevin Atkinson - - PowerPoint PPT Presentation

1

ABI Compatibility Through A Customizable Language

Kevin Atkinson PhD Thesis University of Utah

2

Application Programmer Interface (API)

Application

int main() { X x; x.f(); } Code Library Library class X { virtual int f(); ... }; API class Y API

3

Application Binary Interface (ABI)

class X { virtual int f(); ... }; int main() { X x; x.f(); }

Application

API Code Library Library gcc ABI

Class Layout Calling Convention

gcc Object Code class Y API gcc ABI

4

Source vs Binary Compatibility

Application

Doesn't Change Recompile class X { virtual int f(); virtual int g(); ... }; API Library gcc ABI int main() { X x; x.f(); } Code gcc Object Code

(ABI Fragility)

5

ABI Incompatibility

Application

Doesn't Change Change Compilers and Recompile class X { virtual int f(); ... }; API Library gcc vc++ ABI int main() { X x; x.f(); } Code vc++ Object Code

6

Previous Work

• Some Standardization

Attempts

• Itanium C++ ABI
• Some Work Towards

Less Fragile ABIs

• Delta C++ (Palay, 1992)
• Object Binary Interface

(Williams and Kindel, 1994)

• Still Two Common ABIs:
• GCC
• Visual C++
• Sacrifice Performance
• Not Commonly Used

7

Thesis Statement

int main() { ... } Object Code Class Impl. Macro Extensible Compiler

ZL

Fragile and incompatible ABIs are a major problem in software maintenance and evolution that can be systematically dealt with and effectively managed using a macro-based system that allows the programmer to control how an API maps to an ABI.

8

Breaking Binary Compatibility

class X { int x; ... }; gcc class X { int x; int y; ... }; gcc class X int x class X int x int y size 4 size 8 API (.h) API (.h)

9

Preserving Binary Compatibility

Dummy Member

class X { int x; int dummy; ... }; gcc class X { int x; int y; ... }; gcc class X int x int dummy size 8 class X int x int y size 8 API (.h) API (.h)

10

Preserving Binary Compatibility

Pointer to Implementation

API (.h) gcc

Implementation

class X { Impl* imp; int get_x(); }; class X::Impl { int x; }; int X::get_x() {imp->x;}

class X

Impl

int x

11

Breaking Binary Compatibility

class X { virtual void foo(); ... }; gcc class X { virtual void foo(); virtual void bar(); ... }; gcc

• For Preserving: Options Limited

vtable void foo() size 4 vtable void foo() void bar() size 8

12

Dummy Member Pointer to Impl.

class X { int x; int dummy; ... }; gcc class X { int x; int y; ... }; gcc class X int x int dummy size 8 class X int x int y size 8

Automate These Patterns

…

API (.h) gcc

Implementation

class X { Impl* imp; int get_x(); }; class X::Impl { int x; }; int X::get_x() {imp->x;}

class X

Impl

int x

13

Automation is Not Enough...

Breaking Binary Compatibility

class X { virtual void foo(); ... }; gcc class X { virtual void foo(); virtual void bar(); ... }; gcc

• For Preserving: Options Limited

vtable void foo() size 4 vtable void foo() void bar() size 8

Breaking Binary Compatibility

class X { virtual void foo(); ... }; gcc class X { virtual void foo(); virtual void bar(); ... }; gcc

• For Preserving: Options Limited

vtable void foo() size 4 vtable void foo() void bar() size 8

14 int main() { ... } Extensible Compiler Object Code int main() { ... } Macro Extensible Compiler Object Code Class Impl. Class Impl. Extension

15 int main() { ... } Object Code Class Impl. import ... int main() { ... } Object Code Class Impl. Library Macro Extensible Compiler Macro Extensible Compiler

16

Two ABI's at Once

class : ABI(gcc) class : ABI(vc++) Object Code Gcc ABI VC++ ABI Macro Extensible Compiler

17

ZL

import ...; int main() { ... } Macro Extensible Compiler Object Code default class implementation

• ther macro

library

C Like Core w/ Minimal Amount of C++

18

How to Parse This Expression?

f(x/2, y)

Function Call? Macro Invocation?

f(x/2, y) x/2 + y

?

int y = x/2

?

19

Parsing Overview

• ZL doesn't parse in a single linear pass
• Iterative-deepening approach

Raw Text

Parse Syntax Object Raw Text AST Node Syntax Object Partly Expand Compile R e p a r s e

20

Parsing Details

inline int f() {int x = 10; return x;} int main() {return f();} Parse Expand & Compile (@ (stmt inline int f ("()" "") ("{}" "int x = 10; return x;") (stmt int main ("()" "") ("{}" "return f();")))

Top Level Environment

(stmt int main ...)

...

(stmt inline int f ...)

...

21

...

(stmt inline int f ...) (fun f "()" int :inline ("{}" "int x = 10; return x;"})

Top Level Environment

Expand Compile ("{}" "int x = 10; return x;")

Function

...

(stmt int main ...)

...

22

...

("{}" "int x = 10; return x;")

Function

(block (stmt int x = 10) (return (exp x))) Expand & Reparse Compile

Block

(stmt int x = 10)) (return (exp x))

... ...

23

Hygienic Pattern Macros

macro or(x, y) { ({typeof(x) t = x; t ? t : y;}); }

• r(0.0,t)

({typeof(0.0) t0 = 0.0; t0 ? t0 : t;});

24

Procedural Macros

Syntax* foreach(Syntax*) {...} make_macro foreach int main() { Container c; ... foreach(el, c, el.print()); }

Syntax Forms: new_mark syntax make_macro Callbacks: match replace ct_value error

25

The Class Macro

import_file "class-simple.zlh"; ... .class ParseClass { ... virtual Syntax * parse(Syntax * p) {...} virtual void init() {...} virtual void init_parent_info() {...} ... }; Syntax * parse_class(Syntax * p) { ParseClass * pc = new ParseClass; return pc->parse(p); } make_macro class parse_class;

900 Lines

• f Code

26

• .f()

C::f(...)

Expanding Method Calls

What is f? C o

27

User Types

(ZL's Notion of Classes)

user_type C { struct Data { int i; }; associate_type struct Data; int foo(int j) macro i(:this ths) {...} macro f(x, :this ths) {...} } void main() { C o; C::f(x, :this = &o); int x = C::i(:this = &o); } void main() { C o;

• .f(x);

int x = o.i; }

28

Classes

class C { int i; int f(int j) {return i + j;} }; user_type C { struct Data { int i; }; associate_type struct Data; int f`internal(C * this, int j) {return i + j;} macro i (:this ths = this) {(*(C *)ths)..i;} macro f(j, :this ths = this) {f`internal(ths, j);} }

29

Inheritance

class D : public C { int j; }; user_type D { import C; struct Data { struct C::Data parent; int j; }; associate_type struct Data; make_subtype C _up_cast _down_cast; macro _up_cast ... macro _down_cast ... macro j (:this ths = this) {...} }

30

Virtual Methods

class D : public C { virtual void f(int j); }; user_type D { ... class VTable : public C::VTable { void (*f)(int j); }; struct Data { VTable* _vptr; ... }; associate_type struct Data; macro f(j, :this ths = this) {_vptr->f(ths, j);} ... }

31

Examples

Using ZL to Mitigate ABI Problems

32

Example: Fixing Size of Class

class C { int x; char dummy[8] }; class X int x dummy[8] class C : fix_size(12) { int x; }; 8 12 class C { int x; int y; char dummy[4] }; class X int x int y dummy[4] class C : fix_size(12) { int x; int y; }; 4 12

(Complete Macro In Dissertation, 36 Lines of Code)

33

Example: Allowing More Fields

class C : fix_size(12) { int x; int y; int i; int j; }; int main() { C obj;

• bj.x;
• bj.i;

} class X int x int y Overflow int i int j 12

(Expanded Macro, Around 100 Lines of Code)

34

Example: Pointer to Impl.

class X Overflow int x int y int i; int j;

(Same Macro, Around 100 Lines of Code)

class C : fix_size(0) { int x; int y; int i; int j; }; int main() { C obj;

• bj.x;
• bj.i;

}

35

Fixing Virtual Table Size

class X : fix_vtable_size(8) { virtual void f(); virtual void g(); virtual void h(); };

(Modified Macro, Around 120 Lines of Code)

macro alt_vtable (name, body) { class name : fix_size(8) {body}; } class X : vtable_class(alt_vtable) { { virtual void f(); virtual void g(); virtual void h(); };

36 extern "C++" class ClassWithZLABI extern "C++" : "gcc" class ClassWithGCCABI

GCC ABI

.class ParseClascGccABI : public ParseClass {...}

Matching GCC's ABI

ZL Default ABI

.class ParseClass {...} int ClassWithZLABI$get_x$$P$_ClassWithZLABI (); int _ZN15ClassWithGCCABI5get_xEv (); Compiles To {int get_x();} {int get_x();}

(Class Macro 150 lines, Mangler 300 lines)

37

Demonstration

Simple Spell

38

Bridge EmailFilter

Simple Spell´s Classes

SessionWFilter Filter

filter(...) reset()

UrlFilter

GCC ABI ZL ABI

Session

... ... reset()

Speller

... personal ... add_to_personal()

GCC ABI

Application

GCC or ZL

39

Speller Class

... Speller * speller = new Speller(); speller->init(new_lang(), new_master_dict()); ... for (...) { // look through document bool correct = speller->check(word); if (!correct) { // maybe Suggestions * sugs = speller->suggest(); // if indeed correct speller->add_to_session(word); } } ...

class Speller { init(...) check(...) suggest() add_to_session(...) }; struct Suggestions; new_lang(); new_master_dict();

GCC ABI

40

class Session { virtual new_line(...) virtual next_misspelling() inline mispelled_word() virtual replace(...) }; class SessionWFilters : public Session { virtual add(Filter *) }; Filter * new_url_filter();

Session Class

GCC ABI

...

Speller * speller = new Speller(); speller->init(new_lang(), new_master_dict());

SessionWFilters * session = new SessionWFilters(speller); session->add(new_url_filter()); for (...) { // look through doc. one line at a time session->new_line(line); while (session->next_misspelling()) { word = session->misspelled_word();

bool correct = speller->check(word); if (!correct) { // maybe

Suggestions * sugs = speller->suggest(); // if indeed correct speller->add_to_session(word);

// if incorrect session->replace(new_word) }

}

...

41

Extension API

class Filter { virtual void filter(char * line) = 0 // blank parts of line not to be checked }

ZL ABI

42

Simple Application

*Teh* dog swm up the stream. 1) The 2) Tea 3) Ted 4) Tee 5) Tel 6) Ten ... i) Ignore I) Ignore all r) Replace a) Abort

Application GCC Library ZL (Using GCC ABI) Application ZL Library GCC

43

Email Filter with GCC ABI

session->add(filter_bridge(new_email_filter()));

> This line skippd This line checkd

class EmailFilter : public Filter {...} Filter * new_email_filter();

GCC ABI

class FilterBridge : public Filter {...} Filter * filter_bridge(/* filter in GCC ABI*/)

ZL ABI

44

Automate Bridge Class

Syntax * parse_mk_bridge(...) { ... } make_macro mk_bridge parse_mk_bridge(); mk_bridge(Filter, filter_bridge, “GCC”) (Macro Just Under 55 Lines of Code)

45

Support For A Personal Dictionary

Speller data_member_1 data_member_2 ... personal Changes Size Speller : fix_size(...) data_member_1 data_member_2 Overflow ... Speller data_member_1 data_member_2 ... Speller data_member_1 data_member_2 ... OK Still Match GCC ABI

46

Fixing Size of Speller Class

class SpellerOld { // original Speller class private data members }; class Speller : fix_size(sizeof(SpellerOld)) { // private data members SavableDict * personal; // original methods void add_to_personal(const char *); void save_personal(); };

47

An ABI Allowing Future Adaptation

class Speller : fix_size(0) { ... }; class Session : fix_size(/* size of members used by inline functions */) : vtable_slots(16) { const char * word ... const char * misspelled_word() {return word;} ... }; class SessionWFilter : fix_size(0) : vtable_slots(16) ... extern “C++” : “better” class Filter { ... }

48

Simple Spell, Version 2

Add (Back) Support for a Personal Dictionary

class Speller : fix_size(0) { ... SavableDict * personal; ... void add_to_personal(...); void save_personal(); };

49

class Session : fix_size(...) : vtable_slots (16) { ... void reset(); }; extern “C++” : “better” class Filter { ... void reset() }

Simple Spell, Version 2

Add Support for Stateful Filters

Filter * new_comment_filter() // checks only C and C++ comments

50

ZL Implementation Status

• Prototype Compiler
• Most Of C
• Enough C++ to Demonstrate Approach
• Compiled Several Real Applications
• C: bzip2, gzip
• C++: randprog
• Compile Time 2-3 slower
• No Impact on Run-time Performance

51

Conclusion

Demonstrated ABI Compatibility Through Macros

int main() { ... } Object Code Class Impl. Macro Extensible Compiler

ZL