Optimization Remarks Update Francis Visoiu Mistrih Optimization - - PowerPoint PPT Presentation

▶
optimization remarks update
SMART_READER_LITE
LIVE PREVIEW

Optimization Remarks Update Francis Visoiu Mistrih Optimization - - PowerPoint PPT Presentation

Dec 30, 2023 260 likes •553 views

Optimization Remarks Update Francis Visoiu Mistrih Optimization Remarks opt-viewer.py Challenges YAML doesnt scale well Discovering remarks Processing remarks __TEXT 0.1 GB __DWARF Remarks (YAML) 8.5 GB 20.7 GB clang /

slide-1
SLIDE 1

Optimization Remarks Update

Francis Visoiu Mistrih

slide-2
SLIDE 2

Optimization Remarks

  • pt-viewer.py
slide-3
SLIDE 3

Challenges

  • YAML doesn’t scale well
  • Discovering remarks
  • Processing remarks
slide-4
SLIDE 4

clang / RelWithDebInfo / macOS

__DWARF

8.5 GB

__TEXT

0.1 GB

Remarks (YAML)

20.7 GB

slide-5
SLIDE 5

clang / RelWithDebInfo / macOS

  • Significant size impact

__DWARF

8.5 GB

__TEXT

0.1 GB

Remarks (YAML)

20.7 GB

slide-6
SLIDE 6

clang / RelWithDebInfo / macOS

  • Significant size impact
  • Slow to parse: ~4 minutes for 27 million remarks

__DWARF

8.5 GB

__TEXT

0.1 GB

Remarks (YAML)

20.7 GB

slide-7
SLIDE 7

clang / RelWithDebInfo / macOS

  • Significant size impact
  • Slow to parse: ~4 minutes for 27 million remarks
  • Compile-time impact: 5% slowdown

__DWARF

8.5 GB

__TEXT

0.1 GB

Remarks (YAML)

20.7 GB

slide-8
SLIDE 8

clang / RelWithDebInfo / macOS

  • Significant size impact
  • Slow to parse: ~4 minutes for 27 million remarks
  • Compile-time impact: 5% slowdown

__DWARF

8.5 GB

Remarks (LLVM Bitstream)

1.7 GB

__DWARF

8.5 GB

__TEXT

0.1 GB

Remarks (YAML)

20.7 GB

slide-9
SLIDE 9

clang / RelWithDebInfo / macOS

  • Significant size impact 12x smaller
  • Slow to parse: ~4 minutes for 27 million remarks
  • Compile-time impact: 5% slowdown

__DWARF

8.5 GB

Remarks (LLVM Bitstream)

1.7 GB

__DWARF

8.5 GB

__TEXT

0.1 GB

Remarks (YAML)

20.7 GB

slide-10
SLIDE 10

clang / RelWithDebInfo / macOS

  • Significant size impact 12x smaller
  • Slow to parse: ~4 minutes 27 seconds (8x faster)
  • Compile-time impact: 5% slowdown

__DWARF

8.5 GB

Remarks (LLVM Bitstream)

1.7 GB

__DWARF

8.5 GB

__TEXT

0.1 GB

Remarks (YAML)

20.7 GB

slide-11
SLIDE 11

clang / RelWithDebInfo / macOS

__DWARF

8.5 GB

Remarks (LLVM Bitstream)

1.7 GB

__DWARF

8.5 GB

__TEXT

0.1 GB

Remarks (YAML)

20.7 GB

  • Significant size impact 12x smaller
  • Slow to parse: ~4 minutes 27 seconds (8x faster)
  • Compile-time impact: 5% slowdown 2% slowdown
slide-12
SLIDE 12

Extra Benefits

  • Well-known format in the community
  • Testing and tooling like llvm-bcanalyzer
  • Versioning
  • Flexible metadata
slide-13
SLIDE 13

libRemarks

#include "llvm-c/Remarks.h" LLVMRemarkParserRef Parser = LLVMRemarkParserCreateYAML(Buf, Size); LLVMRemarkEntryRef Remark = NULL; while ((Remark = LLVMRemarkParserGetNext(Parser))) { // use Remark }

slide-14
SLIDE 14

libRemarks

#include "llvm-c/Remarks.h" LLVMRemarkParserRef Parser = LLVMRemarkParserCreateBitstream(Buf, Size); LLVMRemarkEntryRef Remark = NULL; while ((Remark = LLVMRemarkParserGetNext(Parser))) { // use Remark } Bitstream

slide-15
SLIDE 15

.opt.yaml .o

{

slide-16
SLIDE 16

.opt.yaml .o

{ {

?

slide-17
SLIDE 17

clang foo.o foo.opt.bitstream ld

  • ut

bar.o bar.opt.bitstream

slide-18
SLIDE 18

clang foo.o foo.opt.bitstream ld

  • ut

bar.o bar.opt.bitstream

slide-19
SLIDE 19

foo.o foo.opt.bitstream bar.o bar.opt.bitstream

slide-20
SLIDE 20

foo.o foo.opt.bitstream

__remarks

52 4d 52 4b 01 08 00 00 17 00 00 00 07 01 b2 40 b4 42 39 d0 43 38 3c 3c c1 28 bc 83 3b d0 43 38 a4 83 3b 94 83 3c 80 41 3a b8 83 39 bc c3 41 80 38 06 08 14 22 1e 9a 61 16 e8 41 1e d2 c1 1d ce 01 0c e8 21 1c c4 81 1d ca 41 71 40 1f 1c a2 14 f0 81 1e ca 41 1e dc 21 1c d8

slide-21
SLIDE 21

foo.o foo.opt.bitstream

__remarks

52 4d 52 4b 01 08 00 00 17 00 00 00 07 01 b2 40 b4 42 39 d0 43 38 3c 3c c1 28 bc 83 3b d0 43 38 a4 83 3b 94 83 3c 80 41 3a b8 83 39 bc c3 41 80 38 06 08 14 22 1e 9a 61 16 e8 41 1e d2 c1 1d ce 01 0c e8 21 1c c4 81 1d ca 41 71 40 1f 1c a2 14 f0 81 1e ca 41 1e dc 21 1c d8

<file path>

slide-22
SLIDE 22

Remarks Section

  • Only in object files
  • Not included in the linked binary
  • Accessible through the Debug Map
  • clang’s remarks section: 0.27GB
slide-23
SLIDE 23

.dSYM

slide-24
SLIDE 24

.dSYM

.o .o .o

❌ ❌ ❌

slide-25
SLIDE 25
  • ut.dSYM

dsymutil DWARF/out

  • ut

Remarks/out

slide-26
SLIDE 26

Get Involved

  • Python bindings for integration with opt-viewer.py
  • llvm-remarkutil: convert, merge, extract, etc.
  • Faster lookup: build an index
  • Remark classes: verbosity, accuracy, etc.
  • Tablegen: better remark structure
  • Per-remark documentation: possible actions, reasons, etc.
slide-27
SLIDE 27

Thanks!