Handling all Facebook requests with JITed C++ code Yuhan Guo, - - PowerPoint PPT Presentation

▶
handling all facebook requests with jited c code
SMART_READER_LITE
LIVE PREVIEW

Handling all Facebook requests with JITed C++ code Yuhan Guo, - - PowerPoint PPT Presentation

Oct 27, 2023 453 likes •703 views

Handling all Facebook requests with JITed C++ code Yuhan Guo, Huapeng Zhou Software Engineers, Facebook User requests L7 User Web servers Loadbalancer User requests L7 User Web servers Loadbalancer Core HTTP stack +

slide-1
SLIDE 1
slide-2
SLIDE 2

Handling all Facebook requests with JITed C++ code

Yuhan Guo, Huapeng Zhou

Software Engineers, Facebook

slide-3
SLIDE 3

User requests

User L7 Loadbalancer Web servers

slide-4
SLIDE 4
  • Core HTTP stack + Business logic
  • Large
  • Stable

User requests

User L7 Loadbalancer Web servers

  • v.s. Small
  • v.s. Fast iteration
slide-5
SLIDE 5
  • Interfacing
  • Testing
  • Debugging
  • Profiling
  • Performance
  • Side-effects

Other scripting language

slide-6
SLIDE 6
  • Interfacing
  • Testing
  • Debugging
  • Profiling
  • Performance

What we want

slide-7
SLIDE 7

Clang/LLVM

slide-8
SLIDE 8

Build time

Main Binary Pre-compiled Header C++ script

JIT Engine

class UrlParser { ... }; int getID(string url) { auto p = UrlParser(url); ... } getID(url);

slide-9
SLIDE 9

Runtime

Main Binary Pre-compiled Header C++ script

JIT Engine

LLVM IR

Clang Compiler options from Clang PCH metadata

Clang/LLVM libraries

Statically linked

slide-10
SLIDE 10

Runtime

Main Binary Pre-compiled Header C++ script

JIT Engine

LLVM IR

Clang

Executable code Exported Symbols

LLVM OrcJit

Clang/LLVM libraries

Statically linked

slide-11
SLIDE 11
  • Interfacing
  • Testing
  • Debugging
  • Profiling
  • Performance

What we want?

slide-12
SLIDE 12

Interfacing

Main Binary Pre-compiled Header C++ script

JIT Engine

LLVM IR

Clang

Executable code Exported Symbols

LLVM OrcJit

Clang/LLVM libraries

Statically linked

slide-13
SLIDE 13

Interfacing

Main Binary Pre-compiled Header C++ script

JIT Engine

LLVM IR

Clang

Executable code Exported Symbols

LLVM OrcJit while (true) { fork(); }

Clang/LLVM libraries

Statically linked

  • -dynamic-list
slide-14
SLIDE 14

Testing

Unit test

Unit Tester Pre-compiled Header C++ script

JIT Engine

Test PCH Test script

slide-15
SLIDE 15

Testing

  • Using same JIT Engine
  • Google Test
  • ASan/UBSan

Unit test

TEST(Foo, bar){ auto id = getID(url); EXPECT_EQUAL(id, 42); }

slide-16
SLIDE 16

Testing

  • Spin up main binary + scripts locally
  • Real HTTP test request against local host

Integration test

slide-17
SLIDE 17

Debugging

  • Register in-memory symbol files with GDB
  • Github JitFromScratch project has been helpful

llvm::JITEventListener::createGDBRegistrationListener

slide-18
SLIDE 18

Profiling

  • PerfJITEventListener added in https://reviews.llvm.org/D44892
  • Based on jitdump
  • Rolled our own PerfMapJITEventListener
  • Based on /tmp/perf-%pid.map

START SIZE symbolname START SIZE symbolname ...

slide-19
SLIDE 19

Performance

Execution time Time distribution

slide-20
SLIDE 20
  • Addition binary size: ~100MB
  • Addition start up time: ~2s
  • Quirks: Thread local storage
  • Adapt to OrcJIT upstream API change

Cost?

slide-21
SLIDE 21
  • Performance tuning based on Perf
  • Clang checker
  • Module?
  • Coroutines-TS?

What's next?

slide-22
SLIDE 22
  • https://llvm.org/docs/tutorial/index.html#building-a-jit-in-

llvm has been extremely helpful

  • Continuous support from LLVM society is awesome
  • Giving back to community https://reviews.llvm.org/D53911

Thank you!

slide-23
SLIDE 23