Benchmarking C++ From video games to algorithmic trading - - PowerPoint PPT Presentation

▶
benchmarking c from video games to algorithmic trading
SMART_READER_LITE
LIVE PREVIEW

Benchmarking C++ From video games to algorithmic trading - - PowerPoint PPT Presentation

Apr 01, 2024 315 likes •829 views

Benchmarking C++ From video games to algorithmic trading Alexander Radchenko Quiz. How long it takes to run ? 3.5GHz Xeon at CentOS 7 Write your name Write your guess as a single number Write time units clearly Answers

slide-1
SLIDE 1

Benchmarking C++
 From video games to algorithmic trading

Alexander Radchenko

slide-2
SLIDE 2
  • Quiz. How long it takes to run ?
  • 3.5GHz Xeon at CentOS 7

2

  • Write your name
  • Write your guess as a single

number

  • Write time units clearly
  • Answers will be collected in the

next 5 minutes

slide-3
SLIDE 3

Outline

  • Performance challenges in games
  • How games tackle performance
  • Performance challenges in trading
  • How trading tackles performance
  • Lightweight tracing use case

3

slide-4
SLIDE 4

My background

  • Game development for 15 years
  • 3D graphics programming and optimisation
  • Shipped 8 titles on various platforms

– PS2, PS3, Xbox 360, Wii, iOS, Android, PC

  • 3 years @ Optiver

– Low latency trading systems

  • Performance matters in both domains

4

slide-5
SLIDE 5

Why performance matters ?

  • Slow running game is no fun to play

– Guess what’s the second most common complaint about any PC game ?

  • Slow trading system is not making money

– In fact, it might lose your money

5

slide-6
SLIDE 6

Games

  • Soft real-time systems
  • Performance is important
  • Normally run at 30 frames per second
  • Consistent CPU/GPU load
  • Occasional spikes
  • Throughput is the king

6

slide-7
SLIDE 7

Game loop

  • Performance as a currency

– Graphics – Animations – Physics

PROCESS INPUT RENDER UPDATE 
 GAME

7

slide-8
SLIDE 8

Performance challenges in games

  • PC and Mobiles

– Fragmented HW

  • Game consoles

– Fixed HW ☺ – They are cheap for a reason ☹ – Proprietary tools and devkits

8

slide-9
SLIDE 9

Performance challenges in games

9

slide-10
SLIDE 10

How games tackle performance

  • Reference game levels
  • Custom profilers
  • Whole game session
  • Single frame

10

slide-11
SLIDE 11

World of Tanks

  • Online MMO shooter
  • Fragmented platform
  • Wide range of HW

– Old laptops – High-end desktops – Everything in between

11

slide-12
SLIDE 12

Replays

  • Record incoming network traffic
  • Initially created to repro bugs
  • Very useful tool for performance testing
  • At some point released to the public

12

slide-13
SLIDE 13

Replays: problems

  • Protocol upgrades
  • Game map changes may invalidate replay
  • Security

13

slide-14
SLIDE 14

Regression testing and replays

  • Avoiding performance degradation
  • Categorize HW: low, medium, high
  • Run replays on a fixed set of HW
  • 2s / 5s window averaged frame rate

14

slide-15
SLIDE 15

Trading

  • Low latency request processing systems
  • Performance is a currency

– Everyone will identify big opportunities – Race to the exchange – Winner takes all

15

slide-16
SLIDE 16

Trading

  • Most of the time system is idle
  • Bursts on big events
  • Latency is the king

– Speed to take profitable trades – Speed to adjust our own orders

16

slide-17
SLIDE 17

Trading

  • Dedicated high end Linux HW
  • Speedlab environment to test performance
  • Lightweight tracing in speedlab and production
  • Using time series DB to store captured data

– Easy data retrieval for given time range – Historical data analysis

17

slide-18
SLIDE 18

TRADING STACK

Money loop

INFORMATION EXECUTION STRATEGY EXCHANGE

18

slide-19
SLIDE 19

Performance challenges in trading

  • Cache !

19

slide-20
SLIDE 20

Cache

  • Generally L3 is shared across all cores
  • Pick your neighbours wisely
  • HT threads share L1.

– This is one of the reasons why we disable HT

  • You want all your data to be in cache !
  • Cache warming techniques

– Keep running – Keep touching memory

20

slide-21
SLIDE 21

How trading measures latency

Information Auto trader Execution

Auto trader

Software timestamps Hardware timestamps

21

TRADING STACK INFORMATION STRATEGY EXCHANGE EXECUTION

slide-22
SLIDE 22

Using timestamps

  • Latency histograms

– simulated environment – production

  • Detecting outliers
  • Drilling down specific events

22

slide-23
SLIDE 23

Lightweight tracing

  • How light it is ?

– HW timestamp cost is a few nanoseconds – SW timestamp is higher, still very cheap

  • Very useful for understanding performance profile
  • Visualizing and recognizing patterns

23

slide-24
SLIDE 24

Low Latency Fizzbuzz

  • https://github.com/phejet/benchmarkingcpp_games_trading
  • C++ server which reads input data
  • Outputs Fizz, Buzz, FizzBuzz or just a number
  • How to make it fast ?
  • Measure first !!!

24

slide-25
SLIDE 25

Fizzbuzz

  • How long do you think it takes run this code ?
  • 3.5GHz Xeon at CentOS 7

25

slide-26
SLIDE 26

Quiz results

26

slide-27
SLIDE 27

Request processing

27

slide-28
SLIDE 28

Timing

28

slide-29
SLIDE 29

Timing

29

slide-30
SLIDE 30

Using Epoch

30

slide-31
SLIDE 31

Timings output

31

slide-32
SLIDE 32

Macro benchmark

32

slide-33
SLIDE 33

Quick feedback

  • Time in nanoseconds

33

slide-34
SLIDE 34

Jupyter notebooks

  • Open-source web application
  • Create and share documents that contain

– Live code – Equations – Visualizations – Narrative text

34

slide-35
SLIDE 35

Jupyter notebook for in-depth analysis

35

slide-36
SLIDE 36

Histogram as text

Looks big

36

slide-37
SLIDE 37

Beware of outliers

Outlier

37

slide-38
SLIDE 38

Discarding outliers

Max value more reasonable

38

slide-39
SLIDE 39

Distribution is strange…

Not unimodal ?

39

slide-40
SLIDE 40

Bimodal distribution

40

slide-41
SLIDE 41

Optiver profiler

41

  • In-house tracing profiler
  • Mark interesting parts of your code

– Scope guards to capture entry/exit timestamps and function name – Single named events

  • Nanosecond precision
  • Multiple tools to view results
  • Tarantula is the most interesting one
slide-42
SLIDE 42

Tarantula

42

slide-43
SLIDE 43

Two codepaths !

Non FizzBuzz code path

43

slide-44
SLIDE 44

Optimisation

44

  • FizzBuzz logic is the most expensive part of our request processing
  • How can we make it faster ?
slide-45
SLIDE 45

Brute force approach

45

  • Write custom function instead of using std::to_string
  • Return result as const char* and use static buffer
slide-46
SLIDE 46

Look at high level

46

slide-47
SLIDE 47

Avoid int->string conversion

47

slide-48
SLIDE 48

Measuring Optimised code

48

slide-49
SLIDE 49

Closing

  • It’s very hard to guess execution time by just looking at code
  • Having a simple and reproducible way to measure performance is

very important

  • Visualising performance data helps to understand it
  • Understanding is a necessary first step before optimization
  • When optimizing code, always look at the high level picture

49

slide-50
SLIDE 50

Questions ?

  • Alexander Radchenko
  • phejet@gmail.com
  • https://github.com/phejet/benchmarkingcpp_games_trading
  • @phejet on Twitter