What did exist before? How does it work? What are the findings? What is the Conclusion? Measuring Performance on OpenBSD Alexander Bluhm bluhm@openbsd.org BSDCan, May 2019
What did exist before? How does it work? What are the findings? What is the Conclusion? Detect Changes in Performance over Commits and Releases What did exist before? 1 How does it work? 2 What are the findings? 3 What is the Conclusion? 4
What did exist before? How does it work? What are the findings? What is the Conclusion? Agenda What did exist before? 1 How does it work? 2 What are the findings? 3 What is the Conclusion? 4
What did exist before? How does it work? What are the findings? What is the Conclusion? genua Firewall Testbed HPF Numbers for Customers Marketing Developers Hardware . . .
What did exist before? How does it work? What are the findings? What is the Conclusion? HPF Result
What did exist before? How does it work? What are the findings? What is the Conclusion? Multi User, Multi Purpose Hardware Setup 1 Gbit 1 Gbit Source 1 Switch Drain 1 Source 2 Drain 2 . . . . . . Target 1 Target 2 Source n Drain n . . . 10 Gbit 10 Gbit Target n
What did exist before? How does it work? What are the findings? What is the Conclusion? Unsuitable HPF too many requirements too much complexity not enough flexibility
What did exist before? How does it work? What are the findings? What is the Conclusion? Performance Hardware Design 10 Gbit Target Target
What did exist before? How does it work? What are the findings? What is the Conclusion? Existing Regression Tests daily test runs using /usr/src/regress multi architecture history of pass and fail useful information web access http://bluhm.genua.de/
What did exist before? How does it work? What are the findings? What is the Conclusion? All Regression Tests
What did exist before? How does it work? What are the findings? What is the Conclusion? Regression Tests with Links setup logs tested obj download version run history cvs web colored status test output
What did exist before? How does it work? What are the findings? What is the Conclusion? Regression History for i386 history severity
What did exist before? How does it work? What are the findings? What is the Conclusion? Regression Test Hardware server remote router local
What did exist before? How does it work? What are the findings? What is the Conclusion? Regression Master web publish configure master control server remote router local console install
What did exist before? How does it work? What are the findings? What is the Conclusion? Agenda What did exist before? 1 How does it work? 2 What are the findings? 3 What is the Conclusion? 4
What did exist before? How does it work? What are the findings? What is the Conclusion? Performance Goals history reproducable details available drill down automatic
What did exist before? How does it work? What are the findings? What is the Conclusion? Performance History advance by step install collect release results run checkout compile tests at date kernel
What did exist before? How does it work? What are the findings? What is the Conclusion? Performance Tests Overview cvs run install checkout date version steps run history run detail test command
What did exist before? How does it work? What are the findings? What is the Conclusion? Performance Run at Date checkout build kernel date quirks commits run and install log average numbers unstable results
What did exist before? How does it work? What are the findings? What is the Conclusion? Performance Repeat at CVS Checkout repeat standard outlier count deviation reboot log single result
What did exist before? How does it work? What are the findings? What is the Conclusion? Weekly from 6.2 to 6.3 TCP Performance A C B D E F G H I J 4 × 10 9 3 . 5 × 10 9 3 × 10 9 2 . 5 × 10 9 bits/sec 2 × 10 9 1 . 5 × 10 9 1 × 10 9 5 × 10 8 0 2017-11-01 2017-12-01 2018-01-01 2018-02-01 2018-03-01 Checkout (date)
What did exist before? How does it work? What are the findings? What is the Conclusion? Quirks from 6.2 to 6.3 A OpenBSD/amd64 6.2 release B fix cvs vendor branch checkout C clang update LLVM to 5.0.0 D pfctl pf packet rate matching E move kernel source file dwiic.c F pfctl pf divert type G sysctl struct vfsconf H clang update LLVM to 5.0.1 I pfctl pf syncookies J OpenBSD/amd64 6.3 release
What did exist before? How does it work? What are the findings? What is the Conclusion? Build Quirks advance build checkout by step toolchain userland run checkout compile tests at date kernel
What did exist before? How does it work? What are the findings? What is the Conclusion? Performance Hardware Future Target 1 Target 1 Target 2 Target 2 Linux
What did exist before? How does it work? What are the findings? What is the Conclusion? Performance Master web publish configure and compile master control remote local target target console install release
What did exist before? How does it work? What are the findings? What is the Conclusion? Agenda What did exist before? 1 How does it work? 2 What are the findings? 3 What is the Conclusion? 4
What did exist before? How does it work? What are the findings? What is the Conclusion? Drilldown from Week to Days TCP Performance 4 × 10 9 3 . 5 × 10 9 3 × 10 9 2 . 5 × 10 9 bits/sec 2 × 10 9 1 . 5 × 10 9 1 × 10 9 5 × 10 8 0 2017-12-28 2017-12-30 2018-01-01 2018-01-03 2018-01-05 2018-01-07 2018-01-09 Checkout (date)
What did exist before? How does it work? What are the findings? What is the Conclusion? Reproduce and Reboot run compile tests kernel run reboot relink again machine kernel
What did exist before? How does it work? What are the findings? What is the Conclusion? 6.4, 5 Days, 10 Tests, Keep Machine Running TCP Performance Y 4 × 10 9 3 . 5 × 10 9 3 × 10 9 2 . 5 × 10 9 bits/sec 2 × 10 9 1 . 5 × 10 9 1 × 10 9 5 × 10 8 0 2018-10-11 2018-10-11 2018-10-12 2018-10-12 2018-10-13 2018-10-13 2018-10-14 2018-10-14 2018-10-15 Checkout (date)
What did exist before? How does it work? What are the findings? What is the Conclusion? 6.4, 5 Days, 10 Tests, Reboot Machine TCP Performance Y 4 × 10 9 3 . 5 × 10 9 3 × 10 9 2 . 5 × 10 9 bits/sec 2 × 10 9 1 . 5 × 10 9 1 × 10 9 5 × 10 8 0 2018-10-11 2018-10-11 2018-10-12 2018-10-12 2018-10-13 2018-10-13 2018-10-14 2018-10-14 2018-10-15 Checkout (date)
What did exist before? How does it work? What are the findings? What is the Conclusion? 6.4, 5 Days, 10 Tests, Link and Reorder Kernel TCP Performance Y 4 × 10 9 3 . 5 × 10 9 3 × 10 9 2 . 5 × 10 9 bits/sec 2 × 10 9 1 . 5 × 10 9 1 × 10 9 5 × 10 8 0 2018-10-11 2018-10-11 2018-10-12 2018-10-12 2018-10-13 2018-10-13 2018-10-14 2018-10-14 2018-10-15 Checkout (date)
What did exist before? How does it work? What are the findings? What is the Conclusion? 6.4, 15 Days, 5 Tests, 2 CPU Sockets, Keep running TCP Performance Y Z ab c 4 × 10 9 3 . 5 × 10 9 3 × 10 9 2 . 5 × 10 9 bits/sec 2 × 10 9 1 . 5 × 10 9 1 × 10 9 5 × 10 8 0 2018-10-12 2018-10-14 2018-10-16 2018-10-18 2018-10-20 2018-10-22 2018-10-24 Checkout (date)
What did exist before? How does it work? What are the findings? What is the Conclusion? 2 CPU Sockets, Repeat, Keep running second cycle
What did exist before? How does it work? What are the findings? What is the Conclusion? from 6.3 to 6.4, 202 Days TCP Performance K M L N O PQ RS TU V W X Y 4 × 10 9 3 . 5 × 10 9 3 × 10 9 2 . 5 × 10 9 bits/sec 2 × 10 9 1 . 5 × 10 9 1 × 10 9 5 × 10 8 0 2018-04-01 2018-05-01 2018-06-01 2018-07-01 2018-08-01 2018-09-01 2018-10-01 Checkout (date)
What did exist before? How does it work? What are the findings? What is the Conclusion? from 6.4 to 6.5, 185 Days TCP Performance Y Z bc a d e f g h i j k l 4 × 10 9 3 . 5 × 10 9 3 × 10 9 2 . 5 × 10 9 bits/sec 2 × 10 9 1 . 5 × 10 9 1 × 10 9 5 × 10 8 0 2018-11-01 2018-12-01 2019-01-01 2019-02-01 2019-03-01 2019-04-01 Checkout (date)
What did exist before? How does it work? What are the findings? What is the Conclusion? from 6.5, 19 Days TCP Performance m 4 × 10 9 3 . 5 × 10 9 3 × 10 9 2 . 5 × 10 9 bits/sec 2 × 10 9 1 . 5 × 10 9 1 × 10 9 5 × 10 8 0 2019-04-14 2019-04-16 2019-04-18 2019-04-20 2019-04-22 2019-04-24 2019-04-26 2019-04-28 2019-04-30 Checkout (date)
Recommend
More recommend
