Not All Coverage Measurements Are Equal Fuzzing by Coverage - PowerPoint PPT Presentation

Not All Coverage Measurements Are Equal Fuzzing by Coverage Accounting for Input Prioritization NDSS Symposium 2020 YanhaoWang, Xiangkun Jia, Yuwei Liu, Kyle Zeng, Tiffany Bao, DinghaoWu, and Purui Su 1

AFL Family and Coverage-based Fuzzing AFLFast FairFuzz CollAFL QSYM AFL-Sensitive AFL Driller 2

AFL Family and Coverage-based Fuzzing Input Coverage Feedback Program Fuzzer Crash Inputs 3

Coverage-based Fuzzing: The Internals Input Prioritization Factors: Execution Time, Input Size, etc. Queue Prioritized input Queue Culling ( isFavor ) Other input Prioritized Queue Favored 4

Coverage Measurements are Treated Equally if len < 256 Spend equal time on security-sensitive paths and security-insensitive paths a b memcpy(x, y, len) print error msg Delay finding vulnerabilities return 5

Anti-Fuzzing if len < 256 b' a Inject fake coverage measurements n fake paths to mislead coverage-based fuzzers memcpy(x, y, len) print error msg return 6

What then? 7

do not We treat coverage measurements equally 8

Coverage Accounting if len < 256 if len < 256 a a b b The prioritization of input reflects secu curity sensitivity memcpy(x, y, len) memcpy(x, y, len) print error msg print error msg return return 9

Coverage Accounting What should be the indicators? function level basic block level loop level Design a new queue culling scheme based on coverage accounting metrics 10

Function Level malloc free ...... memcpy memmove memset Some functions are inherently likely to be involved in memory corruptions We crawled call-stacks from webpages of all CVEs in the latest 4 years Funct ction Nu Number ber Funct ction Number Nu ber 80 12 memcpy free 35 12 strlen memset 17 11 ReadImage delete 15 10 malloc memcmp 12 9 memmove getString 11

Loop Level 1 Incorrect looping condition is often the root 2 3 cause of memory corruption vulnerabilities 4 5 12

Basic Block Level read 1 shl [rbp+var1], 4 1 shl [rbp+var1], 4 write 2 mov edx, [rbp+var1] 2 mov edx, [rbp+var1] 3 mov eax, edx 3 mov eax, edx 4 shl eax, 4 4 shl eax, 4 5 add eax, edx 5 add eax, edx 6 mov [rbp+var1], eax 6 mov [rbp+var1], eax 7 mov rdx, [rbp+var2] 7 mov rdx, [rbp+var2] 8 mov rax, [rbp+i] 8 mov rax, [rbp+i] 9 add rax, rdx 9 add rax, rdx 10 movzx edx, byte ptr [rax] 10 movzx edx, byte ptr [rax] 11 movzx eax, [rbp+var3] 11 movzx eax, [rbp+var3] 12 xor eax, edx 12 xor eax, edx 13 movzx eax, al 13 movzx eax, al 14 add [rbp+var1], eax 14 add [rbp+var1], eax 15 movzx edx, [rbp+var3] 15 movzx edx, [rbp+var3] 16 mov eax, edx 16 mov eax, edx 17 shl eax, 3 17 shl eax, 3 13

Design Coverage Accounting Information Queue Security-sensitive prioritized input Queue Culling Security-insensitive ( isFavor ) prioritized input Other input Prioritized Queue Favored 14

TortoiseFuzz: Coverage-based Fuzzer with Coverage Accounting FairFuzz AFLFast CollAFL AFL QSYM TortoiseFuzz AFL-Sensitive Driller 15

TortoiseFuzz: Coverage-based Fuzzer with Coverage Accounting The Hare and The Tortoise Story, Bedtime Story by Kids Hut https://www.youtube.com/watch?v=eMXmMHVNx4U 16

Implementation We implement co accounting on AFL as To coverag age acco TortoiseFuzz TortoiseFuzz for both source code and binaries We implement To 17

Experiment Setup We ran TortoiseFuzz on 30 real-world programs Each experiment lasted for 140 hours Each experiment was done 10 times We performed Mann-Whitney U test to measure statistical significance 18

Vulnerability Discovery Average # of discovered vulnerabilities 45 40 35 TortoiseFuzz outperforms 5 30 state-of-the-art fuzzers and 25 achieves comparable results 20 with QSYM 15 10 5 0 TortoiseFuzz AFL AFLFast FairFuzz MOPT Angora QSYM 19

Comparison with QSYM TortoiseFuzz uses 2 % of QSYM’s memory usage on average 20

Complementary to Other Fuzzers Coverage accounting helps improve QSYM in discovering vulnerabilities Averag age # of disco covered vulnerab abilities QSYM QSYM + coverage accounting 39.8 51.2 28.6 % improvement 21

Robustness to Anti-fuzzing if len < 256 b' a Fake paths do not contain many coverage n fake paths accounting info memcpy(x, y, len) print error msg return 22

Robustness to Anti-fuzzing # of covered edges over time 12000 10000 8000 Coverage accounting metrics are more robust to anti-fuzzing 6000 4000 2000 0 0 12 24 36 48 60 72 AFL AFL+anti-fuzzing TortoiseFuzz TortoiseFuzz+anti-fuzzing 23

Conclusion We propose coverage accounting which is complementary to other coverage-based fuzzers We design and implement TortoiseFuzz, and we are going to release it at https://github.com/TortoiseFuzz/TortoiseFuzz We evaluate TortoiseFuzz on 30 real-world programs and find 20 zero-day vulnerabilities TortoiseFuzz outperforms 5 state-of-the-art fuzzers and achieves comparable results with QSYM with 2 % of its memory usage 24

Not All Coverage Measurements Are Equal Fuzzing by Coverage Accounting for Input Prioritization Thank you! Q & A Kyle Zeng zengyhkyle@asu.edu 25