Software Security Dynamic analysis and fuzz testing Bhargava Shastry Prof. Jean-Pierre Seifert Security in Telecommunications TU Berlin SoSe 2016 1 / 21
Introduction What is this lecture about? How do we find bugs in programs? ... by executing an open-source program ... applies to binary code in principle Let’s recap Last lecture, we discussed static analysis → Find bugs without program execution How does it compare with Dynamic analysis? Pros and cons? 2 / 21
Motivation Why do this? Find an input to the program that leads to undefined/insecure behavior Typically, we are looking for an input that crashes a program Program crashes provide an entry point to craft exploits → Not all program crashes are exploitable Technically, this is just another way to find bugs in programs 3 / 21
Outline Fuzz testing Black-box vs. White-box testing Dynamic analysis Limitations Conclusion 4 / 21
Example 1 Which input make this program crash? 1 #i n c l u d e < s t d i o . h > 2 #i n c l u d e < u n i s t d . h > 3 #i n c l u d e < s t d l i b . h > 4 #i n c l u d e < s t r i n g . h > 5 6 i n t main ( i n t argc , char ∗ argv [ ] ) { 7 8 char buf [ 2 0 ] ; 9 10 while ( AFL LOOP (1000) ) { 11 memset ( buf , 0 , 20) ; 12 i f ( read (0 , buf , 19) < 0) { 13 p e r r o r ( ” read ” ) ; 14 return 1 ; 15 } 16 17 i f ( buf [ 0 ] != ’ p ’ ) 18 p r i n t f ( ” f i r s t l e t t e r i s not p \ n” ) ; 19 e l s e i f ( buf [ 1 ] != ’w ’ ) 20 p r i n t f ( ” second l e t t e r i s not w \ n” ) ; 21 e l s e i f ( buf [ 2 ] != ’ n ’ ) 22 p r i n t f ( ” t h i r d l e t t e r i s not n \ n” ) ; 23 e l s e 24 abort () ; 25 26 p r i n t f ( ” buf c o n t a i n s %s \ n” , buf ) ; 27 } 28 0 ; return 29 } 5 / 21
1 #i n c l u d e < s t d i o . h > 2 #i n c l u d e < u n i s t d . h > 3 #i n c l u d e < s t d l i b . h > 4 #i n c l u d e < s t r i n g . h > 5 6 i n t main ( i n t argc , char ∗ argv [ ] ) { 7 8 char buf [ 2 0 ] ; 9 10 while ( AFL LOOP (1000) ) { 11 memset ( buf , 0 , 20) ; 12 ( read (0 , buf , 19) < 0) { i f 13 p e r r o r ( ” read ” ) ; 14 1 ; return 15 } 16 17 ( buf [ 0 ] != ’ p ’ ) i f 18 p r i n t f ( ” f i r s t l e t t e r i s not p \ n” ) ; 19 ( buf [ 1 ] != ’w ’ ) e l s e i f 20 p r i n t f ( ” second l e t t e r i s not w \ n” ) ; 21 ( buf [ 2 ] != ’ n ’ ) e l s e i f 22 p r i n t f ( ” t h i r d l e t t e r i s not n \ n” ) ; 23 e l s e 24 abort () ; 25 26 p r i n t f ( ” buf c o n t a i n s %s \ n” , buf ) ; 27 } 28 return 0 ; 29 } pwn 1 1 Pwn is a leetspeak slang term derived from the verb own, as meaning to appropriate or to conquer to gain ownership. - Wikipedia 6 / 21
Fuzz testing aka Fuzzing Idea attributed to Prof.Barton Miller (Miller, Fredriksen, & So, 1990) . . . i n the F a l l of 1988 , t h e r e was a w i l d midwest thunderstorm . . . With the heavy rain , t h e r e was n o i s e on the ( d i a l − up ) l i n e and that n o i s e was i n t e r f e r i n g with my a b i l i t y to type s e n s i b l e commands to the s h e l l . − Prof . M i l l e r Feed random inputs to a program → No knowledge of program internals Idea bore very good initial results → crash or hang between 25-33% of the Unix utility programs 7 / 21
Let’s fuzz! 8 / 21
Black-box testing Black-box testing treats the program under test as a black-box (opaque to tester) This means the testing framework feeds random inputs to the program However, random inputs alone are unlikely to trigger bugs x = 1000 / ( input + 100000 ) , how long will it take the fuzzer to feed input = − 100000? Fuzzing, in its initial form, was conceived as a black-box testing technique Ever since, it has evolved into a white box testing technique State of the art fuzzers like afl (Zalewski, n.d.) “learn” from program behavior 9 / 21
White-box testing While-box testing treats the PUT as a white-box (transparent to tester) Program internals can be used to reduce input search space Techniques: Program coverage guided fuzzing, Concolic execution etc. However, these need a model that abstracts program behavior for generating inputs Uses program instrumentation to realize the model = ⇒ Access to program binary or source code Instrumentation may be static (compile-time) or dynamic (run-time) 10 / 21
Coverage guided fuzzing Fuzz inputs based on program coverage information For instance, retain only those input mutations that lead to new coverage 11 / 21
Example Each colour indicates a different program path 1 #i n c l u d e < s t d i o . h > 2 #i n c l u d e < u n i s t d . h > 3 #i n c l u d e < s t d l i b . h > 4 #i n c l u d e < s t r i n g . h > 5 6 i n t main ( i n t argc , char ∗ argv [ ] ) { 7 8 . . . 9 . . . 10 11 i f ( buf [ 0 ] != ’ p ’ ) 12 // buf [ 0 ] != ’ p ’ 13 p r i n t f ( ” f i r s t l e t t e r i s not p \ n” ) ; 14 e l s e i f ( buf [ 1 ] != ’w ’ ) 15 // buf [ 0 ] == ’ p ’ && buf [ 1 ] != ’w ’ 16 p r i n t f ( ” second l e t t e r i s not w \ n” ) ; 17 e l s e i f ( buf [ 2 ] != ’ n ’ ) 18 // buf [ 0 ] == ’ p ’ && buf [ 1 ] == ’w ’ && buf [ 2 ] != ’ n ’ 19 p r i n t f ( ” t h i r d l e t t e r i s not n \ n” ) ; 20 e l s e 21 // buf [ 0 ] == ’ p ’ && buf [ 1 ] == ’w ’ && buf [ 2 ] == ’ n ’ 22 abort () ; 23 24 . . . 25 . . . 26 } 12 / 21
Instrumentation But how does the fuzzer know which program path was taken? Answer: Instrument the program What does the instrumentation look like? cur_location = random-number; shared_mem[cur_location ˆ prev_location]++; prev_location = cur_location >> 1; Each branch point indexes a byte in global state State captures number of times branch point taken Comparing state changes per input, it is possible to filter seemingly redundant inputs 13 / 21
Thought experiment Is the state captured by the instrumentation good enough? In other words, can you think of an example where the instrumentation might discard potentially crashing inputs Only a heuristic → But, turns out to be very good in practice 14 / 21
Type of instrumentation Program instrumentation can be done at compile-time or at run-time Compile-time instrumentation incurs low performance overhead Checking logic can be optimized before-hand Slow down only because of additional instructions and associated I/O But, hard to encode checking logic for dynamic language features in general Run-time instrumentations incurs relatively high overhead Checking logic needs to be inserted at program run time Typically implemented using binary translation = ⇒ very slow Since instrumentation occurs at run time, dynamism (e.g., indirect branches) can be handled relatively easily Rest of the lecture looks at dynamic analyses that use compile-time instrumentation 15 / 21
Dynamic analysis Why bother if a good fuzzer gives us crashing inputs? A normal program crash happens only when there is a serious problem Example: Segmentation fault, assertion failure, stack canary failures etc. But, it does not uncover latent faults Example: Heap corruption, stack corruption but canary intact etc. Dynamic analysis can help uncover latent faults Can be used in conjunction with a fuzzer for proactive security audits 16 / 21
AddressSanitizer AddressSanitizer (Serebryany, Bruening, Potapenko, & Vyukov, 2012) is an instrumentation framework for detecting multiple classes of memory corruption faults Heap/stack buffer overflows, use-after-free bugs Idea is similar to coverage guided fuzzing i.e., insert bounds checks by instrumenting the program Each word of addressable heap maps to a byte of shadow memory Uses custom memory allocation functions to set shadow byte during allocation Uses shadow lookups during load and store instructions to detect faults e.g. loading from/storing to unaddressable memory location 17 / 21
ASan Instrumentation Every 8-byte aligned word on x86 64 has 9 states First n bytes 0 ≤ n ≤ 8 are addressable, rest not = ⇒ 1 shadow byte for 8 bytes of application memory Instrumentation looks like char *shadow = MemToShadow(addr); if (*shadow && *shadow <= func(addr)) ReportError(addr) ... 18 / 21
Limitations Fuzz testing and dynamic analysis covered in the lecture are program-input centric This means, if a fuzzed input cannot uncover a program path, bugs in that program path cannot be found Data on false negatives (bugs missed) by fuzzers is hard to come by Exception: Recent research on bug insertion (Dolan-Gavitt et al., 2016) Preliminary results show that only 10% of inserted bugs found by a SotA fuzzer Future: Use symbolic execution to maximize path coverage? Awaiting a public study of efficacy 19 / 21
Conclusions Dynamic analysis attempts to uncover bugs during program execution We discussed fuzz testing and program instrumentation Discussed techniques and tools remain relevant for real-world security audits Anecdotal evidence shows that over 90% of C/C++ bugs found using a white-box fuzzer Recent data on bugs missed by fuzzing tells us that there is room for improvement 20 / 21
Recommend
More recommend
