Fuzzing the Solidity Compiler Bhargava Shastry @ibags Ethereum - - PowerPoint PPT Presentation

▶
fuzzing the solidity compiler
SMART_READER_LITE
LIVE PREVIEW

Fuzzing the Solidity Compiler Bhargava Shastry @ibags Ethereum - - PowerPoint PPT Presentation

Dec 19, 2022 153 likes •441 views

Fuzzing the Solidity Compiler Bhargava Shastry @ibags Ethereum Foundation bshastry Fuzzcon Europe 2020 | Fuzzing the Solidity Compiler 2 whoami Security engineer, Solidity team Semantic testing of Solidity compiler Find

slide-1
SLIDE 1

Fuzzcon Europe 2020 | Fuzzing the Solidity Compiler

Fuzzing the Solidity Compiler

Bhargava Shastry

Ethereum Foundation

@ibags bshastry

slide-2
SLIDE 2

Fuzzcon Europe 2020 | Fuzzing the Solidity Compiler

whoami

  • Security engineer, Solidity team
  • Semantic testing of Solidity compiler

Find security-critical bugs in the compiler before it is shipped

2

slide-3
SLIDE 3

Fuzzcon Europe 2020 | Fuzzing the Solidity Compiler

tl;dr:

  • Threat model: Incorrect code generation
  • Randomly generated valid Solidity (yul) programs

test compiler

  • Found 10 bugs using semantic fuzzing
  • Continuous fuzzing for early bug discovery

3

slide-4
SLIDE 4

Fuzzcon Europe 2020 | Fuzzing the Solidity Compiler

Introduction

4

slide-5
SLIDE 5

Fuzzcon Europe 2020 | Fuzzing the Solidity Compiler

Threat model

  • Compiler user (programmer) is not malicious
  • Bugs introduced by the optimizer

5

slide-6
SLIDE 6

Fuzzcon Europe 2020 | Fuzzing the Solidity Compiler

Fuzz testing in a nutshell

while not ctrl + c do input=gen_input() runProgram(input) done

6

slide-7
SLIDE 7

Fuzzcon Europe 2020 | Fuzzing the Solidity Compiler

Limitation of random fuzzing

contract C { function foo() public { do_something(); } } contract C { fu#!3ion foo() puX^&c { do_something(); } }

Accepted by parser Rejected by parser Mutation

7

slide-8
SLIDE 8

Fuzzcon Europe 2020 | Fuzzing the Solidity Compiler

Fuzzing a compiler requires generating valid programs... … generating a valid program requires structure awareness

8

slide-9
SLIDE 9

Fuzzcon Europe 2020 | Fuzzing the Solidity Compiler

Approach

9

slide-10
SLIDE 10

Fuzzcon Europe 2020 | Fuzzing the Solidity Compiler

Write a specification

Specification written in protobuf language

message Block { repeated Statement stmts; } ... message program { repeated Block blocks; }

Full spec: https://github.com/ethereum/solidity/blob/develop/test/tools/ossfuzz/yulProto.proto

10

slide-11
SLIDE 11

Fuzzcon Europe 2020 | Fuzzing the Solidity Compiler

Input generation

  • Input generated and mutated by libprotobuf-mutator
  • Each input is a tree

blocks { stmts { ifstmt { condition { binaryOp { eq { op1: varref{id: 0} op2: 0} } } } } }

11

slide-12
SLIDE 12

Fuzzcon Europe 2020 | Fuzzing the Solidity Compiler

Input conversion

  • Converter is source-to-source translator
  • Input: protobuf serialization format
  • Output: yul program

12

slide-13
SLIDE 13

Fuzzcon Europe 2020 | Fuzzing the Solidity Compiler

Example

blocks { stmts { ifstmt { condition { binaryOp { eq { op1: varref{id: 0} op2: 0} } } } } } if x_0 == 0

Conversion

13

slide-14
SLIDE 14

Fuzzcon Europe 2020 | Fuzzing the Solidity Compiler

Test program generation

Protobuf Converter

Libprotobuf + mutator

{ function f() { ... } } Message func { Block b = 1; }

Protobuf specification Test program

14

slide-15
SLIDE 15

Fuzzcon Europe 2020 | Fuzzing the Solidity Compiler

Correctness testing requires encoding expectation somehow

15

slide-16
SLIDE 16

Fuzzcon Europe 2020 | Fuzzing the Solidity Compiler

Differential fuzzing

  • Track side-effects of execution
  • Run program
  • Run optimized program
  • Compare side-effects

16

slide-17
SLIDE 17

Fuzzcon Europe 2020 | Fuzzing the Solidity Compiler

Yul interpreter

  • Interprets arbitrary yul program
  • Outputs side-effects as a trace (string)

17

slide-18
SLIDE 18

Fuzzcon Europe 2020 | Fuzzing the Solidity Compiler

Yul interpreter

Interpreter

MLOAD MSTORE … DATACOPY { function f() { ... } }

Test program Execution trace

18

slide-19
SLIDE 19

Fuzzcon Europe 2020 | Fuzzing the Solidity Compiler

Fuzzing Setup

Program generator Interpreter Optimizer

MLOAD MSTORE … DATACOPY MLOAD MSTORE … DATACOPY

==

Trace

19

slide-20
SLIDE 20

Fuzzcon Europe 2020 | Fuzzing the Solidity Compiler

Custom Fuzz Mutator

Program Generator Custom Mutator

if x_0 == 0 if x_0 != 0

20

slide-21
SLIDE 21

Fuzzcon Europe 2020 | Fuzzing the Solidity Compiler

Results

21

slide-22
SLIDE 22

Fuzzcon Europe 2020 | Fuzzing the Solidity Compiler

Bugs by component

7 3

22

slide-23
SLIDE 23

Fuzzcon Europe 2020 | Fuzzing the Solidity Compiler

Bugs by impact

5 5

23

slide-24
SLIDE 24

Fuzzcon Europe 2020 | Fuzzing the Solidity Compiler

Bugs by severity

1 2 6 1

Found via custom mutation

24

slide-25
SLIDE 25

Fuzzcon Europe 2020 | Fuzzing the Solidity Compiler

Current work

Antlr based custom mutator

{ function f() { ... } }

Test program

Antlr unparser Solidity mutator

{ function g() { g() } }

Mutation

25

slide-26
SLIDE 26

Fuzzcon Europe 2020 | Fuzzing the Solidity Compiler

Conclusion

26

slide-27
SLIDE 27

Fuzzcon Europe 2020 | Fuzzing the Solidity Compiler

Conclusion

  • Continuous structure-aware fuzzing for early bug

discovery

  • Useful for testing optimizer and data en/decoding
  • Decent assurance

○ Evidence that it works ○ No formal guarantees though

27

slide-28
SLIDE 28

Fuzzcon Europe 2020 | Fuzzing the Solidity Compiler

Thank you!

28

ethereum/solidity.git gitter.im/ethereum/solidity-dev