Detecting Concurrency Errors of Erlang Programs Using Systematic - - PowerPoint PPT Presentation

Detecting Concurrency Errors

• f Erlang Programs

Using Systematic Testing

Kostis Sagonas

kostis@it.uu.se

2017-02-01 Dagstuhl 2017

Outline

Erlang Concurrency errors and their sources Systematic Concurrency Testing Concuerror & Demo Techniques to fight combinatorial explosion Some experiences

2017-02-01 Dagstuhl 2017

Erlang

Concurrent functional language Implements the actor model of concurrency

• lightweight processes (“green threads”)
• asynchronous message passing
• selective receive

Conceptually no shared memory but

• various built-ins that manipulate shared memory

2017-02-01 Dagstuhl 2017

Concurrent programming is HARD

Concurrent execution is difficult to reason about

and get right (even for experts!)

Rare process interleaving results in bugs that are

hard to anticipate difficult to find, reproduce, and debug (“Heisenbugs”) hard to be sure whether they are really fixed

Big productivity problem: it can waste significant

developers’ time and resources

Can have severe consequences

2017-02-01 Dagstuhl 2017

Systematic Concurrency Testing

aka Stateless Model Checking

Technique to find concurrency errors or verify

their absence

by exploring all possible ways that concurrent

execution can influence a program’s outcome

Fully automatic Low memory requirements Applicable to programs with finite executions

2017-02-01 Dagstuhl 2017

Sources of non-determinism

Scheduling non-determinism

Interleaving non-determinism

Processes can race to access shared resources Processes can be preempted at arbitrary points

Timing non-determinism

Sleeping processes can wake up at any point Timers can fire in arbitrary points/orders

Memory model effects Input/data non-determinism

Programs can be used in a variety of ways Non-deterministic calls (e.g. random())

2017-02-01 Dagstuhl 2017

x := 1; y := 1; x := 1; y := 1; x := 2; y := 2; x := 2; y := 2;

2,1 2,1 1,0 1,0 0,0 0,0 1,1 1,1 2,2 2,2 2,2 2,2 2,1 2,1 2,0 2,0 2,1 2,1 2,2 2,2 1,2 1,2 2,0 2,0 2,2 2,2 1,1 1,1 1,1 1,1 1,2 1,2 1,0 1,0 1,2 1,2 1,1 1,1

y := 1; y := 1; x := 1; x := 1; y := 2; y := 2; x := 2; x := 2;

Scheduling non-determinism

2017-02-01 Dagstuhl 2017

Concuerror

A SCT tool for Erlang programs Given a program and its test suite

• systematically explores process interleaving and
• presents detailed interleaving information about

errors during the execution of these tests

Errors detected

• Process crashes and abnormal termination
• Assertion violations
• “Deadlocks”: lack of progress for processes

2017-02-01 Dagstuhl 2017

www.concuerror.com

2017-02-01 Dagstuhl 2017

Concuerror's properties

Easy to use Scalable

Applicable to “real-world” programs

Precise

Any error found is possible to occur Does not introduce new behaviors

Coverage

All concurrency errors (for a test) can be found Captures all scheduling non-determinism Exhaustively explores this non-determinism

2017-02-01 Dagstuhl 2017

Erlang program and its unit test

• module(ping_pong).
• export([pong/0]).

pong() -> Self = self(), Pid = spawn(fun() -> ping(Self) end), register(ping_pong, Pid), receive ping -> ok end. ping(P) -> P ! ping.

• module(ping_pong_test).
• export([test/0]).

pong_test() ->

• k = ping_pong:pong().

2017-02-01 Dagstuhl 2017

Error discovered by Concuerror

Checked 5 interleaving( s) . 1 error found. Error type : Exception Details : {badarg,[{erlang,register,[ping_pong,<...>],[]}, ... Process P1 spawns process P1.1 Process P1.1 sends message `ping` to process P1 Process P1.1 exits (normal) Process P1 registers process P1.1 (dead) as `ping_pong` Process P1 exits ("Exception")

2017-02-01 Dagstuhl 2017

Another Erlang program

• module(identity_theft).
• export([action/0]).

action() -> Bank = self(), register(bank, Bank), Cust = spawn(fun() -> bank ! money end), God = spawn(fun() -> receive Msg -> ok end end), receive money -> God ! bank_got_money end.

• module(identity_theft).
• export([action/0]).

action() -> Bank = self(), register(bank, Bank), Cust = spawn(fun() -> bank ! money end), God = spawn(fun() -> receive Msg -> ok end end), Thief = spawn(fun() -> unregister(bank), register(bank, self()), receive money -> God ! thief_got_money after 0 -> God ! theft_failed end end), receive money -> God ! bank_got_money end.

2017-02-01 Dagstuhl 2017

Interleaving Explosion

Combinatorial explosion in the number of interleavings

Initially: x = y = … = z = 0 Thread 1: x := 1 Thread 2: y := 1

…

Thread N: z := 1

• Interleavings under naïve exploration: N!
• Interleavings needed to cover all behaviors: 1

Partial Order Reduction (POR)

Explore just a subset of all interleavings Still cover all behaviors

2017-02-01 Dagstuhl 2017

Partial Order Reduction

Thread 1: x := 1 Thread 2: y := 1

The order of independent events does not matter

x= 1 y= 0 x= 0 y= 0

Thread 1: x := 1 Thread 2: y:= 1

x= 0 y= 1 x= 1 y= 1

Thread 1: x := 1 Thread 2: y := 1

2017-02-01 Dagstuhl 2017

Dynamic Partial Order Reduction

• 1. Run the program, recording events that

affect shared data as they happen

• 2. Determine pairs of conflicting events
• 3. Find suitable rescheduling points
• 4. Backtrack

2017-02-01 Dagstuhl 2017

New DPOR Algorithms

Source DPOR

• Identical to “Classic” DPOR with one test replaced
• Significantly less explored traces and time

Optimal DPOR

• Achieves optimality with Wakeup Trees

Completely prevent exploration of traces that DPOR

algorithms will eventually discover as redundant

• Memory overhead is very reasonable in practice

2017-02-01 Dagstuhl 2017

POPL ‘05

Benchmark Traces explored Time Classic Source Optimal Classic Source Optimal

filesystem (14) 4 2 2 0.54s 0.36s 0.35s filesystem (16) 64 8 8 8.13s 1.82s 1.78s filesystem (18) 1024 32 32 2m11s 8.52s 8.86s filesystem (19) 4096 64 64

8m33s 18.62s 19.57s

Classic

Our work Our work

2017-02-01 Dagstuhl 2017

POPL ‘14

Benchmark Traces explored Time Classic Source Optimal Classic Source Optimal

indexer (12) 78 8 8 0.74s 0.11s 0.10s indexer (15) 341832 4096 4096

56m20s 50.24s 52.35s

Optimal

2017-02-01 Dagstuhl 2017

Evaluation: Real programs

Benchmark Traces explored Time Classic Source Optimal Classic Source Optimal

dialyzer 12436 3600 3600 14m46s 5m17s 5m46s gproc 14080 8328 8104 3m3s 1m45s 1m57s poolboy 6018 3120 2680 3m2s 1m28s 1m20s rushhour 793375 536118 528984 145m 102m 106m

LOC: 44596 (dialyzer), 9446 (gproc), 79732 (poolboy), 917 (rushhour)