Beating the No Win Scenario Joe DeVivo @joedevivo Tuesday, 26 - - PowerPoint PPT Presentation

▶
beating the no win scenario
SMART_READER_LITE
LIVE PREVIEW

Beating the No Win Scenario Joe DeVivo @joedevivo Tuesday, 26 - - PowerPoint PPT Presentation

Nov 28, 2022 163 likes •1.89k views

Beating the No Win Scenario Joe DeVivo @joedevivo Tuesday, 26 March 2013 Beating the No Win Scenario Joe DeVivo @joedevivo Maybe youve heard of me Tuesday, 26 March 2013 Things to know about Joe Tuesday, 26 March 2013 Things to know

slide-1
SLIDE 1

Beating the No Win Scenario

Joe DeVivo @joedevivo

Tuesday, 26 March 2013

slide-2
SLIDE 2

Beating the No Win Scenario

Joe DeVivo @joedevivo Maybe you’ve heard of me

Tuesday, 26 March 2013

slide-3
SLIDE 3

Things to know about Joe

Tuesday, 26 March 2013

slide-4
SLIDE 4

Things to know about Joe

  • I’ve never done this before

Tuesday, 26 March 2013

slide-5
SLIDE 5

Things to know about Joe

  • I’ve never done this before
  • Riak User since 2010

Tuesday, 26 March 2013

slide-6
SLIDE 6

Things to know about Joe

  • I’ve never done this before
  • Riak User since 2010
  • Started in Basho Professional Services

Tuesday, 26 March 2013

slide-7
SLIDE 7

Things to know about Joe

  • I’ve never done this before
  • Riak User since 2010
  • Started in Basho Professional Services
  • Moved to Basho Engineering

Tuesday, 26 March 2013

slide-8
SLIDE 8

Things to know about Joe

  • I’ve never done this before
  • Riak User since 2010
  • Started in Basho Professional Services
  • Moved to Basho Engineering
  • OTP Express Alumni (last year!)

Tuesday, 26 March 2013

slide-9
SLIDE 9

Why are you here?

Tuesday, 26 March 2013

slide-10
SLIDE 10

HANDBOOK FOR NEW EMPLOYEES

A fearless adventure in knowing what to do when no one’s there telling you what to do

FIRST EDITION

2012

========================================================

Tuesday, 26 March 2013

slide-11
SLIDE 11

HANDBOOK FOR NEW EMPLOYEES

A fearless adventure in knowing what to do when no one’s there telling you what to do

FIRST EDITION

2012

========================================================

Tuesday, 26 March 2013

slide-12
SLIDE 12

I can do what I want

Tuesday, 26 March 2013

slide-13
SLIDE 13

A Call to Adventure!

Tuesday, 26 March 2013

slide-14
SLIDE 14

The needs of the many, bro

Tuesday, 26 March 2013

slide-15
SLIDE 15

It was a painful process

Tuesday, 26 March 2013

slide-16
SLIDE 16

Python, Virtual Machines, and Bears

Tuesday, 26 March 2013

slide-17
SLIDE 17

Ooooh My

Tuesday, 26 March 2013

slide-18
SLIDE 18

I needed this pain

there are no hi-res ST:V images

Tuesday, 26 March 2013

slide-19
SLIDE 19

Choose your own adventure

Tuesday, 26 March 2013

slide-20
SLIDE 20

The needs of the few, bro

Tuesday, 26 March 2013

slide-21
SLIDE 21

I felt like I couldn’t win

Tuesday, 26 March 2013

slide-22
SLIDE 22

I don’t like to lose

Tuesday, 26 March 2013

slide-23
SLIDE 23

OMG That’s Complicated!

Tuesday, 26 March 2013

slide-24
SLIDE 24

Testing A Distributed System is Hard

Tuesday, 26 March 2013

slide-25
SLIDE 25

Testing A Distributed System is Hard

Tuesday, 26 March 2013

slide-26
SLIDE 26

Testing A Distributed System is Hard ...but not impossible

Tuesday, 26 March 2013

slide-27
SLIDE 27

Were you in the shit?

Tuesday, 26 March 2013

slide-28
SLIDE 28

Is it really a no- win scenario?

Tuesday, 26 March 2013

slide-29
SLIDE 29

Y So Hard?

Tuesday, 26 March 2013

slide-30
SLIDE 30

Y So Hard?

  • Multiple Machines!

Tuesday, 26 March 2013

slide-31
SLIDE 31

Y So Hard?

  • Multiple Machines!
  • Networks!

Tuesday, 26 March 2013

slide-32
SLIDE 32

Y So Hard?

  • Multiple Machines!
  • Networks!
  • Nodes!

Tuesday, 26 March 2013

slide-33
SLIDE 33

Y So Hard?

  • Multiple Machines!
  • Networks!
  • Nodes!
  • Timing!

Tuesday, 26 March 2013

slide-34
SLIDE 34

Y So Hard?

  • Multiple Machines!
  • Networks!
  • Nodes!
  • Timing!
  • Load!

Tuesday, 26 March 2013

slide-35
SLIDE 35

What would Cap’n Kirk Do?

Tuesday, 26 March 2013

slide-36
SLIDE 36

Change the Conditions of the Test

Tuesday, 26 March 2013

slide-37
SLIDE 37

Change the Conditions of the Test

  • We can run a ‘devrel’ to simulate a

distributed system locally

Tuesday, 26 March 2013

slide-38
SLIDE 38

Change the Conditions of the Test

  • We can run a ‘devrel’ to simulate a

distributed system locally

  • Don’t focus on what’s right, focus on what

works

Tuesday, 26 March 2013

slide-39
SLIDE 39

I see you favor riak_test

Tuesday, 26 March 2013

slide-40
SLIDE 40

tonight I do

Tuesday, 26 March 2013

slide-41
SLIDE 41

riak_test

Tuesday, 26 March 2013

slide-42
SLIDE 42

riak_test

  • A simple abstraction of riak cluster

functions

Tuesday, 26 March 2013

slide-43
SLIDE 43

riak_test

  • A simple abstraction of riak cluster

functions

  • Based on a QuickCheck test

Tuesday, 26 March 2013

slide-44
SLIDE 44

riak_test

  • A simple abstraction of riak cluster

functions

  • Based on a QuickCheck test
  • Erlang!

Tuesday, 26 March 2013

slide-45
SLIDE 45

riak_test

  • A simple abstraction of riak cluster

functions

  • Based on a QuickCheck test
  • Erlang!
  • A Machine! Your Machine!

Tuesday, 26 March 2013

slide-46
SLIDE 46

Embracing the Most Important Feature

Tuesday, 26 March 2013

slide-47
SLIDE 47

A User Base

Tuesday, 26 March 2013

slide-48
SLIDE 48

riak_test is a ghetto

Tuesday, 26 March 2013

slide-49
SLIDE 49

riak_test is a ghetto

  • Not a traditional OTP Application

Tuesday, 26 March 2013

slide-50
SLIDE 50

riak_test is a ghetto

  • Not a traditional OTP Application
  • uses rebar and app.src (I’m not an

animal!)

Tuesday, 26 March 2013

slide-51
SLIDE 51

riak_test is a ghetto

  • Not a traditional OTP Application
  • uses rebar and app.src (I’m not an

animal!)

  • Procedural

Tuesday, 26 March 2013

slide-52
SLIDE 52

State and the Main Process

Tuesday, 26 March 2013

slide-53
SLIDE 53

WHOSE DATA IS THIS?

Tuesday, 26 March 2013

slide-54
SLIDE 54

Test Metadata

  • Each test is spawned into its own process
  • riak_test enters a receive loop
  • riak_test answers metadata questions

Tuesday, 26 March 2013

slide-55
SLIDE 55

Main Process

Spawns Test

Test Process

receive loop query metadata repeat query metadata

Tuesday, 26 March 2013

slide-56
SLIDE 56

rec_loop(Pid, TestModule, TestMetaData) -> receive metadata -> Pid ! {metadata, TestMetaData}, rec_loop(Pid, TestModule, TestMetaData) end.

Tuesday, 26 March 2013

slide-57
SLIDE 57

Main Process

Spawns Test

Test Process

receive loop query metadata Spawn sub process repeat

Test Subprocess

query metadata

Tuesday, 26 March 2013

slide-58
SLIDE 58

rec_loop(Pid, TestModule, TestMetaData) -> receive metadata -> Pid ! {metadata, TestMetaData}, rec_loop(Pid, TestModule, TestMetaData); {metadata, P} -> P ! {metadata, TestMetaData}, rec_loop(Pid, TestModule, TestMetaData) end.

Tuesday, 26 March 2013

slide-59
SLIDE 59

?assert

Tuesday, 26 March 2013

slide-60
SLIDE 60

Main Process

Spawns Test

Test Process

receive loop query metadata repeat query metadata ?assert(fail).

Tuesday, 26 March 2013

slide-61
SLIDE 61

rec_loop(Pid, TestModule, TestMetaData) -> receive metadata -> Pid ! {metadata, TestMetaData}, rec_loop(Pid, TestModule, TestMetaData); {metadata, P} -> P ! {metadata, TestMetaData}, rec_loop(Pid, TestModule, TestMetaData); {'EXIT', Pid, normal} -> {pass, undefined}; {'EXIT', Pid, Error} -> lager:warning("~s failed: ~p", [TestModule, Error]), {fail, Error} end.

Tuesday, 26 March 2013

slide-62
SLIDE 62

even if I fail, I don’t fail

Tuesday, 26 March 2013

slide-63
SLIDE 63

Main Process

Spawns Test

Test Process

receive loop query metadata repeat query metadata exit, normal

Tuesday, 26 March 2013

slide-64
SLIDE 64

Collecting Output

Tuesday, 26 March 2013

slide-65
SLIDE 65

Who is group_leader and what does he do?

Tuesday, 26 March 2013

slide-66
SLIDE 66

OldGroupLeader = group_leader(), NewGroupLeader = riak_test_group_leader:new_group_leader(self()), group_leader(NewGroupLeader, self()), Pid = spawn_link(TestModule, confirm, []), {Status, Reason} = rec_loop(Pid, TestModule, TestMetaData), group_leader(OldGroupLeader, self()),

Tuesday, 26 March 2013

slide-67
SLIDE 67

OldGroupLeader = group_leader(), NewGroupLeader = riak_test_group_leader:new_group_leader(self()), group_leader(NewGroupLeader, self()), Pid = spawn_link(TestModule, confirm, []), {Status, Reason} = rec_loop(Pid, TestModule, TestMetaData), group_leader(OldGroupLeader, self()), OldGroupLeader = group_leader(), NewGroupLeader = riak_test_group_leader:new_group_leader(self()), group_leader(NewGroupLeader, self()), Pid = spawn_link(TestModule, confirm, []), {Status, Reason} = rec_loop(Pid, TestModule, TestMetaData), group_leader(OldGroupLeader, self()),

Tuesday, 26 March 2013

slide-68
SLIDE 68

OldGroupLeader = group_leader(), NewGroupLeader = riak_test_group_leader:new_group_leader(self()), group_leader(NewGroupLeader, self()), Pid = spawn_link(TestModule, confirm, []), {Status, Reason} = rec_loop(Pid, TestModule, TestMetaData), group_leader(OldGroupLeader, self()), OldGroupLeader = group_leader(), NewGroupLeader = riak_test_group_leader:new_group_leader(self()), group_leader(NewGroupLeader, self()), Pid = spawn_link(TestModule, confirm, []), {Status, Reason} = rec_loop(Pid, TestModule, TestMetaData), group_leader(OldGroupLeader, self()),

Tuesday, 26 March 2013

slide-69
SLIDE 69

OldGroupLeader = group_leader(), NewGroupLeader = riak_test_group_leader:new_group_leader(self()), group_leader(NewGroupLeader, self()), Pid = spawn_link(TestModule, confirm, []), {Status, Reason} = rec_loop(Pid, TestModule, TestMetaData), group_leader(OldGroupLeader, self()), OldGroupLeader = group_leader(), NewGroupLeader = riak_test_group_leader:new_group_leader(self()), group_leader(NewGroupLeader, self()), Pid = spawn_link(TestModule, confirm, []), {Status, Reason} = rec_loop(Pid, TestModule, TestMetaData), group_leader(OldGroupLeader, self()),

Tuesday, 26 March 2013

slide-70
SLIDE 70

OldGroupLeader = group_leader(), NewGroupLeader = riak_test_group_leader:new_group_leader(self()), group_leader(NewGroupLeader, self()), Pid = spawn_link(TestModule, confirm, []), {Status, Reason} = rec_loop(Pid, TestModule, TestMetaData), group_leader(OldGroupLeader, self()), OldGroupLeader = group_leader(), NewGroupLeader = riak_test_group_leader:new_group_leader(self()), group_leader(NewGroupLeader, self()), Pid = spawn_link(TestModule, confirm, []), {Status, Reason} = rec_loop(Pid, TestModule, TestMetaData), group_leader(OldGroupLeader, self()),

Tuesday, 26 March 2013

slide-71
SLIDE 71

OldGroupLeader = group_leader(), NewGroupLeader = riak_test_group_leader:new_group_leader(self()), group_leader(NewGroupLeader, self()), Pid = spawn_link(TestModule, confirm, []), {Status, Reason} = rec_loop(Pid, TestModule, TestMetaData), group_leader(OldGroupLeader, self()), OldGroupLeader = group_leader(), NewGroupLeader = riak_test_group_leader:new_group_leader(self()), group_leader(NewGroupLeader, self()), Pid = spawn_link(TestModule, confirm, []), {Status, Reason} = rec_loop(Pid, TestModule, TestMetaData), group_leader(OldGroupLeader, self()),

Tuesday, 26 March 2013

slide-72
SLIDE 72

OldGroupLeader = group_leader(), NewGroupLeader = riak_test_group_leader:new_group_leader(self()), group_leader(NewGroupLeader, self()), Pid = spawn_link(TestModule, confirm, []), {Status, Reason} = rec_loop(Pid, TestModule, TestMetaData), group_leader(OldGroupLeader, self()), OldGroupLeader = group_leader(), NewGroupLeader = riak_test_group_leader:new_group_leader(self()), group_leader(NewGroupLeader, self()), Pid = spawn_link(TestModule, confirm, []), {Status, Reason} = rec_loop(Pid, TestModule, TestMetaData), group_leader(OldGroupLeader, self()),

Tuesday, 26 March 2013

slide-73
SLIDE 73

io:format custom group_leader group_leader

Tuesday, 26 March 2013

slide-74
SLIDE 74

io:format custom group_leader group_leader

group_leader(NewGroupLeader, self()),

Tuesday, 26 March 2013

slide-75
SLIDE 75

io:format custom group_leader group_leader

group_leader(NewGroupLeader, self()),

Tuesday, 26 March 2013

slide-76
SLIDE 76

io:format custom group_leader group_leader

Tuesday, 26 March 2013

slide-77
SLIDE 77

io:format custom group_leader group_leader

group_leader(OldGroupLeader, self()),

Tuesday, 26 March 2013

slide-78
SLIDE 78

io:format custom group_leader group_leader

group_leader(OldGroupLeader, self()),

Tuesday, 26 March 2013

slide-79
SLIDE 79

console interaction

Tuesday, 26 March 2013

slide-80
SLIDE 80

Who is port_command and what does he do?

Tuesday, 26 March 2013

slide-81
SLIDE 81

Port = open_port({spawn, Cmd}, [stream, use_stdio, exit_status, binary, stderr_to_stdout]), port_command(Port, Msg), receive {Port, {data, Data}} -> %% do something end

Tuesday, 26 March 2013

slide-82
SLIDE 82

Port = open_port({spawn, Cmd}, [stream, use_stdio, exit_status, binary, stderr_to_stdout]), port_command(Port, Msg), receive {Port, {data, Data}} -> %% do something end

Tuesday, 26 March 2013

slide-83
SLIDE 83

Port = open_port({spawn, Cmd}, [stream, use_stdio, exit_status, binary, stderr_to_stdout]), port_command(Port, Msg), receive {Port, {data, Data}} -> %% do something end

Tuesday, 26 March 2013

slide-84
SLIDE 84

Port = open_port({spawn, Cmd}, [stream, use_stdio, exit_status, binary, stderr_to_stdout]), port_command(Port, Msg), receive {Port, {data, Data}} -> %% do something end

Tuesday, 26 March 2013

slide-85
SLIDE 85

Port = open_port({spawn, Cmd}, [stream, use_stdio, exit_status, binary, stderr_to_stdout]), port_command(Port, Msg), receive {Port, {data, Data}} -> %% do something end

Tuesday, 26 March 2013

slide-86
SLIDE 86

rt:attach(Node, CommandExpectations),

Tuesday, 26 March 2013

slide-87
SLIDE 87

rt:attach(Node, CommandExpectations), [{expect, "\(^D to exit\)"}, {send, "riak_core_ring_manager:get_my_ring()."}, {expect, "dict,"}, {send, [4]}] %% 4 = ^D

Tuesday, 26 March 2013

slide-88
SLIDE 88

rt:attach(Node, CommandExpectations), [{expect, "\(^D to exit\)"}, {send, "riak_core_ring_manager:get_my_ring()."}, {expect, "dict,"}, {send, [4]}] %% 4 = ^D

Tuesday, 26 March 2013

slide-89
SLIDE 89

rt:attach(Node, CommandExpectations), [{expect, "\(^D to exit\)"}, {send, "riak_core_ring_manager:get_my_ring()."}, {expect, "dict,"}, {send, [4]}] %% 4 = ^D

Tuesday, 26 March 2013

slide-90
SLIDE 90

rt:attach(Node, CommandExpectations), [{expect, "\(^D to exit\)"}, {send, "riak_core_ring_manager:get_my_ring()."}, {expect, "dict,"}, {send, [4]}] %% 4 = ^D

Tuesday, 26 March 2013

slide-91
SLIDE 91

rt:attach(Node, CommandExpectations), [{expect, "\(^D to exit\)"}, {send, "riak_core_ring_manager:get_my_ring()."}, {expect, "dict,"}, {send, [4]}] %% 4 = ^D

Tuesday, 26 March 2013

slide-92
SLIDE 92

rt:attach(Node, CommandExpectations), [{expect, "\(^D to exit\)"}, {send, "riak_core_ring_manager:get_my_ring()."}, {expect, "dict,"}, {send, [4]}] %% 4 = ^D

Tuesday, 26 March 2013

slide-93
SLIDE 93

commands

Tuesday, 26 March 2013

slide-94
SLIDE 94

13:14:17.861 [info] Cmd: java -Dcom.basho.riak.host=127.0.0.1 - Dcom.basho.riak.http.port=10018 -Dcom.basho.riak.pbc.port=10017 -cp / tmp/riak_test_scratch/riak-client-1.1.0-SNAPSHOT-jar-with- dependencies-and-tests.jar:/tmp/riak_test_scratch/riak-client-1.1.0- SNAPSHOT-tests.jar org.junit.runner.JUnitCore com.basho.riak.client.AllTests 13:16:54.580 [info] JUnit version 4.4 ...................................................................... ..........I.................I......................................... ...................................................................... ...................................................................... ...................................................................... ...................................................................... ...................................................................... ...................................................................... .. Time: 153.93 OK (583 tests)

Tuesday, 26 March 2013

slide-95
SLIDE 95

13:14:17.861 [info] Cmd: java -Dcom.basho.riak.host=127.0.0.1 - Dcom.basho.riak.http.port=10018 -Dcom.basho.riak.pbc.port=10017 -cp / tmp/riak_test_scratch/riak-client-1.1.0-SNAPSHOT-jar-with- dependencies-and-tests.jar:/tmp/riak_test_scratch/riak-client-1.1.0- SNAPSHOT-tests.jar org.junit.runner.JUnitCore com.basho.riak.client.AllTests 13:16:54.580 [info] JUnit version 4.4 ...................................................................... ..........I.................I......................................... ...................................................................... ...................................................................... ...................................................................... ...................................................................... ...................................................................... ...................................................................... .. Time: 153.93 OK (583 tests)

Tuesday, 26 March 2013

slide-96
SLIDE 96

11:29:38.463 [info] Cmd: java -Dcom.basho.riak.host=127.0.0.1 - Dcom.basho.riak.http.port=10018 -Dcom.basho.riak.pbc.port=10017 -cp / tmp/riak_test_scratch/riak-client-1.1.0-SNAPSHOT-jar-with- dependencies-and-tests.jar:/tmp/riak_test_scratch/riak-client-1.1.0- SNAPSHOT-tests.jar org.junit.runner.JUnitCore com.basho.riak.client.AllTests 11:29:38.865 [info] 11:29:38.865 [info] JUnit version 4.4 11:29:39.301 [info] 11:29:40.234 [info] .................................................. 11:29:41.049 [info] ........ 11:29:42.158 [info] . 11:29:43.303 [info] . 11:29:45.009 [info] ...... 11:29:46.328 [info] ...... 11:29:47.286 [info] ...... 11:29:49.044 [info] ...I. 11:29:50.333 [info] .... 11:29:51.190 [info] .. 11:29:52.070 [info] ...... 11:29:53.845 [info] ....I. 11:29:54.939 [info] .... 11:29:55.160 [info] . 11:29:56.939 [info] ....... 11:29:57.098 [info] ...

Tuesday, 26 March 2013

slide-97
SLIDE 97

11:29:38.463 [info] Cmd: java -Dcom.basho.riak.host=127.0.0.1 - Dcom.basho.riak.http.port=10018 -Dcom.basho.riak.pbc.port=10017 -cp / tmp/riak_test_scratch/riak-client-1.1.0-SNAPSHOT-jar-with- dependencies-and-tests.jar:/tmp/riak_test_scratch/riak-client-1.1.0- SNAPSHOT-tests.jar org.junit.runner.JUnitCore com.basho.riak.client.AllTests 11:29:38.865 [info] 11:29:38.865 [info] JUnit version 4.4 11:29:39.301 [info] 11:29:40.234 [info] .................................................. 11:29:41.049 [info] ........ 11:29:42.158 [info] . 11:29:43.303 [info] . 11:29:45.009 [info] ...... 11:29:46.328 [info] ...... 11:29:47.286 [info] ...... 11:29:49.044 [info] ...I. 11:29:50.333 [info] .... 11:29:51.190 [info] .. 11:29:52.070 [info] ...... 11:29:53.845 [info] ....I. 11:29:54.939 [info] .... 11:29:55.160 [info] . 11:29:56.939 [info] ....... 11:29:57.098 [info] ...

Tuesday, 26 March 2013

slide-98
SLIDE 98

Prerequisites

Tuesday, 26 March 2013

slide-99
SLIDE 99
  • prereq("java").
  • prereq("curl").

check_prereqs(Module) -> Prereqs = proplists:get_all_values(prereq,

  • Module:module_info(attributes)),

P2 = [ {Prereq, rt:which(Prereq)} || Prereq <- Prereqs], lager:info("~s prereqs: ~p", [Module, P2]), [ lager:warning("~s prereq '~s' not installed.", [Module, P]) || {P, false} <- P2], GoodToGo = lists:all(fun({_, Present}) -> Present end, P2), ?assertEqual({all_prereqs_present, true},

  • {all_prereqs_present, GoodToGo}).

Tuesday, 26 March 2013

slide-100
SLIDE 100
  • prereq("java").
  • prereq("curl").

check_prereqs(Module) -> Prereqs = proplists:get_all_values(prereq,

  • Module:module_info(attributes)),

P2 = [ {Prereq, rt:which(Prereq)} || Prereq <- Prereqs], lager:info("~s prereqs: ~p", [Module, P2]), [ lager:warning("~s prereq '~s' not installed.", [Module, P]) || {P, false} <- P2], GoodToGo = lists:all(fun({_, Present}) -> Present end, P2), ?assertEqual({all_prereqs_present, true},

  • {all_prereqs_present, GoodToGo}).
  • prereq("java").
  • prereq("curl").

check_prereqs(Module) -> Prereqs = proplists:get_all_values(prereq,

  • Module:module_info(attributes)),

P2 = [ {Prereq, rt:which(Prereq)} || Prereq <- Prereqs], lager:info("~s prereqs: ~p", [Module, P2]), [ lager:warning("~s prereq '~s' not installed.", [Module, P]) || {P, false} <- P2], GoodToGo = lists:all(fun({_, Present}) -> Present end, P2), ?assertEqual({all_prereqs_present, true},

  • {all_prereqs_present, GoodToGo}).

Tuesday, 26 March 2013

slide-101
SLIDE 101
  • prereq("java").
  • prereq("curl").

check_prereqs(Module) -> Prereqs = proplists:get_all_values(prereq,

  • Module:module_info(attributes)),

P2 = [ {Prereq, rt:which(Prereq)} || Prereq <- Prereqs], lager:info("~s prereqs: ~p", [Module, P2]), [ lager:warning("~s prereq '~s' not installed.", [Module, P]) || {P, false} <- P2], GoodToGo = lists:all(fun({_, Present}) -> Present end, P2), ?assertEqual({all_prereqs_present, true},

  • {all_prereqs_present, GoodToGo}).
  • prereq("java").
  • prereq("curl").

check_prereqs(Module) -> Prereqs = proplists:get_all_values(prereq,

  • Module:module_info(attributes)),

P2 = [ {Prereq, rt:which(Prereq)} || Prereq <- Prereqs], lager:info("~s prereqs: ~p", [Module, P2]), [ lager:warning("~s prereq '~s' not installed.", [Module, P]) || {P, false} <- P2], GoodToGo = lists:all(fun({_, Present}) -> Present end, P2), ?assertEqual({all_prereqs_present, true},

  • {all_prereqs_present, GoodToGo}).

Tuesday, 26 March 2013

slide-102
SLIDE 102
  • prereq("java").
  • prereq("curl").

check_prereqs(Module) -> Prereqs = proplists:get_all_values(prereq,

  • Module:module_info(attributes)),

P2 = [ {Prereq, rt:which(Prereq)} || Prereq <- Prereqs], lager:info("~s prereqs: ~p", [Module, P2]), [ lager:warning("~s prereq '~s' not installed.", [Module, P]) || {P, false} <- P2], GoodToGo = lists:all(fun({_, Present}) -> Present end, P2), ?assertEqual({all_prereqs_present, true},

  • {all_prereqs_present, GoodToGo}).
  • prereq("java").
  • prereq("curl").

check_prereqs(Module) -> Prereqs = proplists:get_all_values(prereq,

  • Module:module_info(attributes)),

P2 = [ {Prereq, rt:which(Prereq)} || Prereq <- Prereqs], lager:info("~s prereqs: ~p", [Module, P2]), [ lager:warning("~s prereq '~s' not installed.", [Module, P]) || {P, false} <- P2], GoodToGo = lists:all(fun({_, Present}) -> Present end, P2), ?assertEqual({all_prereqs_present, true},

  • {all_prereqs_present, GoodToGo}).

Tuesday, 26 March 2013

slide-103
SLIDE 103
  • prereq("java").
  • prereq("curl").

check_prereqs(Module) -> Prereqs = proplists:get_all_values(prereq,

  • Module:module_info(attributes)),

P2 = [ {Prereq, rt:which(Prereq)} || Prereq <- Prereqs], lager:info("~s prereqs: ~p", [Module, P2]), [ lager:warning("~s prereq '~s' not installed.", [Module, P]) || {P, false} <- P2], GoodToGo = lists:all(fun({_, Present}) -> Present end, P2), ?assertEqual({all_prereqs_present, true},

  • {all_prereqs_present, GoodToGo}).
  • prereq("java").
  • prereq("curl").

check_prereqs(Module) -> Prereqs = proplists:get_all_values(prereq,

  • Module:module_info(attributes)),

P2 = [ {Prereq, rt:which(Prereq)} || Prereq <- Prereqs], lager:info("~s prereqs: ~p", [Module, P2]), [ lager:warning("~s prereq '~s' not installed.", [Module, P]) || {P, false} <- P2], GoodToGo = lists:all(fun({_, Present}) -> Present end, P2), ?assertEqual({all_prereqs_present, true},

  • {all_prereqs_present, GoodToGo}).

Tuesday, 26 March 2013

slide-104
SLIDE 104

14:27:32.712 [info] Checking for presence of java 14:27:32.721 [info] Checking for presence of curl 14:27:32.727 [info] client_java_verify prereqs: [{"java",true},{"curl",true}] 14:27:32.728 [notice] Running Test client_java_verify

Tuesday, 26 March 2013

slide-105
SLIDE 105

14:27:32.712 [info] Checking for presence of java 14:27:32.721 [info] Checking for presence of curl 14:27:32.727 [info] client_java_verify prereqs: [{"java",true},{"curl",true}] 14:27:32.728 [notice] Running Test client_java_verify 14:27:32.712 [info] Checking for presence of java 14:27:32.721 [info] Checking for presence of curl 14:27:32.727 [info] client_java_verify prereqs: [{"java",true},{"curl",true}] 14:27:32.728 [notice] Running Test client_java_verify

Tuesday, 26 March 2013

slide-106
SLIDE 106

14:27:32.712 [info] Checking for presence of java 14:27:32.721 [info] Checking for presence of curl 14:27:32.727 [info] client_java_verify prereqs: [{"java",true},{"curl",true}] 14:27:32.728 [notice] Running Test client_java_verify 14:27:32.712 [info] Checking for presence of java 14:27:32.721 [info] Checking for presence of curl 14:27:32.727 [info] client_java_verify prereqs: [{"java",true},{"curl",true}] 14:27:32.728 [notice] Running Test client_java_verify 14:27:32.712 [info] Checking for presence of java 14:27:32.721 [info] Checking for presence of curl 14:27:32.727 [info] client_java_verify prereqs: [{"java",true},{"curl",true}] 14:27:32.728 [notice] Running Test client_java_verify

Tuesday, 26 March 2013

slide-107
SLIDE 107

14:27:32.712 [info] Checking for presence of java 14:27:32.721 [info] Checking for presence of curl 14:27:32.727 [info] client_java_verify prereqs: [{"java",true},{"curl",true}] 14:27:32.728 [notice] Running Test client_java_verify 14:27:32.712 [info] Checking for presence of java 14:27:32.721 [info] Checking for presence of curl 14:27:32.727 [info] client_java_verify prereqs: [{"java",true},{"curl",true}] 14:27:32.728 [notice] Running Test client_java_verify 14:27:32.712 [info] Checking for presence of java 14:27:32.721 [info] Checking for presence of curl 14:27:32.727 [info] client_java_verify prereqs: [{"java",true},{"curl",true}] 14:27:32.728 [notice] Running Test client_java_verify 14:27:32.712 [info] Checking for presence of java 14:27:32.721 [info] Checking for presence of curl 14:27:32.727 [info] client_java_verify prereqs: [{"java",true},{"curl",true}] 14:27:32.728 [notice] Running Test client_java_verify

Tuesday, 26 March 2013

slide-108
SLIDE 108

14:27:32.712 [info] Checking for presence of java 14:27:32.721 [info] Checking for presence of curl 14:27:32.727 [info] client_java_verify prereqs: [{"java",true},{"curl",true}] 14:27:32.728 [notice] Running Test client_java_verify 14:27:32.712 [info] Checking for presence of java 14:27:32.721 [info] Checking for presence of curl 14:27:32.727 [info] client_java_verify prereqs: [{"java",true},{"curl",true}] 14:27:32.728 [notice] Running Test client_java_verify 14:27:32.712 [info] Checking for presence of java 14:27:32.721 [info] Checking for presence of curl 14:27:32.727 [info] client_java_verify prereqs: [{"java",true},{"curl",true}] 14:27:32.728 [notice] Running Test client_java_verify 14:27:32.712 [info] Checking for presence of java 14:27:32.721 [info] Checking for presence of curl 14:27:32.727 [info] client_java_verify prereqs: [{"java",true},{"curl",true}] 14:27:32.728 [notice] Running Test client_java_verify 14:27:32.712 [info] Checking for presence of java 14:27:32.721 [info] Checking for presence of curl 14:27:32.727 [info] client_java_verify prereqs: [{"java",true},{"curl",true}] 14:27:32.728 [notice] Running Test client_java_verify

Tuesday, 26 March 2013

slide-109
SLIDE 109
  • prereq("java").
  • prereq("curl").
  • prereq("missing_prereq").

Tuesday, 26 March 2013

slide-110
SLIDE 110
  • prereq("java").
  • prereq("curl").
  • prereq("missing_prereq").
  • prereq("java").
  • prereq("curl").
  • prereq("missing_prereq").

Tuesday, 26 March 2013

slide-111
SLIDE 111

10:59:31.402 [info] Checking for presence of java 10:59:31.414 [info] Checking for presence of curl 10:59:31.423 [info] Checking for presence of missing_prereq 10:59:31.435 [warning] `missing_prereq` is not installed 10:59:31.436 [info] client_java_verify prereqs: [{"java",true},{"curl",true},{"missing_prereq",false}] 10:59:31.437 [warning] client_java_verify prereq 'missing_prereq' not installed. escript: exception error: {assertEqual_failed, [{module,riak_test_runner}, {line,122}, {expression,"{ all_prereqs_present , GoodToGo }"}, {expected,{all_prereqs_present,true}}, {value,{all_prereqs_present,false}}]}

Tuesday, 26 March 2013

slide-112
SLIDE 112

10:59:31.402 [info] Checking for presence of java 10:59:31.414 [info] Checking for presence of curl 10:59:31.423 [info] Checking for presence of missing_prereq 10:59:31.435 [warning] `missing_prereq` is not installed 10:59:31.436 [info] client_java_verify prereqs: [{"java",true},{"curl",true},{"missing_prereq",false}] 10:59:31.437 [warning] client_java_verify prereq 'missing_prereq' not installed. escript: exception error: {assertEqual_failed, [{module,riak_test_runner}, {line,122}, {expression,"{ all_prereqs_present , GoodToGo }"}, {expected,{all_prereqs_present,true}}, {value,{all_prereqs_present,false}}]} 10:59:31.402 [info] Checking for presence of java 10:59:31.414 [info] Checking for presence of curl 10:59:31.423 [info] Checking for presence of missing_prereq 10:59:31.435 [warning] `missing_prereq` is not installed 10:59:31.436 [info] client_java_verify prereqs: [{"java",true},{"curl",true},{"missing_prereq",false}] 10:59:31.437 [warning] client_java_verify prereq 'missing_prereq' not installed. escript: exception error: {assertEqual_failed, [{module,riak_test_runner}, {line,122}, {expression,"{ all_prereqs_present , GoodToGo }"}, {expected,{all_prereqs_present,true}}, {value,{all_prereqs_present,false}}]}

Tuesday, 26 March 2013

slide-113
SLIDE 113

?assertEqual(true, GoodToGo).

Tuesday, 26 March 2013

slide-114
SLIDE 114

?assertEqual(true, GoodToGo).

escript: exception error: {assertEqual_failed, [{module,riak_test_runner}, {line,122}, {expression,"GoodToGo"}, {expected,true}, {value,false}]}

Tuesday, 26 March 2013

slide-115
SLIDE 115

?assertEqual(true, GoodToGo).

escript: exception error: {assertEqual_failed, [{module,riak_test_runner}, {line,122}, {expression,"GoodToGo"}, {expected,true}, {value,false}]} escript: exception error: {assertEqual_failed, [{module,riak_test_runner}, {line,122}, {expression,"GoodToGo"}, {expected,true}, {value,false}]}

Tuesday, 26 March 2013

slide-116
SLIDE 116

?assertEqual( {all_prereqs_present, true}, {all_prereqs_present, GoodToGo}).

Tuesday, 26 March 2013

slide-117
SLIDE 117

?assertEqual( {all_prereqs_present, true}, {all_prereqs_present, GoodToGo}).

escript: exception error: {assertEqual_failed, [{module,riak_test_runner}, {line,122}, {expression,"{ all_prereqs_present , GoodToGo }"}, {expected,{all_prereqs_present,true}}, {value,{all_prereqs_present,false}}]}

Tuesday, 26 March 2013

slide-118
SLIDE 118

?assertEqual( {all_prereqs_present, true}, {all_prereqs_present, GoodToGo}).

escript: exception error: {assertEqual_failed, [{module,riak_test_runner}, {line,122}, {expression,"{ all_prereqs_present , GoodToGo }"}, {expected,{all_prereqs_present,true}}, {value,{all_prereqs_present,false}}]} escript: exception error: {assertEqual_failed, [{module,riak_test_runner}, {line,122}, {expression,"{ all_prereqs_present , GoodToGo }"}, {expected,{all_prereqs_present,true}}, {value,{all_prereqs_present,false}}]}

Tuesday, 26 March 2013

slide-119
SLIDE 119

?assertEqual( {all_prereqs_present, true}, {all_prereqs_present, GoodToGo}).

escript: exception error: {assertEqual_failed, [{module,riak_test_runner}, {line,122}, {expression,"{ all_prereqs_present , GoodToGo }"}, {expected,{all_prereqs_present,true}}, {value,{all_prereqs_present,false}}]} escript: exception error: {assertEqual_failed, [{module,riak_test_runner}, {line,122}, {expression,"{ all_prereqs_present , GoodToGo }"}, {expected,{all_prereqs_present,true}}, {value,{all_prereqs_present,false}}]} escript: exception error: {assertEqual_failed, [{module,riak_test_runner}, {line,122}, {expression,"{ all_prereqs_present , GoodToGo }"}, {expected,{all_prereqs_present,true}}, {value,{all_prereqs_present,false}}]}

Tuesday, 26 March 2013

slide-120
SLIDE 120

?assertEqual( {all_prereqs_present, true}, {all_prereqs_present, GoodToGo}).

escript: exception error: {assertEqual_failed, [{module,riak_test_runner}, {line,122}, {expression,"{ all_prereqs_present , GoodToGo }"}, {expected,{all_prereqs_present,true}}, {value,{all_prereqs_present,false}}]} escript: exception error: {assertEqual_failed, [{module,riak_test_runner}, {line,122}, {expression,"{ all_prereqs_present , GoodToGo }"}, {expected,{all_prereqs_present,true}}, {value,{all_prereqs_present,false}}]} escript: exception error: {assertEqual_failed, [{module,riak_test_runner}, {line,122}, {expression,"{ all_prereqs_present , GoodToGo }"}, {expected,{all_prereqs_present,true}}, {value,{all_prereqs_present,false}}]} escript: exception error: {assertEqual_failed, [{module,riak_test_runner}, {line,122}, {expression,"{ all_prereqs_present , GoodToGo }"}, {expected,{all_prereqs_present,true}}, {value,{all_prereqs_present,false}}]}

Tuesday, 26 March 2013

slide-121
SLIDE 121

Reporting

Tuesday, 26 March 2013

slide-122
SLIDE 122

Tuesday, 26 March 2013

slide-123
SLIDE 123

riak_test lager backend

Tuesday, 26 March 2013

slide-124
SLIDE 124

Tuesday, 26 March 2013

slide-125
SLIDE 125

Test Suites

Tuesday, 26 March 2013

slide-126
SLIDE 126

Tuesday, 26 March 2013

slide-127
SLIDE 127

Returning to a Pristine State

Tuesday, 26 March 2013

slide-128
SLIDE 128

10:59:27.995 [info] Resetting nodes to fresh state 10:59:27.995 [debug] Running: git --git- dir="~/riak_test/.git" --work-tree="~/ riak_test/" reset HEAD --hard 10:59:29.662 [debug] Running: git --git- dir="~/riak_test/.git" --work-tree="~/ riak_test/" clean -fd

Tuesday, 26 March 2013

slide-129
SLIDE 129

10:59:27.995 [info] Resetting nodes to fresh state 10:59:27.995 [debug] Running: git --git- dir="~/riak_test/.git" --work-tree="~/ riak_test/" reset HEAD --hard 10:59:29.662 [debug] Running: git --git- dir="~/riak_test/.git" --work-tree="~/ riak_test/" clean -fd 10:59:27.995 [info] Resetting nodes to fresh state 10:59:27.995 [debug] Running: git --git- dir="~/riak_test/.git" --work-tree="~/ riak_test/" reset HEAD --hard 10:59:29.662 [debug] Running: git --git- dir="~/riak_test/.git" --work-tree="~/ riak_test/" clean -fd

Tuesday, 26 March 2013

slide-130
SLIDE 130

Race Conditions?

It’s all wibbly wobbly timey whimey stuff

Tuesday, 26 March 2013

slide-131
SLIDE 131

Race Conditions?

It’s all wibbly wobbly timey whimey stuff

Tuesday, 26 March 2013

slide-132
SLIDE 132

wait_until(Node, Fun, Retry, Delay, TimeoutFun) -> Pass = Fun(Node), case {Retry, Pass} of {_, true} ->

  • k;

{0, _} -> TimeoutFun(Node); _ -> timer:sleep(Delay), wait_until(Node, Fun, Retry-1, Delay, TimeoutFun) end.

Tuesday, 26 March 2013

slide-133
SLIDE 133

wait_until(Node, Fun, Retry, Delay, TimeoutFun) -> Pass = Fun(Node), case {Retry, Pass} of {_, true} ->

  • k;

{0, _} -> TimeoutFun(Node); _ -> timer:sleep(Delay), wait_until(Node, Fun, Retry-1, Delay, TimeoutFun) end. wait_until(Node, Fun, Retry, Delay, TimeoutFun) -> Pass = Fun(Node), case {Retry, Pass} of {_, true} ->

  • k;

{0, _} -> TimeoutFun(Node); _ -> timer:sleep(Delay), wait_until(Node, Fun, Retry-1, Delay, TimeoutFun) end.

Tuesday, 26 March 2013

slide-134
SLIDE 134

wait_until(Node, Fun, Retry, Delay, TimeoutFun) -> Pass = Fun(Node), case {Retry, Pass} of {_, true} ->

  • k;

{0, _} -> TimeoutFun(Node); _ -> timer:sleep(Delay), wait_until(Node, Fun, Retry-1, Delay, TimeoutFun) end. wait_until(Node, Fun, Retry, Delay, TimeoutFun) -> Pass = Fun(Node), case {Retry, Pass} of {_, true} ->

  • k;

{0, _} -> TimeoutFun(Node); _ -> timer:sleep(Delay), wait_until(Node, Fun, Retry-1, Delay, TimeoutFun) end. wait_until(Node, Fun, Retry, Delay, TimeoutFun) -> Pass = Fun(Node), case {Retry, Pass} of {_, true} ->

  • k;

{0, _} -> TimeoutFun(Node); _ -> timer:sleep(Delay), wait_until(Node, Fun, Retry-1, Delay, TimeoutFun) end.

Tuesday, 26 March 2013

slide-135
SLIDE 135

wait_until(Node, Fun, Retry, Delay, TimeoutFun) -> Pass = Fun(Node), case {Retry, Pass} of {_, true} ->

  • k;

{0, _} -> TimeoutFun(Node); _ -> timer:sleep(Delay), wait_until(Node, Fun, Retry-1, Delay, TimeoutFun) end. wait_until(Node, Fun, Retry, Delay, TimeoutFun) -> Pass = Fun(Node), case {Retry, Pass} of {_, true} ->

  • k;

{0, _} -> TimeoutFun(Node); _ -> timer:sleep(Delay), wait_until(Node, Fun, Retry-1, Delay, TimeoutFun) end. wait_until(Node, Fun, Retry, Delay, TimeoutFun) -> Pass = Fun(Node), case {Retry, Pass} of {_, true} ->

  • k;

{0, _} -> TimeoutFun(Node); _ -> timer:sleep(Delay), wait_until(Node, Fun, Retry-1, Delay, TimeoutFun) end. wait_until(Node, Fun, Retry, Delay, TimeoutFun) -> Pass = Fun(Node), case {Retry, Pass} of {_, true} ->

  • k;

{0, _} -> TimeoutFun(Node); _ -> timer:sleep(Delay), wait_until(Node, Fun, Retry-1, Delay, TimeoutFun) end.

Tuesday, 26 March 2013

slide-136
SLIDE 136

wait_until(Node, Fun, Retry, Delay, TimeoutFun) -> Pass = Fun(Node), case {Retry, Pass} of {_, true} ->

  • k;

{0, _} -> TimeoutFun(Node); _ -> timer:sleep(Delay), wait_until(Node, Fun, Retry-1, Delay, TimeoutFun) end. wait_until(Node, Fun, Retry, Delay, TimeoutFun) -> Pass = Fun(Node), case {Retry, Pass} of {_, true} ->

  • k;

{0, _} -> TimeoutFun(Node); _ -> timer:sleep(Delay), wait_until(Node, Fun, Retry-1, Delay, TimeoutFun) end. wait_until(Node, Fun, Retry, Delay, TimeoutFun) -> Pass = Fun(Node), case {Retry, Pass} of {_, true} ->

  • k;

{0, _} -> TimeoutFun(Node); _ -> timer:sleep(Delay), wait_until(Node, Fun, Retry-1, Delay, TimeoutFun) end. wait_until(Node, Fun, Retry, Delay, TimeoutFun) -> Pass = Fun(Node), case {Retry, Pass} of {_, true} ->

  • k;

{0, _} -> TimeoutFun(Node); _ -> timer:sleep(Delay), wait_until(Node, Fun, Retry-1, Delay, TimeoutFun) end. wait_until(Node, Fun, Retry, Delay, TimeoutFun) -> Pass = Fun(Node), case {Retry, Pass} of {_, true} ->

  • k;

{0, _} -> TimeoutFun(Node); _ -> timer:sleep(Delay), wait_until(Node, Fun, Retry-1, Delay, TimeoutFun) end.

Tuesday, 26 March 2013

slide-137
SLIDE 137

wait_until(Node, Fun, Retry, Delay, TimeoutFun) -> Pass = Fun(Node), case {Retry, Pass} of {_, true} ->

  • k;

{0, _} -> TimeoutFun(Node); _ -> timer:sleep(Delay), wait_until(Node, Fun, Retry-1, Delay, TimeoutFun) end. wait_until(Node, Fun, Retry, Delay, TimeoutFun) -> Pass = Fun(Node), case {Retry, Pass} of {_, true} ->

  • k;

{0, _} -> TimeoutFun(Node); _ -> timer:sleep(Delay), wait_until(Node, Fun, Retry-1, Delay, TimeoutFun) end. wait_until(Node, Fun, Retry, Delay, TimeoutFun) -> Pass = Fun(Node), case {Retry, Pass} of {_, true} ->

  • k;

{0, _} -> TimeoutFun(Node); _ -> timer:sleep(Delay), wait_until(Node, Fun, Retry-1, Delay, TimeoutFun) end. wait_until(Node, Fun, Retry, Delay, TimeoutFun) -> Pass = Fun(Node), case {Retry, Pass} of {_, true} ->

  • k;

{0, _} -> TimeoutFun(Node); _ -> timer:sleep(Delay), wait_until(Node, Fun, Retry-1, Delay, TimeoutFun) end. wait_until(Node, Fun, Retry, Delay, TimeoutFun) -> Pass = Fun(Node), case {Retry, Pass} of {_, true} ->

  • k;

{0, _} -> TimeoutFun(Node); _ -> timer:sleep(Delay), wait_until(Node, Fun, Retry-1, Delay, TimeoutFun) end. wait_until(Node, Fun, Retry, Delay, TimeoutFun) -> Pass = Fun(Node), case {Retry, Pass} of {_, true} ->

  • k;

{0, _} -> TimeoutFun(Node); _ -> timer:sleep(Delay), wait_until(Node, Fun, Retry-1, Delay, TimeoutFun) end.

Tuesday, 26 March 2013

slide-138
SLIDE 138

wait_until(Node, Fun, Retry, Delay, TimeoutFun) -> Pass = Fun(Node), case {Retry, Pass} of {_, true} ->

  • k;

{0, _} -> TimeoutFun(Node); _ -> timer:sleep(Delay), wait_until(Node, Fun, Retry-1, Delay, TimeoutFun) end. wait_until(Node, Fun, Retry, Delay, TimeoutFun) -> Pass = Fun(Node), case {Retry, Pass} of {_, true} ->

  • k;

{0, _} -> TimeoutFun(Node); _ -> timer:sleep(Delay), wait_until(Node, Fun, Retry-1, Delay, TimeoutFun) end. wait_until(Node, Fun, Retry, Delay, TimeoutFun) -> Pass = Fun(Node), case {Retry, Pass} of {_, true} ->

  • k;

{0, _} -> TimeoutFun(Node); _ -> timer:sleep(Delay), wait_until(Node, Fun, Retry-1, Delay, TimeoutFun) end. wait_until(Node, Fun, Retry, Delay, TimeoutFun) -> Pass = Fun(Node), case {Retry, Pass} of {_, true} ->

  • k;

{0, _} -> TimeoutFun(Node); _ -> timer:sleep(Delay), wait_until(Node, Fun, Retry-1, Delay, TimeoutFun) end. wait_until(Node, Fun, Retry, Delay, TimeoutFun) -> Pass = Fun(Node), case {Retry, Pass} of {_, true} ->

  • k;

{0, _} -> TimeoutFun(Node); _ -> timer:sleep(Delay), wait_until(Node, Fun, Retry-1, Delay, TimeoutFun) end. wait_until(Node, Fun, Retry, Delay, TimeoutFun) -> Pass = Fun(Node), case {Retry, Pass} of {_, true} ->

  • k;

{0, _} -> TimeoutFun(Node); _ -> timer:sleep(Delay), wait_until(Node, Fun, Retry-1, Delay, TimeoutFun) end. wait_until(Node, Fun, Retry, Delay, TimeoutFun) -> Pass = Fun(Node), case {Retry, Pass} of {_, true} ->

  • k;

{0, _} -> TimeoutFun(Node); _ -> timer:sleep(Delay), wait_until(Node, Fun, Retry-1, Delay, TimeoutFun) end.

Tuesday, 26 March 2013

slide-139
SLIDE 139

wait_until(Node, Fun, Retry, Delay, TimeoutFun) -> Pass = Fun(Node), case {Retry, Pass} of {_, true} ->

  • k;

{0, _} -> TimeoutFun(Node); _ -> timer:sleep(Delay), wait_until(Node, Fun, Retry-1, Delay, TimeoutFun) end. wait_until(Node, Fun, Retry, Delay, TimeoutFun) -> Pass = Fun(Node), case {Retry, Pass} of {_, true} ->

  • k;

{0, _} -> TimeoutFun(Node); _ -> timer:sleep(Delay), wait_until(Node, Fun, Retry-1, Delay, TimeoutFun) end. wait_until(Node, Fun, Retry, Delay, TimeoutFun) -> Pass = Fun(Node), case {Retry, Pass} of {_, true} ->

  • k;

{0, _} -> TimeoutFun(Node); _ -> timer:sleep(Delay), wait_until(Node, Fun, Retry-1, Delay, TimeoutFun) end. wait_until(Node, Fun, Retry, Delay, TimeoutFun) -> Pass = Fun(Node), case {Retry, Pass} of {_, true} ->

  • k;

{0, _} -> TimeoutFun(Node); _ -> timer:sleep(Delay), wait_until(Node, Fun, Retry-1, Delay, TimeoutFun) end. wait_until(Node, Fun, Retry, Delay, TimeoutFun) -> Pass = Fun(Node), case {Retry, Pass} of {_, true} ->

  • k;

{0, _} -> TimeoutFun(Node); _ -> timer:sleep(Delay), wait_until(Node, Fun, Retry-1, Delay, TimeoutFun) end. wait_until(Node, Fun, Retry, Delay, TimeoutFun) -> Pass = Fun(Node), case {Retry, Pass} of {_, true} ->

  • k;

{0, _} -> TimeoutFun(Node); _ -> timer:sleep(Delay), wait_until(Node, Fun, Retry-1, Delay, TimeoutFun) end. wait_until(Node, Fun, Retry, Delay, TimeoutFun) -> Pass = Fun(Node), case {Retry, Pass} of {_, true} ->

  • k;

{0, _} -> TimeoutFun(Node); _ -> timer:sleep(Delay), wait_until(Node, Fun, Retry-1, Delay, TimeoutFun) end. wait_until(Node, Fun, Retry, Delay, TimeoutFun) -> Pass = Fun(Node), case {Retry, Pass} of {_, true} ->

  • k;

{0, _} -> TimeoutFun(Node); _ -> timer:sleep(Delay), wait_until(Node, Fun, Retry-1, Delay, TimeoutFun) end.

Tuesday, 26 March 2013

slide-140
SLIDE 140

Network Partitions

Tuesday, 26 March 2013

slide-141
SLIDE 141

The Prefix Code?

Tuesday, 26 March 2013

slide-142
SLIDE 142

The Cookie!

Tuesday, 26 March 2013

slide-143
SLIDE 143

dev1 dev3 dev2 dev4 riak test

Tuesday, 26 March 2013

slide-144
SLIDE 144

dev1 dev3 dev2 dev4 riak test

reverse the cookie on dev3 and dev4

Tuesday, 26 March 2013

slide-145
SLIDE 145

dev1 dev3 dev2 dev4 riak test

reverse the cookie on dev3 and dev4 disconnect dev3 and dev4 from dev1 and dev2

Tuesday, 26 March 2013

slide-146
SLIDE 146

dev1 dev3 dev2 dev4 riak test

reverse the cookie on dev3 and dev4 disconnect dev3 and dev4 from dev1 and dev2 leave riak_test node connected to all 4

Tuesday, 26 March 2013

slide-147
SLIDE 147

Intercepts

Tuesday, 26 March 2013

slide-148
SLIDE 148

riak_kv_vnode Some Code get/3 del/3 put/6 Some Code

Tuesday, 26 March 2013

slide-149
SLIDE 149

riak_kv_vnode_orig riak_kv_vnode Some Code get/3 del/3 put/6 get_orig/3 del_orig/3 put_orig/6 Some Code

Tuesday, 26 March 2013

slide-150
SLIDE 150

riak_kv_vnode_orig riak_kv_vnode_intercepts riak_kv_vnode Some Code get/3 del/3 put/6 get_orig/3 del_orig/3 put_orig/6 Some Code failed_get/3

Tuesday, 26 March 2013

slide-151
SLIDE 151

riak_kv_vnode_orig riak_kv_vnode_intercepts riak_kv_vnode Some Code get/3 del/3 put/6 get_orig/3 del_orig/3 put_orig/6 Some Code failed_get/3

rt_intercept:add(Node, {riak_kv_vnode, [{{get,3}, failed_get}])

Tuesday, 26 March 2013

slide-152
SLIDE 152

riak_kv_vnode_orig riak_kv_vnode_intercepts riak_kv_vnode Some Code get/3 del/3 put/6 get_orig/3 del_orig/3 put_orig/6 Some Code failed_get/3

rt_intercept:add(Node, {riak_kv_vnode, [{{get,3}, failed_get}])

Tuesday, 26 March 2013

slide-153
SLIDE 153

A new slide, this way comes ENOTENUFPIXELS

Tuesday, 26 March 2013

slide-154
SLIDE 154

My Melty-Faced Spock Moment

Tuesday, 26 March 2013

slide-155
SLIDE 155

10x, per node dev1 dev3 dev2 dev4 riak test loaded upgrade worker sup KV listkeys search MapReduce 2i

Spawns

Tuesday, 26 March 2013

slide-156
SLIDE 156

Tuesday, 26 March 2013

slide-157
SLIDE 157

Where do we go from here?

Tuesday, 26 March 2013

slide-158
SLIDE 158

Successes!

Tuesday, 26 March 2013

slide-159
SLIDE 159

Successes!

  • less people spend less hours on release

validation

Tuesday, 26 March 2013

slide-160
SLIDE 160

Successes!

  • less people spend less hours on release

validation

  • For Erlangers by Erlangers

Tuesday, 26 March 2013

slide-161
SLIDE 161

Successes!

  • less people spend less hours on release

validation

  • For Erlangers by Erlangers
  • Features can be developed with integration

testing

Tuesday, 26 March 2013

slide-162
SLIDE 162

Failure was an

Tuesday, 26 March 2013

slide-163
SLIDE 163

Failure was an

  • Release Validation took a long calendar time

Tuesday, 26 March 2013

slide-164
SLIDE 164

Failure was an

  • Release Validation took a long calendar time
  • A process problem Nightlies can’t solve

Tuesday, 26 March 2013

slide-165
SLIDE 165

Failure was an

  • Release Validation took a long calendar time
  • A process problem Nightlies can’t solve
  • Performance Testing still Manual

Tuesday, 26 March 2013

slide-166
SLIDE 166

Failure was an

  • Release Validation took a long calendar time
  • A process problem Nightlies can’t solve
  • Performance Testing still Manual
  • Test Environment can cause issues with

results, giving us a “giddyup who cried wolf”

Tuesday, 26 March 2013

slide-167
SLIDE 167

tl;dr best release yet; will only get better

Tuesday, 26 March 2013

slide-168
SLIDE 168

Thanks!

  • Joe Blomstedt
  • Sean Cribbs
  • Chris Meiklejohn
  • Jared Morrow
  • John Newman
  • Dave “Chest Burster”

Parfitt

  • Engel Sanchez
  • Ryan Zezeski

And many more!

Tuesday, 26 March 2013

slide-169
SLIDE 169

QUESTIONS?

Tuesday, 26 March 2013

slide-170
SLIDE 170

WHO’S TAKING QUESTIONS? WHO’S TAKING

Tuesday, 26 March 2013

slide-171
SLIDE 171

Tuesday, 26 March 2013

slide-172
SLIDE 172

J O E D E V I V O

Tuesday, 26 March 2013