Erlang's Open Telecom Platform (OTP) Framework Steve Vinoski - - PowerPoint PPT Presentation

Erlang's Open Telecom Platform (OTP) Framework

Steve Vinoski

Architect, Basho Technologies Cambridge, MA USA vinoski@ieee.org @stevevinoski http://steve.vinoski.net

1

1 Monday, June 18, 12

Erlang

2

2 Monday, June 18, 12

Functional Programming Everywhere

3

3 Monday, June 18, 12

Functional Programming Everywhere

4

4 Monday, June 18, 12

Functional Programming Everywhere

4

4 Monday, June 18, 12

Verivue MDX 9200

Functional Programming Everywhere

4

4 Monday, June 18, 12

Verivue MDX 9200

Functional Programming Everywhere

4

4 Monday, June 18, 12

Verivue MDX 9200

Functional Programming Everywhere

4

4 Monday, June 18, 12

Erlang/OTP

5

5 Monday, June 18, 12

O T P

6

6 Monday, June 18, 12

O T P elecom pen latform

6

6 Monday, June 18, 12

What is OTP?

• libraries of standard modules
• behaviors: implementations of

common patterns for concurrent fault-tolerant Erlang apps

7

7 Monday, June 18, 12

OTP Behavior Modules

• A behavior is sort of like an abstract

base class in OO terms, providing:

• a message handling loop
• integration with underlying OTP

system (for code upgrade, tracing, process management, etc.)

8

8 Monday, June 18, 12

OTP Behaviors

• application
• supervisor
• gen_server
• gen_fsm
• gen_event

More on these later...

9

9 Monday, June 18, 12

Ericsson AXD 301

10

10 Monday, June 18, 12

• Large number of concurrent activities
• Large software systems distributed

across multiple computers

• Continuous operation for years
• Live updates and maintenance
• Tolerance for both hardware and

software faults

Ericsson CSL Telecom Switch Requirements

11

11 Monday, June 18, 12

• Large number of concurrent activities
• Large software systems distributed

across multiple computers

• Continuous operation for years
• Live updates and maintenance
• Tolerance for both hardware and

software faults

Today’s Data/Web/ Cloud/Service Apps

12

12 Monday, June 18, 12

13

13 Monday, June 18, 12

Concurrency

13

13 Monday, June 18, 12

Erlang Processes

• Lightweight, much lighter than OS

threads

• Host hundreds of thousands or even

millions per Erlang VM instance

14

14 Monday, June 18, 12

Concurrency For Reliability

• Isolation: Erlang processes

communicate via message passing

• Distribution: Erlang process model

works across nodes

• Monitoring/supervision: allow an

Erlang process to take action when another fails

15

15 Monday, June 18, 12

Erlang Process Architecture

16

16 Monday, June 18, 12

CPU Core 1

. . . . . .

CPU Core N

Erlang Process Architecture

16

16 Monday, June 18, 12

OS + kernel threads

CPU Core 1

. . . . . .

CPU Core N

Erlang Process Architecture

16

16 Monday, June 18, 12

SMP Schedulers

Erlang VM

OS + kernel threads

CPU Core 1

. . . . . .

CPU Core N

Erlang Process Architecture

16

16 Monday, June 18, 12

SMP Schedulers

Erlang VM

Run Queues OS + kernel threads

CPU Core 1

. . . . . .

CPU Core N

Erlang Process Architecture

16

16 Monday, June 18, 12

SMP Schedulers

Erlang VM

Run Queues

Process Process Process Process Process Process

OS + kernel threads

CPU Core 1

. . . . . .

CPU Core N

Erlang Process Architecture

16

16 Monday, June 18, 12

• spawn: create a new Erlang process
• ! (exclamation point) or send: send a

message to another Erlang process

• receive: receive a message from another

Erlang process

• All Erlang code runs within some process,

always

• processes are not “extra” like threads in
• ther languages

Concurrency Primitives

17

17 Monday, June 18, 12

OTP Building Blocks

• process spawning
• sending and receiving messages
• process linking and monitoring

18

18 Monday, June 18, 12

• Pid ! Term

Pid = spawn(fun() -> ... end), Pid ! {hello, world}.

• Works across distributed nodes too

Sending a Message

19

19 Monday, June 18, 12

receive Pattern1 [when Guard1] -> Expressions; Pattern2 [when Guard2] -> Expressions; ... end

• Each process has a message queue
• receive blocks until a matching message

is received

• Also variant with a receive timeout

(receive ... after Milliseconds -> Expr(s) end)

Receiving a Message

20

20 Monday, June 18, 12

Selective Receive

• The receive call examines messages in the

message queue, even pre-existing ones

• If one matches, receive runs matching

expressions and returns

• If none matches, waits to get one that

does

• If timeout present and none arrives, run

after expressions

21

21 Monday, June 18, 12

ping(N, PongPid) -> PongPid ! ping, receive pong -> ping(N-1, PongPid) end; ping(0, _PongPid) -> ok.

Ping-Pong Example

22

22 Monday, June 18, 12

ping(N, PongPid) -> PongPid ! ping, receive pong -> ping(N-1, PongPid) end; ping(0, _PongPid) -> ok.

Ping-Pong Example

23

23 Monday, June 18, 12

ping(N, PongPid) -> PongPid ! ping, receive pong -> ping(N-1, PongPid) end; ping(0, _PongPid) -> ok.

Ping-Pong Example

24

24 Monday, June 18, 12

ping(N, PongPid) -> PongPid ! ping, receive pong -> ping(N-1, PongPid) end; ping(0, _PongPid) -> ok.

Ping-Pong Example

25

25 Monday, June 18, 12

ping(N, PongPid) -> PongPid ! ping, receive pong -> ping(N-1, PongPid) end; ping(0, _PongPid) -> ok.

Ping-Pong Example

26

26 Monday, June 18, 12

ping(N, PongPid) -> PongPid ! ping, receive pong -> ping(N-1, PongPid) end; ping(0, _PongPid) -> ok.

Ping-Pong Example

27

27 Monday, June 18, 12

pong(0, _PingPid) -> ok; pong(N, PingPid) -> receive ping -> PingPid ! pong, pong(N-1, PingPid) end.

Ping-Pong Example

28

28 Monday, June 18, 12

pong(0, _PingPid) -> ok; pong(N, PingPid) -> PingPid ! pong, pong(N-1, PingPid) end.

Ping-Pong Example

29

29 Monday, June 18, 12

pong(0, _PingPid) -> ok; pong(N, PingPid) -> receive ping -> pong(N-1, PingPid) end.

Ping-Pong Example

30

30 Monday, June 18, 12

pong(0, _PingPid) -> ok; pong(N, PingPid) -> receive ping -> PingPid ! pong, pong(N-1, PingPid) end.

Ping-Pong Example

31

31 Monday, June 18, 12

pong(0, _PingPid) -> ok; pong(N, PingPid) -> receive ping -> PingPid ! pong, pong(N-1, PingPid) end.

Ping-Pong Example

32

32 Monday, June 18, 12

pong(0, _PingPid) -> ok; pong(N, PingPid) -> receive ping -> PingPid ! pong, pong(N-1, PingPid) end.

Ping-Pong Example

33

33 Monday, June 18, 12

start(N) -> PingPid = self(), PongPid = spawn(fun() -> pong(N, PingPid) end), ping(N, PongPid).

Ping-Pong Example

34

34 Monday, June 18, 12

start(N) -> PingPid = self(), PongPid = spawn(fun() -> pong(N, PingPid) end), ping(N, PongPid).

Ping-Pong Example

35

35 Monday, June 18, 12

start(N) -> PingPid = self(), PongPid = spawn(fun() -> pong(N, PingPid) end), ping(N, PongPid).

Ping-Pong Example

36

36 Monday, June 18, 12

start(N) -> PingPid = self(), PongPid = spawn(fun() -> pong(N, PingPid) end), ping(N, PongPid).

Ping-Pong Example

37

37 Monday, June 18, 12

start(N) -> PingPid = self(), PongPid = spawn(fun() -> pong(N, PingPid) end), ping(N, PongPid).

Ping-Pong Example

38

38 Monday, June 18, 12

start(N) -> PingPid = self(), PongPid = spawn(fun() -> pong(N, PingPid) end), ping(N, PongPid).

Ping-Pong Example

39

39 Monday, June 18, 12

Message Handling

40

40 Monday, June 18, 12

PingPid PongPid

Message Handling

40

40 Monday, June 18, 12

PingPid PongPid message queue message queue

Message Handling

40

40 Monday, June 18, 12

PingPid PongPid ping messages message queue message queue

Message Handling

40

40 Monday, June 18, 12

PingPid PongPid ping messages pong messages message queue message queue

Message Handling

40

40 Monday, June 18, 12

PingPid PongPid ping messages pong messages message queue message queue

The receive statements pull messages from the queues

Message Handling

40

40 Monday, June 18, 12

• One process can monitor another by:
• Linking: bidirectional, each process

is tied to the other (erlang:link/1,

erlang:spawn_link/1,2,3,4)

• Monitoring: one process monitors

another (erlang:monitor/2)

• Allows processes to take action when

another one dies

Process Linking

41

41 Monday, June 18, 12

• Processes can be registered by

name either locally (same VM) or globally (across distributed node set)

• Registered names can be used

instead of pids as message targets

Process Registration

42

42 Monday, June 18, 12

OTP Building Blocks

• process spawning
• sending and receiving messages
• process linking and monitoring

43

43 Monday, June 18, 12

And one more:

Tail Recursion

44

44 Monday, June 18, 12

Maintaining State

loop(State) -> NewState = receive % handle messages here, % messages may affect State end, loop(NewState).

45

45 Monday, June 18, 12

Maintaining State

loop(State) -> receive % handle messages here, % messages may affect State end, loop(NewState).

46

46 Monday, June 18, 12

Maintaining State

loop(State) -> receive % handle messages here, % messages may affect State end, loop(NewState).

47

47 Monday, June 18, 12

But State is Immutable!

48

48 Monday, June 18, 12

Erlang Immutability

49

• Erlang assignment is pattern matching,

not mutation

• If a variable is unbound, it is bound to

the value of the right-hand side

• If bound, its value is matched against

the right-hand side

• if match, return value
• if no match, throw badmatch exception

49 Monday, June 18, 12

Maintaining State

loop(State) -> NewState = receive % handle messages here, % messages may affect State end, loop(NewState).

50

50 Monday, June 18, 12

Maintaining State

loop(State) -> NewState = receive % handle messages here, % messages may affect State end, loop(NewState).

51

51 Monday, June 18, 12

Maintaining State

loop(State) -> NewState = receive % handle messages here, % messages may affect State end, loop(NewState).

52

52 Monday, June 18, 12

Common vs. App- Specific

Common App-Specific Loop Management Message Reception and Dispatch Invoking Callbacks Handling Callbacks Message Handling State Management

53

53 Monday, June 18, 12

Callbacks with State

loop(Callbacks, State) -> NState = receive {message, Val} -> Callbacks:handle(Val, State) end, loop(Callbacks, NState).

54

54 Monday, June 18, 12

Callbacks with State

loop(Callbacks, State) -> NState = receive {message, Val} -> Callbacks:handle(Val, State) end, loop(Callbacks, NState).

55

55 Monday, June 18, 12

Callbacks with State

loop(Callbacks, State) -> NState = receive {message, Val} -> Callbacks:handle(Val, State) end, loop(Callbacks, NState).

56

56 Monday, June 18, 12

Multiple Messages

loop(Callbacks, State) -> NState = receive {message1, M1} -> Callbacks:handle1(M1,State); {message2, M2} -> Callbacks:handle2(M2,State); Other -> Callbacks:handle_other( Other,State) end, loop(Callbacks, NState).

57

57 Monday, June 18, 12

Controlling the Loop

loop(Callbacks, State) -> {LoopAction, NState} = receive {message1, M1} -> Callbacks:handle_m1(M1,State); {message2, M2} -> Callbacks:handle_m2(M2,State); Other -> Callbacks:handle_other(Other,State) end, case LoopAction of stop -> ok; _ -> loop(Callbacks, NState) end.

58

58 Monday, June 18, 12

Controlling the Loop

loop(Callbacks, State) -> {LoopAction, NState} = receive {message1, M1} -> Callbacks:handle_m1(M1,State); {message2, M2} -> Callbacks:handle_m2(M2,State); Other -> Callbacks:handle_other(Other,State) end, case LoopAction of stop -> ok; _ -> loop(Callbacks, NState) end.

59

59 Monday, June 18, 12

• These are the basics
• Actual OTP behavior loops are more

sophisticated, integrating with underlying OTP system

• gen_server, gen_fsm, supervisor, etc. all

assume specific callbacks, checked by the compiler

Behavior Loops

60

60 Monday, June 18, 12

gen_server

61

61 Monday, June 18, 12

gen_server

• Generic server behavior
• Supports server-like components,

not necessarily distributed

• “Business logic” lives in app-specific

callback module

62

62 Monday, June 18, 12

gen_server Process Startup

• start_link(Module, Args, Options) -> Result
• or

start_link(ServerName, Module, Args, Options) -> Result

• ServerName is typically {local, Name}

where Name becomes the registered process name

• Module is the callback module

63

63 Monday, June 18, 12

gen_server Process Startup

• start_link calls Module:init/1 callback
• init typically returns {ok, State} where

State is the new app-specific state of the gen_server process

• State is usually an Erlang record (a

collection of fields, can get and set by name)

64

64 Monday, June 18, 12

gen_server Call

• Allows caller to make a synchronous

call to the server

• call(ServerRef, Request) -> Reply

call(ServerRef, Request, Timeout) -> Reply

• ServerRef is the name or pid of the

gen_server process

• Request is any term, typically an atom
• r tuple

65

65 Monday, June 18, 12

gen_server Call

• Call invokes Module:handle_call/3 callback
• Arguments are Request, From, State
• Request is passed thru from

gen_server:call

• From identifies the calling client

process

• State is the gen_server app state

66

66 Monday, June 18, 12

gen_server Call

• handle_call returns reply, noreply, or stop
• {reply, Reply, NewState} or
• {noreply, NewState} or
• {stop, Reason, Reply, NewState} or

{stop, Reason, NewState}

• Replies go back to calling process
• If no reply, it must be supplied later via

gen_server:reply/2

67

67 Monday, June 18, 12

gen_server Cast

• Allows caller to make a fire-and-

forget call to the server (no reply)

• cast(ServerRef, Request) -> ok
• ServerRef is the name or pid of the

gen_server process

• Request is any term, typically an

atom or tuple

68

68 Monday, June 18, 12

gen_server Cast

• Call invokes Module:handle_cast/2

callback

• Arguments are Request, State
• Request is passed thru from

gen_server:cast

• State is the gen_server state

69

69 Monday, June 18, 12

gen_server Cast

• handle_cast returns noreply or stop
• {noreply, NewState} or
• {stop, Reason, NewState}
• Either way, no reply to calling

process

70

70 Monday, June 18, 12

handle_info Callback

• The Module:handle_info/2 callback is

called when a message arrives for the gen_server process

• {noreply, NewState} or
• {stop, Reason, NewState}

71

71 Monday, June 18, 12

terminate Callback

• Opposite of init/1, called when

gen_server process is exiting

• Allows application to clean up

72

72 Monday, June 18, 12

Module:init gen_server:start_link

gen_server process State Loop

gen_server:call Module:handle_call gen_server:cast Module:handle_cast

message

Module:handle_info

73

73 Monday, June 18, 12

gen_server Interface

• Provide a set of app-specific

functions in your callback module as your interface

• Hide the fact that your module is a

gen_server

• Allows you to change the module

later without breaking clients

74

74 Monday, June 18, 12

gen_server Example

• module(key_value).
• behavior(gen_server).
• export([init/1, handle_call/3, handle_cast/2,

handle_info/2, terminate/2, code_change/3]).

• export([start_link/0, stop/0,

store/2, lookup/1, delete/1]).

75

75 Monday, June 18, 12

key_value Start & Stop

start_link() -> gen_server:start_link({local, ?MODULE}, ?MODULE, [], []). stop() -> gen_server:cast(?MODULE, stop).

76

76 Monday, June 18, 12

key_value Start & Stop

start_link() -> gen_server:start_link({local, ?MODULE}, ?MODULE, [], []). stop() -> gen_server:cast(?MODULE, stop).

76

76 Monday, June 18, 12

key_value Start & Stop

start_link() -> gen_server:start_link({local, ?MODULE}, ?MODULE, [], []). stop() -> gen_server:cast(?MODULE, stop).

76

76 Monday, June 18, 12

key_value Start & Stop

start_link() -> gen_server:start_link({local, ?MODULE}, ?MODULE, [], []). stop() -> gen_server:cast(?MODULE, stop).

76

76 Monday, June 18, 12

key_value Start & Stop

start_link() -> gen_server:start_link({local, ?MODULE}, ?MODULE, [], []). stop() -> gen_server:cast(?MODULE, stop).

76

76 Monday, June 18, 12

key_value Start & Stop

start_link() -> gen_server:start_link({local, ?MODULE}, ?MODULE, [], []). stop() -> gen_server:cast(?MODULE, stop).

76

76 Monday, June 18, 12

key_value Start & Stop

start_link() -> gen_server:start_link({local, ?MODULE}, ?MODULE, [], []). stop() -> gen_server:cast(?MODULE, stop).

76

76 Monday, June 18, 12

key_value API

store(Key, Value) -> gen_server:call(?MODULE, {store, Key, Value}). lookup(Key) -> gen_server:call(?MODULE, {lookup, Key}). delete(Key) -> gen_server:cast(?MODULE, {delete, Key}).

77

77 Monday, June 18, 12

key_value API

store(Key, Value) -> gen_server:call(?MODULE, {store, Key, Value}). lookup(Key) -> gen_server:call(?MODULE, {lookup, Key}). delete(Key) -> gen_server:cast(?MODULE, {delete, Key}).

77

77 Monday, June 18, 12

key_value API

store(Key, Value) -> gen_server:call(?MODULE, {store, Key, Value}). lookup(Key) -> gen_server:call(?MODULE, {lookup, Key}). delete(Key) -> gen_server:cast(?MODULE, {delete, Key}).

77

77 Monday, June 18, 12

key_value API

store(Key, Value) -> gen_server:call(?MODULE, {store, Key, Value}). lookup(Key) -> gen_server:call(?MODULE, {lookup, Key}). delete(Key) -> gen_server:cast(?MODULE, {delete, Key}).

77

77 Monday, June 18, 12

key_value API

store(Key, Value) -> gen_server:call(?MODULE, {store, Key, Value}). lookup(Key) -> gen_server:call(?MODULE, {lookup, Key}). delete(Key) -> gen_server:cast(?MODULE, {delete, Key}).

77

77 Monday, June 18, 12

key_value init

• record(state, {table}).

init([]) -> process_flag(trap_exit, true), Tbl = ets:new(?MODULE, []), {ok, #state{table = Tbl}}.

78

78 Monday, June 18, 12

key_value init

• record(state, {table}).

init([]) -> process_flag(trap_exit, true), Tbl = ets:new(?MODULE, []), {ok, #state{table = Tbl}}.

78

78 Monday, June 18, 12

key_value init

• record(state, {table}).

init([]) -> process_flag(trap_exit, true), Tbl = ets:new(?MODULE, []), {ok, #state{table = Tbl}}.

78

78 Monday, June 18, 12

key_value handle_call

handle_call({store,Key,Value},_From,#state{table=Tbl}=State) -> true = ets:insert(Tbl, {Key, Value}), {reply, ok, State};

79

79 Monday, June 18, 12

key_value handle_call

handle_call({store,Key,Value},_From,#state{table=Tbl}=State) -> true = ets:insert(Tbl, {Key, Value}), {reply, ok, State};

79

79 Monday, June 18, 12

key_value handle_call

handle_call({store,Key,Value},_From,#state{table=Tbl}=State) -> true = ets:insert(Tbl, {Key, Value}), {reply, ok, State};

79

79 Monday, June 18, 12

key_value handle_call

handle_call({store,Key,Value},_From,#state{table=Tbl}=State) -> true = ets:insert(Tbl, {Key, Value}), {reply, ok, State};

79

79 Monday, June 18, 12

key_value handle_call

handle_call({lookup, Key}, _From, #state{table = Tbl} = State) -> Reply = case ets:lookup(Tbl, Key) of [{Key, Value}] -> {ok, Value}; [] -> not_found end, {reply, Reply, State}; handle_call(_, _From, State) -> {reply, ok, State}.

80

80 Monday, June 18, 12

key_value handle_call

handle_call({lookup, Key}, _From, #state{table = Tbl} = State) -> Reply = case ets:lookup(Tbl, Key) of [{Key, Value}] -> {ok, Value}; [] -> not_found end, {reply, Reply, State}; handle_call(_, _From, State) -> {reply, ok, State}.

80

80 Monday, June 18, 12

key_value handle_call

handle_call({lookup, Key}, _From, #state{table = Tbl} = State) -> Reply = case ets:lookup(Tbl, Key) of [{Key, Value}] -> {ok, Value}; [] -> not_found end, {reply, Reply, State}; handle_call(_, _From, State) -> {reply, ok, State}.

80

80 Monday, June 18, 12

key_value handle_call

handle_call({lookup, Key}, _From, #state{table = Tbl} = State) -> Reply = case ets:lookup(Tbl, Key) of [{Key, Value}] -> {ok, Value}; [] -> not_found end, {reply, Reply, State}; handle_call(_, _From, State) -> {reply, ok, State}.

80

80 Monday, June 18, 12

key_value handle_call

handle_call({lookup, Key}, _From, #state{table = Tbl} = State) -> Reply = case ets:lookup(Tbl, Key) of [{Key, Value}] -> {ok, Value}; [] -> not_found end, {reply, Reply, State}; handle_call(_, _From, State) -> {reply, ok, State}.

80

80 Monday, June 18, 12

key_value handle_cast

handle_cast({delete, Key}, #state{table=Tbl}=State) -> ets:delete(Tbl, Key), {noreply, State}; handle_cast(stop, State) -> {stop, normal, State}; handle_cast(_, State) -> {noreply, State}.

81

81 Monday, June 18, 12

key_value handle_cast

handle_cast({delete, Key}, #state{table=Tbl}=State) -> ets:delete(Tbl, Key), {noreply, State}; handle_cast(stop, State) -> {stop, normal, State}; handle_cast(_, State) -> {noreply, State}.

81

81 Monday, June 18, 12

key_value handle_cast

handle_cast({delete, Key}, #state{table=Tbl}=State) -> ets:delete(Tbl, Key), {noreply, State}; handle_cast(stop, State) -> {stop, normal, State}; handle_cast(_, State) -> {noreply, State}.

81

81 Monday, June 18, 12

key_value handle_cast

handle_cast({delete, Key}, #state{table=Tbl}=State) -> ets:delete(Tbl, Key), {noreply, State}; handle_cast(stop, State) -> {stop, normal, State}; handle_cast(_, State) -> {noreply, State}.

81

81 Monday, June 18, 12

key_value handle_info

handle_info(_, State) -> {noreply, State}.

82

82 Monday, June 18, 12

key_value terminate

terminate(_Reason, #state{table = Tbl}) -> ets:delete(Tbl),

• k.

83

83 Monday, June 18, 12

key_value Usage

20> key_value:start_link(). {ok,<0.33.0>} 21> key_value:store("foo", "bar").

• k

22> key_value:lookup("foo"). {ok, "bar"} 23> key_value:lookup("baz"). not_found 24> key_value:delete("foo").

• k

25> key_value:lookup("foo"). not_found

84

84 Monday, June 18, 12

key_value Usage

20> key_value:start_link(). {ok,<0.33.0>} 21> key_value:store("foo", "bar").

• k

22> key_value:lookup("foo"). {ok, "bar"} 23> key_value:lookup("baz"). not_found 24> key_value:delete("foo").

• k

25> key_value:lookup("foo"). not_found

84

84 Monday, June 18, 12

key_value Usage

20> key_value:start_link(). {ok,<0.33.0>} 21> key_value:store("foo", "bar").

• k

22> key_value:lookup("foo"). {ok, "bar"} 23> key_value:lookup("baz"). not_found 24> key_value:delete("foo").

• k

25> key_value:lookup("foo"). not_found

84

84 Monday, June 18, 12

key_value Usage

20> key_value:start_link(). {ok,<0.33.0>} 21> key_value:store("foo", "bar").

• k

22> key_value:lookup("foo"). {ok, "bar"} 23> key_value:lookup("baz"). not_found 24> key_value:delete("foo").

• k

25> key_value:lookup("foo"). not_found

84

84 Monday, June 18, 12

key_value Usage

20> key_value:start_link(). {ok,<0.33.0>} 21> key_value:store("foo", "bar").

• k

22> key_value:lookup("foo"). {ok, "bar"} 23> key_value:lookup("baz"). not_found 24> key_value:delete("foo").

• k

25> key_value:lookup("foo"). not_found

84

84 Monday, June 18, 12

key_value Usage

20> key_value:start_link(). {ok,<0.33.0>} 21> key_value:store("foo", "bar").

• k

22> key_value:lookup("foo"). {ok, "bar"} 23> key_value:lookup("baz"). not_found 24> key_value:delete("foo").

• k

25> key_value:lookup("foo"). not_found

84

84 Monday, June 18, 12

key_value Usage

20> key_value:start_link(). {ok,<0.33.0>} 21> key_value:store("foo", "bar").

• k

22> key_value:lookup("foo"). {ok, "bar"} 23> key_value:lookup("baz"). not_found 24> key_value:delete("foo").

• k

25> key_value:lookup("foo"). not_found

84

84 Monday, June 18, 12

key_value Stopping

26> key_value:stop().

• k

27> key_value:lookup("foo"). ** exception exit: {noproc, {gen_server,call, [key_value,{lookup,"foo"}]}} in function gen_server:call/2 (gen_server.erl, line 180)

85

85 Monday, June 18, 12

gen_fsm

86

86 Monday, June 18, 12

gen_fsm

• Behavior supporting finite state

machines

• Not used much but helpful for e.g.

protocols

87

87 Monday, June 18, 12

• Current state S1
• Event arrives
• Perform actions based on event and

current state

• Move to new state S2

88

88 Monday, June 18, 12

gen_fsm Process Start

• start_link: start a gen_fsm process
• calls Module:init, same as with

gen_server

89

89 Monday, June 18, 12

Async Events

• send_event: asynchronously send event into

gen_fsm process

• calls Module:StateName/2,where StateName is a

function named for the current state

• send_all_state_event: asynchronously send event

into gen_fsm process

• calls Module:handle_event,allows handling event

regardless of current state

• current state name is passed into handle_event

90

90 Monday, June 18, 12

Sync Events

• sync_send_event: synchronously send event into

gen_fsm process

• calls Module:StateName/3,where StateName is a

function named for the current state

• sync_send_all_state_event: synchronously send event

into gen_fsm process

• calls Module:handle_sync_event,allows handling event

regardless of current state

• current state name is passed into handle_sync_event

91

91 Monday, June 18, 12

• You can also write an FSM with gen_server
• Keep current state as part of loop state
• But it’s much harder to write and debug

92

92 Monday, June 18, 12

application & supervisor

93

93 Monday, June 18, 12

application Behavior

• application provides an entry point for

an OTP-compliant app

• Allows multiple Erlang components

to be combined into a system

• Erlang apps can declare their

dependencies on other apps

94

94 Monday, June 18, 12

App Startup Sequence

• Hierarchical sequence
• Erlang system application controller

starts the app

• App starts supervisor(s)
• Each supervisor starts workers
• Workers are typically instances of

OTP behaviors

95

95 Monday, June 18, 12

• module(bitcask_app).
• behaviour(application).

%% Application callbacks

• export([start/2, stop/1]).

start(_StartType, _StartArgs) -> bitcask_sup:start_link(). stop(_State) ->

• k.

Example application

96

96 Monday, June 18, 12

• module(bitcask_app).
• behaviour(application).

%% Application callbacks

• export([start/2, stop/1]).

start(_StartType, _StartArgs) -> bitcask_sup:start_link(). stop(_State) ->

• k.

Example application

97

97 Monday, June 18, 12

• module(bitcask_app).
• behaviour(application).

%% Application callbacks

• export([start/2, stop/1]).

start(_StartType, _StartArgs) -> bitcask_sup:start_link(). stop(_State) ->

• k.

Example application

98

98 Monday, June 18, 12

• module(bitcask_app).
• behaviour(application).

%% Application callbacks

• export([start/2, stop/1]).

start(_StartType, _StartArgs) -> bitcask_sup:start_link(). stop(_State) ->

• k.

Example application

99

99 Monday, June 18, 12

• module(bitcask_app).
• behaviour(application).

%% Application callbacks

• export([start/2, stop/1]).

start(_StartType, _StartArgs) -> bitcask_sup:start_link(). stop(_State) ->

• k.

Example application

100

100 Monday, June 18, 12

Workers & Supervisors

• Workers implement application logic
• Supervisors:
• start child workers and sub-

supervisors

• link to the children and trap exits
• take action when a child dies, typically

restarting one or more children

101

101 Monday, June 18, 12

Let It Crash

• In his doctoral thesis, Joe Armstrong,

creator of Erlang, wrote:

• Let some other process do the error recovery.
• If you can’t do what you want to do, die.
• Let it crash.
• Do not program defensively.

102

102 Monday, June 18, 12

Application, Supervisors, Workers

Simple Core

103

103 Monday, June 18, 12

Application, Supervisors, Workers

Application Simple Core

103

103 Monday, June 18, 12

Application, Supervisors, Workers

Application Supervisors Simple Core

103

103 Monday, June 18, 12

Application, Supervisors, Workers

Application Workers Supervisors Simple Core

103

103 Monday, June 18, 12

supervisor Behavior

• module(bitcask_sup).
• behaviour(supervisor).

%% API

• export([start_link/0]).

%% Supervisor callbacks

• export([init/1]).

start_link() -> supervisor:start_link({local, ?MODULE}, ?MODULE, []).

104

104 Monday, June 18, 12

supervisor init

init([]) -> {ok, {{RestartStrategy, MaxRestarts, MaxTime}, [ChildSpecifications]}}.

105

105 Monday, June 18, 12

supervisor init

init([]) -> {ok, {{one_for_one, 5, 10}, [{bitcask_merge_worker, {bitcask_merge_worker, start_link, []}, permanent, 5000, worker, [bitcask_merge_worker]}]}}.

106

106 Monday, June 18, 12

supervisor init

init([]) -> {ok, {{one_for_one, 5, 10}, [{bitcask_merge_worker, {bitcask_merge_worker, start_link, []}, permanent, 5000, worker, [bitcask_merge_worker]}]}}.

Restart Strategy

106

106 Monday, June 18, 12

supervisor init

init([]) -> {ok, {{one_for_one, 5, 10}, [{bitcask_merge_worker, {bitcask_merge_worker, start_link, []}, permanent, 5000, worker, [bitcask_merge_worker]}]}}.

Max Restart Frequency

106

106 Monday, June 18, 12

supervisor init

init([]) -> {ok, {{one_for_one, 5, 10}, [{bitcask_merge_worker, {bitcask_merge_worker, start_link, []}, permanent, 5000, worker, [bitcask_merge_worker]}]}}.

Child Spec

106

106 Monday, June 18, 12

Restart Strategies

• one_for_one: child dies, restart that

child

• one_for_all: any child dies, terminate

and then restart all children

• rest_for_one: a child dies, terminate

all children started after it, then restart the failed one and the rest

107

107 Monday, June 18, 12

Restart Frequency

• More than MaxRestarts in MaxTime

seconds terminates all children and the supervisor

• Limits the number of times a child

can die and restart

init([]) -> {ok, {{RestartStrategy, MaxRestarts, MaxTime}, [ChildSpecifications]}}.

108

108 Monday, June 18, 12

Child Specs

[{bitcask_merge_worker, {bitcask_merge_worker, start_link, []}, permanent, 5000, worker, [bitcask_merge_worker]}]

109

109 Monday, June 18, 12

Child Specs

[{bitcask_merge_worker, {bitcask_merge_worker, start_link, []}, permanent, 5000, worker, [bitcask_merge_worker]}] child identifer

109

109 Monday, June 18, 12

Child Specs

[{bitcask_merge_worker, {bitcask_merge_worker, start_link, []}, permanent, 5000, worker, [bitcask_merge_worker]}] start function

109

109 Monday, June 18, 12

Child Specs

[{bitcask_merge_worker, {bitcask_merge_worker, start_link, []}, permanent, 5000, worker, [bitcask_merge_worker]}] restart type

109

109 Monday, June 18, 12

Child Specs

[{bitcask_merge_worker, {bitcask_merge_worker, start_link, []}, permanent, 5000, worker, [bitcask_merge_worker]}] how to terminate

109

109 Monday, June 18, 12

Child Specs

[{bitcask_merge_worker, {bitcask_merge_worker, start_link, []}, permanent, 5000, worker, [bitcask_merge_worker]}] process type

109

109 Monday, June 18, 12

Child Specs

[{bitcask_merge_worker, {bitcask_merge_worker, start_link, []}, permanent, 5000, worker, [bitcask_merge_worker]}] module list

109

109 Monday, June 18, 12

Child Specs

[{bitcask_merge_worker, {bitcask_merge_worker, start_link, []}, permanent, 5000, worker, [bitcask_merge_worker]}]

109

109 Monday, June 18, 12

Also: http://learnyousomeerlang.com/

For More Info

110

110 Monday, June 18, 12

OTP System Facilities

111

• Status
• Process info
• Tracing
• These work with OTP-compliant

behaviors, very useful for debug

• Releases and upgrades (not covered

here)

111 Monday, June 18, 12

31> {ok, Pid} = key_value:start_link(). {ok,<0.71.0>} 32> sys:get_status(key_value). {status,<0.71.0>, {module,gen_server}, [[{'$ancestors',[<0.67.0>]}, {'$initial_call',{key_value,init,1}}], running,<0.67.0>,[], [{header,"Status for generic server key_value"}, {data,[{"Status",running}, {"Parent",<0.67.0>}, {"Logged events",[]}]}, {data,[{"State",{state,24592}}]}]]}

112

sys:get_status

112 Monday, June 18, 12

Process Info

33> process_info(Pid). [{registered_name,key_value}, {current_function,{gen_server,loop,6}}, {initial_call,{proc_lib,init_p,5}}, {status,waiting}, {message_queue_len,0}, {messages,[]}, {links,[<0.67.0>]}, ... (more here, truncated)

113

113 Monday, June 18, 12

sys:trace

38> sys:trace(key_value, true).

• k

39> key_value:lookup("foo"). *DBG* key_value got call {lookup,"foo"} from <0.67.0> *DBG* key_value sent {ok,"bar"} to <0.67.0>, new state {state,24592} {ok,"bar"}

114

114 Monday, June 18, 12

Trace Events

• VM has trace events built into it, no

need for special build

• Can trace function calls, arguments,

results, the caller, time of call

• Overhead low enough for production

use

• but care is needed, you can kill a VM

if not careful

115

115 Monday, June 18, 12

dbg Tracing

40> dbg:tracer(). {ok,<0.125.0>} 41> dbg:p(all, call). {ok,[{matched,nonode@nohost,27}]} 42> dbg:p(key_value, [c,m]). {ok,[{matched,nonode@nohost,1}]} 43> dbg:tpl(key_value, c). {ok,[{matched,nonode@nohost,13},{saved,c}]}

116

116 Monday, June 18, 12

dbg Tracing

40> dbg:tracer(). {ok,<0.125.0>} 41> dbg:p(all, call). {ok,[{matched,nonode@nohost,27}]} 42> dbg:p(key_value, [c,m]). {ok,[{matched,nonode@nohost,1}]} 43> dbg:tpl(key_value, c). {ok,[{matched,nonode@nohost,13},{saved,c}]}

116

116 Monday, June 18, 12

dbg Tracing

40> dbg:tracer(). {ok,<0.125.0>} 41> dbg:p(all, call). {ok,[{matched,nonode@nohost,27}]} 42> dbg:p(key_value, [c,m]). {ok,[{matched,nonode@nohost,1}]} 43> dbg:tpl(key_value, c). {ok,[{matched,nonode@nohost,13},{saved,c}]}

116

116 Monday, June 18, 12

dbg Tracing

40> dbg:tracer(). {ok,<0.125.0>} 41> dbg:p(all, call). {ok,[{matched,nonode@nohost,27}]} 42> dbg:p(key_value, [c,m]). {ok,[{matched,nonode@nohost,1}]} 43> dbg:tpl(key_value, c). {ok,[{matched,nonode@nohost,13},{saved,c}]}

116

116 Monday, June 18, 12

dbg Tracing

40> dbg:tracer(). {ok,<0.125.0>} 41> dbg:p(all, call). {ok,[{matched,nonode@nohost,27}]} 42> dbg:p(key_value, [c,m]). {ok,[{matched,nonode@nohost,1}]} 43> dbg:tpl(key_value, c). {ok,[{matched,nonode@nohost,13},{saved,c}]}

116

116 Monday, June 18, 12

dbg Tracing

44> key_value:store("foo", "bar").

• k

(<0.67.0>) call key_value:store("foo","bar") ({erl_eval,do_apply,6}) (<0.71.0>) << {'$gen_call',{<0.67.0>,#Ref<0.0.0.490>}, {store,"foo","bar"}} (<0.71.0>) call key_value:handle_call({store,"foo","bar"}, {<0.67.0>,#Ref<0.0.0.490>},{state,36880}) ({gen_server,handle_msg, 5}) (<0.71.0>) <0.67.0> ! {#Ref<0.0.0.490>,ok}

117

117 Monday, June 18, 12

dbg Tracing

44> key_value:store("foo", "bar").

• k

(<0.67.0>) call key_value:store("foo","bar") ({erl_eval,do_apply,6}) (<0.71.0>) << {'$gen_call',{<0.67.0>,#Ref<0.0.0.490>}, {store,"foo","bar"}} (<0.71.0>) call key_value:handle_call({store,"foo","bar"}, {<0.67.0>,#Ref<0.0.0.490>},{state,36880}) ({gen_server,handle_msg, 5}) (<0.71.0>) <0.67.0> ! {#Ref<0.0.0.490>,ok}

117

117 Monday, June 18, 12

dbg Tracing

44> key_value:store("foo", "bar").

• k

(<0.67.0>) call key_value:store("foo","bar") ({erl_eval,do_apply,6}) (<0.71.0>) << {'$gen_call',{<0.67.0>,#Ref<0.0.0.490>}, {store,"foo","bar"}} (<0.71.0>) call key_value:handle_call({store,"foo","bar"}, {<0.67.0>,#Ref<0.0.0.490>},{state,36880}) ({gen_server,handle_msg, 5}) (<0.71.0>) <0.67.0> ! {#Ref<0.0.0.490>,ok}

117

117 Monday, June 18, 12

dbg Tracing

44> key_value:store("foo", "bar").

• k

(<0.67.0>) call key_value:store("foo","bar") ({erl_eval,do_apply,6}) (<0.71.0>) << {'$gen_call',{<0.67.0>,#Ref<0.0.0.490>}, {store,"foo","bar"}} (<0.71.0>) call key_value:handle_call({store,"foo","bar"}, {<0.67.0>,#Ref<0.0.0.490>},{state,36880}) ({gen_server,handle_msg, 5}) (<0.71.0>) <0.67.0> ! {#Ref<0.0.0.490>,ok}

117

117 Monday, June 18, 12

dbg Tracing

44> key_value:store("foo", "bar").

• k

(<0.67.0>) call key_value:store("foo","bar") ({erl_eval,do_apply,6}) (<0.71.0>) << {'$gen_call',{<0.67.0>,#Ref<0.0.0.490>}, {store,"foo","bar"}} (<0.71.0>) call key_value:handle_call({store,"foo","bar"}, {<0.67.0>,#Ref<0.0.0.490>},{state,36880}) ({gen_server,handle_msg, 5}) (<0.71.0>) <0.67.0> ! {#Ref<0.0.0.490>,ok}

117

117 Monday, June 18, 12

dbg Tracing

44> key_value:store("foo", "bar").

• k

(<0.67.0>) call key_value:store("foo","bar") ({erl_eval,do_apply,6}) (<0.71.0>) << {'$gen_call',{<0.67.0>,#Ref<0.0.0.490>}, {store,"foo","bar"}} (<0.71.0>) call key_value:handle_call({store,"foo","bar"}, {<0.67.0>,#Ref<0.0.0.490>},{state,36880}) ({gen_server,handle_msg, 5}) (<0.71.0>) <0.67.0> ! {#Ref<0.0.0.490>,ok}

117

117 Monday, June 18, 12

Tracing

• Call tracing can also match function

args

• I use tracing instead of a debugger
• Erlang’s trace facilities are very rich and

extensive, we’ve barely scratched the surface

• See also redbug, easier and safer than dbg

(https://github.com/massemanet/eper)

118

118 Monday, June 18, 12

Summary: Erlang/OTP

119

• Flexible, rich facilities for reliable

concurrent distributed applications

• Behaviors implement common patterns,

take care of nitty-gritty details, let you focus on your application

• Features honed and proven across a

number of projects with years in production

119 Monday, June 18, 12

Also: http://learnyousomeerlang.com/

Read These

120

120 Monday, June 18, 12

121

• Erlang website: http://erlang.org
• Erlang mailing list:

http://erlang.org/mailman/listinfo/ erlang-questions

See Also

121 Monday, June 18, 12

Thanks

122

122 Monday, June 18, 12