Network Serialization and Routing in World of Warcraft Joe Rumsey - - PowerPoint PPT Presentation

▶
network serialization and routing in world of warcraft
SMART_READER_LITE
LIVE PREVIEW

Network Serialization and Routing in World of Warcraft Joe Rumsey - - PowerPoint PPT Presentation

May 17, 2023 459 likes •1.78k views

Network Serialization and Routing in World of Warcraft Joe Rumsey jrumsey@blizzard.com Twitter: @joerumz What is JAM? J oes A utomated M essages The Problem Game servers need to communicate with each other Manual serialization is

slide-1
SLIDE 1

Network Serialization and Routing in World of Warcraft

Joe Rumsey jrumsey@blizzard.com Twitter: @joerumz

slide-2
SLIDE 2

What is JAM?

Joe’s Automated Messages

slide-3
SLIDE 3

The Problem

Game servers need to communicate with each other

slide-4
SLIDE 4

void Serialize(stream &msg) { vector<int> values; // ...Fill in some values... msg << values.size(); for(int i = values.size(); --i;) { msg << values[i]; } }

void Deserialize(stream &msg) { vector<int> values; int size; msg >> size; values.resize(size); for(int i = size; i--;) { msg >> values[i]; } }

Manual serialization is error-prone

slide-5
SLIDE 5

void Deserialize(stream &msg) { vector<int> values; int size; msg >> size; values.resize(size); for(int i = size; i--;) { msg >> values[i]; } }

Manual serialization is error-prone

void Serialize(stream &msg) { vector<int> values; // ...Fill in some values... msg << values.size(); for(int i = values.size(); --i;) { msg << values[i]; } }

slide-6
SLIDE 6

Manual serialization doesn’t scale

slide-7
SLIDE 7

Manual serialization doesn’t scale

World Of Checkers

slide-8
SLIDE 8

Manual serialization doesn’t scale

World Of Checkers

slide-9
SLIDE 9

Manual serialization doesn’t scale

  • s

e r v e r b

  • u

n d a r y

  • World Of Checkers
slide-10
SLIDE 10

Manual serialization doesn’t scale

World Of Checkers

slide-11
SLIDE 11

Manual serialization doesn’t scale

World Of Checkers

slide-12
SLIDE 12

Goals

slide-13
SLIDE 13

Goals

  • DRY - Don’t Repeat

Yourself

slide-14
SLIDE 14

Goals

  • DRY - Don’t Repeat

Yourself

  • Eliminate boilerplate to reduce bugs
slide-15
SLIDE 15

Goals

  • DRY - Don’t Repeat

Yourself

  • Eliminate boilerplate to reduce bugs
  • No more hand-coded serialize/deserialize
slide-16
SLIDE 16

Goals

  • DRY - Don’t Repeat

Yourself

  • Eliminate boilerplate to reduce bugs
  • No more hand-coded serialize/deserialize
  • Spend more time on the game, not the protocol
slide-17
SLIDE 17

Goals

  • DRY - Don’t Repeat

Yourself

  • Eliminate boilerplate to reduce bugs
  • No more hand-coded serialize/deserialize
  • Spend more time on the game, not the protocol
  • Build a helpful robot that writes our code for us
slide-18
SLIDE 18

Goal: Human readable code

struct CheckerCaptured { CheckerID id; CheckerID capturedBy; u8 jumpType; }; void Capture(CheckerID id, CheckerID by, JUMP_TYPE jumpType) { CheckerCaptured msg; msg.id = id; msg.capturedBy = by; msg.jumpType = jumpType; Send(&msg); }

slide-19
SLIDE 19

Goal: Human readable code

struct CheckerCaptured { CheckerID id; CheckerID capturedBy; u8 jumpType; }; void Capture(CheckerID id, CheckerID by, JUMP_TYPE jumpType) { CheckerCaptured msg; msg.id = id; msg.capturedBy = by; msg.jumpType = jumpType; Send(&msg); }

slide-20
SLIDE 20

Goal: Human readable code

struct CheckerCaptured { CheckerID id; CheckerID capturedBy; u8 jumpType; }; void Capture(CheckerID id, CheckerID by, JUMP_TYPE jumpType) { CheckerCaptured msg; msg.id = id; msg.capturedBy = by; msg.jumpType = jumpType; Send(&msg); }

slide-21
SLIDE 21

Goal: Human readable code

struct CheckerCaptured { CheckerID id; CheckerID capturedBy; u8 jumpType; }; void Capture(CheckerID id, CheckerID by, JUMP_TYPE jumpType) { CheckerCaptured msg; msg.id = id; msg.capturedBy = by; msg.jumpType = jumpType; Send(&msg); }

slide-22
SLIDE 22

Goal: Human readable code

struct CheckerCaptured { CheckerID id; CheckerID capturedBy; u8 jumpType; }; void Capture(CheckerID id, CheckerID by, JUMP_TYPE jumpType) { CheckerCaptured msg; msg.id = id; msg.capturedBy = by; msg.jumpType = jumpType; Send(&msg); }

slide-23
SLIDE 23

Goal: Human readable code

struct CheckerCaptured { CheckerID id; CheckerID capturedBy; u8 jumpType; }; void Capture(CheckerID id, CheckerID by, JUMP_TYPE jumpType) { CheckerCaptured msg; msg.id = id; msg.capturedBy = by; msg.jumpType = jumpType; Send(&msg); }

slide-24
SLIDE 24

Implementation Details

slide-25
SLIDE 25

Development Cycle

  • Describe the protocol
  • Generate serialization and dispatch
  • Send messages
  • Receive messages
  • Configure routing info
slide-26
SLIDE 26

1-to-1 mapping of .jam messages to C++ classes

// From Checkers.jam message CheckerCaptureCredit { CheckerID capturedCheckerID; CheckerID capturedBy; u8 jumpType; };

slide-27
SLIDE 27

1-to-1 mapping of .jam messages to C++ classes

// From Checkers.jam message CheckerCaptureCredit { CheckerID capturedCheckerID; CheckerID capturedBy; u8 jumpType; };

// 100% Generated code in JamCheckers.cpp class CheckerCaptureCredit : public JamMessage { public: // Message decoders BOOL Get(BinaryDecoder &decoder); BOOL Get(JSONDecoder &decoder); // Message encoders BOOL Put(BinaryEncoder &encoder) const; BOOL Put(JSONEncoder &encoder) const; /**** DATA START ****/ CheckerID capturedCheckerID; CheckerID capturedBy; u8 jumpType; /**** DATA STOP ****/ // Lots more stuff... };

slide-28
SLIDE 28

Development Cycle

  • Describe the protocol
  • Generate serialization and dispatch
  • Send messages
  • Receive messages
  • Configure routing info
slide-29
SLIDE 29

Auto-generated serialization code

//NOTICE: This is generated code. DO NOT EDIT! BOOL CheckerCaptureCredit::Put(BinaryEncoder &_encoder) const { _encoder.BeginMessage(CODE, NAME); _encoder.Put("capturedCheckerID", capturedCheckerID); _encoder.Put("capturedBy", capturedBy); _encoder.Put("jumpType", jumpType); _encoder.EndMessage(CODE, NAME); return TRUE; }

slide-30
SLIDE 30

Flex and Bison make writing parsers easy

Flex & Bison - parser generators

Other tools

  • ANTLR
  • GOLD
  • PLY (Python Lex &

Yacc)

  • Boost.Spirit
slide-31
SLIDE 31

JAM File syntax is described to Bison

part of jam.y

slide-32
SLIDE 32

From .jam to .cpp

2004 2013

slide-33
SLIDE 33

TRL Turns .jam into C++

TRL to generate a message class definition

{@ define OutputMessage(msg, encoders, decoders) @} // // NOTICE: This is generated code. DO NOT EDIT! // class {{ msg.structName }} : public JamMessage { public: static u32 CRC; static u16 CODE; static cchar *NAME; // No argument constructor: {{ msg.structName }}() { {@ foreach f in msg.fields @} {@ if f.hasDefault @} {{ f.name }} = {{ f.defValue }}; {@ end if @} {@ end foreach @} }

slide-34
SLIDE 34

TRL Turns .jam into C++

TRL to generate a message class definition See Also

  • CTemplate
  • ngTemplate
  • Django (HTML focused)
  • Jinja (Python)

{@ define OutputMessage(msg, encoders, decoders) @} // // NOTICE: This is generated code. DO NOT EDIT! // class {{ msg.structName }} : public JamMessage { public: static u32 CRC; static u16 CODE; static cchar *NAME; // No argument constructor: {{ msg.structName }}() { {@ foreach f in msg.fields @} {@ if f.hasDefault @} {{ f.name }} = {{ f.defValue }}; {@ end if @} {@ end foreach @} }

slide-35
SLIDE 35

Fill out a dictionary and feed it to TRL

slide-36
SLIDE 36

Fill out a dictionary and feed it to TRL

{@ foreach f in msg.fields @} {@ if f.hasDefault @} {{ f.name }} = {{ f.defValue }}; {@ end if @} {@ end foreach @}

slide-37
SLIDE 37

Fill out a dictionary and feed it to TRL

{@ foreach f in msg.fields @} {@ if f.hasDefault @} {{ f.name }} = {{ f.defValue }}; {@ end if @} {@ end foreach @}

slide-38
SLIDE 38

Fill out a dictionary and feed it to TRL

{@ foreach f in msg.fields @} {@ if f.hasDefault @} {{ f.name }} = {{ f.defValue }}; {@ end if @} {@ end foreach @}

slide-39
SLIDE 39

Global Feature addition using TRL

TODO: Figure out how to illustrate this. Looking through history of our TRL files and jamgen would be useful. In fact, a screenshot of a diff of one of those changes would work well here.

slide-40
SLIDE 40

Development Cycle

  • Describe the protocol
  • Generate serialization and dispatch
  • Send messages
  • Receive messages
  • Configure routing info
slide-41
SLIDE 41

Create a message, fill in data, call send

void Checker::OnCaptured(CheckerID capturedBy, JUMP_TYPE how) { CheckerCapturedCredit msg; msg.capturedCheckerID = GetID(); msg.capturedBy = capturedBy; msg.jumpType = how; JamID destination = GetRouter()->GetCreditManagerID(); GetRouter()->Send(destination, &msg); }

slide-42
SLIDE 42

Create a message, fill in data, call send

void Checker::OnCaptured(CheckerID capturedBy, JUMP_TYPE how) { CheckerCapturedCredit msg; msg.capturedCheckerID = GetID(); msg.capturedBy = capturedBy; msg.jumpType = how; JamID destination = GetRouter()->GetCreditManagerID(); GetRouter()->Send(destination, &msg); }

slide-43
SLIDE 43

Create a message, fill in data, call send

void Checker::OnCaptured(CheckerID capturedBy, JUMP_TYPE how) { CheckerCapturedCredit msg; msg.capturedCheckerID = GetID(); msg.capturedBy = capturedBy; msg.jumpType = how; JamID destination = GetRouter()->GetCreditManagerID(); GetRouter()->Send(destination, &msg); }

slide-44
SLIDE 44

Create a message, fill in data, call send

void Checker::OnCaptured(CheckerID capturedBy, JUMP_TYPE how) { CheckerCapturedCredit msg; msg.capturedCheckerID = GetID(); msg.capturedBy = capturedBy; msg.jumpType = how; JamID destination = GetRouter()->GetCreditManagerID(); GetRouter()->Send(destination, &msg); }

slide-45
SLIDE 45

Create a message, fill in data, call send

void Checker::OnCaptured(CheckerID capturedBy, JUMP_TYPE how) { CheckerCapturedCredit msg; msg.capturedCheckerID = GetID(); msg.capturedBy = capturedBy; msg.jumpType = how; JamID destination = GetRouter()->GetCreditManagerID(); GetRouter()->Send(destination, &msg); }

slide-46
SLIDE 46

Create a message, fill in data, call send

void Checker::OnCaptured(CheckerID capturedBy, JUMP_TYPE how) { CheckerCapturedCredit msg; msg.capturedCheckerID = GetID(); msg.capturedBy = capturedBy; msg.jumpType = how; JamID destination = GetRouter()->GetCreditManagerID(); GetRouter()->Send(destination, &msg); }

slide-47
SLIDE 47

Structs and arrays in messages

message GroupUpdate { GroupID group; array<.CheckerID> checkers; }; /*** DATA START ***/ GroupID group; vector<CheckerID> checkers; /*** DATA STOP ***/ void GroupService::SendUpdate(GroupID id) { GroupUpdate msg; msg.group = id; msg.checkers.resize(MAX_GROUP_SIZE); // ... }

slide-48
SLIDE 48

Structs and arrays in messages

message GroupUpdate { GroupID group; array<.CheckerID> checkers; }; /*** DATA START ***/ GroupID group; vector<CheckerID> checkers; /*** DATA STOP ***/ void GroupService::SendUpdate(GroupID id) { GroupUpdate msg; msg.group = id; msg.checkers.resize(MAX_GROUP_SIZE); // ... }

slide-49
SLIDE 49

Structs and arrays in messages

message GroupUpdate { GroupID group; array<.CheckerID> checkers; }; /*** DATA START ***/ GroupID group; vector<CheckerID> checkers; /*** DATA STOP ***/ void GroupService::SendUpdate(GroupID id) { GroupUpdate msg; msg.group = id; msg.checkers.resize(MAX_GROUP_SIZE); // ... }

slide-50
SLIDE 50

Structs and arrays in messages

message GroupUpdate { GroupID group; array<.CheckerID> checkers; }; /*** DATA START ***/ GroupID group; vector<CheckerID> checkers; /*** DATA STOP ***/ void GroupService::SendUpdate(GroupID id) { GroupUpdate msg; msg.group = id; msg.checkers.resize(MAX_GROUP_SIZE); // ... }

slide-51
SLIDE 51

Structs and arrays in messages

message GroupUpdate { GroupID group; array<.CheckerID> checkers; }; /*** DATA START ***/ GroupID group; vector<CheckerID> checkers; /*** DATA STOP ***/ void GroupService::SendUpdate(GroupID id) { GroupUpdate msg; msg.group = id; msg.checkers.resize(MAX_GROUP_SIZE); // ... }

slide-52
SLIDE 52

Definitions

slide-53
SLIDE 53

Definitions

  • Message - serialized structure defined in a .jam file
slide-54
SLIDE 54

Definitions

  • Message - serialized structure defined in a .jam file
  • Protocol - a collection of messages
slide-55
SLIDE 55

Definitions

  • Message - serialized structure defined in a .jam file
  • Protocol - a collection of messages
  • Service - a module of code that implements message handlers

for one or more protocols

slide-56
SLIDE 56

Definitions

  • Message - serialized structure defined in a .jam file
  • Protocol - a collection of messages
  • Service - a module of code that implements message handlers

for one or more protocols

  • Program - can be composed of multiple services
slide-57
SLIDE 57

Message Destinations

slide-58
SLIDE 58

Message Destinations

void MatchService::CreateBoard(u64 width, u64 height) { BoardID = GenerateBoard(); // Send to a known, connected, service m_pServer->Send(m_boardServerID, &msg); }

slide-59
SLIDE 59

void MatchService::GameOver(u32 gameID, u64 winnerID) { msg.gameID = gameID; msg.winner = winnerID(); // Send to a service type, non-specified ID m_pServer->Send(JAM_SERVER_STATS_TRACKER, &msg); }

Message Destinations

void MatchService::CreateBoard(u64 width, u64 height) { BoardID = GenerateBoard(); // Send to a known, connected, service m_pServer->Send(m_boardServerID, &msg); }

slide-60
SLIDE 60

void MatchService::GameOver(u32 gameID, u64 winnerID) { msg.gameID = gameID; msg.winner = winnerID(); // Send to a service type, non-specified ID m_pServer->Broadcast(JAM_SERVER_STATS_TRACKER, &msg); }

Message Destinations

void MatchService::CreateBoard(u64 width, u64 height) { BoardID = GenerateBoard(); // Send to a known, connected, service m_pServer->Send(m_boardServerID, &msg); }

slide-61
SLIDE 61

void MatchService::GameOver(u32 gameID, u64 winnerID) { msg.gameID = gameID; msg.winner = winnerID(); // Send to a service type, non-specified ID m_pServer->Broadcast(JAM_SERVER_STATS_TRACKER, &msg); }

Message Destinations

void MatchService::CreateBoard(u64 width, u64 height) { BoardID = GenerateBoard(); // Send to a known, connected, service m_pServer->Send(m_boardServerID, &msg); } void Checker::HealChecker(CheckerID toHeal, u32 amount) { CheckerHeal msg; msg.healedBy = GetID(); msg.amount = amount; // Send a message to a specific object m_pServer->Send(toHeal, &msg); }

slide-62
SLIDE 62

Message routing by type

MatchmakerAddPlayer addMsg; addMsg.player = GetPlayerID(); addMsg.rank = GetRank(); // No JamID needed, send to any Matchmaker // May be queued until a Matchmaker is available m_pService->Send(JAM_SERVER_MATCHMAKER, &addMsg);

slide-63
SLIDE 63

Send a message and expect a response

MatchmakerAddPlayer addMsg; addMsg.player = GetPlayerID(); addMsg.level = GetLevel(); // Send to any Matchmaker, PlayerAddedHandler // will be called with response when complete m_pService->SendRegistered<PlayerAdded>( JAM_SERVER_MATCHMAKER, &addMsg );

slide-64
SLIDE 64

Send a message and expect a response

MatchmakerAddPlayer addMsg; addMsg.player = GetPlayerID(); addMsg.level = GetLevel(); // Send to any Matchmaker, PlayerAddedHandler // will be called with response when complete m_pService->SendRegistered<PlayerAdded>( JAM_SERVER_MATCHMAKER, &addMsg );

slide-65
SLIDE 65

Send a message to an object

void CheckerGroup::ChangeBoards(u32 newBoard) { CheckerChangeBoard msg; msg.boardID = newBoard; for(int i = 0; i < m_checkers.size(); i++) { m_pServer->Send(m_checkers[i]->GetID(), &msg); } }

slide-66
SLIDE 66

Each object is owned by one server

class Checker { //... CheckerID m_id; JamID m_serverID; JamID GetServer() { return m_serverID; } CheckerID GetID() { return m_id; } //... };

slide-67
SLIDE 67

Each object is owned by one server

class Checker { //... CheckerID m_id; JamID m_serverID; JamID GetServer() { return m_serverID; } CheckerID GetID() { return m_id; } //... };

slide-68
SLIDE 68

Each object is owned by one server

class Checker { //... CheckerID m_id; JamID m_serverID; JamID GetServer() { return m_serverID; } CheckerID GetID() { return m_id; } //... };

slide-69
SLIDE 69

How messages get routed

void BoardServer::Send(Checker *pChecker, JamMessage *pMessage) { m_pJamServer->Send(pChecker->GetServer(), pChecker->GetID(), pMessage); }

slide-70
SLIDE 70

Development Cycle

  • Describe the protocol
  • Generate serialization and dispatch
  • Send messages
  • Receive messages
  • Configure routing info
slide-71
SLIDE 71

On receipt, look up and dispatch

// static callback registered with JAM by protocol ID // called for each incoming message void BoardServer::CheckerDispatch(JamLink &link, JamMessage *pMessage) { CheckerID destID = pMessage->GetDestination(); Checker *pChecker = GetCheckerObject(destID); pChecker->QueueMessage(pMessage); switch(pMessage->GetProtocolCRC()) { case JAMCheckerProtocol_CRC: JamCheckerProtocol::Dispatch<Checker>(pMessage, pChecker); } }

slide-72
SLIDE 72

On receipt, look up and dispatch

// static callback registered with JAM by protocol ID // called for each incoming message void BoardServer::CheckerDispatch(JamLink &link, JamMessage *pMessage) { CheckerID destID = pMessage->GetDestination(); Checker *pChecker = GetCheckerObject(destID); pChecker->QueueMessage(pMessage); switch(pMessage->GetProtocolCRC()) { case JAMCheckerProtocol_CRC: JamCheckerProtocol::Dispatch<Checker>(pMessage, pChecker); } }

slide-73
SLIDE 73

On receipt, look up and dispatch

// static callback registered with JAM by protocol ID // called for each incoming message void BoardServer::CheckerDispatch(JamLink &link, JamMessage *pMessage) { CheckerID destID = pMessage->GetDestination(); Checker *pChecker = GetCheckerObject(destID); pChecker->QueueMessage(pMessage); switch(pMessage->GetProtocolCRC()) { case JAMCheckerProtocol_CRC: JamCheckerProtocol::Dispatch<Checker>(pMessage, pChecker); } }

slide-74
SLIDE 74

JamLink

void BoardServer::CheckerDispatch(JamLink &link, JamMessage *pMessage) {

slide-75
SLIDE 75

Generated Dispatch methods

//NOTICE: This is generated code. DO NOT EDIT! template<typename HANDLER_T> static JAM_RESULT Dispatch(JamMessage *pMessage, HANDLER_T *pHandler) { switch(pMessage->GetCode()) { case JAM_MSG_CheckerHeal: result = pHandler->CheckerHealHandler(link, (CheckerHeal *)pMessage); break; // cases for rest of protocol's messages...

slide-76
SLIDE 76

Generated Dispatch methods

//NOTICE: This is generated code. DO NOT EDIT! template<typename HANDLER_T> static JAM_RESULT Dispatch(JamMessage *pMessage, HANDLER_T *pHandler) { switch(pMessage->GetCode()) { case JAM_MSG_CheckerHeal: result = pHandler->CheckerHealHandler(link, (CheckerHeal *)pMessage); break; // cases for rest of protocol's messages...

slide-77
SLIDE 77

Generated Dispatch methods

//NOTICE: This is generated code. DO NOT EDIT! template<typename HANDLER_T> static JAM_RESULT Dispatch(JamMessage *pMessage, HANDLER_T *pHandler) { switch(pMessage->GetCode()) { case JAM_MSG_CheckerHeal: result = pHandler->CheckerHealHandler(link, (CheckerHeal *)pMessage); break; // cases for rest of protocol's messages...

slide-78
SLIDE 78

Generated message handler prototypes

#include this in the middle of a class

// A message handler prototype is auto-generated for each message // in the protocol. #include these declarations in the middle // of your hand constructed class. JAM_RESULT CheckerHealHandler(JamLink &link, CheckerHeal *msg); JAM_RESULT CheckerDamageHandler(JamLink &link, CheckerDamage *msg); JAM_RESULT CheckerPowerupHandler(JamLink &link, CheckerPowerup *msg); JAM_RESULT CheckerKingHandler(JamLink &link, CheckerKing *msg);

slide-79
SLIDE 79

Message handler methods

JAM_RESULT Checker::CheckerHealHandler(CheckerHeal *pMessage) { m_health += pMessage->amount; LOG("Checker %d was healed for %d by checker %d", GetID(), pMessage->amount, pMessage->healedBy); return JAM_OK; }

slide-80
SLIDE 80

Send and Receive

void Checker::HealChecker(CheckerID toHeal, u32 amount) { CheckerHeal msg; msg.healedBy = GetID(); msg.amount = amount; // Send a message to a specific object m_pServer->Send(toHeal, &msg); } JAM_RESULT Checker::CheckerHealHandler(CheckerHeal *pMessage) { m_health += pMessage->amount; LOG("Checker %d was healed for %d by checker %d", GetID(), pMessage->amount, pMessage->healedBy); return JAM_OK; }

slide-81
SLIDE 81

Development Cycle

  • Describe the protocol
  • Generate serialization and dispatch
  • Send messages
  • Receive messages
  • Configure routing info
slide-82
SLIDE 82

Define services

Configure protocols the Matchmaker service sends and receives

void Matchmaker::Configure(JamServer *pServer) { JamRouteConfig &routeConfig = pServer->GetRouteConfig(); routeConfig.ConfigureInbound<MatchmakerProtocol>( this, Matchmaker::DispatchMessage); routeConfig.ConfigureOutbound<MatchmakerResponseProtocol>(); }

slide-83
SLIDE 83

Define services

Configure protocols the Matchmaker service sends and receives

void Matchmaker::Configure(JamServer *pServer) { JamRouteConfig &routeConfig = pServer->GetRouteConfig(); routeConfig.ConfigureInbound<MatchmakerProtocol>( this, Matchmaker::DispatchMessage); routeConfig.ConfigureOutbound<MatchmakerResponseProtocol>(); }

slide-84
SLIDE 84

RouteConfig maintains a protocol to handler mapping

slide-85
SLIDE 85

Handlers have access to sender and

  • ther metadata about received messages

JAM_RESULT BoardServer::AddPlayerHandler(JamLink &link, AddPlayer *msg) { LOG("Adding player %s from server %s", IDSTR(msg->playerID), link.Describe().c_str()); // Do stuff return JAM_OK; }

slide-86
SLIDE 86

Handlers have access to sender and

  • ther metadata about received messages

JAM_RESULT BoardServer::AddPlayerHandler(JamLink &link, AddPlayer *msg) { LOG("Adding player %s from server %s", IDSTR(msg->playerID), link.Describe().c_str()); // Do stuff return JAM_OK; }

slide-87
SLIDE 87

Coarse and fine-grained queueing and

Race Condition

slide-88
SLIDE 88

Receiving via Message Queue

void Matchmaker::Configure() { // Messages received at any time are placed into a queue routeConfig.ConfigureInbound<MatchmakerProtocol>( this, &m_messageQueue); } void Matchmaker::Idle() { // Queue is processed in one thread at a known time pServer->ProcessQueue(&m_messageQueue, this); }

slide-89
SLIDE 89

Receiving via Message Queue

void Matchmaker::Configure() { // Messages received at any time are placed into a queue routeConfig.ConfigureInbound<MatchmakerProtocol>( this, &m_messageQueue); } void Matchmaker::Idle() { // Queue is processed in one thread at a known time pServer->ProcessQueue(&m_messageQueue, this); }

slide-90
SLIDE 90

Receiving via Message Queue

void Matchmaker::Configure() { // Messages received at any time are placed into a queue routeConfig.ConfigureInbound<MatchmakerProtocol>( this, &m_messageQueue); } void Matchmaker::Idle() { // Queue is processed in one thread at a known time pServer->ProcessQueue(&m_messageQueue, this); }

slide-91
SLIDE 91

Receiving via Message Queue

void Matchmaker::Configure() { // Messages received at any time are placed into a queue routeConfig.ConfigureInbound<MatchmakerProtocol>( this, &m_messageQueue); } void Matchmaker::Idle() { // Queue is processed in one thread at a known time pServer->ProcessQueue(&m_messageQueue, this); }

slide-92
SLIDE 92

Global lock dispatching

slide-93
SLIDE 93

Raw concurrent handlers

slide-94
SLIDE 94

Raw concurrent handlers

slide-95
SLIDE 95

Lock Policies

class MatchmakerLockPolicy { Matchmaker *m_owner; void Lock(JamMessage *msg, JamMessageQueue **ppQueue) { // Adding a player requires a write lock if(msg->GetCode() == JAM_MSG_MatchmakerAddPlayer) { m_owner->AcquireWriteLock(); } else { m_owner->AcquireReadLock(); } } void Unlock(JamMessage *msg) { /* Same logic, release lock */ } }

slide-96
SLIDE 96

Lock Policies

class MatchmakerLockPolicy { Matchmaker *m_owner; void Lock(JamMessage *msg, JamMessageQueue **ppQueue) { // Adding a player requires a write lock if(msg->GetCode() == JAM_MSG_MatchmakerAddPlayer) { m_owner->AcquireWriteLock(); } else { m_owner->AcquireReadLock(); } } void Unlock(JamMessage *msg) { /* Same logic, release lock */ } }

slide-97
SLIDE 97

Incoming messages are refcounted

slide-98
SLIDE 98

Incoming messages are refcounted

  • Message passed to handler is a refcounted object
slide-99
SLIDE 99

Incoming messages are refcounted

  • Message passed to handler is a refcounted object
  • Possible to retain a message pointer until later
slide-100
SLIDE 100

Incoming messages are refcounted

  • Message passed to handler is a refcounted object
  • Possible to retain a message pointer until later
  • Smart pointers are available
slide-101
SLIDE 101

Incoming messages are refcounted

  • Message passed to handler is a refcounted object
  • Possible to retain a message pointer until later
  • Smart pointers are available
  • Messages contain no pointers to any other objects
slide-102
SLIDE 102

Incoming messages are refcounted

  • Message passed to handler is a refcounted object
  • Possible to retain a message pointer until later
  • Smart pointers are available
  • Messages contain no pointers to any other objects
  • No circular references are possible
slide-103
SLIDE 103

CPU And Bandwidth Efficiency

slide-104
SLIDE 104

JAM is either efficient or backwards compatible

2004 - Assumed binary compatibility

slide-105
SLIDE 105

Negotiation means dead-simple binary serialization most of the time

slide-106
SLIDE 106

In some cases, can just memcpy it onto the wire

// This message could easily be memcpy'ed onto the wire class CreateChecker : public JamMessage { /**** DATA START ****/ u32 checkerType; u32 owner; /**** DATA STOP ****/ // Code... };

slide-107
SLIDE 107

Generated code means easy

  • ptimizations
slide-108
SLIDE 108

Generated code means easy

  • ptimizations

_encoder.Put("capturedCheckerID", capturedCheckerID); _encoder.Put("capturedBy", capturedBy); _encoder.Put("jumpType", jumpType);

slide-109
SLIDE 109

Fallback to JSON Serialization

slide-110
SLIDE 110

Fallback to JSON Serialization

  • Switch to JSON serialization when binary CRC check fails
slide-111
SLIDE 111

Fallback to JSON Serialization

  • Switch to JSON serialization when binary CRC check fails
  • Great for programmers
slide-112
SLIDE 112

Fallback to JSON Serialization

  • Switch to JSON serialization when binary CRC check fails
  • Great for programmers
  • Way more expensive (CPU and Bandwidth)
slide-113
SLIDE 113

Fallback to JSON Serialization

  • Switch to JSON serialization when binary CRC check fails
  • Great for programmers
  • Way more expensive (CPU and Bandwidth)
  • Never allowed on public facing protocols
slide-114
SLIDE 114

Fallback to JSON Serialization

  • Switch to JSON serialization when binary CRC check fails
  • Great for programmers
  • Way more expensive (CPU and Bandwidth)
  • Never allowed on public facing protocols
  • Even internally it’s sometimes unreasonable
slide-115
SLIDE 115

Fallback to JSON Serialization

  • Switch to JSON serialization when binary CRC check fails
  • Great for programmers
  • Way more expensive (CPU and Bandwidth)
  • Never allowed on public facing protocols
  • Even internally it’s sometimes unreasonable
slide-116
SLIDE 116

JSON

{ "_msgID":10, "type":6, "error":0, "desc":{ "m_id":"T2R00S40.00E14815726P10987H127.0.0.1:14001", "m_host":"127.0.0.1", "m_partitionID":0, "m_configID":0, "m_buildNum":0, "m_type":40, "m_subType":0 } }

slide-117
SLIDE 117

Protocol Negotiation

slide-118
SLIDE 118

Message overhead

slide-119
SLIDE 119

JSON vs. Binary performance

40 80

Two 32-bit ints u32+vec3

75 49 21 13 Wire size including overhead

Binary JSON

slide-120
SLIDE 120

Still highly successful - some network tools run on old versions frequently

slide-121
SLIDE 121

Google's Protocol Buffers and

slide-122
SLIDE 122

For both speed AND inter-version compatibility, there are better choices

slide-123
SLIDE 123

protobufs sometimes wins on bandwidth, but JAM is faster

ConstructRequest SerializeOpenDirToString SerializeOpenDirToArray DeserializeOpenDirRequest SerializeDirContents 10000 20000 30000 40000

Time in ms

Protobufs JAM

SmallStatFS LargeStatFS 25 50 75 100

Size in Bytes

slide-124
SLIDE 124

Writing our own gives us ultimate control over everything

slide-125
SLIDE 125

Writing our own gives us ultimate control over everything

  • Automated serialization
slide-126
SLIDE 126

Writing our own gives us ultimate control over everything

  • Automated serialization
  • Easy yet flexible message dispatching
slide-127
SLIDE 127

Writing our own gives us ultimate control over everything

  • Automated serialization
  • Easy yet flexible message dispatching
  • High performance
slide-128
SLIDE 128

Writing our own gives us ultimate control over everything

  • Automated serialization
  • Easy yet flexible message dispatching
  • High performance
  • Inter-version compatibility
slide-129
SLIDE 129

Writing our own gives us ultimate control over everything

  • Automated serialization
  • Easy yet flexible message dispatching
  • High performance
  • Inter-version compatibility
  • Less tedium = more awesome
slide-130
SLIDE 130

Thanks! Questions?

Joe Rumsey jrumsey@blizzard.com Twitter: @joerumz

slide-131
SLIDE 131

Thanks! Questions?

Joe Rumsey jrumsey@blizzard.com Twitter: @joerumz

Disclaimer: Blizzard is not really making World of Checkers