Bringing Riak to the Mobile Platform Kresten Krab Thorup Hacker - - PowerPoint PPT Presentation

▶
bringing riak to the mobile platform
SMART_READER_LITE
LIVE PREVIEW

Bringing Riak to the Mobile Platform Kresten Krab Thorup Hacker - - PowerPoint PPT Presentation

Jan 08, 2023 538 likes •964 views

Bringing Riak to the Mobile Platform Kresten Krab Thorup Hacker @drkrab Bringing Riak to the Mobile Platform Client Kresten Krab Thorup Hacker @drkrab About the Speaker Language Geek Emacs/TeX Hacker, Objective C, NeXT, GNU Compiled

slide-1
SLIDE 1

Bringing Riak to the Mobile Platform

@drkrab Kresten Krab Thorup Hacker

slide-2
SLIDE 2

Bringing Riak to the Mobile Platform

Kresten Krab Thorup Hacker @drkrab

Client

slide-3
SLIDE 3

About the Speaker

3

■ Language Geek Emacs/TeX Hacker, Objective C, NeXT, GNU Compiled Java, Java Generics, Ph.D., Erlang/Erjang ■ Developer J2EE AppServer, CORBA/RMI, XA-TM ■ Trifork CTO QCon & GOTO Conferences Technology Adoption; Riak; Erlang

slide-4
SLIDE 4

■ Riak and it’s Data Model ■ Protocol for Key/Value synchronization ■ BucketDB, a Riak “client peer”

Outline

@drkrab

4

slide-5
SLIDE 5

What is Riak, anyway?

@drkrab

slide-6
SLIDE 6

6

slide-7
SLIDE 7

7

slide-8
SLIDE 8

8

coordinate

slide-9
SLIDE 9

9

sync

slide-10
SLIDE 10

10

scalable and available system captures write-write conflicts

slide-11
SLIDE 11

Common Medicine Card

11

slide-12
SLIDE 12

4

[ola:1]

  • la: foo=4

foo 6

[ola:2]

  • la: foo=6
slide-13
SLIDE 13

4

[ola:1]

  • la: foo=4

foo 7

[ola:1, joe:1] joe: foo=7

slide-14
SLIDE 14

4

[ola:1]

  • la: foo=4

foo 6

[ola:2]

  • la: foo=6

7

[ola:1, joe:1] joe: foo=7 merge merge

6 or 7

[ola:2, joe:1]

slide-15
SLIDE 15

connect(Server) -> Client Client:insert(BKey, Datum) -> {ok, VClock} | ... Client:lookup(BKey) -> {ok, VClock, [Datum]} | ... Client:update(BKey, VClock, Datum) -> ok | ... Client:delete(BKey, VClock) -> ok | ...

Riak API review

Pseudo Code!

BKey :: {Bucket, Key} Datum :: {ContentType, RawData}

15

slide-16
SLIDE 16

16

system captures write-write conflicts resolve lazily (read repair)

slide-17
SLIDE 17

17

sync sync

slide-18
SLIDE 18

■ Riak and it’s Data Model ■ Protocol for Key/Value synchronization ■ BucketDB, a Riak “client peer”

Outline

@drkrab

18

slide-19
SLIDE 19

riak cluster riak cluster riak_sync peer riak_sync peer

bucket sync

Not to be confused with riak_repl, part of the commercial Riak EDS

slide-20
SLIDE 20

%% “client” peer > LocalStore = riak:client_connect(‘riak@127.0.0.1’) > riak_sync:sync(LocalStore, <<“bucket”>>, “172.1.35.204”, 8082) %% Other site has riak_sync running... (riak@172.1.35.204)1> riak_sync:start() {riak_sync, [{pb_ip, “172.1.35.204”}, {pb_port, 8082}, {riak, ‘dev@127.0.0.1’}]}

slide-21
SLIDE 21

Riak Sync Protocol

■ Riak bucket replication ■ Asymmetric work load ■ Designed for high-latency networks

21

slide-22
SLIDE 22

"" "4" "46"

46D47F3C5B C275DA8A33 EEC7B5EA3F 0FDBC95D0D 467B5EA3F0 FDBC95D0DD 47F3C5BC27 5DA8A33EEC 49DA8A33EE C7B5EA3F0F DBC95D0DD4 7F3C5BC275

"B"

B8EEC7B5EA 3F0FDBC95D 0DD47F3C5B C275DA8A33 Inner Nodes Leaf Nodes

StorageKey = sha1(Key)

CHash CHash CHash CHash CHash CHash CHash CHash

CHash = sha1(Key, Version,Value)

slide-23
SLIDE 23

"" "4" "46"

46D47F3C5B C275DA8A33 EEC7B5EA3F 0FDBC95D0D 467B5EA3F0 FDBC95D0DD 47F3C5BC27 5DA8A33EEC 49DA8A33EE C7B5EA3F0F DBC95D0DD4 7F3C5BC275

"B"

B8EEC7B5EA 3F0FDBC95D 0DD47F3C5B C275DA8A33

"" "4" "46"

46D47F3C5B C275DA8A33 EEC7B5EA3F 0FDBC95D0D 467B5EA3F0 FDBC95D0DD 47F3C5BC27 5DA8A33EEC 49DA8A33EE C7B5EA3F0F DBC95D0DD4 7F3C5BC275

"B"

B8EEC7B5EA 3F0FDBC95D 0DD47F3C5B C275DA8A33

riaksync peer riaksync peer

CHash CHash CHash CHash CHash CHash CHash CHash

slide-24
SLIDE 24

"" "4" "46"

46D47F3C5B C275DA8A33 EEC7B5EA3F 0FDBC95D0D 467B5EA3F0 FDBC95D0DD 47F3C5BC27 5DA8A33EEC 49DA8A33EE C7B5EA3F0F DBC95D0DD4 7F3C5BC275

"B"

B8EEC7B5EA 3F0FDBC95D 0DD47F3C5B C275DA8A33

"" "4" "46"

46D47F3C5B C275DA8A33 EEC7B5EA3F 0FDBC95D0D 467B5EA3F0 FDBC95D0DD 47F3C5BC27 5DA8A33EEC 49DA8A33EE C7B5EA3F0F DBC95D0DD4 7F3C5BC275

"B"

B8EEC7B5EA 3F0FDBC95D 0DD47F3C5B C275DA8A33

riaksync peer riaksync peer

Slice Slice Slice Slice Slice Slice Slice Slice

slide-25
SLIDE 25

"" "4" "46"

46D47F3C5B C275DA8A33 EEC7B5EA3F 0FDBC95D0D 467B5EA3F0 FDBC95D0DD 47F3C5BC27 5DA8A33EEC 49DA8A33EE C7B5EA3F0F DBC95D0DD4 7F3C5BC275

"B"

B8EEC7B5EA 3F0FDBC95D 0DD47F3C5B C275DA8A33

"" "4" "46"

46D47F3C5B C275DA8A33 EEC7B5EA3F 0FDBC95D0D 467B5EA3F0 FDBC95D0DD 47F3C5BC27 5DA8A33EEC 49DA8A33EE C7B5EA3F0F DBC95D0DD4 7F3C5BC275

"B"

B8EEC7B5EA 3F0FDBC95D 0DD47F3C5B C275DA8A33

riaksync peer riaksync peer

slide-26
SLIDE 26

"" "4" "46"

46D47F3C5B C275DA8A33 EEC7B5EA3F 0FDBC95D0D 467B5EA3F0 FDBC95D0DD 47F3C5BC27 5DA8A33EEC 49DA8A33EE C7B5EA3F0F DBC95D0DD4 7F3C5BC275

"B"

B8EEC7B5EA 3F0FDBC95D 0DD47F3C5B C275DA8A33

"" "4" "46"

46D47F3C5B C275DA8A33 EEC7B5EA3F 0FDBC95D0D 467B5EA3F0 FDBC95D0DD 47F3C5BC27 5DA8A33EEC 49DA8A33EE C7B5EA3F0F DBC95D0DD4 7F3C5BC275

"B"

B8EEC7B5EA 3F0FDBC95D 0DD47F3C5B C275DA8A33

riaksync peer riaksync peer

slide-27
SLIDE 27

Sync Protocol

■ Parallelizes quickly (client does breadth first) ■ Many small messages (will benefit from Nagle’s alg) ■ Server peer reorders requests to reduce session/buffer state ■ Client needs (almost) no session state

27

slide-28
SLIDE 28

riak_sync_pb_socket riak_sync_peer riak_sync_bucket_server riak_client

wire protocol handler session management merkle_tree server

slide-29
SLIDE 29

riak_sync_pb_socket riak_sync_peer riak_sync_bucket_server riak_client listener bucket_server_manager postcommit_hook

slide-30
SLIDE 30

■ Riak and it’s Data Model ■ Protocol for Key/Value synchronization ■ BucketDB, a Riak “client peer”

Outline

@drkrab

30

slide-31
SLIDE 31

riak cluster riaksync peer

Bucket DB

sync

Client Riak: BucketDB

iOS / Android C++ / Java HTML / Desktop

31

slide-32
SLIDE 32

riak cluster riaksync peer

Bucket DB

sync

Content Distribution

32

slide-33
SLIDE 33

riak cluster riaksync peer

Bucket DB

sync

Reliable Queueing

33

slide-34
SLIDE 34

Bucket DB

sync

Peer Sync

34

Bucket DB

slide-35
SLIDE 35

"" "4" "46"

46D47F3C5B C275DA8A33 EEC7B5EA3F 0FDBC95D0D 467B5EA3F0 FDBC95D0DD 47F3C5BC27 5DA8A33EEC 49DA8A33EE C7B5EA3F0F DBC95D0DD4 7F3C5BC275

"B"

B8EEC7B5EA 3F0FDBC95D 0DD47F3C5B C275DA8A33

"" "4" "46"

46D47F3C5B C275DA8A33 EEC7B5EA3F 0FDBC95D0D 467B5EA3F0 FDBC95D0DD 47F3C5BC27 5DA8A33EEC 49DA8A33EE C7B5EA3F0F DBC95D0DD4 7F3C5BC275

"B"

B8EEC7B5EA 3F0FDBC95D 0DD47F3C5B C275DA8A33

BucketDB riak_sync peer

slide-36
SLIDE 36

"" "4" "46"

46D47F3C5B C275DA8A33 EEC7B5EA3F 0FDBC95D0D 467B5EA3F0 FDBC95D0DD 47F3C5BC27 5DA8A33EEC 49DA8A33EE C7B5EA3F0F DBC95D0DD4 7F3C5BC275

"B"

B8EEC7B5EA 3F0FDBC95D 0DD47F3C5B C275DA8A33

BucketDB

  • Client stores data in

local data store as a HASH TREE

  • Always “ready to

sync” (no session state, no cpu/memory intensive computation)

slide-37
SLIDE 37

ROW_ID KEY CONTENT TYPE DATA

DATA

HKEY VCLOCK CHASH KEY

MERKLE_LEAF

PATH CHASH CHILDREN

MERKLE_INNER

TRIGGER

slide-38
SLIDE 38

Other Items to Discuss

■ Security models ■ Provide own “socket factory” / credentials ■ Document- or Record-oriented ■ Handling Deleted data ■ Future: Index/Search, local M/R

38

slide-39
SLIDE 39

■ Riak and it’s Data Model ■ Protocol for Key/Value synchronization ■ BucketDB, a Riak “client peer”

Summary

@drkrab

39

slide-40
SLIDE 40

Thank You

40

@drkrab

slide-41
SLIDE 41

Thank You

41

@drkrab