Bob Lee @crazybob 2 ,000,000 Dim Sum Accepting a payment - - PowerPoint PPT Presentation

▶
bob lee
SMART_READER_LITE
LIVE PREVIEW

Bob Lee @crazybob 2 ,000,000 Dim Sum Accepting a payment - - PowerPoint PPT Presentation

Dec 05, 2023 540 likes •1.01k views

Bob Lee @crazybob 2 ,000,000 Dim Sum Accepting a payment Authorization 1 Capture 2 Persistent Queues ACID Atomicity Durability Filesystem Promises Renaming is atomic Fsync is durable* Segment/block writes Traditional

slide-1
SLIDE 1
slide-2
SLIDE 2

Bob Lee

@crazybob

slide-3
SLIDE 3
slide-4
SLIDE 4
slide-5
SLIDE 5

2 ,000,000

slide-6
SLIDE 6

Dim Sum

slide-7
SLIDE 7

Accepting a payment

1

Authorization Capture

2

slide-8
SLIDE 8

Persistent Queues

slide-9
SLIDE 9

ACID

  • Atomicity
  • Durability
slide-10
SLIDE 10

Filesystem Promises

  • Renaming is atomic
  • Fsync is durable*
  • Segment/block writes
slide-11
SLIDE 11

Traditional Strategies

  • Write, fsync, rename, fsync
  • Rollback log
  • Journal
slide-12
SLIDE 12

Tape

github.com/square/tape

slide-13
SLIDE 13 Header Elements
slide-14
SLIDE 14

Header (16 bytes)

  • File length
  • Element count
  • First offset
  • Last offset
slide-15
SLIDE 15

Element

  • Length
  • Data
slide-16
SLIDE 16

Enqueueing

1

Write data Fsync Write header Fsync

2 3 4

slide-17
SLIDE 17

Dequeueing

1

Read data Write header Fsync

2 3

slide-18
SLIDE 18

Expansion

1

Set file length Fsync Copy data Write header Fsync

2 3 4 5

slide-19
SLIDE 19

What’s next?

  • Tape on the server
  • Elide the filesystem
slide-20
SLIDE 20

Server Stack

slide-21
SLIDE 21

Ruby

slide-22
SLIDE 22

Java

slide-23
SLIDE 23

One Repository

slide-24
SLIDE 24

One-Jar

slide-25
SLIDE 25

Self Executing Jars

#!/usr/bin/java -Xmx2G -jar [jar file]

slide-26
SLIDE 26

Java Stack

  • Jetty
  • JAX-RS (Jersey)
  • JPA (Hibernate)
  • Guice
  • Dropwizard
slide-27
SLIDE 27

Rethinking Pub/Sub

slide-28
SLIDE 28 Payment Processing Capture Settlement Business Intelligence Risk etc.
slide-29
SLIDE 29 Producer Messaging Server Consumer
slide-30
SLIDE 30 Messaging Server Producer Messaging Server Consumer Messaging Server
slide-31
SLIDE 31

Synchronization Feeds

Producer Consumer Retrieve all records Return all records and latest version V Retrieve all records since version V Return all records since V and latest version ...
slide-32
SLIDE 32

Constraints

  • Immutable sequence of events
  • Total order
  • Centralized server
slide-33
SLIDE 33

Benefits

  • Email vs. RSS
  • Stateless
  • Fewer moving parts
  • Free bootstrapping!
slide-34
SLIDE 34

Further Ideas

  • Partitioning feeds
  • Caching and replicating feeds
  • Pubsubhubbub
slide-35
SLIDE 35

Dependency Injection

slide-36
SLIDE 36

Guice

  • Launched in ’06
  • Used by Google, Facebook, Twitter, etc.
  • On version 3
slide-37
SLIDE 37

Dagger

  • Supports JSR-330
  • Provider methods only
  • Uses code generation
  • Compile time validation
  • No custom scopes
  • No circular dependencies

github.com/square/dagger

slide-38
SLIDE 38

corner.squareup.com

slide-39
SLIDE 39

github.com/square

slide-40
SLIDE 40
slide-41
SLIDE 41
slide-42
SLIDE 42
slide-43
SLIDE 43

@crazybob

slide-44
SLIDE 44

?

Questions

slide-45
SLIDE 45