MILLIONS OF TRANSACTIONS PER SECOND ON A SINGLE MACHINE CASE FOR A - - PowerPoint PPT Presentation

▶
millions of transactions per second on a single machine
SMART_READER_LITE
LIVE PREVIEW

MILLIONS OF TRANSACTIONS PER SECOND ON A SINGLE MACHINE CASE FOR A - - PowerPoint PPT Presentation

Mar 04, 2024 310 likes •683 views

MILLIONS OF TRANSACTIONS PER SECOND ON A SINGLE MACHINE CASE FOR A VIRTUALIZED DATABASE AND SCALE-IN ROGER JOHANSSON Who am I? Roger Johansson Actor Model, Scalability, Distributed Systems, C#, Senior Solution Architect Starcounter Go,

slide-1
SLIDE 1

MILLIONS OF TRANSACTIONS PER SECOND ON A SINGLE MACHINE CASE FOR A VIRTUALIZED DATABASE AND SCALE-IN

ROGER JOHANSSON

slide-2
SLIDE 2

Who am I?

Roger Johansson

Senior Solution Architect – Starcounter http://StarCounter.io Proto.Actor Founder – Ultra Fast Distributed Actors (Go, .NET, JVM) http://Proto.Actor https://github.com/AsynkronIT Twitter: @rogeralsing Github: rogeralsing Mail: roger@starcounter.com http://Github.com/rogeralsing/presentations

Actor Model, Scalability, Distributed Systems, C#, Go, Kotlin

slide-3
SLIDE 3

Agenda

Setting the stage - Why we need in memory computing:

  • Application Platform
  • Micro-Apps

The Starcounter information operating system:

  • Our approach to in memory computing
  • The future of hardware
  • The Mars project
slide-4
SLIDE 4
slide-5
SLIDE 5

Starcounter Communication Palindrom - REST, Web Sockets Front-end Framework React, Polymer Application View Models, Entities, App Logic In Memory App Platform Mapping, Persistence, Queries

slide-6
SLIDE 6

Starcounter Front-end Framework Application In Memory App Platform Traditional Stack Services + Contracts Client Side Code Application Code Database Front-end Framework O/R Mapper Network Network View Models Network

slide-7
SLIDE 7
slide-8
SLIDE 8

Starcounter Model A Model C Model B App A App C App B UI A UI C UI B

slide-9
SLIDE 9

Starcounter Model A Model C Model B App A App C App B UI A UI C UI B Client Side Blending UI A UI B UI C

slide-10
SLIDE 10
slide-11
SLIDE 11
slide-12
SLIDE 12

ObjectId User LastName FirstName 123 Edqvist Stefan ObjectId Profile LastName FirstName 789 Edqvist Stefan

slide-13
SLIDE 13

Starcounter Model C App A App C App B UI A UI C UI B Client Side Blending UI A UI B UI C Model A Model B Mapper ACID, Snapshot Isolation

slide-14
SLIDE 14
slide-15
SLIDE 15

Starcounter Model C App A App C App B UI A UI C UI B Model A Model B Mapper

slide-16
SLIDE 16

Demo Dataflow

slide-17
SLIDE 17
slide-18
SLIDE 18

Database Process Disk Data is stored on disk, optimized for spinning disks and moving arms Application Process ObjectId Object LastName FirstName Age 123 Turing Alan 41 Offline copy of database data Network / IPC Transfer Data over network Object Mapper Map to and from Object / Storage model 123 Alan Turing 41 Ada Lovelace 36 Grace Hopper 86 FirstName LastName Age ObjectId 124 125

slide-19
SLIDE 19

Database Process Memory Application Process ObjectId Object LastName FirstName Age 123 Turing Alan 41 Offline copy of database data Network / IPC Transfer Data over network Object Mapper Map to and from Object / Relational model Data is mainly kept in memory, enabling faster reads and writes 123 Alan Turing 41 Ada Lovelace 36 Grace Hopper 86 FirstName LastName Age ObjectId 124 125

slide-20
SLIDE 20

Database Process Application Process Shared Memory ObjectId Object 123 123 Alan Turing 41 Ada Lovelace 36 Grace Hopper 86 FirstName LastName Age ObjectId 124 125 Stateless Entity maps directly to shared memory Data is kept in memory and shared across processes

slide-21
SLIDE 21

Database Process Node.JS C# Shared Memory 123 Alan Turing 41 Ada Lovelace 36 Grace Hopper 86 FirstName LastName Age ObjectId 124 125 ObjectId Object 123 ObjectId Object 123 Multiple processes can read the same data

slide-22
SLIDE 22

C# Source Code

public class Person { public string FirstName { get; set; }

slide-23
SLIDE 23

C# Compiler Output

public class Person { private string _firstName; public string FirstName { get => _firstName; set value => _firstName = value; }

slide-24
SLIDE 24

Starcounter Weaver Output

public class Person { private string _firstName; private ulong _id; public string FirstName { get => DbState.GetString(_id, FirstNameId); set value => DbState.SetString(_id, FirstNameId, value) }

slide-25
SLIDE 25
slide-26
SLIDE 26

O/R mapper – Object Graphs

Order OrderDetail Product Customer

Lazy Load Execute SQL Query per traversed relation Eager Load Carefully defining load boundaries

Ripple load, Hundreds/Thousands

  • f queries per view

Hard to maintain, inefficient joined queries

slide-27
SLIDE 27

Starcounter – Object Graphs

Order OrderDetail Product Customer

Chasing Pointers Traversing relations are simple pointer traversals

slide-28
SLIDE 28
slide-29
SLIDE 29

Demo - Write 1 mil transactions

slide-30
SLIDE 30

In this demo we provided:

  • ACID Transactions
  • Snapshot Isolation
  • Guaranteed disk persistency

Can we go even faster?

  • We are capped by disk IO
  • The Mars project

Demo - Write 1 mil transactions

slide-31
SLIDE 31
slide-32
SLIDE 32

0,00001 0,00010 0,00100 0,01000 0,10000 1,00000 10,00000 100,00000 1000,00000 1980 1985 1990 1995 2000 2005 2010 2015 2020 USD Year Price of 1MB in USD over time

slide-33
SLIDE 33

Current Main Memory Cache Solid State Disk Registers Magnetic Disk Future Main Memory Cache Non Volatile Memory Registers Solid State Disk Magnetic Disk

slide-34
SLIDE 34

Volatile DRAM Non Volatile DRAM Main Store (batch update) Delta Store (incremental updates)

CPU

Starcounter on the CPUs of 2018

Read Write Merge

slide-35
SLIDE 35

Server Server

Few Big Servers, not many small

slide-36
SLIDE 36

Server Server

Few Big Servers, not many small

slide-37
SLIDE 37