Dr. Strange- Todd L. Montgomery @toddlmontgomery Haskell - - PowerPoint PPT Presentation

▶
dr strange
SMART_READER_LITE
LIVE PREVIEW

Dr. Strange- Todd L. Montgomery @toddlmontgomery Haskell - - PowerPoint PPT Presentation

Dec 01, 2022 290 likes •1.14k views

How I Learned to Stop Worrying & Love the or Dr. Strange- Todd L. Montgomery @toddlmontgomery Haskell Erlang Haskell Clojure Groovy Erlang Scala Haskell Clojure C# Groovy Erlang C++11 Scala Haskell

slide-1
SLIDE 1

How I Learned to Stop Worrying & Love the λ

  • r
  • Dr. Strange-λ

Todd L. Montgomery @toddlmontgomery

slide-2
SLIDE 2
slide-3
SLIDE 3

λ

slide-4
SLIDE 4

Haskell Erlang

λ

slide-5
SLIDE 5

λ

Clojure Haskell Erlang Groovy Scala

slide-6
SLIDE 6

λ

Clojure Haskell Erlang Groovy C# C++11 Scala

slide-7
SLIDE 7

λ

Scala Clojure Haskell Erlang Groovy C# C++11 C11

slide-8
SLIDE 8
slide-9
SLIDE 9

9

“Mr. President, we must not allooooow a Lambda Gap!”

slide-10
SLIDE 10

Java 8

slide-11
SLIDE 11

What could we do… green field…

slide-12
SLIDE 12

Perfect Timing!

slide-13
SLIDE 13

Aeron

https://github.com/real-logic/Aeron

slide-14
SLIDE 14

Low Latency High Throughput Open Source Messaging Transport

slide-15
SLIDE 15
slide-16
SLIDE 16

“Mr. President, the technology required is easily within the means of even the smallest nuclear power. It requires only the will to do so.”

slide-17
SLIDE 17

Todd Montgomery Richard Warburton Martin Thompson

The Aeron Team

slide-18
SLIDE 18

Why?

slide-19
SLIDE 19

Latency is [lost] opportunity

In finance & other places…

slide-20
SLIDE 20

Capital Markets

slide-21
SLIDE 21

The FX Trader

$ / £ = 0.6375 £ / ¥ = 123.77 ¥ / $ = 0.0127 $1,000,000 = £637,500

  • £637,500 = 78,903,375¥
  • 78,903,375¥ = $1,002,072
  • CHA-CHING!

Currency Pairs Trades

¥ / $ = 0.0126

78,903,375¥ = $994,182.53 !!!

slide-22
SLIDE 22

Ad Market

slide-23
SLIDE 23

Sports & Entertainment

slide-24
SLIDE 24

Logistics

slide-25
SLIDE 25

Why Java?

Why even an OS? Why even …

Don't Settle! Don’t Compromise!

slide-26
SLIDE 26
slide-27
SLIDE 27
slide-28
SLIDE 28

Design Principles

  • 1. Garbage free in steady state running
  • 2. Apply Smart Batching in the message path
  • 3. Wait-free algorithms in the message path
  • 4. Non-blocking IO in the message path
  • 5. No exceptional cases in the message path
  • 6. Apply the Single Writer Principle
  • 7. Prefer unshared state
  • 8. Avoid unnecessary data copies

https://github.com/real-logic/Aeron/wiki/Design-Principles

slide-29
SLIDE 29

And, yes, Unsafe

slide-30
SLIDE 30
slide-31
SLIDE 31

But… We have the launch codes so, it’s cool

slide-32
SLIDE 32

Publisher Subscriber Subscriber Publisher

Architecture

IPC Log Buffer

slide-33
SLIDE 33

Sender Receiver Receiver Sender Publisher Subscriber Subscriber Publisher

Architecture

Media IPC Log Buffer Media (UDP, InfiniBand, PCI-e 3.0)

slide-34
SLIDE 34

Conductor Sender Receiver Conductor Receiver Sender Publisher Subscriber Subscriber Publisher Admin Events

Architecture

Admin Events Media IPC Log Buffer Media (UDP, InfiniBand, PCI-e 3.0) Function/Method Call Volatile Fields & Queues

slide-35
SLIDE 35

Client Media Driver Media Driver Conductor Sender Receiver Conductor Receiver Sender Client Publisher Conductor Conductor Subscriber Subscriber Publisher Admin Events

Architecture

Admin Events Media IPC Log Buffer IPC Ring/Broadcast Buffer Media (UDP, InfiniBand, PCI-e 3.0) Function/Method Call Volatile Fields & Queues

slide-36
SLIDE 36

Much, much more… but…

slide-37
SLIDE 37

Java 8…

slide-38
SLIDE 38

What have we learned?

slide-39
SLIDE 39

Stream API

Too much garbage! Immature for now…

slide-40
SLIDE 40

JMC

Very handy. Use it!

slide-41
SLIDE 41

LOCK XADD

AtomicLong.getAndIncrement() Unsafe.getAndAddLong(…) Replacing a CAS on x86!! HOT!!

slide-42
SLIDE 42

And, of course…

slide-43
SLIDE 43

Lambdas [& Allocation]

slide-44
SLIDE 44

Design Principles

  • 1. Garbage free in steady state running
  • 2. Apply Smart Batching in the message path
  • 3. Wait-free algorithms in the message path
  • 4. Non-blocking IO in the message path
  • 5. No exceptional cases in the message path
  • 6. Apply the Single Writer Principle
  • 7. Prefer unshared state
  • 8. Avoid unnecessary data copies

https://github.com/real-logic/Aeron/wiki/Design-Principles

slide-45
SLIDE 45

public void something() { List<Object> l = new ArrayList<>(); final int value = 10;

  • l.forEach(this::func);

l.forEach((o) -> staticMethod(value)); l.forEach((o) -> this.func()); }

slide-46
SLIDE 46

http://mail.openjdk.java.net/pipermail/lambda-dev/2014-August/012097.html

public void something() { List<Object> l = new ArrayList<>(); final int value = 10;

  • l.forEach(this::func);

l.forEach((o) -> staticMethod(value)); l.forEach((o) -> this.func()); }

http://www.infoq.com/articles/Java-8-Lambdas-A-Peek-Under-the-Hood

Each time something called, it allocates 3 lambdas!!!

slide-47
SLIDE 47

http://mail.openjdk.java.net/pipermail/lambda-dev/2014-August/012097.html

public void something() { List<Object> l = new ArrayList<>(); final int value = 10;

  • l.forEach(this::func);

l.forEach((o) -> staticMethod(value)); l.forEach((o) -> this.func()); }

http://www.infoq.com/articles/Java-8-Lambdas-A-Peek-Under-the-Hood

Why? Capturing this or value

slide-48
SLIDE 48

But… Lambdas Optimize Well

slide-49
SLIDE 49

More things about Good Ol’ Java…

(not Java 8 specific)

slide-50
SLIDE 50

Lack of Unsigned Types

slide-51
SLIDE 51

51

slide-52
SLIDE 52

Lack of [Efficient] Primitive Maps

  • Map<int,? extends Object>

Map<long,? extends Object>

http://openjdk.java.net/jeps/8046267

slide-53
SLIDE 53

public static void main(final String[] args) { byte a = 0b0000_0001; byte b = 0b0000_0010;

  • byte flags = a | b;
  • System.out.printf(

"flags=%s\n", Integer.toBinaryString(flags));
 }

slide-54
SLIDE 54

public static void main(final String[] args) { byte a = 0b0000_0001; byte b = 0b0000_0010;

  • byte flags = a | b;
  • System.out.printf(

"flags=%s\n", Integer.toBinaryString(flags));
 }

slide-55
SLIDE 55

public void main(final String[] args) { byte a = 0b0000_0001; byte b = 0b0000_0010;

  • byte flags = a | b;
  • System.out.printf(

"flags=%s\n", Integer.toBinaryString(flags));
 }

Error:(8, ¡24) ¡java: ¡incompatible ¡types: ¡ ¡ possible ¡lossy ¡conversion ¡from ¡int ¡to ¡byte

Does not inspire confidence…

slide-56
SLIDE 56

NIO

slide-57
SLIDE 57

To an old network hacker, feels a bit “phoned” in…

slide-58
SLIDE 58

ByteBuffer & “Strangs”

slide-59
SLIDE 59

public void send(final String s) { // b be reused ByteBuffer

  • // allocating hot mess

b.put(s.getBytes());

  • // what could be… but JDK playin…

b.put(s); }

http://mail.openjdk.java.net/pipermail/core-libs-dev/2014-September/028837.html

Allocate mutable from immutable…

slide-60
SLIDE 60

public String receive(final ByteBuffer b) { // could reuse, but that’s HARD byte[] tmp = new byte[b.remaining()];

  • // allocating hot mess

b.get(tmp, offset, length); return new String(tmp);

  • // what could be… but JDK playin…

return new String(b, offset, length); }

http://mail.openjdk.java.net/pipermail/core-libs-dev/2014-September/028837.html

Yes, Charset, yes…

slide-61
SLIDE 61

STRINGS!

slide-62
SLIDE 62

NIO & Locks

slide-63
SLIDE 63

When Threads Collide…

slide-64
SLIDE 64

“You can’t fight in here, this is the War Room!”

slide-65
SLIDE 65

DatagramChannelImpl

http://grepcode.com/file/repository.grepcode.com/java/root/jdk/openjdk/8-b132/sun/nio/ch/DatagramChannelImpl.java#DatagramChannelImpl

public int write(ByteBuffer buf) { synchronized (writeLock) { synchronized (stateLock) { … } … } }

  • public int read(ByteBuffer buf)

{ synchronized (readLock) { synchronized (stateLock) { … } … } } send & receive are similar

slide-66
SLIDE 66

Why would we care?

slide-67
SLIDE 67

Firewall Traversal

slide-68
SLIDE 68

SelectorImpl & Locks

http://grepcode.com/file/repository.grepcode.com/java/root/jdk/openjdk/8-b132/sun/nio/ch/SelectorImpl.java#SelectorImpl

Second verse, same as the first…

slide-69
SLIDE 69

FileChannel.transferTo & DatagramChannel

slide-70
SLIDE 70

FileChannel.transferTo & DatagramChannel

Observed on Mac: call sendfile, get EINVAL, call sendto

  • ver and over

Platform Dependent, sure

slide-71
SLIDE 71

SelectorImpl & HashSet Garbage

http://grepcode.com/file/repository.grepcode.com/java/root/jdk/openjdk/8-b132/sun/nio/ch/SelectorImpl.java#SelectorImpl

slide-72
SLIDE 72
slide-73
SLIDE 73

public void handleFds(Selector selector) { final Set<SelectionKey> keys = selector.selectedKeys();

  • if (!keys.isEmpty())

{ final Iterator<SelectionKey> iter = keys.iterator(); while (iter.hasNext()) { … } } }

slide-74
SLIDE 74

Whoa!… Java 8-ize!

slide-75
SLIDE 75

public void handleFds(Selector selector) { final Set<SelectionKey> keys = selector.selectedKeys();

  • keys.forEach((key) -> { … });

} HashSet usage means node garbage so… Borrow from netty.io & replace publicSelectedKeys & selectedKeys

https://github.com/netty/netty/blob/master/transport/src/main/java/io/netty/channel/nio/NioEventLoop.java

https://github.com/netty/netty/blob/master/transport/src/main/java/io/netty/channel/nio/SelectedSelectionKeySet.java

slide-76
SLIDE 76
slide-77
SLIDE 77

public void handleFds(Selector selector) { // optimize underneath, more usable API selector.forEachSelectedKey( (key) -> { … }); } What it could be… and something optimized underneath

slide-78
SLIDE 78

Just the tip … but some times …

slide-79
SLIDE 79

But what can happen when you get it right?

slide-80
SLIDE 80
slide-81
SLIDE 81

20+ million msgs/sec

single stream (NO contention) 40 byte messages

slide-82
SLIDE 82
slide-83
SLIDE 83

Not Just Java

C++, Erlang, Go, … InfiniBand, PCI-e 3.0, …

slide-84
SLIDE 84

@toddlmontgomery

Questions?

  • Aeron https://github.com/real-logic/Aeron
  • Kaazing http://www.kaazing.com
  • SlideShare http://www.slideshare.com/toddleemontgomery
  • Twitter @toddlmontgomery

Thank You!