High Performance Network Applications in the Capital Markets Todd - - PowerPoint PPT Presentation

1 1

Todd L. Montgomery VP Architecture, Messaging Business Unit @toddlmontgomery

High Performance Network Applications in the Capital Markets

2 2

Why do Developers use Messaging?

Message-Oriented Middleware (MOM)

• Abstraction (Pub-Sub, Req/Resp, Queuing)
• Separate physical systems from communication
• Easily modify logic and scale applications
• Functionality
• Guaranteed delivery, fault tolerance, load balancing…
• Efficiency
• Well designed messaging systems reduce infrastructure
• Leverage broad, deep and detailed expertise
• Focus on core competencies, Faster Time-to-Market

2

3 3

Market Data Growth

Data Deluge

5 7 10 13 26 120 160 310 559 696 1,100 1,562 1,925 2,562 3,410 4,380 5,957 7,174 Dec-00 Dec-01 Dec-02 Dec-03 Dec-04 Dec-05 Dec-06 Dec-07 Dec-08 Dec-09 Dec-10 Dec-11

Total Options Equities

Aggregated One Minute Peak Messages Per Second Rates Arca, CTS, CQS, OPRA, NQDS (in thousands) > 1Terabyte of Data per Day

4 4

The Trader

Why Latency Matters

Execution Execution

Fast Slow Starting Line

Feed Handler Feed Handler

Ultra Messaging TIBCO RV and EMS

Got INFA at 40.00 Got INFA at 41.00

You Lost!

INFA at 40.00

Market Data Market Data

5 5

The Exchange

Why Latency Matters

Exchanges

Trader Order Cancel

Alpha Ultra Messaging Tango TIBCO RV / EMS Hotel Homegrown

You Both Lost! Acknowledge

6 6

(Ultra) Low Latency Timeline

Race to Zero – Less than 8 years, 10,000x-100,000x decrease! ≤2003 (Ethernet) 4-5 ms 2004 (Ethernet) 200 μs 2008 (IB) 10 μs 2010 (10G,IB) 2 μs 2010 (IPC) <400 ns 2011 (ITC) <50 ns Application to Application Latency

Predictions – Technology

• <1 μs Eth (2012)
• <500 ns Eth (2015)
• <100 ns Eth (2020)

Predictions – Technique

• <100 ns IPC (2012)
• 1G mps ITC (2012)

IPC/ITC only Limited by CPU!

7 7

Legacy Messaging Designs

Before 2004

6 Data Hops 4 Data Hops Daemon Based Design Broker Based Design

8 8

2004 – Need for a State Change

More Efficient, More Scalable, More MORE…

• Motivations / Challenges
• Systems not scaling to todays (yet alone tomorrows!)

demands

• Systems not resilient to failure
• Trends:
• Need Efficiency, Need Consolidation, More with Less, Need

Competitive Advantage (No Vendor Innovation)

• Broker-based Solutions are a Bottleneck
• Broker is a source of contention that limits scaling
• Broker failure disastrous to latency and stability

Remove the Broker from the Message Path!

9 9

Shared Nothing Messaging

MOM for Todays Demands

• Peer-to-Peer Messaging
• No broker, No daemons
• Direct connectivity between sources and receivers
• Parallel Persistence
• Broker out of message path and off to the side
• Broker consulted only for recovery
• Evolution of Queuing
• Single Messaging API across all Use Cases
• Source-based (vs. Immediate), Event Driven
• No need for separate Queuing (or PTP) API

10 10

Topic Resolution

Connecting Sources and Receivers (Peer-to-Peer)

SZ SY RZ RZ SX RY RX RX SZ

“Service” Location Paradigms

• Static – manual, difficult scaling with topics
• Server-based – (non)caching variants
• Multicast – (un)reliable variants

Traditionally, brokers handled the task of providing transparent connectivity between sources and receivers Separate the message delivery path and the topic discovery mechanism! Avoid including topic string in each message!

11 11

Data Transport Choices

Customization of Connectivity

• Transport Types – No One Size Fits All!
• Unicast (Optimize for single receivers)
• TCP (with varying buffering behaviors), Reliable Unicast (without

congestion control)

• Multicast (Optimize for multiple receivers)
• (Un)Reliable Multicast (NAK-based)
• Intra-Host (Optimize for lowest latency)
• IPC (Shared Memory), Inter-Thread (ITC)
• Source Configuration
• Runtime choice

12 12

Less Controlled Infrastructures

Architecture for Conflation and Rate Adaptation

SZ DS RZ RZ RZ

…

RZ

TCP

All Receivers are Not Equal!

• Desktops
• Web (HTML5/WebSockets Ideally)
• Mobile Apps

Conflation

• Conflate Data from multiple

buffered messages into one

• Data Representation Specific

Rate Adaptation

• “Non-”Intelligent Data Drops
• Tail, Oldest, Head, etc.
• Per-Topic vs. Per-Receiver vs. Per-

Connection Need Per-Receiver backpressure in

• rder to adapt. TCP provides ideal

flow and congestion control in these environments and thus ideal backpressure signaling.

13 13

Traditional Persistence

Store and Forward Architecture

SZ

Broker

RZ RZ

Brokered Architecture Limits

• Broker is point of contention
• Slow receiver impacts source and, more

importantly, other receivers

• Broker typically SAN backed (scaling limited)
• Recovery is “pushed” to receiver by broker

Receiver/Delivery Durability

• Receiver can crash or go down gracefully

without loss of messages upon restart

• Recovery is the act of restarting and

recovering missed messages

• Durability can be extended to Sources also

Deployments can only scale by adding brokers and splitting the topic space

14 14

Parallel Persistence

Durable Delivery without Penalty

SZ

Store

RZ RZ

Store Store

Store not in the Message Path

• Stores receive data in parallel to receivers
• Consumption Feedback (ACKs) are out-of-band
• Recovery can occur in parallel to “live” data delivery
• Receiver-driven recovery
• Receivers pull data from stores
• Stores maintain much less state and do much less
• No need to track receiver recovery, for example
• Recovery does not impact source or other non-

recovering receivers

• Dissemination from source to stores and receivers

uses normal peer-to-peer messaging

Consumption Information

Store ≠ Broker

Stores do less work, maintain less state, and can scale!

15 15

Quorum

Shared Nothing Approach to Persistence

Resiliency

• Avoids “Split-Brain” (majority must be reachable post failure)
• Stores persist locally independently
• Only need Quorum (majority) to withstand failure of minority
• Zero Latency Failover – no need to stop or change behavior

Performance

• Per-Message Striping (+50% per store as shown)

SZ

Store3 Store2 Store1

Message 1 Message 2 Message 3 Message 4

… … …

16 16

Consensus

Receiver Recovery and Arbitration

SZ

Store

RZ RZ

Store Store

Receiver Recovery

• Receivers ask Stores for message consumption

status and take majority or highest (arbitration)

• Receivers “pull” messages from stores
• Load balancing across Stores to spread out

impact of recovery

• Rate of recovery up to individual receivers
• Rate of recovery not bound by individual store
• Handling the “live” stream from the Source
• Ignore it or Buffer it (up to individual receiver)
• Seamless cutover from recovery to live
• Source too fast?
• Receiver can ignore live stream and pull from

stores at slower pace

Live Recovering

17 17

Messaging API – Sending

Simplifying the Semantics – Publish/Subscribe

send(“topic A”, data, length); send(“topic B”, dataB, lengthB); srcA = create_src(“topic A”); srcB = create_src(“topic B”); … send(srcA, data, length); send(srcB, dataB, lengthB); … delete_src(srcA); delete_src(srcB);

Immediate Sends Source-Based Sends JMS

Create MessageProducer without Destination and specify Destination on each send

JMS

Create Topic and TopicPublisher

Source-Based APIs

Can leverage Topic Resolution in order to reduce message path latency

18 18

Messaging API – Receiving

Simplifying the Semantics – Publish/Subscribe

int msg_proc(msg *m, void *cd) { /* handle m based on cd value (rA_state or rB_state) and/or m contents */ } … rcv1 = create_rcv(“topic A”, msg_proc, rA_state); Rcv2 = create_rcv(“topic B”, msg_proc, rB_state); …

Event-Driven Reception

How do you handle receiving on thousands to millions of topics?

JMS

Create Topic and TopicSubscriber Attach MessageListener

19 19

Queuing Semantics

Load Balancing + De-Coupling

• What semantics are needed for Queuing?
• Load Balancing (Once-and-Only-Once)
• Decoupling
• Source Rate vs. Receiver Consumption Rate
• Source Lifetime vs. Receiver Lifetime
• What APIs are needed for Queuing?
• JMS has the Point-to-Point API
• PTP and Pub/Sub share most calls and interfaces

Does this need to be different than Pub/Sub?!?

20 20

Queuing is Dead, Long Live Queuing!

No Need For Point-to-Point to be Different

Queuing

Sources send to Queues Receivers receive from Queues

Publish/Subscribe

Sources send to Topics Receivers receive from Topics

Replace “Queue” with “Topic”

Single Semantic – Publish/Subscribe

• A queue can be considered a topic
• Need Load Balancing per topic
• Need Rate and Lifetime Decoupling per topic

Point-to-Point API – Redundant

• Subsume the PTP receive call into Pub/Sub

21 21

Persistence + Queuing Semantics

Load Balancing + De-Coupling

SZ

Store

RZ RZ

Store Store

Assignment and Consumption

Load Balancing

• Assignment separate from Data Dissemination
• Source Assigned
• Receivers up-to-pace
• Consumption can backpressure source
• Store Assigned
• Receivers request messages (i.e. pull)
• Assignments sent out-of-band from Data

Rate and Lifetime Decoupling Already Done by Parallel Persistence!

22 22

Next-Generation APIs

MOM Evolution

• Right Value
• Not more layers of Abstraction, it’s about Complimentary

Functionality

• Actor Model
• Built on message passing
• Most developers using messaging APIs are already doing it

without knowing it

• Right metaphors for truly useful ESBs

23 23