From HFT to Laplace Demon @abifet When timed data technology - - PowerPoint PPT Presentation

▶
from hft to laplace demon
SMART_READER_LITE
LIVE PREVIEW

From HFT to Laplace Demon @abifet When timed data technology - - PowerPoint PPT Presentation

Mar 19, 2024 451 likes •934 views

From HFT to Laplace Demon @abifet When timed data technology curves the market @erichoresnyi HFT in the hey days High Frequency Trading 5ms=20m$ Source: Tabb Group $100trn HFT context Fidelity StateStreet GS BoNY Blackrock* JPM

slide-1
SLIDE 1

From HFT to Laplace Demon

When timed data technology curves the market

@erichoresnyi @abifet

slide-2
SLIDE 2

HFT in the hey days

5ms=20m$

Source: Tabb Group

High Frequency Trading

slide-3
SLIDE 3

$100trn

CapGroup

Vanguard

StateStreet

BoNY

JPM

Pimco

Fidelity GS

Prudential

AUM>$1trn, source: Towers Watson

Blackrock*

*Blackrock is actually headquartered in NY, main AUM coming from ETF/ passive originally BGI in SF

Approx 3xGDP in USA ie 155k$/hab

HFT context

slide-4
SLIDE 4

Liquidity Flow

Buy Side 2 Buy Side 1 NASDAQ NYSE NSX Sell Side 2 Sell Side 1 70% Algo PCX

HFT context

slide-5
SLIDE 5

Order Flow

NY

NJ

CT

IL

Sell Side 2 Sell Side 1 Market Maker

CA

HFT context

slide-6
SLIDE 6

PCX INET BRUT

NASDAQ

CME

CBOT

ICE

BATS

NYSE ARCA

CBOE NYMEX IEX

Cambrian Explosion

Reg.ATS'98-Reg.NMS'05

HFT context

slide-7
SLIDE 7

Infra view

NJ NJ NY IL CO POP Fiber 20ms 2ms 2ms 4ms

HFT context

slide-8
SLIDE 8

1

HFT: Proximity

NY NJ CT IL 16 2 1 1

Host in Network Nodes, then Exchanges

HFT

slide-9
SLIDE 9

+ Serialization Latency = + Processing Propagation

slide-10
SLIDE 10
slide-11
SLIDE 11

HFT: Ultra

NY NJ CT IL HFT

Dark Fiber

1 1 15

slide-12
SLIDE 12

Buy-Side view of HFT

slide-13
SLIDE 13

It's not a ghost...

slide-14
SLIDE 14

HFT: Straight Fiber

1,000 miles > 825 miles 14.5 ms > 11.5 ms

slide-15
SLIDE 15

HFT: Microwaves

11.5 > 8.5ms N:1.33 > 1.0003 v = c/n

slide-16
SLIDE 16

HFT: FPGA

Nanosecs

slide-17
SLIDE 17

Choose your lane

HFT <> Algo Trading "Once you get into milliseconds it's almost not HFT any more"

slide-18
SLIDE 18

Spacetime is relative

Market Events: [ct,x,y,z]

slide-19
SLIDE 19

Speed curves spacetime

HFT built a wormhole to win on [ct',x,y,z] events

slide-20
SLIDE 20

Mass curves spacetime

AI builds a blackhole by massively processing [ct,x,y,z] events G + Λg = T

µν µν c4 8πG µν

{

slide-21
SLIDE 21

Laplace Demon

The endgame of Determinism

∀ [ct,x,y,z] ∈ Rn ⊢ ∀ [ct',x',y',z']

slide-22
SLIDE 22

The Endgame 1/3

Event Machine View

slide-23
SLIDE 23

The Endgame 2/3

Loss aka Cost Function = J(θ) : distance points to line

Graph View : Regression

slide-24
SLIDE 24

⎣ ⎢ ⎢ ⎡ w w ...w

1,1 1,2 1,n

w w ...w

2,1 2,2 2,n

w w ...w

...,1 ...,2 ...,n

w w ...w

n,1 n,2 n,n ⎦

⎥ ⎥ ⎤

The Endgame 3/3

[x , x , ..., x

1 n]

Features Labels $AAPL $GOOG . $FB

⎣ ⎢ ⎢ ⎡y1 y2 . yn⎦ ⎥ ⎥ ⎤

Matrix view

Matrices of Weights

slide-25
SLIDE 25

AI not news to trading

+35% yoy for 20 years : $2,500 > $1,000,000

PhD Mathematics, Berkeley - String Theory Chern-Simons Form

slide-26
SLIDE 26

AI age:Gradient Descent

Follow the steepest slope, 100m+ features

α : Learning Rate, ∇ J : Gradient

slide-27
SLIDE 27

AI age:Back Propagation

Adapt weight to control error from previous layer's input, 150+ layers

Source: Neural Networks simulation by Matt Mazur at Emergent Mind

slide-28
SLIDE 28

AI age: GPU

From Final Fantasy to Autonomous Car

"The implementation of streaming algorithms, typied by highly parallel computations with little reuse of input data, has been widely explored on GPUs."(Stanford, 2004)

slide-29
SLIDE 29

Bullish Fitness Drill

3-Test 2-Validate 1-Train Overfitting?

slide-30
SLIDE 30

Bearish Fitness Drill

3-Test 2-Validate 1-Train Overfitting?

slide-31
SLIDE 31

Standard Approach

Batch-based, finite training sets, static models

Dataset Model

slide-32
SLIDE 32

Data Stream Approach

Infinite training sets, dynamic models

D D D D D D M M M M M M

slide-33
SLIDE 33

Approximation Algo

What is the largest number that we can store in 8 bits?

slide-34
SLIDE 34

Approximation Algo

What is the largest number that we can store in 8 bits?

slide-35
SLIDE 35

Approximation Algorithm

slide-36
SLIDE 36

Massive Online Analysis

slide-37
SLIDE 37

Stream Setting

Process an example at a time

Inspect it only once (at most)

Use a limited amount of memory Work in a limited amount of time Be ready to predict at any point

slide-38
SLIDE 38

Prequential Evaluation

Sequence of examples > Error of a model

slide-39
SLIDE 39

Command Line

java -cp .:moa.jar:weka.jar -javaagent:sizeofag.jar moa.DoTask EvaluatePrequential

  • l DecisionStump //training DecisionStump classifier ...
  • s generators.WaveformGenerator //...on WaveformGenerator data
  • n 100000 //using the first 100 thousand examples for testing
  • i 100000000 //training on a total of 100 million examples
  • f 1000000 //testing every one million examples

> dsresult.csv

slide-40
SLIDE 40

Resourceful

Classification Regression Concept Drift Sentiment Analysis Stock Price Alerting

slide-41
SLIDE 41

Simple

learner.getVotesForInstance(instance) learner.trainOnInstance(instance)

slide-42
SLIDE 42

Scalable

http://samoa-project.net

slide-43
SLIDE 43

An experiment

Public Stock Dataset MOA Regression Stock Price Error

slide-44
SLIDE 44

An Experiment

slide-45
SLIDE 45

The New HF Frontier: AI {API}

Sentiment Analysis Alerts Regression/Perceptron

slide-46
SLIDE 46

Fast vs Smart

Data Stream a compromise ct x,y,z HFT AI Data Stream

slide-47
SLIDE 47

Thanks!

Apache & Wikipedia Foundation : please donate! MOA, Kaggle & Giphy : please contribute!

Books & Lectures

Data Stream Mining, MOA team Yann LeCun Deep Learning Class, NYU Matt Mazure, Emergent Mind & Andrew Ng, Coursera on AI My Life as a Quant:Reflections on Physics&Finance, E.Derman The Value of a Millisecond: Finding the Optimal Speed of a Trading Infra., TabbGroup Flashboys, M.Lewis

Movies & Games

The Big Short, Back to the Future, Interstellar, The Black Hole, Harry Potter, Rocky, Into the Mind, Star Wars, Matrix; Final Fantasy