Visual Programming: Cooking the Spaghetti GOTO Chicago '13 Rick - - PowerPoint PPT Presentation

SLIDE 1

Visual Programming: Cooking the Spaghetti

GOTO Chicago '13

Rick Lane

rick.lane@tradingtechnologies.com @R1ck_L4n3 @trading_tech @GOTOChgo

SLIDE 2

First things first

• Visual Programming < Regular Programming (usually)
• But, there's a time and a place
• How to make it work, when it makes sense to work

SLIDE 3

Who I am

• Chief technology officer at Trading Technologies

○ TT makes software for professional traders around the world

• Background in wargaming: terrorist

network modeling and analysis, agent- based modeling, etc.

• Many similarities between these

industries: a knowledge gap between the idea generators and the technologists

SLIDE 4

Who I am not

• I am not a visual programming academic
• There is a sea of research on the various forms,

families, perils and merits of visual programming ○ You won’t find any of that here

Boursier, P. and M. Mainguenaud, Spatial Query Languages : Extended SQL vs. Visual Languages vs. Hypermaps, 5th International Symposium on Spatial Data Handling, Charleston, USA, August 3-7, 1992. Keywords and Phrases: VPL-II: Language Classifications Glinert, Ephraim P. and Charles D. Norton, Novis: A Visual Laboratory for Exploring the Design of Processor Arrays, Journal of Visual Languages and Computing, Vol. 3, Number 2, 135-159, June 1992. Keywords and Phrases: Novis, VPL-II.A.1: Concurrent languages, VPL-II.B.2: Iconic languages, VPL-IV.A: Computational approaches, VPL-VI.C.6: Visual representation issues Norton, Charles D. and Ephraim P. Glinert, A Visual Environment for Designing and Simulating Execution of Processor Arrays, 1990 IEEE Workshop on Visual Languages, Skokie, Illinois, 227-232, October 1990. Keywords and Phrases: Novis, VPL-II.A.1: Concurrent languages, VPL-II.B.2: Iconic languages, VPL-IV.A: Computational approaches, VPL-VI.C.6: Visual representation issues

• M. Paltrinieri, A Visual Environment for Constraint Programming (Poster Paper), 11th IEEE International Symposium on

Visual Languages, Darmstadt, Germany, September 5-9, 1995. Keywords and Phrases: VPL-II.A.2: Constraint-based languages Harvey, N. and J. Morris, NL: A General Purpose Visual Dataflow Language, Technical Report, University of Tasmania, October 1993. Keywords and Phrases: VPL-II.A.3: Data-flow languages, VPL-V.A: General-purpose languages Koelma, D., R. van Balen, and A. Smeulders, SCIL-VP: a multi-purpose visual programming environment, Proceedings of the 1992 ACM/SIGAPP Symposium on Applied Computing, 1188-1198, 1992. Keywords and Phrases: VPL-II.A.3: Data-flow languages, VPL-III.A.2: Procedural abstraction, VPL-VI.C.4: Scope Ambler, Allen L. and Margaret M. Burnett, Visual Languages and the Conflict Between Single Assignment and Iteration, 1989 IEEE Workshop on Visual Languages, Rome, Italy, 138-143, Oct. 1989. Keywords and Phrases: Forms/2, VPL-II.A.4: Form-based and spreadsheet-based languages, VPL-III.B: Control flow Ambler, Allen L. and Margaret M. Burnett, Visual Forms of Iteration that Preserve Single Assignment, Journal of Visual Languages and Computing, 1(2), Academic Press, June 1990. Keywords and Phrases: Forms/2, VPL-II.A.4: Form-based and spreadsheet-based languages, VPL-III.B: Control flow B u r n e t t , M a r g a r e t M . a n d A l l e n L . A m b l e r , G e n e r a l i z i n g E v e n t D e t e c t i

• n

a n d R e s p

• n

s e i n V i s u a l P r

• g

r a m m i n g L a n g u a g e s , 1 9 9 2 P r

• c

e e d i n g s

• f

A d v a n c e d V i s u a l I n t e r f a c e s I n t e r n a t i

• n

a l W

• r

k s h

• p

, R

• m

e , I t a l y , M a y 2 7

• 2

9 , 1 9 9 2 . K e y w

• r

d s a n d P h r a s e s : F

• r

m s / 3 , V P L

• I

I . A . 4 : F

• r

m

• b

a s e d a n d s p r e a d s h e e t

• b

a s e d l a n g u a g e s , V P L

• I

I I . A : A b s t r a c t i

• n

, V P L

• I

I I . E : E v e n t h a n d l i n g B u r n e t t , M a r g a r e t M . a n d A l l e n L . A m b l e r , A D e c l a r a t i v e A p p r

• a

c h t

• E

v e n t

• H

a n d l i n g i n V i s u a l P r

• g

r a m m i n g L a n g u a g e s , 1 9 9 2 I E E E W

• r

k s h

• p
• n

V i s u a l L a n g u a g e s , S e a t t l e , W A , 3 4

• 4

, S e p t . 1 5

• 1

8 , 1 9 9 2 . K e y w

• r

d s a n d P h r a s e s : F

• r

m s / 3 , V P L

• I

I . A . 4 : F

• r

m

• b

a s e d a n d s p r e a d s h e e t

• b

a s e d l a n g u a g e s , V P L

• I

I I . A : A b s t r a c t i

• n

, V P L

• I

I I . E : E v e n t h a n d l i n g Burnett, Margaret M. and Allen L. Ambler, Interactive Visual Data Abstraction in a Declarative Visual Programming Language, Journal of Visual Languages and Computing, 29-60, March 1994. Keywords and Phrases: Forms/3, VPL-II.A.4: Form-based and spreadsheet-based languages, VPL-III.A.1: Data abstraction, VPL-III.E: Event handling Carlson, Paul and Margaret Burnett, Integrating Algorithm Animation into a Declarative Visual Programming Language (poster paper), 1995 IEEE Symposium on Visual Languages, Darmstadt, Germany, September 1995. Keywords and Phrases: Forms/3, VPL-II.A.4: Form-based and spreadsheet-based languages Hays, Judith G. and Margaret M. Burnett, A Guided Tour of Forms/3, Oregon State University, Dept. of Computer Science, TR 95-60-6, June 1995. Keywords and Phrases: Forms/3, VPL-II.A.4: Form-based and spreadsheet-based languages Pandey, Rajeev and Margaret Burnett, Is It Easier to Write Matrix Manipulation Programs Visually or Textually? An Empirical Study, 1993 IEEE Symposium on Visual Languages, Bergen, Norway, August 24-27, 1993. Keywords and Phrases: Forms/3, VPL-II.A.4: Form-based and spreadsheet-based languages, VPL-VI.C.1: Cognitive and user-interface design issues Yang, S. and M. Burnett, From Concrete Forms to Generalized Abstractions through Perspective-Oriented Analysis of Logical Relationships, 1994 IEEE Symposium on Visual Languages, St. Louis, MO, 6-14, October 4-7, 1994. Keywords and Phrases: Forms/3, VPL-II.A.4: Form-based and spreadsheet-based languages Cordy, James R. and T. C. Nicholas Graham, GVL: Visual Specification of Graphical Output, Journal of Visual Languages and Computing, 3, 1992. Keywords and Phrases: VPL-II.A.5: Functional languages, VPL-V.E: User-interface generation languages Graham, T. C. Nicholas and Tore Urnes, Relational Views as a Model for Automatic Distributed Implementation of Multi- User Applications, Proceedings of the Fourth Conference on Computer-Supported Cooperative Work, Toronto, October 1992. Keywords and Phrases: VPL-II.A.5: Functional languages, VPL-V.E: User-interface generation languages Lakin, Fred, Computing with Text-Graphic Forms, Proceedings of the LISP Conference, Stanford University, August 1980. Keywords and Phrases: computing with text-graphic forms, visual pattern manipulation language, visual lisp, VPL-II.A.5: Functional languages, VPL-V.A: General-purpose languages, VPL-IV.D: Translators (interpreters and compilers) Standing, C. and G. Roy, A Functional, Visual Programming Interface to Geographical Information Systems, Proceedings of OZCHI, Melbourne, Australia, 209-214, Nov. 28 - Dec. 1, 1994. Keywords and Phrases: VPL-II.A.5: Functional languages Schuerr, Andreas, Introduction to PROGRES, an Attribute Graph Grammar Based Specification Language, Proc. WG'89 Workshop on Graphtheoretic Concepts in Computer Science, LNCS 411, (Nagl M., ed.), Springer Verlag, 151-165, 1989. Keywords and Phrases: VPL-II.A.6: Imperative languages, VPL-II.A.8: Multi-paradigm languages, VPL-II.A.11: Rule-based languages, VPL-II.B.1: Diagrammatic languages Schuerr, Andreas, PROGRES: A VHL-Language Based on Graph Grammars, Proc. 4th Int. Workshop on Graph-Grammars and Their Application to Computer Science, LNCS 532, Springer Verlag, 641-659, 1991. also: Technical Report AIB 90-16, RWTH Aachen, Germany, 1990. Keywords and Phrases: VPL-II.A.6: Imperative languages, VPL-II.A.8: Multi-paradigm languages, VPL-II.A.11: Rule-based languages, VPL-II.B.1: Diagrammatic languages S c h u e r r , A n d r e a s , R a p i d P r

• g

r a m m i n g w i t h G r a p h R e w r i t e R u l e s , U S E N I X S y m p

• s

i u m P r

• c

.

• n

V e r y H i g h L e v e l L a n g u a g e s ( V H L L ) , U S E N I X A s s

• c

i a t i

• n

, 8 3

• 1

, 1 9 9 4 . K e y w

• r

d s a n d P h r a s e s : V P L

• I

I . A . 6 : I m p e r a t i v e l a n g u a g e s , V P L

• I

I . A . 8 : M u l t i

• p

a r a d i g m l a n g u a g e s , V P L

• I

I . A . 1 1 : R u l e

• b

a s e d l a n g u a g e s , V P L

• I

I . B . 1 : D i a g r a m m a t i c l a n g u a g e s S c h u e r r , A n d r e a s , A . Z u e n d

• r

f , N

• n

d e t e r m i n i s t i c C

• n

t r

• l

S t r u c t u r e s f

• r

G r a p h R e w r i t i n g S y s t e m s , P r

• c

. W G ' 9 1 W

• r

k s h

• p

i n G r a p h

• t

h e

• r

e t i c C

• n

c e p t s i n C

• m

p u t e r S c i e n c e , L N C S 5 7 , S p r i n g e r V e r l a g , 4 8

• 6

2 , 1 9 9 1 . a l s

• :

T e c h n i c a l R e p

• r

t A I B 9 1

• 1

7 , R W T H A a c h e n , G e r m a n y , 1 9 9 1 . K e y w

• r

d s a n d P h r a s e s : V P L

• I

I . A . 6 : I m p e r a t i v e l a n g u a g e s , V P L

• I

I . A . 8 : M u l t i

• p

a r a d i g m l a n g u a g e s , V P L

• I

I . A . 1 1 : R u l e

• b

a s e d l a n g u a g e s , V P L

• I

I . B . 1 : D i a g r a m m a t i c l a n g u a g e s Lakin, Fred, Visual Grammars for Visual Languages, Proceedings of the American Association for Artificial Intelligence, Seattle, Washington, July 1987. Keywords and Phrases: visual grammar notation, declarative programming, visual rules, formal visual languages, informal conversational graphics, visual lisp, visual emacs, vmacs, VPL-IV.C: Parsing, VPL-VI.A: Formal definition of VPLs, VPL-II. A.7: Logic languages Meyer, Bernd, Beyond Icons: Towards New Metaphors for Visual Query Languages for Spatial Information Systems, Proceedings of the International Workshop on Interfaces to Database Systems, Glasgow, 113-135, July 1992. (published by Springer, London, 1993). Keywords and Phrases: VPL-II.A.7: Logic languages, VPL-II.B.1: Diagrammatic languages, VPL-V.B: Database languages

SLIDE 5

Low-latency trading & VPL

Communication Barriers Safety

SLIDE 6

Why I decided to build a VPL

• I didn't! - I stumbled upon it
• I was still learning the business
• I had insanely aggressive deadlines (see: minutes)

for writing and deploying production code It was built out of necessity

SLIDE 7

I got tired of these guys yelling at me

SLIDE 8

int qty = currOrderQty - _officialQty; if (qty > 0) { OrderProfileBase orderProfile = null; AutospreaderSyntheticOrderProfile aop = null; if (currInstrument.IsASEInstrument) { //aop = new AutospreaderSyntheticOrderProfile(currInstrument.OrderFeed, (AutospreaderInstrument)m_CurrInstrument.Instrument); aop = TopMostAlgorithm.CreateAutospreaderOrderProfile(currInstrument, routingInfo);

• rderProfile = aop;

} else {

• rderProfile = TopMostAlgorithm.CreateAlgoExchangeChildOrderProfile(currInstrument, routingInfo);

//orderProfile = new AlgoExchangeChildOrderProfile(currInstrument.OrderFeed, currInstrument.Instrument.InstrumentDefinition); AlgoExchangeChildOrderProfile algoExchangeChildOrderProfile = (AlgoExchangeChildOrderProfile)orderProfile; algoExchangeChildOrderProfile.LeaveWhenPaused = leaveOnPause; algoExchangeChildOrderProfile.LeaveWhenCanceled = leaveOnCancel; if (IsFlagColorSet) algoExchangeChildOrderProfile.Color = flagColor; }

• rderProfile.Action = OrderAction.Add;
• rderProfile.QuantityToWork = Quantity.FromInt(m_CurrInstrument.Instrument, qty);
• rderProfile.BuySell = m_Buy ? BuySell.Buy : BuySell.Sell;
• rderProfile.OrderTag = m_Flag;
• rderProfile.OrderType = m_OrderType;

if (_isGtc) {

• rderProfile.TimeInForce = new TimeInForce(TimeInForceCode.GoodTillCancel);

} if (m_LimitPriceSet) { m_CurrRoundedLimitPrice = FPOps.RoundedPrice(m_CurrInstrument.Instrument.InstrumentDetails, m_CurrLimitPrice, m_Buy);

• rderProfile.LimitPrice = m_CurrRoundedLimitPrice;

if (!orderProfile.LimitPrice.IsValid) { TopMostAlgorithm.StopStrategy(StrategyInterfaces.BaseStrategy.StopStrategyReason.UnrecoverableStrategyProblem, "Attempted to place an

• rder with an invalid price.");

return; } }

O r d e r Q u

• t

e r T i m e d H e d g e 2

• L

e g g e d S p r e a d Arb Routine

Synthetic Stop

Market Maker

Modules made my life easier

SLIDE 9

But modules weren't enough

• Bugs could still spell disaster
• I wanted the traders to be in control

○ Just piecing them together took work and programming know-how ○ This "work" wasn't going to add any business value

SLIDE 10

The VPL was born

• I built a GUI that let them visually link these modules

together

• I didn't know I was building a VPL
• Algo Design Lab (ADL)

SLIDE 11

The "Deutsch Limit"

“The problem with visual programming languages is that you can't have more than 50 visual primitives on the screen at the same time.”

• Peter Deutsch

SLIDE 12

The "Deutsch Limit"

• The limit of 50 primitives is arbitrary
• We rely on our IDE to help us solve this problem

○ "Find References/Usages" ○ "Go to Definition/Declaration" ○ "Find in Files" ○ Refactoring

• Mental Map

SLIDE 13

The solution is threefold

• 1. Domain specificity
• 2. Hiding the "dirty stuff"
• 3. Mitigating the "Deutsch Limit"

SLIDE 14

• 1. Domain specificity

"The ability for the general public to understand a VPL is inversely proportional to the likelihood that it will actually do something useful."

• me
• This is really no different from a well-made API

○ Abstracts away unnecessary granularity

• Unspecific: Scratch, AppInventor, etc.
• Specific: Lego Mindstorms

SLIDE 15

• 2. Hiding the "dirty stuff"
• All VPLs hide some of the generic dirty stuff
• Goes hand-in-hand with domain specificity

○ If not domain-specific, it's hard to identify the dirty stuff

• Examples of dirty stuff in automated trading

○ In-flight Orders ○ Fairness ○ "It just works"

SLIDE 16

• 3. Mitigating the "Deutsch Limit"
• Browser-based navigation
• Show Up/Downstream Blocks
• Block searching
• Componentization
• Breakpoints
• "Jump Blocks"

SLIDE 17

We want our users to eat fruit

Kathy Sierra - http://goo.gl/hKd95

• Cake vs. Fruit
• We want them to focus on idea generation, not

implementation

• It's also about safety

SLIDE 18

Let's do an experiment

• Build a simple trading strategy:

○ Place an order to buy 100 shares of stock (e.g. GOOG) at the last-traded-price (LTP) ○ Whenever the LTP of GOOG changes, we should change our order's price to match ○ If the LTP of GOOG *ever* falls below $770, we should cancel our order

• Scratch-->Domain Specific Language-->ADL

SLIDE 19

stock=GOOG;volume=5433728;LTP=782.03;

SLIDE 20

Domain Specific

SLIDE 21

Domain Specific

SLIDE 22

Domain Specific

SLIDE 23

Domain Specific

SLIDE 24

Domain Specific

SLIDE 25

Domain Specific

SLIDE 26

Domain Specific

SLIDE 27

Domain Specific

SLIDE 28

Domain Specific

SLIDE 29

Domain Specific

SLIDE 30

Domain Specific

SLIDE 31

Domain Specific

SLIDE 32

Domain Specific

SLIDE 33

Domain Specific

SLIDE 34

Domain Specific

SLIDE 35

Domain Specific

SLIDE 36

Algo Design Lab (ADL)

(that's it.)

SLIDE 37

Domain Specific

SLIDE 38

Domain Specific

SLIDE 39

• VPLs can work for a very narrow set of

industries and applications

• Domain specificity
• Hiding the dirty stuff
• Mitigating the "Deutsch Limit"
• Are our users fruit or cake eaters?

* Please exit through the back for processing

Wrap-up*

SLIDE 40

Questions?

SLIDE 41

Appendix

SLIDE 42

Flowcharts are evil

“The flowchart is a very poor abstraction of software structure.”

• Fred Brooks

SLIDE 43

Flowcharts are evil

• Poor representation of the actual effort needed to

realize the intended goal

• Flowcharts tend to be written after you write code,

not before

• Too many assumptions; too abstract