[PPT] - String-diagram semantics for functional languages with data-flow PowerPoint Presentation

SLIDE 1

String-diagram semantics for functional languages with data-flow

Steven Cheung & Dan Ghica & Koko Moruya University of Birmingham 9th Sep 2017

SLIDE 2

GoI machine

[Mackie ’95] [Danos & Regnier ’99]

Geometry of interaction [Girard ’89]
a λ-term → a graph (proof-net)
evaluation = specific path in the fixed graph
✔ call-by-name, ? call-by-value

SLIDE 3

Dynamic GoI machine

[Muroya & Ghica ’17]

Framework for defining FPL semantics
Combine token passing & graph rewriting
different interleaving = different strategy

SLIDE 4

Dynamic GoI machine

[Muroya & Ghica ’17]

@ D + λ + C D D ! D D 1 ! 1 !

(λx.x + x) (1 + 1)

SLIDE 5

Dynamic GoI machine

[Muroya & Ghica ’17]

@ D + λ + C D D ! D D 1 ! 1 !

(λx.x + x) (1 + 1)

SLIDE 6

Dynamic GoI machine

[Muroya & Ghica ’17]

@ D + λ + C D D ! D D 1 ! 1 !

(λx.x + x) (1 + 1) → (λx.x + x) 2

@ D ! λ + C D D ! 2

SLIDE 7

Dynamic GoI machine

[Muroya & Ghica ’17]

@ D ! λ + C D D ! 2

(λx.x + x) (1 + 1) → (λx.x + x) 2

SLIDE 8

Dynamic GoI machine

[Muroya & Ghica ’17]

@ D ! λ + C D D ! 2

(λx.x + x) (1 + 1) → (λx.x + x) 2 → x + x [x ↦ 2]

+ 2 ! D D C

SLIDE 9

Dynamic GoI machine

[Muroya & Ghica ’17]

+ 2 ! D D C

(λx.x + x) (1 + 1) → (λx.x + x) 2 → x + x [x ↦ 2]

SLIDE 10

Dynamic GoI machine

[Muroya & Ghica ’17]

+ 2 ! D D C

(λx.x + x) (1 + 1) → (λx.x + x) 2 → x + x [x ↦ 2] → x + 2 [x ↦ 2]

+ 2 ! D D C ! 2

SLIDE 11

Dynamic GoI machine

[Muroya & Ghica ’17]

+ 2 ! D D C ! 2

(λx.x + x) (1 + 1) → (λx.x + x) 2 → x + x [x ↦ 2] → x + 2 [x ↦ 2] → 2 + 2

SLIDE 12

Dynamic GoI machine

[Muroya & Ghica ’17]

+ 2 ! D D C ! 2

(λx.x + x) (1 + 1) → (λx.x + x) 2 → x + x [x ↦ 2] → x + 2 [x ↦ 2] → 2 + 2 → 4

! 4

SLIDE 13

Dynamic GoI machine

[Muroya & Ghica ’17]

@ D + λ + C D D ! D D 1 ! 1 !

(λx.x + x) (1 + 1)

SLIDE 14

DGoIM & data-flow model

Machine graph ~ data-flow graph
GoI-style semantics
natural & intuitive
make the complex operational semantics tractable
Examples:
PL for Machine learning
PL for Self-adjusting computation

SLIDE 15

Machine learning

m(x) = W * x + b
init: W = 1, b = 0
tune the value of W and b to minimise the loss function
TensorFlow

SLIDE 16

TensorFlow

import tensorflow as tf # Model parameters W = tf.Variable([1], dtype=tf.float32) b = tf.Variable([0], dtype=tf.float32) # Model input and output x = tf.placeholder(tf.float32) linear_model = W * x + b y = tf.placeholder(tf.float32) # loss loss = tf.reduce_sum(tf.square(linear_model - y)) # sum of the squares # optimizer

ptimizer = tf.train.GradientDescentOptimizer(0.01)

train = optimizer.minimize(loss) # training data x_train = [1, 2, 3, 4] y_train = [0, -1, -2, -3] # training loop init = tf.global_variables_initializer() sess = tf.Session() sess.run(init) # reset values to wrong for i in range(1000): sess.run(train, {x: x_train, y: y_train}) # evaluate training accuracy curr_W, curr_b, curr_loss = sess.run([W, b, loss], {x: x_train, y: y_train}) print("W: %s b: %s loss: %s"%(curr_W, curr_b, curr_loss))

SLIDE 17

TensorFlow

import tensorflow as tf # Model parameters W = tf.Variable([1], dtype=tf.float32) b = tf.Variable([0], dtype=tf.float32) # Model input and output x = tf.placeholder(tf.float32) linear_model = W * x + b y = tf.placeholder(tf.float32) # loss loss = tf.reduce_sum(tf.square(linear_model - y)) # sum of the squares # optimizer

ptimizer = tf.train.GradientDescentOptimizer(0.01)

train = optimizer.minimize(loss) # training data x_train = [1, 2, 3, 4] y_train = [0, -1, -2, -3] # training loop init = tf.global_variables_initializer() sess = tf.Session() sess.run(init) # reset values to wrong for i in range(1000): sess.run(train, {x: x_train, y: y_train}) # evaluate training accuracy curr_W, curr_b, curr_loss = sess.run([W, b, loss], {x: x_train, y: y_train}) print("W: %s b: %s loss: %s"%(curr_W, curr_b, curr_loss))

SLIDE 18

TensorFlow

import tensorflow as tf # Model parameters W = tf.Variable([1], dtype=tf.float32) b = tf.Variable([0], dtype=tf.float32) # Model input and output x = tf.placeholder(tf.float32) linear_model = W * x + b y = tf.placeholder(tf.float32) # loss loss = tf.reduce_sum(tf.square(linear_model - y)) # sum of the squares # optimizer

ptimizer = tf.train.GradientDescentOptimizer(0.01)

train = optimizer.minimize(loss) # training data x_train = [1, 2, 3, 4] y_train = [0, -1, -2, -3] # training loop init = tf.global_variables_initializer() sess = tf.Session() sess.run(init) # reset values to wrong for i in range(1000): sess.run(train, {x: x_train, y: y_train}) # evaluate training accuracy curr_W, curr_b, curr_loss = sess.run([W, b, loss], {x: x_train, y: y_train}) print("W: %s b: %s loss: %s"%(curr_W, curr_b, curr_loss))

SLIDE 19

TensorFlow

import tensorflow as tf # Model parameters W = tf.Variable([1], dtype=tf.float32) b = tf.Variable([0], dtype=tf.float32) # Model input and output x = tf.placeholder(tf.float32) linear_model = W * x + b y = tf.placeholder(tf.float32) # loss loss = tf.reduce_sum(tf.square(linear_model - y)) # sum of the squares # optimizer

ptimizer = tf.train.GradientDescentOptimizer(0.01)

train = optimizer.minimize(loss) # training data x_train = [1, 2, 3, 4] y_train = [0, -1, -2, -3] # training loop init = tf.global_variables_initializer() sess = tf.Session() sess.run(init) # reset values to wrong for i in range(1000): sess.run(train, {x: x_train, y: y_train}) # evaluate training accuracy curr_W, curr_b, curr_loss = sess.run([W, b, loss], {x: x_train, y: y_train}) print("W: %s b: %s loss: %s"%(curr_W, curr_b, curr_loss))

SLIDE 20

DSL for machine learning

let loss f p = … in let grad_desc f p loss rate = … in let linear_model x = {1} * x + {0} in let f@p = linear_model in let linear_model’ = f (grad_desc f p loss rate) in …

SLIDE 21

DSL for machine learning

let loss f p = … in let grad_desc f p loss rate = … in let linear_model x = {1} * x + {0} in let f@p = linear_model in let linear_model’ = f (grad_desc f p loss rate) in …

SLIDE 22

DSL for machine learning

let f@p = linear_model in … f ≜ λp. λx. p[0] * x + p[1] p ≜ [1, 0] “Abductive” decoupling let loss f p = … in let grad_desc f p loss rate = … in let linear_model x = {1} * x + {0} in let f@p = linear_model in let linear_model’ = f (grad_desc f p loss rate) in …

SLIDE 23

DSL for machine learning

let f@p = linear_model in … f ≜ λp. λx. p[0] * x + p[1] p ≜ [1, 0] “Abductive” decoupling let loss f p = … in let grad_desc f p loss rate = … in let linear_model x = {1} * x + {0} in let f@p = linear_model in let linear_model’ = f (grad_desc f p loss rate) in …

SLIDE 24

Abductive decoupling

let f@p = λx. {1} * x + {0} in f p

@ λ ! A D @ D ! ? D ? ? ! λ ? D + * ᗡ ᗡ D ¿ ¿ ! ¿ ¿ ¡ ¡ 1

SLIDE 25

Abductive decoupling

@ λ ! A D @ D ! ? D ? ? ! λ ? D + * ᗡ ᗡ D ¿ ¿ ! ¿ ¿ ¡ ¡ 1

D A ! @ D ! ? D ? ? ! λ ? D + * ᗡ ᗡ D ¿ ¿ ! ¿ ¿ ¡ ¡ 1

SLIDE 26

Abductive decoupling

D A ! @ D ! ? D ? ? ! λ ? D + * ᗡ ᗡ D ¿ ¿ ! ¿ ¿ ¡ ¡ 1

SLIDE 27

Abductive decoupling

D A ! @ D ! ? D ? ? ! λ ? D + * ᗡ ᗡ D ¿ ¿ ! ¿ ¿ ¡ ¡ 1

SLIDE 28

Abductive decoupling

D A ! @ D ! ? D ? ? ! λ ? D + * ᗡ ᗡ D ¿ ¿ ! ¿ ¿ ¡ ¡ 1 D @ D ! ! D ? ! λ D P2 ! λ ? D + * D D D ? ? ! ? ? [1,0] ! 1 ¡ ¡ Ɔ0 Ɔ0

SLIDE 29

Abductive decoupling

D A ! @ D ! ? D ? ? ! λ ? D + * ᗡ ᗡ D ¿ ¿ ! ¿ ¿ ¡ ¡ 1 D @ D ! ! D ? ! λ D P2 ! λ ? D + * D D D ? ? ! ? ? [1,0] ! 1 ¡ ¡ Ɔ0 Ɔ0

f ≜ λp. λx. p[0] * x + p[1] p ≜ [1, 0]

SLIDE 30

Self-adjusting Computation

[Acar ’05]

Spreadsheet meets functional programming
Adjust the output with minimal re-computation

let x = 1, y = 2, m = x + 1, n = y + 1 in m + n

1

x

2

m

+ 1 2

y

3

n

+ 1 5

z

+

SLIDE 31

Self-adjusting Computation

[Acar ’05]

Spreadsheet meets functional programming
Adjust the output with minimal re-computation

let x = 3, y = 2, m = x + 1, n = y + 1 in m + n

3

x

4

m

+ 1 2

y

3

n

+ 1 7

z

+

SLIDE 32

Self-adjusting Computation

[Acar ’05]

Spreadsheet meets functional programming
Adjust the output with minimal re-computation
↓ re-computation = ↑ performance
Dynamic dependency graph + memoisation

let x = 3, y = 2, m = x + 1, n = y + 1 in m + n

3

x

4

m

+ 1 2

y

3

n

+ 1 7

z

+

SLIDE 33