SLIDE 1
ON THE COST OF TYPE-TAG SOUNDNESS
Zeina Migeed Ben Greenman
ON THE COST OF TYPE-TAG SOUNDNESS
- 1. Tag soundness
- 2. Performance cost of soundness
- 3. Evaluation method
- 4. Conclusions
ON THE COST OF TYPE-TAG SOUNDNESS Ben Greenman Zeina Migeed ON - - PowerPoint PPT Presentation
ON THE COST OF TYPE-TAG SOUNDNESS Ben Greenman Zeina Migeed ON - - PowerPoint PPT Presentation
ON THE COST OF TYPE-TAG SOUNDNESS Ben Greenman Zeina Migeed ON THE COST OF TYPE-TAG SOUNDNESS 1. Tag soundness 2. Performance cost of soundness 3. Evaluation method 4. Conclusions TYPE-TAG SOUNDNESS Type Soundness e : If
SLIDE 2
SLIDE 3
TYPE-TAG SOUNDNESS
SLIDE 4
Type Soundness
No undefined behavior Type-based reasoning If then either:
- and
- diverges
- (division by zero, etc.)
⊢ e : τ ⊢ v : τ e v * e Error * e
SLIDE 5
If then either:
- and
- diverges
- (division by zero, etc.)
Type Soundness
⊢ e : τ e v * e Error * e ⊢ v : τ
SLIDE 6
If then either:
- and
- diverges
- (division by zero, etc.)
Tag Soundness
⊢ e : τ e v * e Error * e ⊢ v : ⌊τ⌋
SLIDE 7
If then either:
- and
- diverges
- (division by zero, etc.)
Tag Soundness
⊢ e : τ e v * e Error * e ⊢ v : ⌊τ⌋ ⌊Int⌋ = Int ⌊τ × τ'⌋ = Pair ⌊τ → τ'⌋ = Fun ... ⌊τ⌋ = K
SLIDE 8
Tag Soundness
No undefined behavior Tag-based reasoning
⌊Int⌋ = Int ⌊τ × τ'⌋ = Pair ⌊τ → τ'⌋ = Fun ... ⌊τ⌋ = K
If then either:
- and
- diverges
- ⊢ e : τ
⊢ v : ⌊τ⌋ e v * e Error * e
SLIDE 9
Types vs. Tags
If and then might be:
⊢ e : Int × Int e v *
Type Soundness Tag Soundness
(0, 0) (3, 2) (-7, 9) (0, 0) (3, 2) (-7, 9) ("A", 0) (0, (1, 2)) v
SLIDE 10
Types vs. Tags
If and then might be:
⊢ e : Int × Int
Type Soundness Tag Soundness
(0, 0) (3, 2) (-7, 9) (0, 0) (3, 2) (-7, 9) ("A", 0) (0, (1, 2)) v e v *
SLIDE 11
* fast * slow
SLIDE 12
Type Sound? Tag Sound?
* fast * slow
SLIDE 13
PERFORMANCE COST OF SOUNDNESS
SLIDE 14
e e e ?
Problem: Safe Interaction
τ τ τ
eτ
SLIDE 15
e e e
Gradual Typing
τ τ τ
eτ
λ
SLIDE 16
read()
Enter a value: >
User Input
e *
τ τ τ
SLIDE 17
unzip()
Deserialization
e *
τ τ τ
0110 1110 1011
SLIDE 18
v + v E[v,v] ... v'
Primitive Operations ( δ )
e *
Int Int Int
SLIDE 19
Unreliable Source
e
Enter a value: >
0110 1110 1011
E[•] ?
τ
eτ
λ
SLIDE 20
e v ?
Option 1: Trust
τ τ
SLIDE 21
e ?
Option 2: Check
τ
vτ ... vτ
SLIDE 22
e ?
Option 2: Check
τ
vτ ...
COST OF SOUNDNESS
vτ
SLIDE 23
?
Cost of Types ( )
Int × Int
(6,1)
Int × Int
(6,1)
Int × Int
(6,1)
Int × Int
(6,1) * slow
SLIDE 24
?
Cost of Tags ( )
Int × Int
(6,1)
Int × Int
(6,1) * fast
SLIDE 25
COST OF SOUNDNESS IN RETICULATED
SLIDE 26
e e e
Retic vs. Python
τ τ τ
eτ
SLIDE 27
Reticulated
def dist(pt : Tuple(Int,Int)) -> Int: x = pt[0] y = pt[1] return abs(x + y)
SLIDE 28
Reticulated
def dist(pt : Tuple(Int,Int)) -> Int: x = pt[0] y = pt[1] return abs(x + y) dist((0, 0)) *
SLIDE 29
Reticulated
def dist(pt : Tuple(Int,Int)) -> Int: x = pt[0] y = pt[1] return abs(x + y) dist("NaN") Expected Tuple *
SLIDE 30
Reticulated
def dist(pt : Tuple(Int,Int)) -> Int: x = pt[0] y = pt[1] return abs(x + y) dist((0, "NaN")) Expected Int *
SLIDE 31
Evaluation Method
SLIDE 32
- 1. Fully-Typed
SLIDE 33
- 2. Configurations
SLIDE 34
7s 11s 5s 4s 9s 2s 5s 24s 9s 14s 5s 21s 9s 6s 9s 8s
- 3. Measure
SLIDE 35
7s 11s 5s 4s 9s 2s 5s 24s 9s 14s 5s 21s 9s 6s 9s 8s
What % have at most Dx overhead? D = 4, vs.
SLIDE 36
What % have at most Dx overhead? D = 4, vs.
SLIDE 37
Evaluation Method
O(2)
2
2
- r
SLIDE 38
EXPERIMENT & RESULTS
SLIDE 39
Benchmarks
DLS 2014 POPL 2017 PEPM 2018 futen call_method espionage http2 call_simple pythonflow slowSHA chaos take5 aespython fannkuch sample_fsm stats go meteor nbody nqueens pidigits pystone spectralnorm
SLIDE 40
# Typed Components
DLS 2014 POPL 2017 PEPM 2018 15 7 12 4 6 12 17 15 16 34 * 1 19 * 79 * 7 8 5 2 5 14 5
SLIDE 41
Exhaustive Results
What % of configurations have at most 4x overhead?
100% 50%
4x
SLIDE 42
Exhaustive Results
What % of configurations have at most Dx overhead?
100% 50%
4x 2x 6x 8x 1x
SLIDE 43
Exhaustive Results
What % of configurations have at most Dx overhead?
100% 50%
4x 2x 6x 8x 1x
SLIDE 44
Approximate Results
What % of configurations have at most 4x overhead, based on R samples of S configurations each?
100% 50%
4x
SLIDE 45
Approximate Results
What % of configurations have at most Dx overhead, based on R samples of S configurations each?
100% 50%
4x 2x 6x 8x 1x
SLIDE 46
Approximate Results
What % of configurations have at most Dx overhead, based on R samples of S configurations each?
100% 50%
4x 2x 6x 8x 1x
SLIDE 47
100% 50%
4x 2x 6x 8x
100% 50%
4x 2x 6x 8x
espionage 4,096 configurations aespython 10 samples
- f 340 configurations
1x 1x
SLIDE 48
SLIDE 49
Cost of Tag Soundness
- Worst-case overhead: under 10x
SLIDE 50
Type Soundness Tag Soundness 1x 1x 1x 1x 1x 1x 2x 2x 2x 3x 3x 3x 3x 5x 7x 8x 7x 7x 6x 6x 2x 43x 27x 1527x 233x 292x 1x 1x 10x 1x 29x 5x 32x 1x 34x 47x 5x 2x 139x 4x This is an APPLES to ORANGES comparison!
SLIDE 51
Cost of Tag Soundness
- Worst-case overhead: under 10x
- Best-case overhead: 1x -- 4x
- adding types never* improves performance
- Slowest configuration: fully-typed
- Overhead number of type annotations
∝
SLIDE 52
SLIDE 53
SLIDE 54
Runtime vs. # Types
SLIDE 55
Runtime vs. # Types
SLIDE 56
Speedup?
- Unsound optimization for
read-only values (tuples)
- Double-checks method
calls
SLIDE 57
Runtime vs. # Types
SLIDE 58
Experiment
- granularity: functions & class-fields
- 10 samples of [10 * (F + C)] configurations
- Karst at Indiana University cluster (32GB RAM, 250GB other)
- Reticulated, master branch, commit e478343
- Python 3.4.3
- 40 iterations per configuration, report average
- 200 values of D on x-axis
SLIDE 59
POPL 2017
SLIDE 60
SLIDE 61
References
- Vitousek, Swords, Siek. Big Types in Little
Runtime: Open-World Soundness and Collaborative Blame for Gradual Type Systems. POPL 2017
- Takikawa, Feltey, Greenman, New, Vitek, Felleisen.
Is Sound Gradual Typing Dead? POPL 2016.
SLIDE 62
UNUSED SLIDES
SLIDE 63
⊢ e : τ ⊢ e' : τ' ???
SLIDE 64
Takikawa Method
- granularity
- experimental modules, fixed modules
- configurations
- baseline
- performance ratio