Full Stack Type Safety Szymon Pyalski Egnyte Inc. Europython 2020 - - PowerPoint PPT Presentation

▶
full stack type safety
SMART_READER_LITE
LIVE PREVIEW

Full Stack Type Safety Szymon Pyalski Egnyte Inc. Europython 2020 - - PowerPoint PPT Presentation

Nov 28, 2023 421 likes •1.02k views

Premise Typing basics Our typical stack Annotations and ORM Enforcing the contract Summary Full Stack Type Safety Szymon Pyalski Egnyte Inc. Europython 2020 Premise Typing basics Our typical stack Annotations and ORM Enforcing the

slide-1
SLIDE 1

Premise Typing basics Our typical stack Annotations and ORM Enforcing the contract Summary

Full Stack Type Safety

Szymon Pyżalski

Egnyte Inc.

Europython 2020

slide-2
SLIDE 2

Premise Typing basics Our typical stack Annotations and ORM Enforcing the contract Summary

Outline

Premise Typing basics Our typical stack Annotations and ORM Enforcing the contract Summary

slide-3
SLIDE 3

Premise Typing basics Our typical stack Annotations and ORM Enforcing the contract Summary

Our goal

slide-4
SLIDE 4

Premise Typing basics Our typical stack Annotations and ORM Enforcing the contract Summary

Our goal

  • Catch typing errors ASAP (not later than in CI)
slide-5
SLIDE 5

Premise Typing basics Our typical stack Annotations and ORM Enforcing the contract Summary

Our goal

  • Catch typing errors ASAP (not later than in CI)
  • Catch typing errors that span layers of stack
slide-6
SLIDE 6

Premise Typing basics Our typical stack Annotations and ORM Enforcing the contract Summary

Problems

slide-7
SLIDE 7

Premise Typing basics Our typical stack Annotations and ORM Enforcing the contract Summary

Problems

  • Type annotation system in Python is new and immature
slide-8
SLIDE 8

Premise Typing basics Our typical stack Annotations and ORM Enforcing the contract Summary

Problems

  • Type annotation system in Python is new and immature
  • Various layers of stack feature different typing paradigms
slide-9
SLIDE 9

Premise Typing basics Our typical stack Annotations and ORM Enforcing the contract Summary

Problems

  • Type annotation system in Python is new and immature
  • Various layers of stack feature different typing paradigms
  • We tend to test layers in separation
slide-10
SLIDE 10

Premise Typing basics Our typical stack Annotations and ORM Enforcing the contract Summary

Weak vs strong

slide-11
SLIDE 11

Premise Typing basics Our typical stack Annotations and ORM Enforcing the contract Summary

Weak vs strong

Weak typing A value can be misinterpreted unless we care about the type by ourselves.

slide-12
SLIDE 12

Premise Typing basics Our typical stack Annotations and ORM Enforcing the contract Summary

Weak vs strong

Weak typing A value can be misinterpreted unless we care about the type by ourselves. Strong typing We are protected from misinterpretations by the type system.

slide-13
SLIDE 13

Premise Typing basics Our typical stack Annotations and ORM Enforcing the contract Summary

Weak typing

# include <stdio.h> short int fun(int* x) { short int y = *(short int*)x; return y + 1; } int main(int argc, char** argv) { int a = -10; int b = 777777; printf("%u\n", a); // prints: 4294967286 printf("%d\n", fun(&b)); // prints: -8654 }

slide-14
SLIDE 14

Premise Typing basics Our typical stack Annotations and ORM Enforcing the contract Summary

Static vs dynamic

slide-15
SLIDE 15

Premise Typing basics Our typical stack Annotations and ORM Enforcing the contract Summary

Static vs dynamic

Static typing The types of objects can be determined during compile time.

slide-16
SLIDE 16

Premise Typing basics Our typical stack Annotations and ORM Enforcing the contract Summary

Static vs dynamic

Static typing The types of objects can be determined during compile time. Dynamic typing The types of objects are determined during runtime.

slide-17
SLIDE 17

Premise Typing basics Our typical stack Annotations and ORM Enforcing the contract Summary

Dynamic typing in python

def sum(xs, init): result = init for x in xs: result += x return result print(sum([1, 2, 3], 0)) # prints 6 print(sum({’a’: ’b’, ’c’: ’d’}, ’Keys: ’)) # prints: Keys: ac

slide-18
SLIDE 18

Premise Typing basics Our typical stack Annotations and ORM Enforcing the contract Summary

Static typing with inference

package main import "fmt"; func fact(n int) int { result := 1 for i := 1; i <=n; i++ { result *= i } return result } func main() { x := 10 y := fact(5) fmt.Println(x) fmt.Println(y) }

slide-19
SLIDE 19

Premise Typing basics Our typical stack Annotations and ORM Enforcing the contract Summary

Strict vs loose

slide-20
SLIDE 20

Premise Typing basics Our typical stack Annotations and ORM Enforcing the contract Summary

Strict vs loose

Strict typing Type conversions must be explicit. Type mismatch exceptions.

slide-21
SLIDE 21

Premise Typing basics Our typical stack Annotations and ORM Enforcing the contract Summary

Strict vs loose

Strict typing Type conversions must be explicit. Type mismatch exceptions. Loose typing Type conversions can be implicit.

slide-22
SLIDE 22

Premise Typing basics Our typical stack Annotations and ORM Enforcing the contract Summary

Stricter than Python

import Data.String.Utils (join) list2Str :: [[Char]] -> [Char] list2Str xs = if xs then "No elements" else (join "," xs) -- main = do putStrLn $ list2Str [] putStrLn 10 -- Error

slide-23
SLIDE 23

Premise Typing basics Our typical stack Annotations and ORM Enforcing the contract Summary

Looser than Python

1 + ’a’ // ’1a’ {} + 2 // 0 ’abc’ + [’d’, ’e’, ’f’] // "abcd,e,f" {} + ’z’ // NaN {} + {} // NaN {} + [] // 0 [] + {} // "[object Object]"

slide-24
SLIDE 24

Premise Typing basics Our typical stack Annotations and ORM Enforcing the contract Summary

Duck vs ???

slide-25
SLIDE 25

Premise Typing basics Our typical stack Annotations and ORM Enforcing the contract Summary

Duck vs ???

Duck typing Interfaces Protocols are implemented implicitly. Object is compatible with a protocol if it implements required methods.

slide-26
SLIDE 26

Premise Typing basics Our typical stack Annotations and ORM Enforcing the contract Summary

Duck vs ???

Duck typing Interfaces Protocols are implemented implicitly. Object is compatible with a protocol if it implements required methods. ??? Classes must inherit from a class in order to be compaticle, or at least be marked as implementing the protocol.

slide-27
SLIDE 27

Premise Typing basics Our typical stack Annotations and ORM Enforcing the contract Summary

Duck vs platonic

Duck typing Interfaces Protocols are implemented implicitly. Object is compatible with a protocol if it implements required methods. Platonic typing Classes must inherit from a class in order to be compaticle, or at lease be marked as implementing the protocol.

slide-28
SLIDE 28

Premise Typing basics Our typical stack Annotations and ORM Enforcing the contract Summary

Structural vs nominal

Structural typing Interfaces Protocols are implemented implicitly. Object is compatible with a protocol if it implements required methods. Nominal typing Classes must inherit from a class in order to be compaticle, or at lease be marked as implementing the protocol.

slide-29
SLIDE 29

Premise Typing basics Our typical stack Annotations and ORM Enforcing the contract Summary

The pythonish language

They say We say

slide-30
SLIDE 30

Premise Typing basics Our typical stack Annotations and ORM Enforcing the contract Summary

The pythonish language

They say We say throw raise

slide-31
SLIDE 31

Premise Typing basics Our typical stack Annotations and ORM Enforcing the contract Summary

The pythonish language

They say We say throw raise array list

slide-32
SLIDE 32

Premise Typing basics Our typical stack Annotations and ORM Enforcing the contract Summary

The pythonish language

They say We say throw raise array list list deque

slide-33
SLIDE 33

Premise Typing basics Our typical stack Annotations and ORM Enforcing the contract Summary

The pythonish language

They say We say throw raise array list list deque blatant abuse of exceptions StopIteration

slide-34
SLIDE 34

Premise Typing basics Our typical stack Annotations and ORM Enforcing the contract Summary

The pythonish language

They say We say throw raise array list list deque blatant abuse of exceptions StopIteration interfaces protocols

slide-35
SLIDE 35

Premise Typing basics Our typical stack Annotations and ORM Enforcing the contract Summary

The pythonish language

They say We say throw raise array list list deque blatant abuse of exceptions StopIteration interfaces protocols

slide-36
SLIDE 36

Premise Typing basics Our typical stack Annotations and ORM Enforcing the contract Summary

Static but duck-typed

package main import "fmt" type Duck interface { swim(x int, y int) quack() string } type Mallard struct { x, y int } func (m *Mallard) swim(x, y int) { m.x += x m.y += y } func (m Mallard) quack() string { return "Quack quaaaack" } func swimThenQuack(d Duck) { d.swim(1, 1) fmt.Println(d.quack()) } func main() { donald := Mallard{x: 0, y: 0} swimThenQuack(&donald) fmt.Println(donald) }

slide-37
SLIDE 37

Premise Typing basics Our typical stack Annotations and ORM Enforcing the contract Summary

Typing models

  • Strong vs weak typing
  • Static vs dynamic typing
  • Strict vs loose typing
  • Structural vs nominal typing
slide-38
SLIDE 38

Premise Typing basics Our typical stack Annotations and ORM Enforcing the contract Summary

Typing models

  • Strong vs weak typing
  • Static vs dynamic typing
  • Strict vs loose typing
  • Structural vs nominal typing
  • Free vs fixed attributes
slide-39
SLIDE 39

Premise Typing basics Our typical stack Annotations and ORM Enforcing the contract Summary

Our typical stack

Javascript Strong Very loose Dynamic Structural Free attributes Python Strong Strict Dynamic Structural Free attributes SQL Weak (foreign keys) Loose Static Nominal Fixed attributes

slide-40
SLIDE 40

Premise Typing basics Our typical stack Annotations and ORM Enforcing the contract Summary

Our typical stack

Javascript Strong Very loose Dynamic Structural Free attributes Python Strong Strict Dynamic Structural Free attributes Models Strong Strict Static Nominal Fixed attributes SQL Weak (foreign keys) Loose Static Nominal Fixed attributes

slide-41
SLIDE 41

Premise Typing basics Our typical stack Annotations and ORM Enforcing the contract Summary

Weakness of SQL foreign keys

UPDATE books set author_id = ( SELECT id FROM publishers WHERE name="Chilton Books" );

slide-42
SLIDE 42

Premise Typing basics Our typical stack Annotations and ORM Enforcing the contract Summary

ORM improving type safety

b = Book.objects.get(id=1) b.author = Publisher.objects.get(name=’Chilton Books’)

slide-43
SLIDE 43

Premise Typing basics Our typical stack Annotations and ORM Enforcing the contract Summary

mypy enters the game

Javascript Strong Very loose Dynamic Structural Free attributes mypy Strong String Static Preference for nominal Fixed attributes Python Strong Strict Dynamic Structural Free attributes Models Strong Strict Static Nominal Fixed attributes SQL Weak (foreign keys) Loose Static Nominal Fixed attributes

slide-44
SLIDE 44

Premise Typing basics Our typical stack Annotations and ORM Enforcing the contract Summary

Demo 1

Django and mypy working together

slide-45
SLIDE 45

Premise Typing basics Our typical stack Annotations and ORM Enforcing the contract Summary

mypy and Django pros and cons

slide-46
SLIDE 46

Premise Typing basics Our typical stack Annotations and ORM Enforcing the contract Summary

mypy and Django pros and cons

  • Pro: Recognizes the relationship between column types and

python types

slide-47
SLIDE 47

Premise Typing basics Our typical stack Annotations and ORM Enforcing the contract Summary

mypy and Django pros and cons

  • Pro: Recognizes the relationship between column types and

python types

  • Pro: Recognizes the idea of null
slide-48
SLIDE 48

Premise Typing basics Our typical stack Annotations and ORM Enforcing the contract Summary

mypy and Django pros and cons

  • Pro: Recognizes the relationship between column types and

python types

  • Pro: Recognizes the idea of null
  • Con: Can’t handle problems with incomplete data
slide-49
SLIDE 49

Premise Typing basics Our typical stack Annotations and ORM Enforcing the contract Summary

mypy and Django pros and cons

  • Pro: Recognizes the relationship between column types and

python types

  • Pro: Recognizes the idea of null
  • Con: Can’t handle problems with incomplete data
  • Con: Requires a mypy plugin
slide-50
SLIDE 50

Premise Typing basics Our typical stack Annotations and ORM Enforcing the contract Summary

Considering the JSON

Javascript Strong Very loose Dynamic Structural Free attributes JSON No typing above primitives mypy Strong String Static Preference for nominal Fixed attributes Python Strong Strict Dynamic Structural Free attributes Models Strong Strict Static Nominal Fixed attributes SQL Weak (foreign keys) Loose Static Nominal Fixed attributes

slide-51
SLIDE 51

Premise Typing basics Our typical stack Annotations and ORM Enforcing the contract Summary

One solution

Typescript Strong Strict Static Structural Fixed attributes JSON No typing above primitives OpenAPI3 Schema Tests Code generation mypy Strong String Static Preference for nominal Fixed attributes Python Strong Strict Dynamic Structural Free attributes Models Strong Strict Static Nominal Fixed attributes SQL Weak (foreign keys) Loose Static Nominal Fixed attributes

slide-52
SLIDE 52

Premise Typing basics Our typical stack Annotations and ORM Enforcing the contract Summary

Demo 2

Enforcing the contract

slide-53
SLIDE 53

Premise Typing basics Our typical stack Annotations and ORM Enforcing the contract Summary

Takeaways

slide-54
SLIDE 54

Premise Typing basics Our typical stack Annotations and ORM Enforcing the contract Summary

Takeaways

  • There are tools for code safety enforcement in a Python stack

that are worth consideration

slide-55
SLIDE 55

Premise Typing basics Our typical stack Annotations and ORM Enforcing the contract Summary

Takeaways

  • There are tools for code safety enforcement in a Python stack

that are worth consideration

  • They are not yet perfect and we can’t expect to catch all

errors

slide-56
SLIDE 56

Premise Typing basics Our typical stack Annotations and ORM Enforcing the contract Summary

Future can bring

slide-57
SLIDE 57

Premise Typing basics Our typical stack Annotations and ORM Enforcing the contract Summary

Future can bring

  • Support for more patterns in type annotations without plugins
slide-58
SLIDE 58

Premise Typing basics Our typical stack Annotations and ORM Enforcing the contract Summary

Future can bring

  • Support for more patterns in type annotations without plugins
  • Tools based on code annotations instead of descriptors (

strawberry-graphql, pydantic, )

slide-59
SLIDE 59

Premise Typing basics Our typical stack Annotations and ORM Enforcing the contract Summary

Tools used

  • django-stubs A distribution of code annotations for django

complete with a mypy plugin

  • spectacular A schema generator for django-rest-framework
  • openapi-generator Code generator that can create boilerplate

code for several languages/frameworks based on OpenAPI3.