Building Flexible Systems with Clojure and Datomic Stuart Sierra - - PowerPoint PPT Presentation

▶
building flexible systems
SMART_READER_LITE
LIVE PREVIEW

Building Flexible Systems with Clojure and Datomic Stuart Sierra - - PowerPoint PPT Presentation

Dec 29, 2022 377 likes •1.19k views

Building Flexible Systems with Clojure and Datomic Stuart Sierra Cognitect We dont want to paint ourselves into a corner Clojure Flexible Systems Fact-based Context-free Non-exclusive Observable Fact Based

slide-1
SLIDE 1

Building Flexible Systems

with Clojure and Datomic Stuart Sierra Cognitect

slide-2
SLIDE 2

“We don’t want to
 paint ourselves
 into a corner”

slide-3
SLIDE 3
slide-4
SLIDE 4
slide-5
SLIDE 5

Clojure

slide-6
SLIDE 6

Flexible Systems

✦ Fact-based ✦ Context-free ✦ Non-exclusive ✦ Observable

slide-7
SLIDE 7

Fact Based

slide-8
SLIDE 8

Fact Based

✦ Fact = statement about the world ✦ Cannot be invalidated

slide-9
SLIDE 9

public class Person { private List<Person> friends; public void addFriend(Person newFriend);

slide-10
SLIDE 10

public class Person { private List<Person> friends; public void addFriend(Person newFriend) { if (friends.length() < 500) friends.add(newFriend); else ... mutable? race condition?

slide-11
SLIDE 11
slide-12
SLIDE 12

["Alice" "Bob"]

Universal Data

{412 "a" 114 "b"} (println user) #{1 7 9 5}

Collection: Set Ordered: Vector Associative: Map Command or stack: List

{:name "Bob" :age 42}

Structured: Map

slide-13
SLIDE 13

{:name "Kelly Q." :friends [{:name "Alice"} {:name "Bob"}]} map vector of maps

slide-14
SLIDE 14

(defn add-friend [person new-friend] (if (< (count (:friends person)) 500) (update person :friends
 conj new-friend) {:error :too-friendly}))

slide-15
SLIDE 15

(defn add-friend [person new-friend] define function parameters name list vector symbol

slide-16
SLIDE 16

(if (< (count (:friends person)) 500) condition

slide-17
SLIDE 17

(if (< (count (:friends person)) 500) (update person :friends
 conj new-friend) then condition

slide-18
SLIDE 18

(update person :friends
 conj new-friend) {:name "Kelly Q." :friends [{:name "Alice"} {:name "Bob"}]} navigation

slide-19
SLIDE 19


 conj new-friend) [{:name "Alice"} {:name "Bob"}] conjoin vector [{:name "Alice"} {:name "Bob"} {:name "Claire"}]}

slide-20
SLIDE 20

update map {:name "Kelly Q." :friends [{:name "Alice"} {:name "Bob"} {:name "Claire"}]} (update person :friends
 conj new-friend) {:name "Kelly Q." :friends [{:name "Alice"} {:name "Bob"}]}

slide-21
SLIDE 21

universal

  • perations

{:name "Kelly Q." :friends [{:name "Alice"} {:name "Bob"} {:name "Claire"}]} (update person :friends
 conj new-friend) {:name "Kelly Q." :friends [{:name "Alice"} {:name "Bob"}]} universal data structures

slide-22
SLIDE 22

{:name "Kelly Q." :friends [{:name "Alice"} {:name "Bob"}]} {:name "Kelly Q." :friends [{:name "Alice"} {:name "Bob"} {:name "Claire"}]} (add-friend person {:name "Claire"}) domain rule in terms of universal data and operations

slide-23
SLIDE 23

get keys vals assoc assoc-in update update-in dissoc merge merge-with select-keys nth map filter remove reduce replace some sort shuffle reverse take drop union difference intersection select subset? superset? join project index rename concat cycle interleave interpose distinct flatten group-by partition split-at split-with frequencies

(A Few) Universal Operations

slide-24
SLIDE 24

(defn add-friend [person new-friend] (if (< (count (:friends person)) 500) (update person :friends
 conj new-friend) ...)) immutable value no race condition

Immutable Values

slide-25
SLIDE 25

value function value

slide-26
SLIDE 26

value function value function value Time

slide-27
SLIDE 27

value function value function value Time identity

slide-28
SLIDE 28

value function value function value Time identity stable live

slide-29
SLIDE 29

(let [current-user (atom {:name "Kelly Q." :friends [{:name "Alice" :name "Bob"}]})]

current-user

{:name "Kelly Q." :friends [{:name "Alice"} {:name "Bob"}]}

slide-30
SLIDE 30

current-user

{:name "Kelly Q." :friends [{:name "Alice"} {:name "Bob"}]}

(let [friends (:friends @current-user)] (html [:p "Your " (count friends) " friends"] [:ul (for [friend friends] [:li (:name friend)])])) dereference

slide-31
SLIDE 31

{:name "Kelly Q." :friends [{:name "Alice"} {:name "Bob"}]}

(let [friends (:friends @current-user)] (html [:p "Your " (count friends) " friends"] [:ul (for [friend friends] [:li (:name friend)])])) dereference immutable value

slide-32
SLIDE 32

current-user

{:name "Kelly Q." :friends [{:name "Alice"} {:name "Bob"}]}

(swap! current-user add-friend {:name "Claire"})

{:name "Kelly Q." :friends [{:name "Alice"} {:name "Bob"} {:name "Claire"}]}

add-friend

change state pure function identity

slide-33
SLIDE 33

storage service connection

slide-34
SLIDE 34

storage service connection

(db connection)

database value

slide-35
SLIDE 35

(query db ...)

database value

(let [db (db connection)] (query db ...) (query db ...)

slide-36
SLIDE 36

connection database value transact database value transact database value Time

slide-37
SLIDE 37

Entity Attribute Value

kelly :name "Kelly Q." kelly :friends alice kelly :friends bob

Universal Data

slide-38
SLIDE 38

Entity Attribute Value Op Tx

kelly :name "Kelly Q." add 1000 kelly :friends alice add 1000 kelly :friends bob add 1000

Universal Data in Time

slide-39
SLIDE 39

Entity Attribute Value Op Tx

kelly :name "Kelly Q." add 1000 kelly :friends alice add 1000 kelly :friends bob add 1000 kelly :friends bob retract 1023 kelly :friends claire add 1023

Universal Data in Time

slide-40
SLIDE 40

Entity Attribute Value Op Tx

kelly :name "Kelly Q." add 1000 kelly :friends alice add 1000 kelly :friends bob add 1000 kelly :friends bob retract 1023 kelly :friends claire add 1023

Current State

{:name "Kelly Q." :friends [{:name "Alice"} {:name "Claire"}]}

slide-41
SLIDE 41

Entity Attribute Value Op Tx

kelly :name "Kelly Q." add 1000 kelly :friends alice add 1000 kelly :friends bob add 1000 kelly :friends bob retract 1023 kelly :friends claire add 1023

State as-of Last Week

{:name "Kelly Q." :friends [{:name "Alice"} {:name "Bob"}]}

slide-42
SLIDE 42

(swap! current-user add-friend ...) (transact connection [[:add-friend ...]]) change state pure function identity

slide-43
SLIDE 43

identity database value transact database value transact database value Time stable live

slide-44
SLIDE 44

Fact Based

✦ Fact = statement about the world ✦ Cannot be invalidated ➡ Immutable values ➡ Using uniform data structures ➡ Incorporating time

slide-45
SLIDE 45

Context Free

slide-46
SLIDE 46

Context Free

✦ Values are complete and self-describing ✦ No out-of-band knowledge required


to handle correctly

slide-47
SLIDE 47

{:id "1234abcd" :name "T.J."} {:id "abcd" :balance 42.00} {:id 7980 :region "AUS"}

slide-48
SLIDE 48

{:customer/id "1234abcd" :customer/name "T.J."} {:billing.customer/id "abcd" :billing.customer/balance 42.00} {:widgetCo.customer/id 7980 :widgetCo/region "AUS"} namespaced keywords

slide-49
SLIDE 49

{:customer/id "1234abcd" :customer/name "T.J." :billing.customer/id "abcd" :billing.customer/balance 42.00 :widgetCo.customer/id 7980 :widgetCo/region "AUS"} (merge customer billing-customer widgetco-customer)

slide-50
SLIDE 50

(s/def :customer/id (s/and string? #(re-matches #"[0-9a-e]{8}" %))) (s/def :customer/name (s/and string? #(not (str/blank? %)))) (s/def ::Customer (s/keys :req [:customer/id] :opt [:customer/name]])) clojure.spec

slide-51
SLIDE 51

(let [user {:customer/name ""}] (s/explain ::Customer user)) val: {:customer/name ""} fails spec: ::Customer predicate: (contains? % :customer/id) In: [:customer/name] val: "" fails spec: :customer/name at: [:customer/name] predicate: (not (blank? %))

slide-52
SLIDE 52

(pull db [:customer/name :customer/start-date {:customer/account [:account/id :account/balance]}] customer) {:customer/name "T.J." :customer/start-date #inst"2012-01-24" :customer/account {:account/id 12345 :account/balance 4200}}

slide-53
SLIDE 53

(defui UserWidget static om/IQuery (query [this] [:user/name {:user/friends [:user/name]}]) Object (render [this] (let [friends (:user/friends (om/props this))] (html [:p "Your " (count friends) " friends"] [:ul (for [friend friends] [:li (:user/name friend)])]))))

  • m.next
slide-54
SLIDE 54

Context Free

✦ Values are complete and self-describing ✦ No out-of-band knowledge required


to handle correctly

➡ Use namespaces for globally-unique labels ➡ Let consumers control interactions

slide-55
SLIDE 55
slide-56
SLIDE 56

Non-exclusive

slide-57
SLIDE 57

Non-exclusive

✦ Do not obstruct data evolution ✦ Continue correct operation in the


presence of unexpected input

slide-58
SLIDE 58

input function input

  • utput
slide-59
SLIDE 59

extra extra input function input extra extra

  • utput
slide-60
SLIDE 60

(defn gold-status [customer] (if (< 100
 (:customer/orders customer)) (assoc customer
 :loyalty/tier :gold) customer)) then: augment and return else: return unchanged condition

slide-61
SLIDE 61

(s/fdef gold-status :args (s/cat :customer (s/keys :req [:customer/id :customer/orders]))) clojure.spec minimal required keys

slide-62
SLIDE 62

(gold-status {:customer/id "abcd1234" :customer/balance 4200}) ExceptionInfo Call to #'user/gold-status did not conform to spec: In: [0] val: {:customer/id "abcd1234", :customer/balance 4200} fails at: [:args :customer] predicate: (contains? % :customer/orders)

slide-63
SLIDE 63

(gold-status {:customer/id "abcd1234" :customer/orders 123 :customer/balance 4200}) {:customer/id "abcd1234" :customer/orders 123 :customer/balance 4200 :status :gold} extra data passed through

slide-64
SLIDE 64

Non-exclusive

➡ Enforce only minimum input requirements ➡ Ignore and pass through “extra” input ✦ Do not obstruct data evolution ✦ Continue correct operation in the


presence of unexpected input

slide-65
SLIDE 65
slide-66
SLIDE 66

<?xml version="1.0" encoding="utf-8" ?> <description xmlns="http://www.w3.org/ns/wsdl" targetNamespace= "http://greath.example.com/2004/wsdl/resSvc" xmlns:tns= "http://greath.example.com/2004/wsdl/resSvc" xmlns:ghns = "http://greath.example.com/2004/schemas/resSvc" xmlns:wsoap= "http://www.w3.org/ns/wsdl/soap" xmlns:soap="http://www.w3.org/2003/05/soap-envelope" xmlns:wsdlx= "http://www.w3.org/ns/wsdl-extensions"> <types> <xs:schema xmlns:xs="http://www.w3.org/2001/XMLSchema" targetNamespace="http://greath.example.com/2004/schemas/ resSvc" xmlns="http://greath.example.com/2004/schemas/resSvc"> <xs:element name="checkAvailability" type="tCheckAvailability"/> <xs:complexType name="tCheckAvailability"> <xs:sequence> <xs:element name="checkInDate" type="xs:date"/> <xs:element name="checkOutDate" type="xs:date"/> <xs:element name="roomType" type="xs:string"/> </xs:sequence> </xs:complexType>

slide-67
SLIDE 67

Observable

slide-68
SLIDE 68

Observable

✦ Internal state of the system can be


recreated at any point

✦ Output of the system is suffjcient to


recover its state

slide-69
SLIDE 69

Internal state

slide-70
SLIDE 70

value function

slide-71
SLIDE 71

current state function new input new state

  • utput
slide-72
SLIDE 72

(defn handle [state request] ;; compute new state ;; decide how to respond (-> (update state ...) (assoc :response ...))

slide-73
SLIDE 73

current state function new input new state

  • utput

log

slide-74
SLIDE 74

{:uri "datomic:ddb:us-east-1..." :t 12345}

slide-75
SLIDE 75

{:uri "datomic:ddb:us-east-1..." :t 12345} (let [conn (connect uri) db (as-of (db conn) t)] (handle db request)) {:method :post :path "/users" :params {:name "Leslie"}}

state input

slide-76
SLIDE 76

[[:db/add user :user/name "Leslie"] [:db/add tx :tx/request-id 1234567890]]

{:request-id 1234567890 :method :post :path "/users" :query-params {:name "Leslie"}}

slide-77
SLIDE 77

Observable

✦ Internal state of the system can be


recreated at any point

✦ Output of the system is suffjcient to


recover its state

➡ Minimize necessary information to

completely describe internal state

➡ Store inputs at boundaries

slide-78
SLIDE 78
slide-79
SLIDE 79

Flexible Systems

✦ Fact-based ➡ Immutable values, in domain terms, with time ✦ Context-free ➡ Namespaces; consumer control ✦ Non-exclusive ➡ Enforce minimum, pass-through extra ✦ Observable ➡ Log input, point-in-time state, provenance

slide-80
SLIDE 80

✦ “The New Normal” blog series, Mike Nygard ✦ Case Studies: cognitect.com, datomic.com ✦ clojure.org ✦ ClojureTV on YouTube ✦ stuartsierra.com