A realtime GraphQL backend as a compiler in Haskell - PowerPoint PPT Presentation

A realtime GraphQL backend as a compiler in Haskell http://bit.ly/graphql-haskell

Heippa! Tanmai Gopal @tanmaigo http://bit.ly/graphql-haskell

Contents 1. The compiler approach 2. Compiler pipeline a. Parse b. GraphQL AST c. Transform to SQL AST <> inject auth d. Optimise SQL AST 3. Realtime (concurrent programming) http://bit.ly/graphql-haskell

Building a GraphQL backend Database GraphQL queries come from app and need to be translated to data calls (with access control) to the database. Goal : - Observability for the team - High performance Hasura - Low footprint App

Resolver approach Resolver based approach: - Parse GraphQL query to AST - Traverse the tree, call resolvers Resolver can: - Make data calls (n+1) where n is the number of results - Make calls via dataloader O(n) where n is number of nodes If there are N authors with N articles each, you’re making 1 + n + n + n 2 queries!

Compiler approach query { SELECT author { author.name, name address.city, articles.title, address { tag.name city } FROM author, address, articles, tags articles { title WHERE tags { author.id = address.author_id, name author.id = articles.author_id, } articles.id = tag.article_id } } }

Approach advantages - Observability: - 1 SQL query to debug/analyse - Performance (on my laptop): - 2000 req/s on nested queries ~ 50MB RAM - 1500 records in 4ms - 400MB data select within 1-5% raw database call joining tables containing 100M rows each

Compiler pipeline AST ExecutableDefinition (OperationDefinition) OperationType: query SelectionSet: ... SQL query { author { SELECT id, name parse SelectionSet: [Selection] id FROM author name WHERE id = <session.user-id> } Selection: } Field: Name: author SelectionSet: [Selection] Selection: Selection: Field: Field: Name: id Name: name SelectionSet: [] SelectionSet: []

Representing the GraphQL AST with Haskell ADTs Algebraic Data Types

Representing the GraphQL AST with Haskell ADTs

Parsing by hand 1. Lex into “tokens” (words and braces) 2. Read the first word query { 3. This should be an ExecutableDefinition. author { 4. Check if this is one of the operation types id name 5. If yes, the rest of the document is a SelectionSet } } 6. In a SelectionSet, we need to assemble an array of fields 7. ...

Parsing with parser combinators query { author { id name } } ExecutableDefinition (OperationDefinition) The Magic Try to parse an operationDefinition or parse a fragmentDefinition.

Try to parse an operationDefinition by 1. Parsing an operationType 2. Optionally parsing a Name 3. Optionally parsing variables 4. Optionally parsing directives 5. Parsing a selectionSet Try to parse an operationType by: 1. Parsing a “query” token OR 2. Parsing a “mutation” token OR 3. Parsing a “subscription” token

Transforming the GraphQL AST to SQL AST ExecutableDefinition (OperationDefinition) OperationType: query SelectionSet: ... SELECT id, name SelectionSet: [Selection] FROM author WHERE id = <session.user-id> Selection: Field: Name: author SelectionSet: [Selection] Selection: Selection: Field: Field: Name: id Name: name SelectionSet: [] SelectionSet: []

Handling realtime (GraphQL subscriptions)

Database Event sourcing threads Thread1 Thread2 Thread3 ... 1. Create a new connection object and attach to the right event-source thread or create a new thread 2. In every thread, take an event and Conn1 Conn2 Conn3 ... deliver to all relevant connection (+Delivery thread) (+Delivery thread) (+Delivery thread) objects 3. In case GraphQL schema or auth Create a connection object + thread for every new connection config changes, close/refresh all connections App App App App App App App App

Concurrent programming

Locks are bad Conn1 Option 1: (+Delivery thread) Thread1 Lock1 && Lock2 Conn2 - Pop (+Delivery thread) - Push Define a thread-safe operation: Release - pop from event queue (lock1) Define a thread-safe operation: - push to connection object queue (lock2) Option 2: Acquire in sequence: Compose: Lock1 then Lock2 - Pop - Pop (lock1) - Push (lock2) - Push Release The composition is not safe!

Concurrent programming in Haskell with STM If pop and push are safe actions, then compose them safely atomically $ do pop push Software transactional memory is the idea of database transactions: - In memory - Across threads

Summary: Writing compilers is easy: - Parser combinators + ADTs https://github.com/Gabriel439/post-rfc/blob/master/sotu.md#compilers Doing concurrent programming is easy: - Software Transactional Memory (STMs) https://github.com/Gabriel439/post-rfc/blob/master/sotu.md#single-machine-concurrency

Hasura hasura.io github.com/hasura/graphql-engine @HasuraHQ Talk to me about: Haskell Postgres GraphQL Subscriptions Serverless https://3factor.app