Human-friendly DNS powered by Golang We love to work in a casual - PowerPoint PPT Presentation

Human-friendly DNS powered by Golang

We love to work in a casual environment that energizes us to revolutionize the fitness industry.

Site Reliability Engineering @ eGym ● Scale

Site Reliability Engineering @ eGym

Site Reliability Engineering @ eGym

Site Reliability Engineering @ eGym

Room for more SREs :)

Site Reliability Engineering @ eGym ● Scale ● Automate ● Own reliability ○ 24/7 Monitoring ○ On-call ● Consult teams

DNS Basics

DNS Basics

Names label www . egym . de . $NULL separator

DNS Hosts host www . egym . de domain Top-level domain

DNS Name Spacing Sub zone int . egym . de zone

DNS Name Spacing https://en.wikipedia.org/wiki/Domain_Name_System

DNS @ eGym Registrar ● 20+ domains Scripts ○ and counting... Scripts ● 18 delegations to sub zones ○ “team spaces” ● 700+ resource records ○ without sub zone records Cloud DNS Hoster Read/Parse Scripts Change Zone Deployment File Delegation (NS RR)

Rethinking DNS @ eGym Single Production DNS Source Tools (Cloud DNS) Of Truth Cloud DNS Tools Single Source of Truth ● Scales ● Rollbacks ● Version Controlled ● API ● Replays ● Human Readable ● Automated (non-interactive mode) ● Non-repetitive ● Safeguards

Rethinking DNS @ eGym Single Production DNS Source Tools (Cloud DNS) Of Truth Cloud DNS Tools Single Source of Truth ● Scales ● Rollbacks ● Version Controlled ● API ● Replays ● Human Readable ● Automated (non-interactive mode) ● Non-repetitive ● Safeguards YAML Templates

DNS Tooling ● PaPuDNS ○ Parses YAML-formatted zone information ○ Parses YAML-formatted templates ○ Applies templates ○ In-Memory database with all resource records ○ Fetches current zone information from Cloud DNS via API ○ Calculates difference ○ Pushes the changes (atomically) ● dns-check ○ Parses YAML-formatted “expectations” ○ Checks via live DNS if expectations are meet ○ Does not (yet) use the same format/database

Zones zones: - zone: egym.coffee description: Test zone. - Sets zone TTL ttl: 300 templates: - Pulls in “names” from templates - gmail - website names: - name: '@' - TXT resource record texts: data: - foobar-site-verification-123456 - name: paloalto - CNAME resource record forwarding: (custom TTL) ttl: 60 target: flaky.cloud.example.com. - name: losangeles - A and AAAA resource records addresses: literals: - 192.0.2.99 - 2001:db8:200::99 github.com/egymgmbh/papudns

Templates templates: - template: gmail description: > This template adds Google mailservers to a zone. www . egym . de . $NULL names: - name: '@' mail: ttl: 604800 # 1 week = 604800 seconds mailservers: - mailserver: aspmx.l.google.com. priority: 10 - mailserver: alt1.aspmx.l.google.com. priority: 20 - name: google._domainkey texts: data: - > v=DKIM1; k=rsa; p=foobar123456 github.com/egymgmbh/papudns

Go and YAML ● Package yaml.v2 ○ gopkg.in/yaml.v2 func Unmarshal(in []byte, out interface{}) (err error) ● ○ Byte stream to custom struct type ○ Struct fields are only unmarshalled if they are exported (have an upper case first letter), and are unmarshalled using the field name lowercased as the default key. ● See demo!

Go and DNS ● Pure Go resolver ○ export GODEBUG=netdns=go # force pure Go resolver ● Cgo resolver ○ export GODEBUG=netdns=cgo # force cgo resolver ● Raw DNS queries ○ github.com/miekg/dns/ By default the pure Go resolver is used, because a blocked DNS request consumes only a goroutine, while a blocked C call consumes an operating system thread. https://golang.org/pkg/net/#Resolver

On the wire: Pure Go resolver vs. Cgo resolver 16:33:52.097709 IP (tos 0x0, ttl 64, 16:36:26.279509 IP (tos 0x0, ttl 64, id 53695, offset 0, flags [DF], proto id 15282, offset 0, flags [DF], proto UDP (17), length 88) UDP (17), length 88) force.59722 > force.55245 > google-public-dns-a.google.com.domain google-public-dns-a.google.com.domain : [bad udp cksum 0x7757 -> 0x35c8!] : [bad udp cksum 0x7757 -> 0x6724!] 18325+ [1au] AAAA? danrl.com. ar: . 10166+ [1au] AAAA? danrl.com. ar: . OPT UDPsize=4096 DO (60) OPT UDPsize=4096 DO (60)

Resolvers ● See demo!

CNAME www.egym.de. CNAME dualstack.egym-server-1779992439.eu-west-1.elb.amazonaws.com. AAAA 2a01:578:3::36d9:cf8b 2a01:578:3::2e89:6e8a 2a01:578:3::36f7:bfb5

Putting it all together ● DNS check ○ Install dns-check ○ Define expectations ○ Reality check ● PaPuDNS ○ Install PaPuDNS ○ Define zone information ○ Deploy!

Summary ● We started building tools in Go ○ SRE ❤ ฀ Go ● We intentionally NOT use goroutines for critical deployments ○ We want humans (slow) to be able to veto the tool’s actions (fast) ● TODO: Use goroutines/channels for dns-check ● TODO: Use single source of truth for monitoring, too ○ And see if that works well (debatable)

- career.egym.com (we are growing!) - code.egym.de (dev blog) - github.com/egymgmbh/papudns (deploy tool) - github.com/egymgmbh/dns-tools (monitoring tool) Time to socialize! - @danrl_com (twitter) - danrl.com (my homepage) - github.com/danrl/playground-2017-08-gopher-meetup (today’s code) What time is it?