PRESIDENTIAL CAMPAIGNS & IMMUTABLE INFRASTRUCTURE Or, how we - - PowerPoint PPT Presentation

▶
presidential campaigns immutable infrastructure
SMART_READER_LITE
LIVE PREVIEW

PRESIDENTIAL CAMPAIGNS & IMMUTABLE INFRASTRUCTURE Or, how we - - PowerPoint PPT Presentation

Sep 20, 2023 216 likes •663 views

PRESIDENTIAL CAMPAIGNS & IMMUTABLE INFRASTRUCTURE Or, how we learned to stop worrying and love the cloud Michael E Fisher JUNE 28, 2017 F r a n k e n b u m p hello BUT FIRST, HOW DID WE GET HERE? MONTH DD, YYYY 6 feel:

slide-1
SLIDE 1

PRESIDENTIAL CAMPAIGNS & IMMUTABLE INFRASTRUCTURE

Michael E Fisher JUNE 28, 2017 Or, how we learned to stop worrying and love the cloud
slide-2
SLIDE 2
slide-3
SLIDE 3
slide-4
SLIDE 4 F r a n k e n b u m p
slide-5
SLIDE 5

hello

slide-6
SLIDE 6 MONTH DD, YYYY

BUT FIRST, HOW DID WE GET HERE?

6
slide-7
SLIDE 7 feel: intreague
slide-8
SLIDE 8 feel: intreague attention
slide-9
SLIDE 9 feel: intreague attention
slide-10
SLIDE 10 feel: intreague attention
slide-11
SLIDE 11
slide-12
SLIDE 12

EDGE EDGE STATIC FRONTENDS API GATEWAY

SERVICE SERVICE SERVICE

SERVICE DISCOVERY LOGS

DD
slide-13
SLIDE 13 feel: intreague attention
slide-14
SLIDE 14

60 elections 80 tech staff 100 immutable backends 150 serverless frontends 577 days 2,500 max QPS 82,759,676 votes

slide-15
SLIDE 15

Chaos, right? Immutable to the rescue.

slide-16
SLIDE 16

Immutable as a handshake

slide-17
SLIDE 17 feel: realization root@really-important-production-instance:~# service nginx restart ==> amazon-ebs: Authorizing SSH access on the temporary security group... ==> amazon-ebs: Launching a source AWS instance... ==> amazon-ebs: Waiting for instance to become ready... ==> amazon-ebs: Connecting to the instance via SSH... ==> amazon-ebs: Stopping the source instance... ==> amazon-ebs: Waiting for the instance to stop... ==> amazon-ebs: Creating the AMI: packer-example 1371856345 ==> amazon-ebs: AMI: ami-19601070 ==> amazon-ebs: Waiting for AMI to become ready... ==> amazon-ebs: Terminating the source AWS instance... ==> amazon-ebs: Deleting temporary security group... ==> amazon-ebs: Deleting temporary keypair... ==> amazon-ebs: Build finished. ==> Builds finished. The artifacts of successful builds are:
  • -> amazon-ebs: AMIs were created:
us-east-1: ami-19601070
slide-18
SLIDE 18

Easy:

  • Build tooling
  • Deployment
  • Resiliency

Hard:

  • Making it work

for everything

  • Getting

everyone on board

slide-19
SLIDE 19 MONTH DD, YYYY

BUILD TOOLS

19
  • Travis CI
  • Packer
  • Continuum
  • Ansible
slide-20
SLIDE 20

Let's focus in on our edge.

slide-21
SLIDE 21

EDGE EDGE STATIC FRONTENDS API GATEWAY

SERVICE SERVICE SERVICE

SERVICE DISCOVERY LOGS

DD
slide-22
SLIDE 22

Everyone is coming for you.

slide-23
SLIDE 23

Scriptkiddies and DDoS

slide-24
SLIDE 24 sub vcl_recv { if (table.lookup(edge_settings, "non_us_ban", "disabled") == "enabled") {if (geoip.country_code != "US") { error 503; } } if (client.ip ~ cc_blacklist && req.url.path == "/api/hamm/ donations" && req.request == "POST") { if (randombool(1, 2)) { error 201; } else { error 402; } } Confusion as a defense strategy
slide-25
SLIDE 25 if (table.lookup(THEWALL, client.ip) && !req.http.Fastly-FF) { error 819 "Bad Taco."; } if (req.http.user-agent ~ "^WordPress") { error 819 {"Forbidden 🌮"}; } Just ban them
slide-26
SLIDE 26 feel: curiosity / excitement solution
slide-27
SLIDE 27
slide-28
SLIDE 28

EDGE STATIC FRONTENDS API GATEWAY

SERVICE SERVICE SERVICE

SERVICE DISCOVERY LOGS

DD
slide-29
SLIDE 29

Onesie

slide-30
SLIDE 30

hillaryclinton.com/calls

  • nesie-web.s3-aws-us-

east-1.amazonaws.com/calls

  • Caching
  • Paths and query params
  • Regional failover
slide-31
SLIDE 31

EDGE S3 TRAVIS GIT

Onesie

slide-32
SLIDE 32

Let's assume the remaining traffic are legitimate folks making API calls. And they really, really want to go to Philadelphia.

slide-33
SLIDE 33

EDGE STATIC FRONTENDS API GATEWAY

SERVICE SERVICE SERVICE

SERVICE DISCOVERY LOGS

DD
slide-34
SLIDE 34 F r a n k e n b u m p
slide-35
SLIDE 35

About that infrastructure diagram...

slide-36
SLIDE 36

EDGE STATIC FRONTENDS API GATEWAY

SERVICE SERVICE SERVICE

SERVICE DISCOVERY LOGS NODEJS HOMEPAGE

DD
slide-37
SLIDE 37

REDIS SERVICE DISCOVERY LOGS

WordPress

NodeJS EDGE

Secretary of Stateless

NodeJS NodeJS NodeJS NodeJS

DD
slide-38
SLIDE 38

Biggest problem to solve: What's the state of your state?

slide-39
SLIDE 39 WORDPRESS SERVICE DISCOVERY MYSQL REDIS NODEJS

Aurora EC2 ElasticCache EC2 CONSUL

TEMPLATES

S3

slide-40
SLIDE 40

Biggest problem to solve: What's the state of your state? Abuse S3, Consul, and ELB health checks to find out.

slide-41
SLIDE 41 export default function(servers, dog = new DD) { return async (ctx) => { let path = '/health'; // only run template version checks on the ELB version of the health check if (ctx.url === '/health-elb') { try { const templateRelease = await getCurrentTemplateRelease(); path = `/health?templateRelease=${templateRelease}`; } catch (err) { // For the initial release consul will be empty, so if the consul check errors with a 404 // there's nothing wrong, just continue on with the normal health check if (err.status === 404) { log.warn('Skipping templateRelease health check since consul is empty'); } else { dog.increment('sos-template-deploy.consul-error', 1); log.error({err}, 'Error reading data from consul'); } } } try { await runHealthChecks(servers, path); } catch (err) { const message = 'Health check failed'; log.error({err}, `${message} : ${err.message}`); ctx.throw(message); } ctx.status = 200; };
slide-42
SLIDE 42 42

Takeaways

  • Immutable infrastructure was key to our technical success
  • We moved quickly but were resilient against failure (most of the

time)

  • It takes more effort to apply immutable to everything you're

doing, but it's worth it

  • Ultimately, developers like the handshake between SRE and dev
slide-43
SLIDE 43 43

Takeaways

  • On a presidential campaign, innovation is a necessity, and there

aren't any hard and fast rules in tech

  • It's difficult to imagine where infrastructure tech will be in four

years, but the next campaigns will be leveraging the most exciting stuff out there

slide-44
SLIDE 44 44

Takeaways

  • You can build cool shit and work in public service