Node.js at Cloudkick Paul Querna July 26, 2011 What is Cloudkick? - - PowerPoint PPT Presentation

node js at cloudkick
SMART_READER_LITE
LIVE PREVIEW

Node.js at Cloudkick Paul Querna July 26, 2011 What is Cloudkick? - - PowerPoint PPT Presentation

Node.js at Cloudkick Paul Querna July 26, 2011 What is Cloudkick? Cloud Server Management Dashboard across providers Metadata container (tags, etc) Monitoring as a Service Acquired by Rackspace in December 2010 Python,


slide-1
SLIDE 1

Node.js at Cloudkick

Paul Querna July 26, 2011

slide-2
SLIDE 2

What is Cloudkick?

  • Cloud Server Management
  • Dashboard across providers
  • Metadata container (tags, etc)
  • Monitoring as a Service
  • Acquired by Rackspace in December 2010
slide-3
SLIDE 3

Python, Python, Python!

  • Django
  • UI
  • API
  • Twisted
  • Services
  • Orbited
  • Realtime
slide-4
SLIDE 4

Cloudkick Classic

Webapp Orbited Rabbit MQ Services User MySQL Solr Cassandra monitoring system (complicated)

slide-5
SLIDE 5

What’s wrong?

  • Django vs Twisted
  • Twisted in general
  • RabbitMQ
  • orbited is dead
  • Multi-Region durability
slide-6
SLIDE 6

Node.js

  • Express
  • API
  • Services
  • Realtime
slide-7
SLIDE 7

New Architecture

Node.js User Cassandra monitoring system (complicated)

slide-8
SLIDE 8

In Production

slide-9
SLIDE 9

Running

  • runit / daemontools
slide-10
SLIDE 10

Configuration

  • Puppet
  • local_settings.js (django style)
slide-11
SLIDE 11

Load Balancer

  • apache 2.2, event mpm, mod_ssl,

mod_proxy_balancer

  • Want to remove it.
  • ssl/tls termination (sad face)
  • consider stunnel+haproxy. for now.
  • examining stud+haproxy:
  • https://github.com/bumptech/stud
slide-12
SLIDE 12

Internals

slide-13
SLIDE 13

package.json

slide-14
SLIDE 14

"async": "0.1.9", "cassandra-client": "0.2.1", "express": "2.4.0", "ipv6": "0.0.1", "jade": "0.10.7", "logmagic": "0.1.4", "long-stack-traces": "0.1.2", "metrics-ck": "0.0.1", "node-dev": "0.1.5", "node-uuid": "1.2.0", "nodelint": "0.4.0", "optimist": "0.2.0", "sax": "0.1.2", "showdown": "0.0.1", "strtok": "0.1.1", "swiz": "0.3.1", "thrift": "0.6.0-1", "whiskey": "0.4.0"

slide-15
SLIDE 15

npm, private registry

  • Replicate Issac’s CouchDb
  • ~/.npmrc
  • always-auth = true
  • registry = https://npm.example.com/
slide-16
SLIDE 16

npm

  • Prevent Publishing:
  • "private" : true,
  • "publishConfig" : { "registry": "http://

127.0.0.1:33333/ __invalid__registry_do_not_publish"},

slide-17
SLIDE 17

Dealing with Patches

  • Fork on github.
  • Patch in branch.
  • Publish to private NPM registry.
  • Pray upstream accepts.
  • Switch back to public package once

released.

slide-18
SLIDE 18

Flow Control

  • Async
  • https://github.com/caolan/async
slide-19
SLIDE 19

Cassandra Client

  • CQL
  • It’s like SQL, but for NoSQL (troll face)
  • Relational and Timeseries data
  • https://github.com/racker/node-cassandra-

client

slide-20
SLIDE 20

Logging

  • Log Magic:
  • https://github.com/pquerna/node-

logmagic/

  • logmagic.route("mylib.*", logmagic.DEBUG,

"stdout")

slide-21
SLIDE 21

Metrics

  • https://github.com/mikejihbe/metrics
  • Metrics Broker:
  • Collect across multiple processes
  • (not open source yet, working on it)
slide-22
SLIDE 22

Testing

  • Whiskey:
  • https://github.com/cloudkick/whiskey
  • http://cast-project.org/test-coverage/
slide-23
SLIDE 23

Linting

  • node-jshint
  • Google Closure Linter:
  • http://code.google.com/p/closure-linter/
slide-24
SLIDE 24

Continuous Integration

  • BuildBot against
  • Production (v0.4.10-99b210d7eac)
  • Stable Branch (v0.4.x)
  • Dev (master / v0.5.x)
  • (need to add --use-uv)
slide-25
SLIDE 25

Questions?

  • Slides:
  • http://paul.querna.org/slides/