Puppet: How and Why Luke Kanies luke@reductivelabs.com Founder, - PowerPoint PPT Presentation

Puppet: How and Why Luke Kanies luke@reductivelabs.com Founder, Reductive Labs Nashville, Tennessee USA Why it exists, how it works, and why it works this way

Automation tools in general

Not exactly modern Image from http://flickr.com/photos/silverwood/593965547/

In fact, they kinda suck Image from http://flickr.com/photos/jefframone/1426716646/ Why?

O SSH

Developer Sysadmin * How many of you have written software to manage computers? * How many have published this software?

Three people Developer Sysadmin * How many of you have written software to manage computers? * How many have published this software?

Somebody has to do something, and it's just incredibly pathetic that it has to be us. -- Jerry Garcia

SSH Cfengine ? We needed something better

SSH Cfengine ? A tool you can’t a � ord not to adopt

Image from http://flickr.com/photos/13035641@N00/270353459/ I want Puppet to be the equivalent of bringing a gun to a knife fight. This analogy works with agriculture, metalworking, or nearly any tech., but it’s easiest with war

But that’s still not enough

What is a sysadmin? Image from http://flickr.com/photos/shirleytwofeathers/2068713495/ Firefighter? Architect? Developer? Tape-changer? All of the above?

Image from http://flickr.com/photos/kenskritters/2128853769/ I want to cause sysadmin speciation. These are house finches, reminding one of the finches Darwin observed in the Galapagos

People are finally figuring out puppet and how it gets you to the pub by 4pm. Note that I've been at this pub since 2pm. -- Jorge Castro

Either you can manage many machines with little effort

Either you can manage many machines with little effort Or you can’t

How do we create that tool?

An Analogy Programming SysAdmin Low-level, non- commands and Assembly portable files Abstract, portable C* Resources * For small values of abstract * The assembly programmers fought the adoption of C * Fear for your career if you’re a bit too fond of assembly * It’s not about few people, it’s about higher quality and productivity * Are there more or fewer programmers today than in the days of assembly?

Infrastructure 2.0 This is a joke, kind of. Talk about going to Web 2.0. We’re stealing their ideas and using them to make our infrastructure better. In general, we need to steal more ideas.

Abstraction

Portable Resources This:

Portable Resources This: Becomes:

Portable Resources This: Becomes:

Portable Resources This: Becomes:

Portable Resources This: Becomes:

Portable Resources This: Becomes:

Resource Providers 23 package types Users in NetInfo, useradd, pw Support for Debian, Ubuntu, Red Hat, Solaris, OS X, Gentoo, SuSE, FreeBSD, and more

Your infrastructure can use µf, too Hang out on this slide, make the point

Reuse

“...we’ve just switched from CVS to SVN, and it’s awesome”

Your Infrastructure is a program

Same concept, different code Debian We’re doing the same thing with different commands on different platforms

Same concept, different code Debian Red Hat We’re doing the same thing with different commands on different platforms

Same concept, different code Debian Red Hat We’re doing the same thing with different commands on different platforms

Portability and Naming

One solution per problem

Network Effects

Completeness

Relationships matter but are often implicit

Relationships matter but are often implicit Package

Relationships matter but are often implicit Configuration should get Package modifed after package installation Configuration

Relationships matter but are often implicit Configuration should get Package modifed after package installation Service should restart when Configuration configuration changes Service

Relationships matter We’ll come back to abstraction

Classes provide Intent This is shareable, releasable code. Classes are analogous with tags

Puppet as a tool

Centralized Management Code Puppetmasterd puppetd puppetd in the OS X puppetd cloud Linux

Each host gets a Resource Catalog

Node Classification

Node Classification

Node Classification

SSH Resources We’ll come back to abstraction

So You’ve Got a Resource Catalog

The Configuration Process * You can change the runinterval * You can trigger runs through puppetrun, SIGUSR1, or puppetd --test

The Configuration Process 1. Retrieve resource catalog from central server * You can change the runinterval * You can trigger runs through puppetrun, SIGUSR1, or puppetd --test

The Configuration Process 1. Retrieve resource catalog from central server 2. Determine resource order * You can change the runinterval * You can trigger runs through puppetrun, SIGUSR1, or puppetd --test

The Configuration Process 1. Retrieve resource catalog from central server 2. Determine resource order 3. Check each resource in turn, fixing if necessary * You can change the runinterval * You can trigger runs through puppetrun, SIGUSR1, or puppetd --test

The Configuration Process 1. Retrieve resource catalog from central server 2. Determine resource order 3. Check each resource in turn, fixing if necessary 4. Rinse and repeat, every 30 minutes * You can change the runinterval * You can trigger runs through puppetrun, SIGUSR1, or puppetd --test

Transactions (for each resource)

Transactions (for each resource) 1. Retrieve current state (e.g., by querying dpkg db or doing a stat)

Transactions (for each resource) 1. Retrieve current state (e.g., by querying dpkg db or doing a stat) 2. Compare to desired state

Transactions (for each resource) 1. Retrieve current state (e.g., by querying dpkg db or doing a stat) 2. Compare to desired state 3. Fix, if necessary (or just log)

Configurations are idempotent Idempotency is what allows us to manage a machine through its whole lifecycle

Configurations are idempotent Idempotency is what allows us to manage a machine through its whole lifecycle

Idempotency allows management through the lifecycle

Resource sorting is done via dependencies In this context, I sometimes call the Resource Catalog the ‘Resource Graph’

A Simple Transaction

A Simple Transaction

A Simple Transaction

Client Reporting

Who’s using Puppet? “...at Google we're currently using Puppet to manage close to 6,000 Macs, and it's likely our deployment will expand dramatically beyond that....” Testimonials . . .at Google we're currently using it to manage close to 6,000 Macs, and it's likely our deployment will expand dramatically beyond that. . . Nigel Kersten MacOps

Puppet vs. Capistrano

Puppet vs. Cfengine

It scales like HTTPS And you don’t even need to centralize it.

All communication is via XMLRPC over HTTPS And moving to REST over HTTPS

Uses SSL , and provides a Certificate Authority * Every connection is encrypted, and the only connection that isn’t authenticated is the one that asks for a signed cert * Client certs * Autosign, manual sign, manual certificate generation * You don’t even have to use it

Logs go to syslog (by default)

Written in Ruby • 1 to 1 test code to real code (and pretty good coverage) • Plugins are nearly always drop-in (resource types, providers, reports, etc.)

Language and Library C was a language and a library, Puppet is a framework and a tool

An api * Discovery * Replace webmin in 20 mins * etc.

ralsh - a thin API wrapper This uses the same model as the rest of puppet -- it chooses the appropriate provider for the local system. You can edit resources, and it even works over the network.

Image from http://flickr.com/photos/pingnews/132543603/ Virtualization

This is all I do

Bad product, hungry Luke

Full time since March 2005

Support, Consulting, Training, and more

Other Software

An Irony Puppet exposes Your Next Big Problem

Puppet is plumbing We’re producing software to take more advantage of it. We’re beginning to build a Puppet ecosystem.

In the Future • Discovery • Node Classification • Probably much more :)

Questions?