Continuous deployment with Jenkins and Salt LinuxCon CloudOpen - - PowerPoint PPT Presentation

Continuous deployment with Jenkins and Salt

LinuxCon CloudOpen ContainerCon North America 2015, Seattle Anirban Saha

About me

Techie… Traveller… Thinker… Author of ‘Salt Cookbook’

Deployment

Should be…

• Simple
• Stable
• Fast (good to have…)
• Reliable
• One click

Problems faced

• Code distribution
• Additional tasks (commands, migrations)
• Latency
• Parallel execution
• Batch deployments
• Ensuring uptime

Methods available

Deployment

Configuration management tools :

• Puppet, Chef, Ansible, Salt

Remote execution tools:

• SSH, Parallel SSH, Rundeck

Packages:

• Code packaged as RPM or DEB files

Code storage

• GIT server (Github, Gitlab, Bitbucket, etc.)
• Object storage (Amazon S3, Swift, etc.)
• Package repositories (YUM/Debian)
• Configuration management repositories
• File servers managed manually

Problems

Deployment:

• Remote execution tools :
• Time consuming (over SSH)
• Need to maintain inventory of hosts

Code storage:

• GIT repositories:
• Usually centrally located, latency when deployed from GIT
• Package repositories (RPM/DEB):
• Problems in simultaneously updating repo metadata

Rollback:

• Version manipulation is a problem with most configuration

management and remote execution tools

Salt

Provides…

• Agent based communication
• Fast execution
• Orchestration
• Batch execution
• Strong Integration with providers and

services

Salt features

Targeting deployment nodes :

• By hostname
• ‘myappweb*’
• By specific grains
• -G ‘node_type:webserver’
• By nodegroups
• -N webgroup

Salt features

Task dependencies :

• Salt requisites to make tasks dependent on each other

get_archive: module.run: . . deploy_app: module.run: . .

• require:
• module: get_archive

Salt features

Orchestration : webgrp2_deploy: salt.state:

• tgt: ' webgrp2'
• tgt_type: nodegroup
• sls:
• webserver.deploy
• require:
• salt: webgrp1_deploy

webgrp1_deploy: salt.state:

• tgt: 'webgrp1'
• tgt_type: nodegroup
• sls:
• webserver.deploy

Salt features

Batch Execution :

• By number of hosts
• --batch-size 5
• Executes on 5 hosts at a time
• By percentage of hosts
• --batch-size 25%
• Executes on 25% of total target

hosts at a time

Salt features

Salt API:

# curl -H "Accept: application/json"

• d secretkey= "mysupersecretkey"
• k https://10.0.0.2:8080/hook/deploy

{"success": true}

Tools

To be used…

• GIT for code repository
• Jenkins for CI
• Ant for build and tasks
• Amazon S3 for code archive storage
• Salt for deployment
• Shell scripts (good old bash) for

additional tasks

Salt deployment methods

• Salt hosted on the Jenkins server.

Jenkins calls Salt binary

• Salt hosted independently.

Jenkins calls Salt via SSH

• Jenkins calls Salt via Salt API

Objective

Steps

• salt-cloud used to launch instances
• Post install actions used to synchronize EC2 grains
• Salt reactor used to run states on new instances ,

perform deployment and register with load balancer

• Jenkins build job used to build new code, create tags

and push new deployment ready archive to Amazon S3

• Jenkins deploy job used to fetch new code version

and deploy code on target servers

Steps explained

• Salt cloud is used to spawn instances
• Following parameter is provided in the profile to push custom EC2 grains,
• sync_after_install: grains
• The node is registered and deregistered from the load balancer with the

following module definition, register: module.run:

• name: boto_elb.register_instances
• m_name: mywebapp
• instances:
• {{ grains['ec2']['instance_id'] }}
• Here ['ec2']['instance_id'] is one of the custom grains pushed to the node

Steps explained

Fetching archive: Salt s3 execution module used to get code archive from Amazon S3 fetch_app_archive: module.run:

• name: s3.get
• bucket: mywebapp-us
• path: mywebapp-{{ app_version }}.tar
• local_file: /tmp/mywebapp-{{ app_version }}.tar

Steps explained

Setting App version grain: After every deploy, a grain is set to record the app version deployed for tracking, app_version: grains.present:

• value: {{ app_version }}
• require:
• cmd: deploy_app

Steps explained

Deployment based on App version: At every deploy, it is checked if the version to be deployed is already on the node using the app_version grain, {% if grains['app_version'] != app_version %} deregister: fetch_app_archive: deploy_app: register: {% endif %}

Lets do it then !!!

Demo repository

Get the demo repository at the following location, https://github.com/rosesnthornz/ cloudopen-na-2015

Questions ?

Contact

Email : sahaanirban1988@gmail.com Twitter : @rosesnthornz

Thank You !!!