Application software configuration using Heat Steve Baker Senior - - PowerPoint PPT Presentation

application software configuration using heat
SMART_READER_LITE
LIVE PREVIEW

Application software configuration using Heat Steve Baker Senior - - PowerPoint PPT Presentation

Application software configuration using Heat Steve Baker Senior Software Engineer, Red Hat sbaker@redhat.com irc stevebaker #heat Application software configuration using Heat Configuration vs Orchestration New heat software config


slide-1
SLIDE 1

Application software configuration using Heat

Steve Baker

Senior Software Engineer, Red Hat

sbaker@redhat.com irc stevebaker #heat

slide-2
SLIDE 2

Application software configuration using Heat

  • Configuration vs Orchestration
  • New heat software config and deployment resources
  • Integrating configuration tools
slide-3
SLIDE 3

Software Configuration Orchestration <>

slide-4
SLIDE 4

Separation of concerns is important

slide-5
SLIDE 5

Choosing an abstraction involves compromise

slide-6
SLIDE 6

CloudFormation cfn-init example

"Resources" : { "WikiDatabase": { "Type": "AWS::EC2::Instance", "Metadata" : { "AWS::CloudFormation::Init" : { "config" : { "packages" : { "yum" : { "mysql" : [], "mysql-server" : [], "httpd" : [], "wordpress" : [] } }, "services" : { "systemd" : { "mysqld" : { "enabled" : "true", "ensureRunning" : "true" }, "httpd" : { "enabled" : "true", "ensureRunning" : "true" } } } } } },

slide-7
SLIDE 7

CloudFormation cfn-init example

"Properties": { "UserData" : { "Fn::Base64" : { "Fn::Join" : ["", [ "#!/bin/bash -v\n", "/opt/aws/bin/cfn-init\n", "# Setup MySQL root password and create a user\n", "mysqladmin -u root password '", { "Ref" : "DBRootPassword" }, "'\n", "cat << EOF | mysql -u root --password='", { "Ref" : "DBRootPassword" }, "'\n", "CREATE DATABASE ", { "Ref" : "DBName" }, ";\n", "GRANT ALL PRIVILEGES ON ", { "Ref" : "DBName" }, ".* TO \"", { "Ref" : "DBUs "IDENTIFIED BY \"", { "Ref" : "DBPassword" }, "\";\n", "FLUSH PRIVILEGES;\n", "EXIT\n", "EOF\n", "sed -i \"/Deny from All/d\" /etc/httpd/conf.d/wordpress.conf\n", "sed -i \"s/Require local/Require all granted/\" /etc/httpd/conf.d/wordpress.con "sed --in-place --e s/database_name_here/", { "Ref" : "DBName" }, "/ --e s/usern "systemctl restart httpd.service\n", "firewall-cmd --add-service=http\n", "firewall-cmd --permanent --add-service=http\n" ]]}}

slide-8
SLIDE 8

Both have roles to play in the stack

slide-9
SLIDE 9

Configuration resource

  • API backed store of configuration data
  • Stores configuration script
  • Defines inputs and outputs schema
  • Tool specific options
  • Are immutable and can be passed by referenced
slide-10
SLIDE 10

Boot configuration with cloud-init

  • ne_init:

type: OS::Heat::CloudConfig properties: cloud_config: write_files:

  • path: /tmp/one

content: "The one is bar" two_init: type: OS::Heat::SoftwareConfig properties: config: | #!/bin/sh echo "The two is bar" > /tmp/two server_init: type: OS::Heat::MultipartMime properties: parts:

  • config: {get_resource: one_init}
  • config: {get_resource: two_init}

server: type: OS::Nova::Server properties: image: {get_param: image} flavor: {get_param: flavor} key_name: {get_param: key_name} user_data_format: RAW user_data: get_resource: server_init

slide-11
SLIDE 11

Deployment resources

  • Maps one config resource to one server resource
  • Allows assignment of server-specific input values
  • Remains in-progress until receiving completed signal
  • Stores outputs for other resources to consume as

resource attributes

  • Can deploy on any heat action, not just CREATE,

UPDATE

  • Stores additional outputs from hook invocation
  • stdin, stdout, status_code
slide-12
SLIDE 12

Deployment illustrated

config deployment server

slide-13
SLIDE 13

Deployment illustrated

config server deployment

slide-14
SLIDE 14

Deployments illustrated

config server deployment config deployment deployment

slide-15
SLIDE 15

Deployments illustrated

config server 1 deployment config deployment deployment config config server 2 deployment config deployment deployment config

slide-16
SLIDE 16

Deployments illustrated

config server 1 deployment config deployment deployment config config server 2 deployment config deployment deployment config

slide-17
SLIDE 17

Deployments illustrated

config server 1 deployment config deployment deployment config config server 2 deployment config deployment deployment config

slide-18
SLIDE 18

Deployments illustrated

config server 1 deployment config deployment deployment config config server 2 deployment config deployment deployment config

slide-19
SLIDE 19

Deployments illustrated

config server 1 deployment config deployment deployment config config server 2 deployment config deployment deployment config

slide-20
SLIDE 20

Deployments illustrated

config server 1 deployment config deployment deployment config config server 2 deployment config deployment deployment config

slide-21
SLIDE 21

Deployments illustrated

config server 1 deployment config deployment deployment config config server 2 deployment config deployment deployment config

slide-22
SLIDE 22

Deployments illustrated

config server 1 deployment config deployment deployment config config server 2 deployment config deployment deployment config

slide-23
SLIDE 23

Deployments illustrated

config server 1 deployment config deployment deployment config config server 2 deployment config deployment deployment config

slide-24
SLIDE 24

Scaling deployments illustrated

deployment server pool member config pool load balancer

slide-25
SLIDE 25

Scaling deployments illustrated

deployment server pool member config pool load balancer deployment server pool member deployment server pool member

slide-26
SLIDE 26

Deployment extra inputs

  • deploy_server_id
  • deploy_action
  • deploy_stack_id, deploy_resource_name
  • deploy_signal_id
  • deploy_auth_url, deploy_username, deploy_password,

deploy_project_id, deploy_user_id

slide-27
SLIDE 27

get_file intrinsic function

  • python-heatclient fetches local files and URLs
  • Contents of get_file calls included in heat stack-create

request

  • Initial support for including binary files
slide-28
SLIDE 28

Script and cfn-init example

config: type: OS::Heat::StructuredConfig properties: group: cfn-init inputs:

  • name: bar

config: config: files: /tmp/foo: content: get_input: bar mode: '000644' check_tmp_foo: type: OS::Heat::SoftwareConfig properties: group: script

  • utputs:
  • name: result

config: {get_file: check_tmp_foo.sh} deployment: type: OS::Heat::StructuredDeployment properties: name: 10_deployment signal_transport: NO_SIGNAL config: get_resource: config server: get_resource: server input_values: bar: baaaaa deploy_check_tmp_foo: type: OS::Heat::SoftwareDeployment properties: name: 30_deploy_check_tmp_foo config: get_resource: check_tmp_foo server: get_resource: server

slide-29
SLIDE 29

Script and cfn-init example

server: type: OS::Nova::Server properties: image: {get_param: image} flavor: {get_param: flavor} key_name: {get_param: key_name} security_groups:

  • {get_resource: the_sg}

user_data_format: SOFTWARE_CONFIG #!/bin/sh echo -n "The file /tmp/foo contains `cat /tmp/foo` for server $deploy_server_id \ during $deploy_action" > $heat_outputs_path.result

slide-30
SLIDE 30

Puppet example

config: type: OS::Heat::SoftwareConfig properties: group: puppet inputs:

  • name: foo
  • name: bar
  • utputs:
  • name: result

config: get_file: puppet-manifest.pp deployment: type: OS::Heat::SoftwareDeployment properties: config: get_resource: config server: get_resource: server input_values: foo: fooooo bar: baaaaa

slide-31
SLIDE 31

Puppet example

server: type: OS::Nova::Server properties: image: {get_param: image} flavor: {get_param: flavor} key_name: {get_param: key_name} security_groups:

  • {get_resource: the_sg}

user_data_format: SOFTWARE_CONFIG file {'barfile': ensure => file, mode => 0644, path => "/tmp/$::bar", content => "$::foo", } file {'output_result': ensure => file, path => "$::heat_outputs_path.result", mode => 0644, content => "The file /tmp/$::bar contains $::foo", }

slide-32
SLIDE 32

Image based example

BlockStorageConfig: type: OS::Heat::StructuredConfig properties: group: os-apply-config config: cinder: db: {get_input: cinder_dsn} volume_size_mb: '5000' service-password: get_param: CinderPassword iscsi-helper: get_param: CinderISCSIHelper admin-password: get_param: AdminPassword BlockStorage0Deployment: type: OS::Heat::StructuredDeployment properties: server: {get_resource: BlockStorage0} config: {get_resource: BlockStorageConfig} input_values: cinder_dsn: str_replace: template: | mysql://cinder:unset@address/cinder params: address: get_attr:

  • controller0
  • networks
  • ctlplane
slide-33
SLIDE 33

Hooks

  • Consumes JSON from stdin, writes JSON to stdout
  • Invokes configuration script with a particular tool
  • Maps config inputs to some tool-specific concepts, e.g.
  • Environment variables (scripts)
  • Facts (puppet)
  • Discovers output values after config tool execution
slide-34
SLIDE 34

Hooks illustrated

  • s-collect-config
  • s-refresh-config
  • s-apply-config

heat-config shell-hook cfn-init-hook your config script cfn-init heat nova puppet-hook puppet apply foo-hook foo script

slide-35
SLIDE 35

Available hooks

  • Script
  • cfn-init
  • Puppet
  • Golden image configuration (not actually a hook!)
slide-36
SLIDE 36

Hooks yet to write

  • Chef
  • Salt
  • Ansible
  • PowerShell
slide-37
SLIDE 37

Mapping config model to tools

tool config inputs

  • utputs
  • ptions

script the script environment variables files

  • cfn-init

declarative yaml heat get_input

  • puppet

manifest facts files extra modules? chef solo cookbook or recipe attributes attributes? databags? cookbooks? salt standalone minion SLS pillar data grains? custom returner?

  • ansible

connection local playbook variables return data?

  • powershell

ps1 scripts variables Out-File?

  • image based

config yaml heat get_input

  • s-refresh-

config curl calls

slide-38
SLIDE 38

Writing a hook

  • Consumes JSON on stdin (inputs, script, options)
  • Invokes the configuration tool to perform config
  • Writes JSON to stdout (outputs)
  • ~100 lines of python
  • Contribute your hook to

https://github.com/openstack/heat-templates

slide-39
SLIDE 39

Golden image requirements

  • os-collect-config
  • os-refresh-config
  • os-apply-config
  • heat-config os-refresh-config scripts
  • Hook for your chosen configuration tool
  • Actual configuration tool
slide-40
SLIDE 40

diskimage-builder for building disk images

git clone https://git.openstack.org/openstack/diskimage-builder.git git clone https://git.openstack.org/openstack/tripleo-image-elements.git git clone https://git.openstack.org/openstack/heat-templates.git export ELEMENTS_PATH=\ tripleo-image-elements/elements:\ heat-templates/hot/software-config/elements diskimage-builder/bin/disk-image-create vm \ fedora \ heat-config \

  • s-collect-config \
  • s-refresh-config \
  • s-apply-config \

heat-config-script \ heat-config-cfn-init \

  • o fedora-software-config.qcow2

glance image-create --disk-format qcow2 --container-format bare \

  • -name fedora-software-config < \

fedora-software-config.qcow2

slide-41
SLIDE 41

Whither the master configuration server?

  • Heat can be the central source of truth, no master

required

  • No need for the complexity of syncing heat<->master,

unless you really want to ;)

  • ...or, minimal heat config could be used to hand off

server to a config master

slide-42
SLIDE 42

Planned improvements

  • Other techniques for heat <-> server communication
  • Swift
  • Marconi
  • Servers in isolated tenant networks
  • Action-aware config resource for alignment with

TOSCA

  • Moar hoooks (chef, salt, ansible, powershell...)
  • Docker integration
  • Windows support
  • Deployments for shutdown in nova rebuild, reboot
slide-43
SLIDE 43

Questions?

sbaker@redhat.com https://wiki.openstack.org/wiki/Heat