CS615 - Aspects of System Administration Configuration Management - - PowerPoint PPT Presentation

▶
cs615 aspects of system administration configuration
SMART_READER_LITE
LIVE PREVIEW

CS615 - Aspects of System Administration Configuration Management - - PowerPoint PPT Presentation

Oct 17, 2023 651 likes •1.19k views

CS615 - Aspects of System Administration Slide 1 CS615 - Aspects of System Administration Configuration Management Department of Computer Science Stevens Institute of Technology Jan Schaumann jschauma@stevens-tech.edu

slide-1
SLIDE 1

CS615 - Aspects of System Administration Slide 1

CS615 - Aspects of System Administration Configuration Management

Department of Computer Science Stevens Institute of Technology Jan Schaumann jschauma@stevens-tech.edu https://www.cs.stevens.edu/~jschauma/615A/

Configuration Management April 9, 2018

slide-2
SLIDE 2

CS615 - Aspects of System Administration Slide 2

To the backups!

Configuration Management April 9, 2018

slide-3
SLIDE 3

CS615 - Aspects of System Administration Slide 3

HW Review

Use your words!

Configuration Management April 9, 2018

slide-4
SLIDE 4

CS615 - Aspects of System Administration Slide 4

Entropy is the Enemy

The entropy of an isolated system never decreases.

Configuration Management April 9, 2018

slide-5
SLIDE 5

CS615 - Aspects of System Administration Slide 5

Entropy is the Enemy

A static system is a useless system. A useful system is being used. data is processed; files are created, modified, removed software is added, upgraded, removed systems are created, copied, decommissioned instances / containers are even more short-lived, coming into existence and disappearing again as needed

Configuration Management April 9, 2018

slide-6
SLIDE 6

CS615 - Aspects of System Administration Slide 6

Single Systems are Fragile

Individual systems created and configured by hand are fragile. Our processes need to be repeatable, automated, reliable. Recall previous lectures: OS installation package management multi-user basics automation recovery / restores

Configuration Management April 9, 2018

slide-7
SLIDE 7

CS615 - Aspects of System Administration Slide 7

Reproducable

“Never trust a computer you can’t throw out the window.” – Woz

Configuration Management April 9, 2018

slide-8
SLIDE 8

CS615 - Aspects of System Administration Slide 8

Evolution of Configuration Management

“I set up a server over here to do X. Replicate that setup on all the

  • thers.”

Configuration Management April 9, 2018

slide-9
SLIDE 9

CS615 - Aspects of System Administration Slide 9

Evolution of Configuration Management

“I set up a server over here to do X. Replicate that setup on all the

  • thers.”

“I know how to do this! Watch me!” $ ssh root@server1 # rsync -e ssh -avz / server2:/ “/etc? What’s that?”

Configuration Management April 9, 2018

slide-10
SLIDE 10

CS615 - Aspects of System Administration Slide 10

Evolution of Configuration Management

shareable content unshareable content static data /usr /boot /opt /etc variable data /home /tmp /var/mail /var/run

Configuration Management April 9, 2018

slide-11
SLIDE 11

CS615 - Aspects of System Administration Slide 11

Every Sysadmin ever...

  • 1. scp(1)

Configuration Management April 9, 2018

slide-12
SLIDE 12

CS615 - Aspects of System Administration Slide 12

Every Sysadmin ever...

  • 1. scp(1)
  • 2. rsync(1)

Configuration Management April 9, 2018

slide-13
SLIDE 13

CS615 - Aspects of System Administration Slide 13

Every Sysadmin ever...

  • 1. scp(1)
  • 2. rsync(1)
  • 3. some sort of parallel ssh(1) of the above

Configuration Management April 9, 2018

slide-14
SLIDE 14

CS615 - Aspects of System Administration Slide 14

Every Sysadmin ever...

  • 1. scp(1)
  • 2. rsync(1)
  • 3. some sort of parallel ssh(1) of the above
  • 4. switch to pull

Configuration Management April 9, 2018

slide-15
SLIDE 15

CS615 - Aspects of System Administration Slide 15

Every Sysadmin ever...

  • 1. scp(1)
  • 2. rsync(1)
  • 3. some sort of parallel ssh(1) of the above
  • 4. switch to pull
  • 5. add mutual authentication

Configuration Management April 9, 2018

slide-16
SLIDE 16

CS615 - Aspects of System Administration Slide 16

Every Sysadmin ever...

  • 1. scp(1)
  • 2. rsync(1)
  • 3. some sort of parallel ssh(1) of the above
  • 4. switch to pull
  • 5. add mutual authentication
  • 6. but effectively ignore mismatches, because doing things the right

way is difficult and inconvenient

Configuration Management April 9, 2018

slide-17
SLIDE 17

CS615 - Aspects of System Administration Slide 17

Every Sysadmin ever...

  • 1. scp(1)
  • 2. rsync(1)
  • 3. some sort of parallel ssh(1) of the above
  • 4. switch to pull
  • 5. add mutual authentication
  • 6. but effectively ignore mismatches, because doing things the right

way is difficult and inconvenient

  • 7. switch to push with remote dæmon

Configuration Management April 9, 2018

slide-18
SLIDE 18

CS615 - Aspects of System Administration Slide 18

Every Sysadmin ever...

  • 1. scp(1)
  • 2. rsync(1)
  • 3. some sort of parallel ssh(1) of the above
  • 4. switch to pull
  • 5. add mutual authentication
  • 6. but effectively ignore mismatches, because doing things the right

way is difficult and inconvenient

  • 7. switch to push with remote dæmon
  • 8. write an inventory database

Configuration Management April 9, 2018

slide-19
SLIDE 19

CS615 - Aspects of System Administration Slide 19

Every Sysadmin ever...

  • 1. scp(1)
  • 2. rsync(1)
  • 3. some sort of parallel ssh(1) of the above
  • 4. switch to pull
  • 5. add mutual authentication
  • 6. but effectively ignore mismatches, because doing things the right

way is difficult and inconvenient

  • 7. switch to push with remote dæmon
  • 8. write an inventory database
  • 9. deploy a well-known CM system

Configuration Management April 9, 2018

slide-20
SLIDE 20

CS615 - Aspects of System Administration Slide 20

Every Sysadmin ever...

  • 1. scp(1)
  • 2. rsync(1)
  • 3. some sort of parallel ssh(1) of the above
  • 4. switch to pull
  • 5. add mutual authentication
  • 6. but effectively ignore mismatches, because doing things the right

way is difficult and inconvenient

  • 7. switch to push with remote dæmon
  • 8. write an inventory database
  • 9. deploy a well-known CM system

Finally: find something it can’t do, goto 1.

Configuration Management April 9, 2018

slide-21
SLIDE 21

CS615 - Aspects of System Administration Slide 21

Base configuration vs. service definition

Your servers have unique, yet predictable properties. E.g.: network configuration critical services: DNS, NTP , Syslog minimum OS / software version user management common service configuration (e.g. sshd(8)) ...

Configuration Management April 9, 2018

slide-22
SLIDE 22

CS615 - Aspects of System Administration Slide 22

Base configuration vs. service definition

Different sets of servers have shared properties. For example, consider an HTTP server: minimum server software appropriate TLS specification shared TLS certificate and key database configuration static content (HTML / JS / CSS files) ...

Configuration Management April 9, 2018

slide-23
SLIDE 23

CS615 - Aspects of System Administration Slide 23

Pets vs. Cattle

“Pets”: unique, cheerful hostnames single systems grown over time, lovingly configured by hand when sick, everybody is very concerned slowly nursed back to life “Cattle”: predictable, boring hostnames almost identical to all others centrally managed, easy to recreate when sick, they get taken out back and shot quickly replaced by another

Configuration Management April 9, 2018

slide-24
SLIDE 24

CS615 - Aspects of System Administration Slide 24

Service definitions

class syslog { include cron include logrotate package { ’syslogng’ : ensure => latest , require => Service[’syslogng’]; } service { ’syslogng’ : ensure => running , enable => true; } file { ’/etc/syslogng/syslogng.conf’: ensure => file, source => ’puppet:///syslog/syslogng.conf’, mode => ’0644’,

  • wner

=> ’root’, group => ’root’, require => Package[’syslog-ng’], notify => Service[’syslog-ng’]; ’/etc/logrotate.d/syslog-ng’: ensure => file, source => ’puppet:///syslog/logrotate-syslogng’, mode => ’0644’,

  • wner

=> ’root’, group => ’root’, require => Package[’logrotate’]; } }

Configuration Management April 9, 2018

slide-25
SLIDE 25

CS615 - Aspects of System Administration Slide 25

Service definitions

package "ldap-utils" do action :upgrade end template "/etc/ldap.conf" do source "ldap.conf.erb" mode 00644

  • wner

"root" group "root" end %w{ account auth password session }.each do |pam| cookbook_file "/etc/pam.d/common-#{pam}" do source "common-#{pam}" mode 00644

  • wner

"root" group "root" notifies :restart, resources(:service => "ssh"), :delayed end end

Configuration Management April 9, 2018

slide-26
SLIDE 26

CS615 - Aspects of System Administration Slide 26

Service definitions

bundle agent sshd(parameter) { files: "/tmp/sshd_config.tmpl" perms => mog("0600","root","root"), copy_from => secure_cp("/templates/etc/ssh/sshd_config", "cf-master.example.com"); "/etc/ssh/sshd_config" perms => mog("0600","root","root"), create => true, edit_line => expand_template("/tmp/sshd_config.tmpl"), classes => if_repaired("restart_sshd"); commands: restart_sshd:: "/etc/rc.d/sshd restart" }

Configuration Management April 9, 2018

slide-27
SLIDE 27

CS615 - Aspects of System Administration Slide 27

CM Requirements

software installation

Configuration Management April 9, 2018

slide-28
SLIDE 28

CS615 - Aspects of System Administration Slide 28

CM Requirements

software installation service management / supervising

Configuration Management April 9, 2018

slide-29
SLIDE 29

CS615 - Aspects of System Administration Slide 29

CM Requirements

software installation service management / supervising file permissions / ownership

Configuration Management April 9, 2018

slide-30
SLIDE 30

CS615 - Aspects of System Administration Slide 30

CM Requirements

software installation service management / supervising file permissions / ownership static files

Configuration Management April 9, 2018

slide-31
SLIDE 31

CS615 - Aspects of System Administration Slide 31

CM Requirements

software installation service management / supervising file permissions / ownership static files host-specific data

Configuration Management April 9, 2018

slide-32
SLIDE 32

CS615 - Aspects of System Administration Slide 32

CM Requirements

software installation service management / supervising file permissions / ownership static files host-specific data command-execution

Configuration Management April 9, 2018

slide-33
SLIDE 33

CS615 - Aspects of System Administration Slide 33

CM Requirements

software installation service management / supervising file permissions / ownership static files host-specific data command-execution data collection

Configuration Management April 9, 2018

slide-34
SLIDE 34

CS615 - Aspects of System Administration Slide 34

One more layer of abstraction...

The objective of a CM system is not to make changes on a system. The objective of a CM system is to assert state.

Configuration Management April 9, 2018

slide-35
SLIDE 35

CS615 - Aspects of System Administration Slide 35

CM States

Configuration Management April 9, 2018

slide-36
SLIDE 36

CS615 - Aspects of System Administration Slide 36

Circles around things

Group your resources into sets. functional groupings services users hosts

Configuration Management April 9, 2018

slide-37
SLIDE 37

CS615 - Aspects of System Administration Slide 37

Circles around things

Configuration Management April 9, 2018

slide-38
SLIDE 38

CS615 - Aspects of System Administration Slide 38

Circles around things

Configuration Management April 9, 2018

slide-39
SLIDE 39

CS615 - Aspects of System Administration Slide 39

Circles around things

Configuration Management April 9, 2018

slide-40
SLIDE 40

CS615 - Aspects of System Administration Slide 40

CMs configure complex systems

CM systems are complex themselves. CM systems are inherently trusted. CM systems can break everything. To the degree that you can’t unbreak things afterwards. Consider: staged rollout of change sets automated error detection and rollback self-healing properties authentication and privilege

Configuration Management April 9, 2018

slide-41
SLIDE 41

CS615 - Aspects of System Administration Slide 41

Idempotence

CM systems assert state. For this, all operations must be idempotent. f(f(x)) ≡ f(x) || − 1|| ≡ | − 1|

Configuration Management April 9, 2018

slide-42
SLIDE 42

CS615 - Aspects of System Administration Slide 42

Idempotence

CM systems assert state. For this, all operations must be idempotent. f(f(x)) ≡ f(x) || − 1|| ≡ | − 1| $ cd etc

Configuration Management April 9, 2018

slide-43
SLIDE 43

CS615 - Aspects of System Administration Slide 43

Idempotence

CM systems assert state. For this, all operations must be idempotent. f(f(x)) ≡ f(x) || − 1|| ≡ | − 1| $ cd etc # not idempotent $ rm resolv.conf

Configuration Management April 9, 2018

slide-44
SLIDE 44

CS615 - Aspects of System Administration Slide 44

Idempotence

CM systems assert state. For this, all operations must be idempotent. f(f(x)) ≡ f(x) || − 1|| ≡ | − 1| $ cd etc # not idempotent $ rm resolv.conf # idempotent $ echo "nameserver 192.168.0.1" > resolv.conf

Configuration Management April 9, 2018

slide-45
SLIDE 45

CS615 - Aspects of System Administration Slide 45

Idempotence

CM systems assert state. For this, all operations must be idempotent. f(f(x)) ≡ f(x) || − 1|| ≡ | − 1| $ cd etc # not idempotent $ rm resolv.conf # idempotent $ echo "nameserver 192.168.0.1" > resolv.conf # idempotent $ echo "nameserver 192.168.0.2" >> resolv.conf

Configuration Management April 9, 2018

slide-46
SLIDE 46

CS615 - Aspects of System Administration Slide 46

Idempotence

CM systems assert state. For this, all operations must be idempotent. f(f(x)) ≡ f(x) || − 1|| ≡ | − 1| $ cd etc # not idempotent $ rm resolv.conf # idempotent $ echo "nameserver 192.168.0.1" > resolv.conf # idempotent $ echo "nameserver 192.168.0.2" >> resolv.conf # not idempotent $ chown root:wheel resolv.conf

Configuration Management April 9, 2018

slide-47
SLIDE 47

CS615 - Aspects of System Administration Slide 47

Idempotence

CM systems assert state. For this, all operations must be idempotent. f(f(x)) ≡ f(x) || − 1|| ≡ | − 1| $ cd etc # not idempotent $ rm resolv.conf # idempotent $ echo "nameserver 192.168.0.1" > resolv.conf # idempotent $ echo "nameserver 192.168.0.2" >> resolv.conf # not idempotent $ chown root:wheel resolv.conf # idempotent $ chmod 0644 resolv.conf # idempotent

Configuration Management April 9, 2018

slide-48
SLIDE 48

CS615 - Aspects of System Administration Slide 48

Convergence and Eventual Consistency

Note: idempotence does not guarantee efficiency! CM systems should ensure changes are:

  • 1. idempotent (well, that part’s on you)
  • 2. only applied if needed
  • 3. eventually consistent

This often requires complexity (oh no!), coordination with and awareness

  • f other systems. Service Orchestration has developed as a separate,

related discipline to help address this.

Configuration Management April 9, 2018

slide-49
SLIDE 49

CS615 - Aspects of System Administration Slide 49

Distributed Systems

CM systems are distributed systems. As such, they are subject to the CAP Theorem: Consistency: all systems managed by the CM are consistent within their respective service definition. Availability: the services managed by the CM are kept available, even if no further updates or change sets can be retrieved. Partition tolerance: the CM system can (continue to) operate despite interruptions between its components; e.g. intermediate (coordinated) changes are not required.

Configuration Management April 9, 2018

slide-50
SLIDE 50

CS615 - Aspects of System Administration Slide 50

More than just servers...

Configuration Management is not just for servers. You also need to manage configurations for: network equipment load balancers containers ...

Configuration Management April 9, 2018

slide-51
SLIDE 51

CS615 - Aspects of System Administration Slide 51

Overlap with other systems

Configuration Management April 9, 2018

slide-52
SLIDE 52

CS615 - Aspects of System Administration Slide 52

Reading

Additional topics to research: Service Orchestration Continuous Deployment / Continuous Integration Infrastructure as Code Information Technology Infrastructure Library (ITIL) Relevant links: http://www.infrastructures.org/bootstrap/recovery.shtml https://is.gd/paZ7qu https://blog.engineyard.com/2014/pets-vs-cattle http://markburgess.org/blog cap.html http://markburgess.org/blog cap2.html https://aws.amazon.com/opsworks/chefautomate/ https://puppet.com/product/managed-technology/aws

Configuration Management April 9, 2018