[PPT] - Transactional Systems: Examples Core OS / RedHat: Various: SUSE: PowerPoint Presentation

SLIDE 1

SLIDE 2

SLIDE 3

Transactional Systems: Examples

Core OS / RedHat: Various: SUSE:

SLIDE 4

Common Properties of Transactional Systems

Transactional / atomic updates

Read-only root file system
Focus on large deployments (clouds)
r embedded systems
Minimal base system
Automatic updates / reboots
Integrity protection

SLIDE 5

What Is a Transactional Update?

A Transactional update is an update that

Is atomic

– Either fully applied or not applied at all – Update does not influence the running system

Can be rolled back

– A failed or incompatible update can be quickly discarded to restore the previous system condition

– ➔ This definition is valid for all atomic distributions ➔ Atomicity guaranteed by rebooting the system ➔ Apart from that, implementations vary widely

SLIDE 6

Transactional Updates in the SUSE World

SLIDE 7

SUSE Tools

Zypper for package management
Snapper for snapshot creation
Btrfs as default file system

SLIDE 8

Updates with snapper

current / Backup / (pre) Backup / (post) Active system

1. Create “pre” snapshot
2. Update the current system
3. Create “post” snapshot
Update is modifying the currently

active file system

Restarts services immediately

SLIDE 9

Updates with snapper

A Transactional Update is an update that

Is atomic

– Either fully applied, or not applied at all – Update does not influence the running system

Can be rolled back

– A failed or incompatible update can be quickly discarded to restore the previous system condition

SLIDE 10

Updates with transactional-update

current / (ro) new /

1. Create new target snapshot
2. Update system, make snapshot

read-only and set as default for next boot

Current root file system is not

modified

No backup snapshots required, due

to ro system

Will not do anything if there are no

updates

Discard snapshot in case of errors

Active system

SLIDE 11

Updates with snapper

A Transactional Update is an update that

Is atomic

– Either fully applied, or not applied at all – Update does not influence the running system

Can be rolled back

– A failed or incompatible update can be quickly discarded to restore the previous system condition ➢ Reboot will provide atomicity

SLIDE 12

Helper Applications: health-checker

Add your own checker scripts to check for system consistency
Automatic rollback if checks fail on first boot
GRUB2: Select old kernel if system wasn’t able to complete initrd

current / (ro) new /

SLIDE 13

Comparison & Usage

SLIDE 14

Comparison to Other Read-Only Systems

No separate tooling or infrastructure needed
Works with any standards-compliant RPM package
General-purpose tool: especially useful for servers and clusters
Fast snapshot switching
Sane /etc and /var handling
Only works with BTRFS root file systems

SLIDE 15

SLIDE 16

SLIDE 17

SLIDE 18

Live Demo

SLIDE 19

Cheat Sheet

Transactional Updates

List repositories zypper lr -d zypper lr -d Refresh repositories zypper ref zypper ref Update installed packages transactional-update up transactional-update up Perform a distribution update transactional-update dup transactional-update dup Install package(s) transactional-update pkg in <name> transactional-update pkg in <name> Update package(s) transactional-update pkg up <name> transactional-update pkg up <name> Remove package(s) transactional-update pkg rm <name> transactional-update pkg rm <name> List snapshots snapper list snapper list Mark snapshots for removal by snapper transactional-update cleanup transactional-update cleanup View default subvolume btrfs subvolume get-default / btrfs subvolume get-default / Open shell transactional-update shell transactional-update shell Request reboot transactional-update reboot transactional-update reboot System rollback transactional-update rollback [number] transactional-update rollback [number]

SLIDE 20

A Deeper Look

SLIDE 21

Handling of Special Directories

Writable directories on an otherwise read-only system:

/var
/etc

Snapshots are only taken of the root file system by default!

SLIDE 22

/var handling /var handling

/var is a special directory because it contains variable data
Has to have read-write permissions
Cannot be rolled back
A rollback would usually delete production data (e.g., databases or

Docker images)

Typically stored on a separate subvolume or partition
/var will not be mounted into the update snapshot (i.e. packages can

not modify it)

Special handling: directories will be created on next boot
Files can be created with systemd-tmpfiles; for more complex

initialization or migration tasks, use systemd services on first boot

SLIDE 23

/etc handling

On read-only systems, /etc has to be writable
Mounted as an overlay file system
Overlays stored in /var/lib/overlays
Each snapshot has one corresponding /etc overlay
Overlay will be mounted during the update
Before: Only one overlay for all snapshots
/etc overlays are stacked
Changes accumulated before the next boot will be visible both in the

current and the next snapshot (if not changed in both overlays)

On rollback, only the layers up to that snapshot will be mounted

SLIDE 24

/etc handling

To prevent the overlayfs stack from growing infinitely, the /etc state of

the oldest available snapshot is synchronized into the new snapshot

The /etc directory in a snapshot (/.snapshots/<num>/snapshot/etc)

alone does not represent a consistent system state!

Do not mess with the files in /var/lib/overlays (although, you can use

them as a backup source if you want)

ve

r l a y / e t c

ve

r l a y de f a ul t s , uppe r di r =/ s ys r

ot

/ va r / l i b/

ve

r l a y/ 18/ e t c , l

we

r di r =/ s ys r

f
r

=/ va r / l i b/

ve

r l a y , x- s ys t e m d. r e qui r e s

m ount

s

f
r

=/ s ys r

ot

/ va r , x- s ys t e m d. r e qui r e s

m ount

s

f
r

=/ s ys r

ot

/ va r / l i b/

ve

r l a y

ve

r l a y / e t c

ve

r l a y de f a ul t s , uppe r di r =/ s ys r

ot

/ va r / l i b/

ve

r l a y/ 18/ e t c , l

we

r di r =/ s ys r

f
r

=/ va r / l i b/

ve

r l a y , x- s ys t e m d. r e qui r e s

m ount

s

f
r

=/ s ys r

ot

/ va r , x- s ys t e m d. r e qui r e s

m ount

s

f
r

=/ s ys r

ot

/ va r / l i b/

ve

r l a y

SLIDE 25

Other Directories

/opt, /var/log and /boot/grub2 will be bind mounted into the update

snapshot

Everything else, including /srv, won’t!

➔ Packages have to follow the FHS and packaging guidelines

SLIDE 26

Mount Summary

/ (ro) / Active system Visible during update /var /etc

layer x

/etc

layer x+1

/opt /opt Bind mounts: /var/ log /var/ log /bo grub arc /bo grub arc /srv /home ...

SLIDE 27

Utilities

SLIDE 28

Helper Applications: rebootmgr

transactional-update.timer triggers daily update including reboot
rebootmgr manages reboot (e.g. in maintenance windows or

synchronized via etcd)

SUSE CaaS Platform users: not used there; use Velum to trigger

cluster-wide reboots instead

SLIDE 29

Other

Configuration file: /etc/transactional-update.conf
Snapper will clean up old snapshots
transactional-update is the only way to update a read-only system
Zypper integration is planned

SLIDE 30

Pitfalls

Snapshots will be branched from the current system

→ snapshots will not contains the previous snapshot’s contents if the system hasn’t been rebooted!

When using transactional-update on a read-write system

→ Don’t forget to reboot your system before making any changes to the root file system!

SLIDE 31

What’s Next?

SLIDE 32

Availability

SUSE Linux Enterprise Server Service Pack 1:

Technology Preview
Compatible with most

packages (~30 packages left out of 10,000)

See documentation for a

list of unsupported packages

SLIDE 33

Availability

SLIDE 34

Future Development

Integration into SUSE Linux Enterprise Server
IMA / EVM support for system verification / integrity
Integrate transactional-update as zypper plugin
Fix remaining RPM packages with scripts modifying /var and /srv

SLIDE 35

SLIDE 36