Nobody ever regretted making a backup AsiaBSDCon 2013 a tutorial, - PDF document

Nobody ever regretted making a backup AsiaBSDCon 2013 a tutorial, by Dan Langille – http://langille.org/ Tutorial Description: Nobody ever got famous writing backup software, yet backups form one of the fundamental basics of system administration. There are many solutions to this simple problem: • do nothing • ad hoc solutions • open source packages • commercial packages Not only is it important to backup data securely and reliably, you need to be able to restore the data. You need to know what was backed up, from what system, when, and where is that backup now. In the past decade, Bacula has been gaining ground on more established solutions, both open source and proprietary. This talk will introduce you to Bacula, show you the main components, give you an outline of how it works, and illustrate why Bacula is becoming so widespread. In this tutorial, you will learn the basics of Bacula, a leading open source backup solution. As a Bacula developer, Dan has some unique insights into the use and deployment of Bacula. An avid user since 2004, he has used Bacula for his own networks and in commercial settings. Topics to cover will include: - overview of Bacula: client, storage, director - Jobs - Schedules - Catalogs - Pools - FileSets - console commands - the various retention settings and how they affect your catalog - which database is best for use with Bacula - the difference between a Bacula volume and a disk volume - running a job - restoring a job - automatic pruning - recycling of Volumes - using tape libraries - copy/migrate jobs from one media to another - spooling

overview of Bacula: client, storage, director Key point: The Director (DIR) contacts the File Daemon (FD/client) and says: backup this FileSet to this Storage Daemon (SD). Vital: allow TCP connections between those components (i.e. hosts.allow, firewall, etc) Heartbeat: keep alive signal for long running jobs under some operating systems / firewalls. Can be specified on both SD & FD. A single DIR can run many hundreds/thousands of clients. Your limiting factor is SD. You can have many SD, easily. You can run many Catalog too.

Command line tools: • bconsole • bcopy • bextract • bregex • bscan • bsmtp • btape • btraceback • bwild • dbcheck

Context • Unless otherwise specified, assume all resources mentioned are entries in bacula-dir.conf

SD and FD configuration • Every SD and FD needs at least one entry like this: Director { Name = your-dir Password = "Jg6JfArzRtVUyNNELcXF6tdrTpRvi" } • The above line identifies the DIR which is allowed to contact this FD or SD and the name and password it must use. • NOTE: the above is the DIR name, not the FD/SD name. But it is the password for the FD/SD as found in the bacula-dir.conf file. • Thus, every password is stored in two locations: • In the FD/SD configuration file • In the bacula-dir.conf file • This is a “shared secret” between the FD/SD and the DIR • The DIR will have a similar entry within a Client (for the FD) or Storage (for the SD) resource within bacula-dir.conf Storage { Name = MySD Address = storage1.example.org SDPort = 9103 Password = "same password as in bacula-sd.conf" Device = FileStorage Media Type = File } Client { Name = laptop-fd Address = laptop.example.org Catalog = MyCatalog Password = "same password as in bacula-fd.conf" File Retention = 3 years Job Retention = 3 years } • Knowing the above will save you much grief when adding new FD/SD.

Client • Entry in bacula-dir.conf file • identifies a client to backup • A Client runs bacula-fd.conf • Client resource identifies the Address of the Client (i.e. hostname) • Catalog to use for that Client • Password that this DIR must use to contact that client • Other DIR might have different password • Retention periods (File & Job; more on retention later) Client { Name = nyi-fd Address = nyi.example.org FDPort = 9102 Catalog = MyCatalog Password = "ICDf+XmbWH3Mw753xbc6hv3gQbNc6WjyAP" TLS Require = yes TLS Enable = yes TLS CA Certificate File = /home/bacula/certificates/ca.crt TLS Certificate = /home/bacula/certificates/client.crt TLS Key = /home/bacula/certificates/client.key File Retention = 3 years Job Retention = 3 years }

Jobs • A job runs on exactly one client • A job consists of exactly one FileSet • A job backs up to exactly one SD • A job has just one schedule, which can be simple or complex • You can have multiple jobs per client Job { Name = "nyi basic" JobDefs = "DefaultJobRemote" Client = "nyi-fd" FileSet = "basic backup" Write Bootstrap = "/home/bacula/working/nyi-fd-basic.bsr" } Job { Name = "nyi" JobDefs = "DefaultJobRemote" Client = "nyi-fd" FileSet = "Production Web Server" Write Bootstrap = "/home/bacula/working/nyi-fd-web.bsr" }

JobDefs • Many jobs are similar • group the similarities into a JobDefs (i.e. Job Defaults) • Common job definitions include • Type – usually backup • Level (full, incremental, differential) • Schedule (when to run Jobs using this JobDefs) • Storage (where to put the backups) • Pool (what Pool to use) • FileSet (what to backup) • Use a JobDef so your Job really specifies how this Job differs from other Jobs, for example, a minimal Job could be: • Name • JobDefs • Client JobDefs { Name = "DefaultJobRemote" Type = Backup Level = Incremental Client = ngaio-fd FileSet = "Full Set" Schedule = "WeeklyCycle" Storage = MegaFile Messages = Standard Pool = FullFile # required parameter for all Jobs Full Backup Pool = FullFile Differential Backup Pool = DiffFile Incremental Backup Pool = IncrFile Priority = 20 Spool Data = no Spool Attributes = yes }

Job Level • Full – backup everything according to the FileSet • Incremental - all files specified in the FileSet that have changed since the last successful backup of the the same Job using the same FileSet and Client • Differential - all files specified in the FileSet that have changed since the last successful Full backup of the same Job and client will be backed up • From the Bacula FAQ: • Before doing an Incremental or a Differential backup, Bacula checks to see if there was a prior Full backup of the same Job that terminated successfully. If so, it uses the date that full backup started as the time for comparing if files have changed. If Bacula does not find a successful full backup, it proceeds to do one. • Don't let your Incremental expiry before your Differential or Full • Don't let your Differential expire before your Full • More on retention periods later.

Schedules • Jobs are run automatically according to the schedule assigned to that job • A Schedules can be used by zero or more jobs • A Schedule can indicate that a job is never run automatically (i.e. manually only) Schedule { Name = "WeeklyCycle" Run = Level=Full 1st sun at 5:55 Run = Level=Differential 2nd-5th sun at 5:55 Run = Level=Incremental mon-sat at 5:55 } Schedule { Name = "Never" }

FileSet • A FileSet is a list of files to backup • A FileSet can be used by zero or more jobs • Exactly one FileSet per job • can specify files to exclude • by default, does not span mount points (can be overridden but use with care; e.g. NFS) • Be sure to enclose path names in “quotes” for Windows clients • test your FileSet using the estimate command in bconsole FileSet { Name = "basic backup" Include { Options { signature=MD5 } Exclude Dir Containing = .NOBACKUP File = /boot File = /usr/src/sys/i386/conf File = /etc File = /usr/local/etc File = /usr/local/info File = /usr/local/libexec/nagios/ File = /usr/local/var File = /root File = /var/db/ports File = /var/log File = /var/cron } } FileSet { Name = "ngaio files" Include { Options { signature=MD5 verify=pnugsmcs5 } File = /usr/websites File = /home/dan File = /home/bacula File = /usr/jails/flavours File = /usr/local/libexec/nagios } Exclude { File = *~ File = /home/dan/tmp File = /home/dan/ports File = /home/bacula/spooling File = /home/bacula/volumes File = /home/bacula/bacula-restores } }

Volume • A Volume is a place to put a backup • It may be disk, tape (DVD – not really supported any more) • Bacula treats disk and tape the same (more or less) • A backup resides in exactly one Volume • May be automatically labeled • Makes most sense for disk Volumes and tape Volumes • To implement: LabelMedia = yes in Device resource in bacula-sd.conf • For disk, you need LabelFormat in Pool resource