The PBI Format Re-implemented for Free/PC-BSD Kris Moore PC-BSD / - PowerPoint PPT Presentation

The PBI Format Re-implemented for Free/PC-BSD Kris Moore PC-BSD / iXsystems kris@pcbsd.org http://www.pcbsd.org/~kris/pbi9-slides.pdf

The PBI Format Re-implemented for Free/PC-BSD The legacy PBI format

The PBI Format Re-implemented for Free/PC-BSD Features ● Method of packaging files into a single, self- contained archive – Eliminates issues with dependency resolution – Makes package management simple, less points of failure.

The PBI Format Re-implemented for Free/PC-BSD Features ● Applications / operating system separate – Applications cannot break operating system or other installed applications – PBIs may be safely updated / downgraded at will – Allows 32bit PBI packages to run on 64bit systems with no complex chroot setups.

The PBI Format Re-implemented for Free/PC-BSD Traditional Package Management

The PBI Format Re-implemented for Free/PC-BSD PBI Managed Applications

The PBI Format Re-implemented for Free/PC-BSD Drawbacks ● Implemented in QT/KDE – Other window manager integration difficult / Impossible – Made headless operation unpractical – Required xorg/qt/kde to always be installed – Running on native FreeBSD not feasible

The PBI Format Re-implemented for Free/PC-BSD Drawbacks ● Duplication of identical files – For each PBI, identical copy of libfoo.so.1 – Wasted disk / run-time memory – But necessary to maintain self-contained functionality

The PBI Format Re-implemented for Free/PC-BSD Drawbacks ● Lack of verification mechanisms – Only bare-minimum md5 check – No digital signature confirming reliability from building – Lacked method to determine where PBI originated

The PBI Format Re-implemented for Free/PC-BSD Drawbacks ● Updating required full download of PBI – Newer releases may contain up to 95% similar data – Wasted time downloading full package – Made updating from low-bandwidth areas troublesome

The PBI Format Re-implemented for Free/PC-BSD Reimplementation ● Written in shell (/bin/sh) – Uses only base-system utilities and devel/xdg-utils/ – Implemented into a single file, “ pbi-manager ” – Able to run on native FreeBSD

The PBI Format Re-implemented for Free/PC-BSD Reimplementation ● Usage – Separated into 15+ hard-linked commands for easy usage, such as pbi_add, pbi_delete, pbi_update, etc – Man pages for each command – Built-in “ pbid ” daemon – 100% of functionality available via command-line without X – Includes tools for building a PBI from a FreeBSD port

The PBI Format Re-implemented for Free/PC-BSD New Feature: Digital Verification ● Digital signatures using openssl ● Archive and installation scripts individually signed during PBI creation ● End user or front-end can verify signatures with pbi_add command.

The PBI Format Re-implemented for Free/PC-BSD New Feature: Repository System ● Provides mechanism for PBI creators to distribute PBI files to end-users ● Integrated with digital signatures ● Allows maintaining both a master INDEX of released PBIs, along with meta-index of PBI information ● Users can browse the meta-index, and download PBIs from repo, or perform updates of older PBIs to newer versions.

The PBI Format Re-implemented for Free/PC-BSD Creating a Repository # pbi_makerepo --desc “My Repository” \ --key [openssl pub keyfile] \ --url ”http://www.example.org/” \ --mirror “http://example.org/downloads,<mirror2>,<etc>” \ /root/repodir/

The PBI Format Re-implemented for Free/PC-BSD Importing a Repository on a client # pbi_addrepo myrepo-test.rpo Listing available Repositories on client # pbi_listrepo [ID] [Description] ----------------------------------------------------- 001 My Repository

The PBI Format Re-implemented for Free/PC-BSD Browsing PBIs from a Repository # pbi_browser 001 Using RepoID: 001 Available Categories: ------------------------------------ Category: Mail Icon: /var/db/pbi/<filename>.jpg Description: E-mail clients and utilities for you to enjoy reading and writing messages

The PBI Format Re-implemented for Free/PC-BSD Browsing PBIs from a Repository (Continued) # pbi_browser -c Mail 001 Using RepoID: 001 Available PBIs for Category: Mail ------------------------------------ Application: Thunderbird Version: 3.1.7 Arch: amd64 Author: Mozilla Corporation URL: http://www.mozilla.org License: GPL Type: Graphical Icon: /var/db/pbi/repo-icons/Thunderbird.png Description: Mozilla Thunderbird is a redesign of the Mozilla mail component. To install this PBI: # pbi_add -r Thunderbird --rArch amd64 --repoid 001

The PBI Format Re-implemented for Free/PC-BSD New Feature: PBID daemon ● ${LOCALBASE}/etc/rc.d/pbid ● Performs periodic updates of repository metadata ● Adds shared libraries into hashdir ● Prunes shared library hashdir after PBI removal ● Performs auto-updating of enabled PBIs

The PBI Format Re-implemented for Free/PC-BSD New Feature: File / Library Sharing ● Allows sharing of identical files / libraries between PBIs with hard-links ● Reduces disk / run-time memory usage ● Tracking of matches with sha256 checksum ● Monitored and managed by the pbid daemon

The PBI Format Re-implemented for Free/PC-BSD Two installed PBIs, pending merge to hashdir

The PBI Format Re-implemented for Free/PC-BSD First library added and hard-linked

The PBI Format Re-implemented for Free/PC-BSD Second library match, hard-linked

The PBI Format Re-implemented for Free/PC-BSD New library found, new hash-file / link created

The PBI Format Re-implemented for Free/PC-BSD A PBI has been removed

The PBI Format Re-implemented for Free/PC-BSD Orphaned hash-file is able to be safely removed

The PBI Format Re-implemented for Free/PC-BSD New Feature: Binary Patch Updates ● Allows updating of PBIs with smaller downloads ● In some cases a 95% size reduction ● Uses bsdiff / bspatch for patching process ● Auto-generation of patch files from port builds ● Auto-fallback to full-patching if bspatch fails

The PBI Format Re-implemented for Free/PC-BSD Inside the binary patching process # pbi_makepatch -o /root --sign <keyfile> foo-1.0-amd64.pbi foo- 1.1-amd64.pbi Generates: /root/foo-1.0_to_1.1-amd64.pbp

The PBI Format Re-implemented for Free/PC-BSD Inside the binary patching process # pbi_makepatch -o <outdir> --sign <keyfile> foo-1.0.pbi foo- 1.1.pbi ● Starts with extraction of both PBIs to temp directories ● Creates archive of NEW files from 1.1 version ● Creates list of REMOVED files from 1.0 version ● Build servers can use options to archive X number of PBIs

The PBI Format Re-implemented for Free/PC-BSD Inside the binary patching process # pbi_makepatch -o <outdir> --sign <keyfile> foo-1.0.pbi foo- 1.1.pbi ● Compares existing files with sha256 / bsdiff , and generates patch data ● Digitally signs archive / install scripts with <keyfile> ● Build servers can use options to archive X number of PBIs for easy patching

The PBI Format Re-implemented for Free/PC-BSD Inside the binary patching process # pbi_patch foo_1.0_to_1.1-amd64.pbp ● First checks that a valid “foo-1.0-amd64” is installed ● Updates the new / removed files ● Unlinks libraries from hash-dir before patching ● Patches existing files with bspatch ● Schedules pbid to re-integrate with hash-dir on next run

The PBI Format Re-implemented for Free/PC-BSD New Feature: PBI Building Tools ● Included with the toolset ● Allows building FreeBSD port into PBI file ● Supports meta-modules of extra data ● Supports running automated build servers

The PBI Format Re-implemented for Free/PC-BSD pbi_makeport ● Builds port in clean chroot environment ● Options for digital key signing ● Example: # pbi_makeport --sign <key> -o <outdir> net/rsync

The PBI Format Re-implemented for Free/PC-BSD pbi_create ● Allows packing a pre configured directory into a PBI file ● Can backup an installed PBI back ● Support for digital key signing ● Example # pbi_create -b firefox-3.6.15

The PBI Format Re-implemented for Free/PC-BSD pbi_autobuild ● Can be used to run build server ● Uses a conf module tree to monitor ports and rebuild updated apps into a PBI file ● Supports digital key signing ● Can archive X number of previous PBI versions, and use to generate binary patches ● Helper script can be integrated to report build status

The PBI Format Re-implemented for Free/PC-BSD Getting pbi-manager Subversion: svn://svn.pcbsd.org/pcbsd/current/src-sh/pbi-manager PC-BSD 9 Snapshots ftp://ftp.pcbsd.org/pub/snapshots/

The PBI Format Re-implemented for Free/PC-BSD Kris Moore PC-BSD / iXsystems kris@pcbsd.org http://www.pcbsd.org/~kris/pbi9-slides.pdf