The PBI Format Re-implemented for Free/PC-BSD Kris Moore PC-BSD / - - PowerPoint PPT Presentation

▶
the pbi format re implemented for free pc bsd
SMART_READER_LITE
LIVE PREVIEW

The PBI Format Re-implemented for Free/PC-BSD Kris Moore PC-BSD / - - PowerPoint PPT Presentation

Apr 28, 2023 325 likes •723 views

The PBI Format Re-implemented for Free/PC-BSD Kris Moore PC-BSD / iXsystems kris@pcbsd.org http://www.pcbsd.org/~kris/pbi9-slides.pdf The PBI Format Re-implemented for Free/PC-BSD The legacy PBI format The PBI Format Re-implemented for

slide-1
SLIDE 1

The PBI Format Re-implemented for Free/PC-BSD

Kris Moore PC-BSD / iXsystems kris@pcbsd.org http://www.pcbsd.org/~kris/pbi9-slides.pdf

slide-2
SLIDE 2

The PBI Format Re-implemented for Free/PC-BSD

The legacy PBI format

slide-3
SLIDE 3

The PBI Format Re-implemented for Free/PC-BSD

Features

  • Method of packaging files into a single, self-

contained archive

– Eliminates issues with dependency resolution – Makes package management simple, less points of

failure.

slide-4
SLIDE 4

The PBI Format Re-implemented for Free/PC-BSD

Features

  • Applications / operating system separate

– Applications cannot break operating system or other

installed applications

– PBIs may be safely updated / downgraded at will – Allows 32bit PBI packages to run on 64bit systems

with no complex chroot setups.

slide-5
SLIDE 5

The PBI Format Re-implemented for Free/PC-BSD

Traditional Package Management

slide-6
SLIDE 6

The PBI Format Re-implemented for Free/PC-BSD

PBI Managed Applications

slide-7
SLIDE 7

The PBI Format Re-implemented for Free/PC-BSD

Drawbacks

  • Implemented in QT/KDE

– Other window manager integration difficult /

Impossible

– Made headless operation unpractical – Required xorg/qt/kde to always be installed – Running on native FreeBSD not feasible

slide-8
SLIDE 8

The PBI Format Re-implemented for Free/PC-BSD

Drawbacks

  • Duplication of identical files

– For each PBI, identical copy of libfoo.so.1 – Wasted disk / run-time memory – But necessary to maintain self-contained

functionality

slide-9
SLIDE 9

The PBI Format Re-implemented for Free/PC-BSD

Drawbacks

  • Lack of verification mechanisms

– Only bare-minimum md5 check – No digital signature confirming reliability from

building

– Lacked method to determine where PBI originated

slide-10
SLIDE 10

The PBI Format Re-implemented for Free/PC-BSD

Drawbacks

  • Updating required full download of PBI

– Newer releases may contain up to 95% similar data – Wasted time downloading full package – Made updating from low-bandwidth areas

troublesome

slide-11
SLIDE 11

The PBI Format Re-implemented for Free/PC-BSD

Reimplementation

  • Written in shell (/bin/sh)

– Uses only base-system utilities and devel/xdg-utils/ – Implemented into a single file, “pbi-manager” – Able to run on native FreeBSD

slide-12
SLIDE 12

The PBI Format Re-implemented for Free/PC-BSD

Reimplementation

  • Usage

– Separated into 15+ hard-linked commands for easy usage,

such as pbi_add, pbi_delete, pbi_update, etc

– Man pages for each command – Built-in “pbid” daemon – 100% of functionality available via command-line without X – Includes tools for building a PBI from a FreeBSD port

slide-13
SLIDE 13

The PBI Format Re-implemented for Free/PC-BSD

New Feature: Digital Verification

  • Digital signatures using openssl
  • Archive and installation scripts individually signed

during PBI creation

  • End user or front-end can verify signatures with

pbi_add command.

slide-14
SLIDE 14

The PBI Format Re-implemented for Free/PC-BSD

New Feature: Repository System

  • Provides mechanism for PBI creators to distribute PBI files to

end-users

  • Integrated with digital signatures
  • Allows maintaining both a master INDEX of released PBIs,

along with meta-index of PBI information

  • Users can browse the meta-index, and download PBIs from

repo, or perform updates of older PBIs to newer versions.

slide-15
SLIDE 15

The PBI Format Re-implemented for Free/PC-BSD

Creating a Repository # pbi_makerepo --desc “My Repository” \

  • -key [openssl pub keyfile] \
  • -url ”http://www.example.org/” \
  • -mirror “http://example.org/downloads,<mirror2>,<etc>” \

/root/repodir/

slide-16
SLIDE 16

The PBI Format Re-implemented for Free/PC-BSD

Importing a Repository on a client # pbi_addrepo myrepo-test.rpo Listing available Repositories on client # pbi_listrepo [ID] [Description]

  • 001

My Repository

slide-17
SLIDE 17

The PBI Format Re-implemented for Free/PC-BSD

Browsing PBIs from a Repository # pbi_browser 001 Using RepoID: 001 Available Categories:

  • Category: Mail

Icon: /var/db/pbi/<filename>.jpg Description: E-mail clients and utilities for you to enjoy reading and writing messages

slide-18
SLIDE 18

The PBI Format Re-implemented for Free/PC-BSD

Browsing PBIs from a Repository (Continued) # pbi_browser -c Mail 001 Using RepoID: 001 Available PBIs for Category: Mail

  • Application: Thunderbird

Version: 3.1.7 Arch: amd64 Author: Mozilla Corporation URL: http://www.mozilla.org License: GPL Type: Graphical Icon: /var/db/pbi/repo-icons/Thunderbird.png Description: Mozilla Thunderbird is a redesign of the Mozilla mail component. To install this PBI: # pbi_add -r Thunderbird --rArch amd64 --repoid 001

slide-19
SLIDE 19

The PBI Format Re-implemented for Free/PC-BSD

New Feature: PBID daemon

  • ${LOCALBASE}/etc/rc.d/pbid
  • Performs periodic updates of repository metadata
  • Adds shared libraries into hashdir
  • Prunes shared library hashdir after PBI removal
  • Performs auto-updating of enabled PBIs
slide-20
SLIDE 20

The PBI Format Re-implemented for Free/PC-BSD

New Feature: File / Library Sharing

  • Allows sharing of identical files / libraries between

PBIs with hard-links

  • Reduces disk / run-time memory usage
  • Tracking of matches with sha256 checksum
  • Monitored and managed by the pbid daemon
slide-21
SLIDE 21

The PBI Format Re-implemented for Free/PC-BSD

Two installed PBIs, pending merge to hashdir

slide-22
SLIDE 22

The PBI Format Re-implemented for Free/PC-BSD

First library added and hard-linked

slide-23
SLIDE 23

The PBI Format Re-implemented for Free/PC-BSD

Second library match, hard-linked

slide-24
SLIDE 24

The PBI Format Re-implemented for Free/PC-BSD

New library found, new hash-file / link created

slide-25
SLIDE 25

The PBI Format Re-implemented for Free/PC-BSD

A PBI has been removed

slide-26
SLIDE 26

The PBI Format Re-implemented for Free/PC-BSD

Orphaned hash-file is able to be safely removed

slide-27
SLIDE 27

The PBI Format Re-implemented for Free/PC-BSD

New Feature: Binary Patch Updates

  • Allows updating of PBIs with smaller downloads
  • In some cases a 95% size reduction
  • Uses bsdiff / bspatch for patching process
  • Auto-generation of patch files from port builds
  • Auto-fallback to full-patching if bspatch fails
slide-28
SLIDE 28

The PBI Format Re-implemented for Free/PC-BSD

Inside the binary patching process # pbi_makepatch -o /root --sign <keyfile> foo-1.0-amd64.pbi foo- 1.1-amd64.pbi Generates: /root/foo-1.0_to_1.1-amd64.pbp

slide-29
SLIDE 29

The PBI Format Re-implemented for Free/PC-BSD

Inside the binary patching process # pbi_makepatch -o <outdir> --sign <keyfile> foo-1.0.pbi foo- 1.1.pbi

  • Starts with extraction of both PBIs to temp directories
  • Creates archive of NEW files from 1.1 version
  • Creates list of REMOVED files from 1.0 version
  • Build servers can use options to archive X number of PBIs
slide-30
SLIDE 30

The PBI Format Re-implemented for Free/PC-BSD

Inside the binary patching process # pbi_makepatch -o <outdir> --sign <keyfile> foo-1.0.pbi foo- 1.1.pbi

  • Compares existing files with sha256 / bsdiff, and generates

patch data

  • Digitally signs archive / install scripts with <keyfile>
  • Build servers can use options to archive X number of PBIs for

easy patching

slide-31
SLIDE 31

The PBI Format Re-implemented for Free/PC-BSD

Inside the binary patching process # pbi_patch foo_1.0_to_1.1-amd64.pbp

  • First checks that a valid “foo-1.0-amd64” is installed
  • Updates the new / removed files
  • Unlinks libraries from hash-dir before patching
  • Patches existing files with bspatch
  • Schedules pbid to re-integrate with hash-dir on next run
slide-32
SLIDE 32

The PBI Format Re-implemented for Free/PC-BSD

New Feature: PBI Building Tools

  • Included with the toolset
  • Allows building FreeBSD port into PBI file
  • Supports meta-modules of extra data
  • Supports running automated build servers
slide-33
SLIDE 33

The PBI Format Re-implemented for Free/PC-BSD

pbi_makeport

  • Builds port in clean chroot environment
  • Options for digital key signing
  • Example:

# pbi_makeport --sign <key> -o <outdir> net/rsync

slide-34
SLIDE 34

The PBI Format Re-implemented for Free/PC-BSD

pbi_create

  • Allows packing a pre configured directory into a

PBI file

  • Can backup an installed PBI back
  • Support for digital key signing
  • Example

# pbi_create -b firefox-3.6.15

slide-35
SLIDE 35

The PBI Format Re-implemented for Free/PC-BSD

pbi_autobuild

  • Can be used to run build server
  • Uses a conf module tree to monitor ports and rebuild

updated apps into a PBI file

  • Supports digital key signing
  • Can archive X number of previous PBI versions, and

use to generate binary patches

  • Helper script can be integrated to report build status
slide-36
SLIDE 36

The PBI Format Re-implemented for Free/PC-BSD

Getting pbi-manager

Subversion: svn://svn.pcbsd.org/pcbsd/current/src-sh/pbi-manager PC-BSD 9 Snapshots ftp://ftp.pcbsd.org/pub/snapshots/

slide-37
SLIDE 37

The PBI Format Re-implemented for Free/PC-BSD

Kris Moore PC-BSD / iXsystems kris@pcbsd.org http://www.pcbsd.org/~kris/pbi9-slides.pdf