with GNU Guix Ricardo Wurmus + Pjotr Prins + Ben Woodcroft + Altuna - - PowerPoint PPT Presentation

▶
with gnu guix
SMART_READER_LITE
LIVE PREVIEW

with GNU Guix Ricardo Wurmus + Pjotr Prins + Ben Woodcroft + Altuna - - PowerPoint PPT Presentation

Aug 14, 2022 523 likes •822 views

reproducible and user-controlled software management in HPC with GNU Guix Ricardo Wurmus + Pjotr Prins + Ben Woodcroft + Altuna Akalin BOSC 2017, Prague System Admins Users prefer mature software, want fresh software, no variants, only apply

slide-1
SLIDE 1

reproducible and user-controlled software management in HPC

with GNU Guix

Ricardo Wurmus + Pjotr Prins + Ben Woodcroft + Altuna Akalin BOSC 2017, Prague

slide-2
SLIDE 2

Users

want fresh software, multiple variants, latest tools, flexibility

System Admins

prefer mature software, no variants, only apply unavoidable updates use stable software for systems manage user software stack on their own

slide-3
SLIDE 3

There is no way to reproduce the environment, even on the same machine at a different point in time. Due to a lack of isolation, the environment will change or even break when the host system changes.

ad-hoc volatile

No safe upgrades or roll-backs. No separation for different workflows. Unportable.

primitive

slide-4
SLIDE 4

?

slide-5
SLIDE 5

...but reproducibility is still out of reach Installing software is easy now!

slide-6
SLIDE 6

...but reproducibility is still out of reach Installing software is easy now!

slide-7
SLIDE 7
slide-8
SLIDE 8
slide-9
SLIDE 9

100% reproducible We have all the bits!

slide-10
SLIDE 10

100% reproducible We have all the bits! 100% stateful We only have the bits!

slide-11
SLIDE 11

App bundles are convenient don't compose well are like giant statically linked binaries lack means of abstraction

slide-12
SLIDE 12
slide-13
SLIDE 13

Functional packaging

headers sources build tools libraries ...

cabba9e-emacs-24.5/ bin emacs lib ...

slide-14
SLIDE 14

Functional packaging

cabba9e-emacs-24.5/ bin emacs lib ... dedbeef-emacs-24.5/ bin emacs lib ...

Different inputs? Different outputs. Same inputs? Same output!

slide-15
SLIDE 15 coreutils-8.24 perl-5.22.1 tar-1.28 gzip-1.6 bzip2-1.0.6 xz-5.2.2 file-5.25 diffutils-3.3 patch-2.7.5 sed-4.2.2 findutils-4.6.0 gawk-4.1.3 grep-2.22 coreutils-8.24 make-4.1 bash-4.3.42 ld-wrapper-0 binutils-2.25.1 gcc-4.9.3 glibc-2.22 glibc-utf8-locales-2.22 acl-2.2.52 gmp-6.1.0 libcap-2.24 glibc-utf8-locales-2.22 gcc-4.9.3 ld-wrapper-boot3-0 binutils-cross-boot0-2.25.1 make-boot0-4.1 diffutils-boot0-3.3 findutils-boot0-4.6.0 file-boot0-5.25 bootstrap-binaries-0 ed-1.12 libsigsegv-2.10 perl-boot0-5.22.1 perl-5.22.1 acl-2.2.52 gmp-6.1.0 libcap-2.24 pkg-config-0.29 guile-2.0.11 bison-3.0.4 readline-6.3 ncurses-6.0 gcc-cross-boot0-wrapped-4.9.3 texinfo-6.0 bash-static-4.3.42 libstdc++-4.9.3 zlib-1.2.8 perl-boot0-5.22.1 gettext-boot0-0.19.7 gcc-cross-boot0-4.9.3 glibc-bootstrap-0 gcc-bootstrap-0 linux-libre-headers-3.14.37 gzip-1.6 gettext-0.19.7 attr-2.4.47 m4-1.4.17 gzip-1.6 guile-bootstrap-2.0 binutils-bootstrap-0 gettext-0.19.7 attr-2.4.47 m4-1.4.17 gcc-cross-boot0-wrapped-4.9.3 glibc-intermediate-2.22 m4-1.4.17 expat-2.1.0 lzip-1.16 pkg-config-0.29 libffi-3.2.1 readline-6.3 libunistring-0.9.6 libltdl-2.4.6 libgc-7.4.2 gmp-6.1.0 ncurses-6.0 libatomic-ops-7.4.2 m4-1.4.17 expat-2.1.0
slide-16
SLIDE 16

guix gc --references /gnu/store/…-foo-0.9 /gnu/store/…-glibc-2.25 /gnu/store/…-gcc-4.9.3-lib /gnu/store/…-bar-0.7b /gnu/store/…-baz-1.4.9 /gnu/store/…-foo-0.9

slide-17
SLIDE 17

guix package -i baz

  • -with-input=foo=bar

guix package -i baz

  • -with-source=baz-1.0.tgz
slide-18
SLIDE 18

guix pack samtools bedtools /gnu/store/…-pack.tar.gz

slide-19
SLIDE 19

guix pack -f docker samtools bedtools /gnu/store/ …-docker-pack.tar.gz

LOL

slide-20
SLIDE 20

guix package

  • -manifest=GeneNetwork

All you need is: Guix version + package manifest

(+ source code)

slide-21
SLIDE 21

The level of abstraction matters

1.

Guix enables reproducible and safe experimentation

2.

Guix makes environment sharing easy

3.

There are ways to use Guix without root access

4.

Like Conda?

  • 5. guix package -i conda
slide-22
SLIDE 22

#guix on irc.freenode.net http://gnu.org/s/guix

ricardo.wurmus@mdc-berlin.de

Learn more!

pjotr.public34@thebird.nl

Poster A-142

slide-23
SLIDE 23

/home/rekado/.guix-profile /var/guix/... guix-profile 42 /gnu/store hscq14x...-profile bin samtools bowtie2 rhrdst11m...-samtools-1.2 bin samtools include lud0v1c...-bowtie-2.2.4 bin bowtie2

...

slide-24
SLIDE 24

/home/rekado/.guix-profile /var/guix/... guix-profile 42 43 /gnu/store hscq14x...-profile bin samtools bowtie2 rhrdst11m...-samtools-1.2 bin samtools include lud0v1c...-bowtie-2.2.4 bin bowtie2 z3braf1sh...-profile bin samtools

...

slide-25
SLIDE 25

/home/rekado/.guix-profile /var/guix/... guix-profile 42 43 /gnu/store hscq14x...-profile bin samtools bowtie2 rhrdst11m...-samtools-1.2 bin samtools include lud0v1c...-bowtie-2.2.4 bin bowtie2 z3braf1sh...-profile bin samtools

...

slide-26
SLIDE 26

/home/rekado/.guix-profile /var/guix/... guix-profile 43 /gnu/store hscq14x...-profile bin samtools bowtie2 rhrdst11m...-samtools-1.2 bin samtools include lud0v1c...-bowtie-2.2.4 bin bowtie2 z3braf1sh...-profile bin samtools

...

slide-27
SLIDE 27

/home/rekado/.guix-profile /var/guix/... guix-profile 43 /gnu/store rhrdst11m...-samtools-1.2 bin samtools include lud0v1c...-bowtie-2.2.4 bin bowtie2 z3braf1sh...-profile bin samtools

...

slide-28
SLIDE 28

/home/rekado/.guix-profile /var/guix/... guix-profile 43 /gnu/store rhrdst11m...-samtools-1.2 bin samtools include z3braf1sh...-profile bin samtools

...