Docker : devops, shared registries, HPC and emerging use cases - - PowerPoint PPT Presentation

▶
docker devops shared registries hpc and emerging use cases
SMART_READER_LITE
LIVE PREVIEW

Docker : devops, shared registries, HPC and emerging use cases - - PowerPoint PPT Presentation

Apr 15, 2023 217 likes •419 views

Docker : devops, shared registries, HPC and emerging use cases Franois Moreews & Olivier Sallou P r e s e n t a t i o n Docker is an open-source engine to easily create lightweight, portable , self-sufficient containers from

slide-1
SLIDE 1

Docker : devops, shared registries, HPC and emerging use cases

François Moreews & Olivier Sallou

slide-2
SLIDE 2

P r e s e n t a t i

  • n

“Docker is an open-source engine to easily create lightweight, portable, self-sufficient containers from any

  • application. The same container that

a developer builds and test on a laptop can run at scale, in production,

  • n Vms,[...], public

clouds and more.”

slide-3
SLIDE 3

P r e s e n t a t i

  • n

chroot

  • Each process/command on Linux has current working directory called root

directory of a process/command. Chroot changes the root directory of a command, which ends up changing the root directory for both current running process and its children.

  • A process/command that is run in such a modified environment cannot access

files outside the root directory. This modified environment is known as "jailed directory" .

slide-4
SLIDE 4

P r e s e n t a t i

  • n

What are Docker “containers” ? Technically: ~chroot on steroids

  • a container is a set of processes (running on top of common kernel)
  • isolated* from the rest of the machine (cannot see/affect/harm host or other

containers)

  • using namespaces to have private view of the system (network interfaces, PID tree,

mountpoints...)

  • and cgroups to have metered/limited/reserved resources (to mitigate “bad neighbor”

effect)

slide-5
SLIDE 5

P r e s e n t a t i

  • n

What are Docker “containers” ? From a distance: looks like a VM

  • I can SSH into my container
  • I can have root access in it
  • I can install packages in it
  • I have my own eth0 interface
  • I can tweak routing table, iptables rules
  • I can mount filesystems

...

slide-6
SLIDE 6

P r e s e n t a t i

  • n

What are Docker “containers” ?

  • boot in milliseconds
  • just a few MB of intrinsic disk/memory usage
  • bare metal performance is possible

An efficient new way to build, ship, deploy & run your apps ! Lightweight, fast, disposable...virtual environments :

slide-7
SLIDE 7

Wh y i t w

  • r

k s — s e p a r a t i

  • n
  • f

c

  • n

c e r n s

  • t

h e D e v e l

  • p

e r w

  • r

r i e s a b

  • u

t w h a t ’ s “ i n s i d e ” t h e c

  • n

t a i n e r

  • H

i s c

  • d

e

  • H

i s L i b r a r i e s

  • H

i s P a c k a g e M a n a g e r

  • H

i s A p p s

  • H

i s D a t a

  • A

l l L i n u x s e r v e r s l

  • k

t h e s a me

  • t

h e O p s ( a d mi n ) w

  • r

r i e s a b

  • u

t w h a t ’ s “

  • u

t s i d e ” t h e c

  • n

t a i n e r

  • L
  • g

g i n g

  • R

e mo t e a c c e s s

  • M
  • n

i t

  • r

i n g

  • N

e t w

  • r

k c

  • n

f i g

  • A

l l c

  • n

t a i n e r s s t a r t , s t

  • p

, c

  • p

y , a t t a c h , mi g r a t e , e t c . t h e s a me w a y

slide-8
SLIDE 8

P r e s e n t a t i

  • n
  • LinuX Containers (LXC)
  • Control Groups & Namespaces
  • AUFS
  • Client – Server with an HTTP API
slide-9
SLIDE 9

Build : Dependencies & Dockerfjle

more api/Dockerfile FROM giltarchitecture/ ubuntu-openjdk-7-jre-headless:12.0.4 ADD . /apidoc ENTRYPOINT ["/apidoc/bin/apidoc-api"]

slide-10
SLIDE 10

Run Docker docker -run

  • -expose 80
  • p 9000:80

mydockercontainer-1-2-3 –Denv.port=90

  • Denv.conffile=file.conf

deamon mode (-d) or interactif mode (-i)

slide-11
SLIDE 11

Docker Registries

DOCKER : Share

slide-12
SLIDE 12

S h a r e d r e g i s t r i e s : y

  • u

r s

slide-13
SLIDE 13

S h a r e d r e g i s t r i e s : y

  • u

r s

slide-14
SLIDE 14

Google Container Engine

A Container based cloud architecture

DOCKER : CLOUD & HPC

slide-15
SLIDE 15

G

  • g

l e C

  • n

t a i n e r E n g i n e

  • A

l p h a

  • t

h e G

  • g

l e C

  • n

t a i n e r E n g i n e i s i n s p i r e d b y G

  • g

l e ’ s e x p e r i e n c e w i t h b u i l d i n g a n d r u n n i n g c

  • n

t a i n e r

  • b

a s e d d i s t r i b u t e d s y s t e ms , C

  • n

t a i n e r E n g i n e r e

  • i

ma g i n e s s

  • me
  • f

G

  • g

l e ’ s mo s t p

  • w

e r f u l i n t e r n a l s y s t e ms , s

  • t

h a t y

  • u

c a n d e v e l

  • p

a n d ma n a g e c

  • n

t a i n e r s t h e w a y G

  • g

l e ’ s e n g i n e e r s d

  • .
  • Wi

t h c

  • n

t a i n e r

  • b

a s e d c

  • mp

u t i n g , a p p l i c a t i

  • n

d e v e l

  • p

e r s c a n f

  • c

u s

  • n

t h e i r a p p l i c a t i

  • n

c

  • d

e , i n s t e a d

  • f
  • n

d e p l

  • y

me n t s a n d i n t e g r a t i

  • n

i n t

  • h
  • s

t i n g e n v i r

  • n

me n t s . A t t h e s a me t i me , a p p l i c a t i

  • n

s c a n b e b u i l t w i t h f e w c

  • n

s t r a i n t s . O p e r a t i

  • n

s c a n p r

  • v

i d e a r

  • b

u s t p l a t f

  • r

m t h a t q u i c k l y p r

  • v

i s i

  • n

s c

  • mp

u t e r e s

  • u

r c e s a n d e a s i l y ma n a g e s a p p l i c a t i

  • n

s . T h e t

  • l

s n e e d t

  • s

u p p

  • r

t t h e r i g h t c

  • n

t r

  • l

s f

  • r

s u c h a p p l i c a t i

  • n

a n d r e s

  • u

r c e ma n a g e me n t .

  • T

h e f

  • c

u s w i t h C

  • n

t a i n e r E n g i n e i s

  • n

b u i l d i n g t h e s e t

  • l

s a n d c

  • n

t r

  • l

s f

  • r
  • p

e r a t i

  • n

s . A t t h e s a me t i me , g

  • g

l e w a n t t

  • a

l l

  • w

f

  • r

w

  • r

k l

  • a

d mo b i l i t y , w h e r e c

  • n

t a i n e r i z e d a p p l i c a t i

  • n

s c a n r u n mu l t i

  • c

l

  • u

d . T h e y h a v e , t h e r e f

  • r

e , d e s i g n e d C

  • n

t a i n e r E n g i n e t

  • s

u p p

  • r

t K u b e r n e t e s , t h e

  • p

e n s

  • u

r c e t e c h n

  • l
  • g

y , s

  • t

h a t c u s t

  • me

r s c a n r u n

  • n

mu l t i p l e c l

  • u

d s .

slide-16
SLIDE 16

Google Kubernetes

A Container based cloud architecture

DOCKER : CLOUD & HPC

slide-17
SLIDE 17

G

  • g

l e K u b e r n e t e s

  • K

u b e r n e t e s i s a n

  • p

e n s

  • u

r c e c

  • n

t a i n e r c l u s t e r ma n a g e r . I t s c h e d u l e s a n y n u mb e r

  • f

c

  • n

t a i n e r r e p l i c a s a c r

  • s

s a g r

  • u

p

  • f

n

  • d

e i n s t a n c e s . A ma s t e r i n s t a n c e e x p

  • s

e s t h e K u b e r n e t e s A P I , t h r

  • u

g h w h i c h t a s k s a r e d e f i n e d . K u b e r n e t e s s p a w n s c

  • n

t a i n e r s

  • n

n

  • d

e s t

  • h

a n d l e t h e d e f i n e d t a s k s .

  • T

h e n u mb e r a n d t y p e

  • f

c

  • n

t a i n e r s c a n b e d y n a mi c a l l y mo d i f i e d a c c

  • r

d i n g t

  • n

e e d . A n a g e n t ( a k u b e l e t )

  • n

e a c h n

  • d

e i n s t a n c e mo n i t

  • r

s c

  • n

t a i n e r s a n d r e s t a r t s t h e m i f n e c e s s a r y .

  • K

u b e r n e t e s i s

  • p

t i mi z e d f

  • r

G

  • g

l e C l

  • u

d P l a t f

  • r

m, b u t c a n r u n

  • n

a n y p h y s i c a l

  • r

v i r t u a l ma c h i n e .

slide-18
SLIDE 18

GO Docker

Batch Scheduler with Docker

DOCKER : CLOUD & HPC

slide-19
SLIDE 19

Job/Interactive context

If user request root access:

  • mount job directory in container
  • mount /softs, /db as read-only
  • execute command
  • chown job directory with user id after command completion

If user does not request root access:

  • mount ~user in container
  • mount job directory in container
  • mount /home/user
  • mount additional user requested directories if user has read access (uid/gid)
  • mount /softs, /db as read-only
  • execute command as user (same uid/gid)