Cont ntaine ner Pods w ds with h Do Dock cker C Compose se - - PowerPoint PPT Presentation

Cont ntaine ner Pods w ds with h Do Dock cker C Compose se in A n Apa pache che Me Meso sos

1

Summary Goals:

• 1. Treating Apache Mesos and docker as first class citizens, the platform needs

to seamlessly run and scale docker container pods in Mesos with a standardized pod spec file.

• 2. Developers can develop and run the pod locally using a spec file and then use

the same spec file to launch it in a QA/Production cluster. Solution: Docker Compose Mesos Executor (https://github.com/PayPal/dce-go)

What are Pods?

• Pod
• ds rep

epres esen ent a a collec llectio ion of c contain iner ers trea eated ed as a sin ingle u le unit it for s sched edulin ling and dep eplo loymen ent.

• Pod
• ds are t

e trea eated ed as s sin ingle le scalin ling unit it.

• Contain

iner ers in in Pod

• ds will

ill gen ener erally lly share o e one o e or m more e names espaces es: n net etwork, pid, , ip ipc etc tc

• Cont
• ntaine

ners in P n Pods ds shou hould d ha have a c com

• mmon
• n cgr

grou

• up to be

e kep ept u under er c chec eck k as a a unit it t to not not s steal resou

• urces f

from

• m ot
• the

her p pods ds i in n the he hos host.

• Col
• location u
• n using

ng cons

• nstraint

nts ! != Pod

Why are Pods needed?

• Migr

grating ng lega gacy w wor

• rkloa
• ads

ds runni nning ng in n a s singl ngle no node de.

• Lift and s

and shi hift.

• Gives ti

es time e to to ex extr tract c t common ser services es duplicated ted in ea each pod into to a a s system stem ser service e whe hen r n relevant ant.

• Po

Pods help elps t to crea eate a e a mod

• dula

lar a applic licatio ion b by c composin ing dif iffer eren ent s ser ervic ices es.

• Si

Side-car, Ad Adapter ter, Am Ambassa ssador a are c e common p patter tterns

• Po

Pods help elps elim elimin inate e pre e and p post d dep eplo loymen ent s step eps.

• Hel

elps m s model el tr transi sien ent sh t short ta t task sks s (sh short l t lived ed c conta tainer ers) s)

Docker Compose

• Compose

e is is a a eleg elegant t too

• ol

l for d def efin inin ing and runnin ing mult ltip iple le doc

• cker

er cont

• ntaine

ners.

• Cher

eris ished ed too

• ol in

l in the c e communit ity over er the y e yea ears for loc local l dev evelo elopmen ent.

• Versi

sion 2. 2.X X pres eser erves es stric ictly ly all ll the loc e local l fea eatures

• es. In t

this is v ver ersio ion, it it in inter eracts wit ith a a sin ingle le doc

• cker

er en engin ine, e, m mostly ly runnin ing loc locally lly.

• Version 3
• n 3.X int

ntrodu duces com

• mpos
• se f

for

• r doc
• cker

er swa warm and d deprecates cer ertain in f fea eatures es of

• f

2.X .X. . For n now, , they remai main as as 2 separ arat ate v version trac acks.

Pods Modelled in Docker Compose

• Pod
• ds are

e contain iner ers bundled led toget ether er loc locally

• lly. S

So, r relies elies on ver ersio ion 2.x c compose e version.

• n.
• Pod
• ds rep

epres esen ented ed by c compose e can pres eser erve a e all ll the e fir irst c cla lass doc

• cker

er vol

• lume a

and nd ne networ

• rk pl

plugi gins ns.

• Pod
• ds can have f

e flex lexib ibili ility on colla llapsin ing names espaces es in in a any c combin inatio ion bet etween een the e cont

• ntaine

ners.

• Cont
• ntaine

ners in n Pods ds c can n ha have strict or

• rde

dering g ng guarant ntees by by using c ng condi

• nditiona
• nal

cons

• nstructs of
• f de

depends nds_on

• n.
• Pod
• ds can ref

efer er to ex exter ernally lly crea eated ed v volu lumes es a and n net etworks ks.

• Mult

ltip iple le files iles can b be e mer erged ed to c construct the r e rig ight p pod

• d d

def efin init itio ion f for a an en envir ironmen ent. Ex: b x: bas ase, , qa, p prod

• d compose

e files iles.

• Easy to s

spawn mult ltip iple le pod

• ds o
• f s

same e applic licatio ion wit ith d dif iffer eren ent ver ersio ions in in same e loc local l env nvironm

• nment

nt w withou hout c conf

• nflicts.

Mesos Architecture

Mesos Key Abstractions

Master

Agent

Framework

Executor Task Task Executor Task

Offer (Resources) Task/Executor Resources Task/Executor

Docker Compose Executor in Mesos

Docker Compose Executor Task (Docker Container Pod spawned as Child Cgroup of Parent Cgroup)

Mesos Parent Cgroup Container

Mesos Agent

Launcher Cgroups MEM Isolator Cgroups CPU Isolator

Mesos Containerizer

Cgroup Hierarchy

• Each cont
• ntaine

ner in n pod ha d has a a chi hild d cgr grou

• up unde

nder t the he pa parent nt mesos

• s task

sk cgr grou

• up. M

Meet eets Pod

• d crit

iter eria ia of c contain iner ers in in p pod

• d sharin

ing a cgr grou

• up.
• Cgr

grou

• ups

ps CFS ha hard d limits ( (ba bandw ndwidt dth h cont

• ntrol
• ls) and

nd memor

• ry limits a

assigne gned d to

• the

he pa parent nt cgr grou

• ups

ps will ill cover er all c ll contain iner ers in in pods.

• Indiv

ivid idual l cont

• ntaine

ners will n ill not be lim e limit ited ed unles less spec ecif ified ied b but cannot nnot go go over pa parent nt.

• Make

ke sure e to en enable m le mem emory h hier ierarchy wit ith use_hi hierarchy hy flag. g.

DCE-GO features

• Implem

lemen ents mesos

• s ex

exec ecutor c callb llbacks ks to m main intain in the lif e lifec ecycle le of a a pod

• d.
• Massa

ssages c s compose se f file to add cgr grou

• up par

arent, , mesos

• s la

labels els a and ed edit it cer ertain in sec ectio ions to resol

• lve a

any ny na naming ng conf

• nflict etc

tc

• Colla

llapses es n net etwork k name amespac ace by d defau ault.

• Provid

ides es pod

• d m

monit itor to not only ly kill kill en entir ire e pod

• d o
• n u

unex expec ected ed contain iner er ex exit it but also w

• whe

hen n a cont

• ntaine

ner becom

• mes unhe

nhealthy hy as p per doc

• cker

er hea ealt lthchec ecks ks.

• Supports runnin

ing m mult ltip iple le compose f e files iles.

• Mesos
• s Mod
• dule

le provid ided ed t to p prev even ent pod

• d lea

leaks ks in in r rare e case o e of ex exec ecutor crashes es.

• Pro

rovi vides pl plugi gins ns.

• Last bu

but not not the he least any ny e existing ng Mesos

• s Fra

ramework rks like A Auro rora ra, Ma Mara rathon etc tc can n use D e DCE dir irec ectly ly wit ithout makin king A ANY f framew ework k changes es.

12

PLUGINS

What are Plugins?

• Plu

Plugin ins p provid vides a wa way to ea easily ext extend end inner nner wo working ngs of DCE. E.

• Plug

ugins ca can be us used to cus customi mize DCE w E with thout t ha having ng t to und nder erstand nd exa exactly how

• w DCE

E is i implem ement ented ed int nter erna

• nally. P

Plugins ns m make e it ea easy t to exp exper eriment ent with th new fe featu ture res. s.

• Plugin

n mec echa hani nism hel helps y you ea easily ena enable a e and nd disable f e fea eatures es.

• Plu

Plugin ins essentia iall lly p y provid vide ho hooks b bef efore e and nd a after er launc nch/k h/kill task me mesos callb llbacks ks to imp mpleme ment cus custom m behavio vior.

• Plugins

ns c can n be e cha hained ned wi with o h order ering ng.

Default Plugin

DCE CE-GO c comes es wi with d h def efault Gener eneral Plugin.

• n. Thi

his Plugin n updates es compose e files es so so th that m t multi tiple pods s are re a able to to launch o

• n a host
• st. I

It t larg rgely covers fo rs following:

• Decora

rate te vari rious s compose se se secti tions to s to re reso solve a all th the confl flicts. ts.

• Label

el ea each h cont ntainer ner wi with s h spec ecific ta task skId and execut cutorId. . This is in informatio ion is is used d to clean an u up p pod. d.

• Addi

dding p g pod d to par parent me mesos ta task sk cg cgroup up.

• Cre

reati ting infra frastru structu ture re conta tainer r in p pod fo for a r allowing to to c collapse se n netw twork rk na names espace e for c cont ntainer ners in n a p pod.

Mesos Hook Module for Compose Pods

• Me

Mesos

• s Modules

es hel help ext extend end inner nner func nctiona nality using ng s sha hared ed libs. Ca Can r n run i n in n Maste ster a r and/or Ag r Agent.

• Me

Mesos

• s Module

les s should ld b be b built ilt again inst the me mesos ver ersion r n runni nning ng i in c n cluster er.

• Dif

Different cla lassif ific icatio ion of M Module les: A : Allo llocator, , Isola lator, , Ho Hooks etc tc

• Hooks Modules

es tie e int nto ev event ents a and nd thei heir cont ntext

• ext. D

DCE CE-GO O leverages es ex exec ecutor r rem emoval ev event ent ho hook in a n an a n agent

• ent. Implem

ement ents Co ComposeP ePodCl Clea eanu nupHook Module le.

• Tha

hat ho hook ens ensures es p pods are c e clea eaned ned u up on n any ny unexp nexpec ected ed ex exec ecutor crash. h.

Current Ecosystem around Pods

1. Docker swarm (as of 1.2.6) does not support local pods.

• Docker compose deploy takes a compose definition and schedules the containers across swarm cluster and

connects them via overlay network.

• Using constraints not the same.
• Most likely to be supported in future.

2. K8 has excellent support for pods but does not treat docker as first class.

• Different volume and network specs
• CRI mostly going to hook to containerd directly. Skip docker engine.
• Pod spec different than compose spec and docker commands do not work (equivalent command provided).
• Image is the only common thing.
• 3. Mesos added in 1.1 pod support via experimental Task Groups and Nested Container.
• Not docker specific and pod can represent any collection of tasks.
• Frameworks needs to make changes to support this.
• Task Groups spec obviously separate from compose spec.
• Universal Containerizer and set of isolators defining a container runtime separate from docker.
• However, Mesos continues to remain extremely flexible!

DCE-GO DEMO

17

• DCE

CE-GO p O proj

• ject (https:/

://git ithub.com/Pa PayPa yPal/ l/dce-go go)

• Depre

recate tes ( s (http ttps: s://github ub.co com/ m/me mesos/dock cker-co comp mpose-execut cutor)

• Me

Mesos

• s Archi

hitec ecture a e and nd Key ey Ab Abstra stracti tions s diagrams(http ttps:// //www ww.slideshare re.ne net/I /Inf nfoQ/mesos-a-stateo eofthea eart- cont ntainer ner-orc rchest stra rator)

Links and R Ref efer eren ences es