SLIDE 1
Caching Caching Demos Caching Limitations Caching Other Registries Caching Gitlab Demo Mirroring Manual Mirroring Summary
Agenda
@sudo_bmitch
Agenda Caching Caching Gitlab Demo Caching Demos Mirroring - - PowerPoint PPT Presentation
Agenda Caching Caching Gitlab Demo Caching Demos Mirroring - - PowerPoint PPT Presentation
Agenda Caching Caching Gitlab Demo Caching Demos Mirroring Caching Limitations Manual Mirroring Caching Other Registries Summary 1 / 35 @sudo_bmitch How to Use Mirroring and Caching to Optimize Your Image Registry Brandon Mitchell
SLIDE 2
SLIDE 3
Ephemeral Build Server?
@sudo_bmitch
3 / 35
SLIDE 4
Cluster Pulling Remote Images?
@sudo_bmitch
4 / 35
SLIDE 5
Worry About Upstream Image Changes?
@sudo_bmitch
5 / 35
SLIDE 6
Build and Deploy Infrastructure Tolerant of Upstream Outages?
@sudo_bmitch
6 / 35
SLIDE 7
Production Resilience
@sudo_bmitch
7 / 35
SLIDE 8
Build Infrastructure
@sudo_bmitch
8 / 35
SLIDE 9
Build Outage
@sudo_bmitch
9 / 35
SLIDE 10
Build Resilience
@sudo_bmitch
10 / 35
SLIDE 11
Faster Builds and Less Bandwidth
@sudo_bmitch
11 / 35
SLIDE 12
Caching
@sudo_bmitch
12 / 35
SLIDE 13
Caching is the Easy Button
https://commons.wikimedia.org/wiki/File:Easy_button.JPG
@sudo_bmitch
13 / 35
SLIDE 14
Cache Architecture
@sudo_bmitch
14 / 35
SLIDE 15
Cache Implementation
Either the dockerd CLI:
dockerd --registry-mirror <cache-url>
Or /etc/docker/daemon.json
{ "registry-mirrors": [ "<cache-url>" ] }
Plus a registry:
docker run -e REGISTRY_PROXY_REMOTEURL=<upstream-url> registry:2
@sudo_bmitch
15 / 35
SLIDE 16
@sudo_bmitch
00:00
16 / 35
SLIDE 17
@sudo_bmitch
00:00
17 / 35
SLIDE 18
@sudo_bmitch
00:00
18 / 35
SLIDE 19
@sudo_bmitch
00:00
19 / 35
SLIDE 20
@sudo_bmitch
00:00
20 / 35
SLIDE 21
So What's the Catch?
@sudo_bmitch
21 / 35
SLIDE 22
Cache Limitations
The "registry-mirror" setting only applies to Docker Hub Only caches pulls not pushes Pulls still check the image manifest on Hub Credentials are in the cache server Docker implementation only supports one authentication method
@sudo_bmitch
22 / 35
SLIDE 23
Options to Cache Other Registries
Congure a squid HTTP caching proxy Pull directly from the cache Use DNS and TLS certs to send pulls to the proxy
@sudo_bmitch
23 / 35
SLIDE 24
@sudo_bmitch
00:00
24 / 35
SLIDE 25
I Want More
@sudo_bmitch
25 / 35
SLIDE 26
Mirroring
@sudo_bmitch
26 / 35
SLIDE 27
Mirror Architecture
@sudo_bmitch
27 / 35
SLIDE 28
Running a Registry
Docker image
docker container run -p 5000:5000 registry:2
Harbor Many Artifact Repositories
@sudo_bmitch
28 / 35
SLIDE 29
Manually Mirroring
docker image pull ${image} docker image tag ${image} local-mirror:5000/${image} docker image push local-mirror:5000/${image}
@sudo_bmitch
29 / 35
SLIDE 30
Manual Mirror Script
docker image pull "$localimg" docker image pull "$remoteimg" remoteid=$(docker image inspect "$remoteimg" --format '{.Id}') localid=$(docker image inspect "$localimg" --format '{.Id}') if [ "$remoteid" != "$localid" ]; then docker image tag "$localimg" "$localimg.$datestamp" docker image tag "$remoteimg" "$localimg" docker image push "$localimg.$datestamp" docker image push "$localimg" fi
@sudo_bmitch
30 / 35
SLIDE 31
Why All the Complication?
@sudo_bmitch
31 / 35
SLIDE 32
Advantages of Manually Mirroring
Over Automatically Syncing Repos: Changes to images happen on your schedule Backout option exists with breaking changes Over Pull Through Cache Those reasons plus... Pushing locally built images to the registry Upstream outage doesn't stop local builds/deploys
@sudo_bmitch
32 / 35
SLIDE 33
Risks of Manually Mirroring
Images go stale if you do not automate the script Adding new images is an added process Recovering from a mirror outage requires populating images FROM line in images needs to point to mirror
ARG REGISTRY=docker.io FROM ${REGISTRY}/alpine:3.9 ... docker build --build-arg REGISTRY=local-mirror:5000 .
@sudo_bmitch
33 / 35
SLIDE 34
Summary
Both
Saves bandwidth Faster builds
Pull Through Cache
Easy to create Little maintenance
Managed Mirror
Control changes Tolerate upstream outages
@sudo_bmitch
34 / 35
SLIDE 35