Crunchy Data PostgreSQL Operator September 2017 Crunchy Data - - PowerPoint PPT Presentation

▶
crunchy data postgresql operator
SMART_READER_LITE
LIVE PREVIEW

Crunchy Data PostgreSQL Operator September 2017 Crunchy Data - - PowerPoint PPT Presentation

Aug 03, 2023 1k likes •1.36k views

Crunchy Data PostgreSQL Operator September 2017 Crunchy Data Commitment to Community and Open Source Emphasis on Data Security and Compliance PostgreSQL for Public, Private and Hybrid Cloud 5 What is an Operator? 3 Operator Basics

slide-1
SLIDE 1

Crunchy Data PostgreSQL Operator

September 2017

slide-2
SLIDE 2

Crunchy Data

Emphasis on Data Security and Compliance PostgreSQL for Public, Private and Hybrid Cloud Commitment to Community and Open Source

5

slide-3
SLIDE 3

3

What is an Operator?

slide-4
SLIDE 4

Operator Basics

4

  • Open Source - project at https://github.com/crunchydata/postgres-operator
  • Controller - an Operator is essentially a controller component you run on your Kubernetes or

OpenShift cluster

  • Automation - the Operator implements the necessary workflow orchestration logic to enable

the automation of PostgreSQL related tasks

  • Leverages the Kube API - the Operator is implemented in golang and leverages the Kubernetes

Client API heavily to look for events, update labels, create containers, etc. https://github.com/kubernetes/client-go

  • Command Line Interface - the Operator has a Command Line Interface that lets users

manipulate the Operator and define metadata used to catalog your PostgreSQL deployments

  • Deployment - the Operator runs as a standard Deployment, it watches for Third Party

Resources that we define to manage PostgreSQL deployments

  • Third Party Resources - the operator makes use of Kubernetes Third Party Resources currently

to store metadata about PostgreSQL deployments, this will change to using Kubernetes Custom Resource Definitions in future versions.

  • Custom PostgreSQL Cluster Definitions - a template based approach for allowing users an

ability to define the makeup of a PostgreSQL cluster rather than the current default

slide-5
SLIDE 5

Operator Basics

5

slide-6
SLIDE 6

Operator Basics

6

slide-7
SLIDE 7

7

Why?

slide-8
SLIDE 8

Why Do I Need This?

8

  • Automation - working with databases requires different workflows that involve multiple steps

and verifications, the Operator allows for automating those sort of workflows in a consistent way and can reduce human errors; working with large numbers of complex database deployments can get time consuming without some sort of automation

  • Standard Practices - advanced users will want to have a standard set of database objects created

so the ability to categorize databases with metadata and also build/apply SQL policies for them is useful to support a set of standards

  • Ease of Use - a simple CLI that is focused solely on deploying and managing a set of PostgreSQL

clusters enables users to focus on getting real work done instead of having to build their own set

  • f scripts or adjust to learning the details of kubectl commands
  • Large Scale Deployments - the Operator supports environments where you might have possibly

hundreds of PostgreSQL clusters and being able to query these assets in a managed way is useful

  • Complex Orchestrations - the Operator provides a means of developing advanced database
  • rchestration logic using a standards-based approach
slide-9
SLIDE 9

9

Building Blocks

slide-10
SLIDE 10

Building Blocks

10

  • https://github.com/crunchydata/crunchy-containers/
  • crunchy-postgres
  • crunchy-proxy
  • crunchy-backup
  • crunchy-upgrade
slide-11
SLIDE 11

11

Installation

slide-12
SLIDE 12

Installation

12

  • https://github.com/CrunchyData/postgres-operator/blob/master/docs/build.asciidoc
  • Requirements

○ Kubernetes 1.5.3+ ○ OpenShift Origin 1.5.1+ and OpenShift Container Platform 3.5 ○ PostgreSQL 9.5+ Container (crunchy-postgres) ○ PostgreSQL Backup Container (crunchy-backup) ○ PostgreSQL Upgrade Container (crunchy-upgrade)

  • Building the project
  • Get packaged dependencies/build from source
  • Bash completion
  • View operator resources

○ kubectl get thirdpartyresources ○ kubectl get pgclusters ○ kubectl get pgbackups ○ kubectl get pgupgrades ○ kubectl get pgpolicies ○ kubectl get pgclones

slide-13
SLIDE 13

Installing Kubernetes

13

  • https://gist.github.com/xenophenes/05552cadbba42c6abcc039aad7273e60

○ (https://goo.gl/22fopq)

  • Minikube

○ minikube addons disable default-storageclass

slide-14
SLIDE 14

14

Features

slide-15
SLIDE 15

Create a Database Cluster

15

The most basic function of the Operator is to let a user create a new Database “Cluster”. Here Cluster refers to a set of resources (services, Deployments, Pods, etc.)

slide-16
SLIDE 16

Test a Database

16

You can test the database cluster with a simple test as follows:

slide-17
SLIDE 17

Delete a Database Cluster

17

You can delete the set of Database Cluster resources by name as follows:

slide-18
SLIDE 18

Backup a Database

18

You can execute a backup job, jobs can run for a very long time so the Operator is able to watch for their completion using the Kubernetes Watch API, when complete the Operator is notified:

slide-19
SLIDE 19

Show Backup PVCs

19

This displays the contents of a PVC (pgo show pvc) - not just the backup pvc as displayed here. This is the equivalent of the kubectl get pvc command.

slide-20
SLIDE 20

Restore from Backup

20

Create a restored database cluster from a previous backup using passed in variables to define the backup pvcs.

slide-21
SLIDE 21

Show Restored Cluster

21

Display information about the cluster we just restored; verify everything looks OK.

slide-22
SLIDE 22

Scale a Database

22

You can scale up the number of read-only replicas as follows:

slide-23
SLIDE 23

Perform a Minor Database Upgrade

23

You can upgrade to a new minor release as follows:

slide-24
SLIDE 24

Perform a Major Database Upgrade

24

You can upgrade to a new major release as follows:

slide-25
SLIDE 25

Create a Database Policy

25

You can define a set of SQL statements which we call a Policy:

slide-26
SLIDE 26

Apply a Database Policy

26

You can apply a Policy against a set of databases as follows:

slide-27
SLIDE 27

Create Policy from GitHub

27

You can define a set of SQL statements which we call a Policy and import the file from GitHub:

slide-28
SLIDE 28

Clone Database

28

Clone an existing PostgreSQL cluster, waits for the replica to complete replication, decouples the replica from the master by triggering a recovery and re-labeling the replica to become a fully functioning master, becomes a fully functioning master, creates a service for the new master and an empty replica deployment.

slide-29
SLIDE 29

Multiple Clusters

29

Command line parameters can be passed in with multiple values, and all clusters can be viewed.

slide-30
SLIDE 30

.pgo.yaml

30

Configuration YAML file for the postgres-operator. https://github.com/CrunchyData/postgres-o perator/blob/master/docs/config.asciidoc https://kubernetes.io/docs/concepts/storage /persistent-volumes/ https://kubernetes.io/docs/concepts/storage /volumes/

slide-31
SLIDE 31

Strategies (Resources)

31

  • https://github.com/CrunchyData/postgres-operator/blob/master/docs/design.asciidoc
  • https://github.com/CrunchyData/postgres-operator/tree/master/conf/postgres-operator/cluster/1
  • https://github.com/CrunchyData/postgres-operator/blob/master/examples/pgo.yaml.emptydir
  • https://github.com/CrunchyData/postgres-operator/blob/master/docs/config.asciidoc
slide-32
SLIDE 32

32

Summary

slide-33
SLIDE 33

33

THANK YOU!

Learn more:

  • https://www.youtube.com/watch?v=HX10WWTRiTY
  • https://github.com/CrunchyData/postgres-operator
  • https://www.crunchydata.com/

Slides are available at:

  • https://www.sarahconway.com/slides/postgres-operator.pdf

Sarah Conway: sarah.conway@crunchydata.com Jeff McCormick: jeff.mccormick@crunchydata.com