THE POWER OF RED HAT CEPH STORAGE And how its essential to your - - PowerPoint PPT Presentation

▶
the power of red hat ceph storage
SMART_READER_LITE
LIVE PREVIEW

THE POWER OF RED HAT CEPH STORAGE And how its essential to your - - PowerPoint PPT Presentation

Oct 29, 2023 356 likes •783 views

THE POWER OF RED HAT CEPH STORAGE And how its essential to your OpenStack environment Jean-Charles Lopez S. Technical Instructor, Global Storage Consulting Practice Red Hat, Inc. jcl@redhat.com May 2017 OpenStack Summit, Boston STORAGE

slide-1
SLIDE 1

THE POWER OF RED HAT CEPH STORAGE

Jean-Charles Lopez

  • S. Technical Instructor, Global Storage Consulting Practice

Red Hat, Inc. jcl@redhat.com May 2017 – OpenStack Summit, Boston

And how it’s essential to your OpenStack environment

slide-2
SLIDE 2

STORAGE CONCEPTS

slide-3
SLIDE 3

3

DIFFERENT KINDS OF STORAGE

FILE STORAGE

File systems allow users to

  • rganize data stored in blocks

using hierarchical folders and files.

OBJECT STORAGE

Object stores distribute data algorithmically throughout a cluster of media, without a rigid structure.

BLOCK STORAGE

Physical storage media appears to computers as a series of sequential blocks of a uniform size.

slide-4
SLIDE 4

4

REPLICATION VS ERASURE CODING

OBJECT OBJECT

COPY COPY COPY

REPLICATED POOL CEPH STORAGE CLUSTER CEPH STORAGE CLUSTER

1 2 3 4

ERASURE CODED POOL

X Y

FULL COPIES OF STORED OBJECTS

  • Very high durability
  • Quicker recovery
  • Performance optimized

ONE COPY PLUS PARITY

  • Cost-effective durability
  • Expensive recovery
  • Capacity optimized
slide-5
SLIDE 5

RED HAT CEPH STORAGE ARCHITECTURAL OVERVIEW

slide-6
SLIDE 6

6

RED HAT CEPH STORAGE ARCHITECTURAL COMPONENTS

RBD

A reliable, fully distributed block device with cloud platform integration

RGW

A web services gateway for object storage, compatible with S3 and Swift

APP HOST/VM

LIBRADOS

A library allowing apps to directly access RADOS (C, C++, Java, Python, Ruby)

RADOS

A software-based reliable, autonomous, distributed object store comprised of self- healing, self-managing, intelligent storage nodes and lightweight monitors

CEPHFS*

A distributed file system with POSIX semantics & scale-out metadata

CLIENT

* CephFS is Tech Preview in RHCS2

slide-7
SLIDE 7

7

RED HAT CEPH STORAGE ARCHITECTURAL COMPONENTS

RBD

A reliable, fully distributed block device with cloud platform integration

RGW

A web services gateway for object storage, compatible with S3 and Swift

APP HOST/VM

LIBRADOS

A library allowing apps to directly access RADOS (C, C++, Java, Python, Ruby)

Reliable Autonomous Distributed Object Store

Software-based, comprised of self-healing, self-managing, intelligent storage nodes and lightweight monitors

CEPHFS*

A distributed file system with POSIX semantics & scale-out metadata

CLIENT

* CephFS is Tech Preview in RHCS2

slide-8
SLIDE 8

RADOS CLUSTER

RADOS CLUSTER

slide-9
SLIDE 9

9

OBJECT STORAGE DAEMONS (OSDs)

OSD FS DISK OSD FS DISK OSD FS DISK OSD FS DISK

OSDs

  • 10s to 10000s in a cluster
  • One per disk (SSD, SAS, SATA, …)
  • Serve stored objects to clients
  • Minimum 3 per cluster
slide-10
SLIDE 10

10

MONITORS (MONs)

Monitors

  • Maintain cluster membership and state
  • Track health of the cluster
  • Provide consensus for distributed decision-making
  • Small, odd number
  • These do not serve stored objects to clients
  • Minimum 3 per cluster
slide-11
SLIDE 11

WHERE DO OBJECTS LIVE?

APPLICATION

OBJECTS

RADOS CLUSTER

slide-12
SLIDE 12

12

INTRODUCTION TO CEPH DATA PLACEMENT : POOLS & PLACEMENT GROUPS

10 10 10 10 10 01 01 01 01 10 01 01 01 11 01 11

POOL A POOL B POOL C POOL D

10 11 10 01 10 11 10 01 10 11 10 01 10 11 10 01 10 11 10 01 01 10 11 10 10 01 01 01 01 10 11 10

slide-13
SLIDE 13

CONTROLLED REPLICATION UNDER SCALABLE HASHING

OBJECTS

CLUSTER

10 01 11

PLACEMENT GROUPS

10 01

10 10 10 10 10 01 01 01 01 10 01 01 01 11 01 11

10 01 01

slide-14
SLIDE 14

10 10 10 10 10 01 01 01 01 10 01 01 01 11 01 11

14

CRUSH IS A QUICK CALCULATION

OBJECTS

CLUSTER

slide-15
SLIDE 15

CRUSH - DYNAMIC DATA PLACEMENT

CRUSH

  • Pseudo-random placement algorithm
  • Fast calculation, no lookup
  • Repeatable, deterministic
  • Statistically uniform distribution
  • Stable mapping
  • Limited data migration on change
  • Rule-based configuration
  • Infrastructure topology aware
  • Adjustable replication
  • Weighting
slide-16
SLIDE 16

16

RED HAT CEPH STORAGE ARCHITECTURAL COMPONENTS

RBD

A reliable, fully distributed block device with cloud platform integration

RGW

A web services gateway for object storage, compatible with S3 and Swift

APP HOST/VM

LIBRADOS

A library allowing apps to directly access RADOS (C, C++, Java, Python, Ruby)

RADOS

A software-based reliable, autonomous, distributed object store comprised of self- healing, self-managing, intelligent storage nodes and lightweight monitors

CEPHFS*

A distributed file system with POSIX semantics & scale-out metadata

CLIENT

* CephFS is Tech Preview in RHCS2

slide-17
SLIDE 17

ACCESSING A RADOS CLUSTER

LIBRADOS APPLICATION

OBJECTS

Socket

RADOS CLUSTER

slide-18
SLIDE 18

18

RED HAT CEPH STORAGE ARCHITECTURAL COMPONENTS

RBD

A reliable, fully distributed block device with cloud platform integration

RGW

A web services gateway for object storage, compatible with S3 and Swift

APP HOST/VM

LIBRADOS

A library allowing apps to directly access RADOS (C, C++, Java, Python, Ruby)

RADOS

A software-based reliable, autonomous, distributed object store comprised of self- healing, self-managing, intelligent storage nodes and lightweight monitors

CEPHFS*

A distributed file system with POSIX semantics & scale-out metadata

CLIENT

* CephFS is Tech Preview in RHCS2

slide-19
SLIDE 19

THE RADOS GATEWAY (RGW)

RADOSGW RADOSGW LIBRADOS REST Socket LIBRADOS

RADOS CLUSTER

slide-20
SLIDE 20

20

RED HAT CEPH STORAGE ARCHITECTURAL COMPONENTS

RBD

A reliable, fully distributed block device with cloud platform integration

RGW

A web services gateway for object storage, compatible with S3 and Swift

APP HOST/VM

LIBRADOS

A library allowing apps to directly access RADOS (C, C++, Java, Python, Ruby)

RADOS

A software-based reliable, autonomous, distributed object store comprised of self- healing, self-managing, intelligent storage nodes and lightweight monitors

CEPHFS*

A distributed file system with POSIX semantics & scale-out metadata

CLIENT

* CephFS is Tech Preview in RHCS2

slide-21
SLIDE 21

21

STORING VIRTUAL DISKS

VM

HYPERVISOR LIBRBD

RADOS CLUSTER

slide-22
SLIDE 22

22

SEPARATE COMPUTE FROM STORAGE

VM

HYPERVISOR LIBRBD HYPERVISOR LIBRBD RADOS CLUSTER

slide-23
SLIDE 23

23

KERNEL MODULE FOR MAX FLEXIBILITY

LINUX HOST

KRBD

RADOS CLUSTER

slide-24
SLIDE 24

24

RED HAT CEPH STORAGE ARCHITECTURAL COMPONENTS

RBD

A reliable, fully distributed block device with cloud platform integration

RGW

A web services gateway for object storage, compatible with S3 and Swift

APP HOST/VM

LIBRADOS

A library allowing apps to directly access RADOS (C, C++, Java, Python, Ruby)

RADOS

A software-based reliable, autonomous, distributed object store comprised of self- healing, self-managing, intelligent storage nodes and lightweight monitors

CEPHFS*

A distributed file system with POSIX semantics & scale-out metadata

CLIENT

* CephFS is Tech Preview in RHCS2

slide-25
SLIDE 25

25

CEPHFS* - SEPARATE METADATA SERVER

KERNEL MODULE LINUX HOST

DATA METADATA

RADOS CLUSTER

* CephFS is Tech Preview in RHCS2

slide-26
SLIDE 26

RED HAT CEPH STORAGE OPENSTACK USE CASE

slide-27
SLIDE 27

RED HAT CEPH STORAGE AND OPENSTACK

KEYSTONE SWIFT CINDER NOVA GLANCE RADOSG W

LIBRADOS

LIBRBD

OPENSTACK

HYPERVISOR

RADOS CLUSTER MANILA

CEPHFS*

* CephFS is Tech Preview in RHCS2

slide-28
SLIDE 28

RED HAT CEPH STORAGE DOING IT!

slide-29
SLIDE 29

RED HAT CEPH STORAGE RBD & GLANCE

On ceph admin node, run: ceph osd pool create {pool_name} 2x ceph auth get-or-create {user_name} ... –o {keyring_file} scp {keyring_file} {unix_user}@{glance_node}:{path} <- Provide read permission for Glance scp /etc/ceph/ceph.conf {unix_user}@{glance_node}:{path} <- Provide read permission for Glance Add the following to /etc/ceph/ceph.conf on Glance node [{user_name}] keyring = {path} Edit /etc/glance/glance-api.conf on Glance node ... [glance_store]
 stores = rbd
 default_store = rbd
 show_image_direct_url = true
 rbd_store_user = {user_id} <- If user name is client.{id}, use {id}
 rbd_store_pool = {pool_name}
 rbd_store_ceph_conf = {Ceph configuration file path} rbd_store_chunk_size = {integer} <- Uses 8 by default for 8MB object RBDs flavor = keystone Restart Glance services

slide-30
SLIDE 30

RED HAT CEPH STORAGE RBD & CINDER

On ceph admin node, run: ceph osd pool create {pool_name} 2x ceph auth get-or-create {user_name} ... –o {keyring_file} scp {keyring_file} {unix_user}@{cinder_node}:{path} <- Provide read permission for Cinder scp /etc/ceph/ceph.conf {unix_user}@{cinder_node}:{path} <- Provide read permission for Cinder Add the following to /etc/ceph/ceph.conf on Cinder node [{user_name}] keyring = {path} Edit /etc/cinder/cinder.conf on Cinder node. Note that you can create multiple storage backends ... [cinder_backend_name] volume_driver = cinder.volume.drivers.rbd.RBDDriver rbd_ceph_conf = {Ceph configuration file path} rbd_pool = {pool_name} rbd_secret_uuid = {UUID} rbd_user = {ceph_userid} Restart Cinder services

slide-31
SLIDE 31

RED HAT CEPH STORAGE RBD & LIBVIRT

Create a file with on compute node <secret ephemeral=”no” private=”no”> <uuid>{UUID}</uuid> <usage type=”ceph”> <name>{username} secret</name> </usage> </secret> Run command virsh secret-define --file ceph.xml virsh secret-set-value --secret {UUID} -base64 $(cat {ceph_user_name}.key)*
 Synchronize libvirt secrets across compute nodes

slide-32
SLIDE 32

RED HAT CEPH STORAGE RBD & NOVA

Edit /etc/nova/nova.conf on Nova nodes [libvirt]
 libvirt_images_type = rbd
 libvirt_images_rbd_pool = {pool_name} libvirt_images_rbd_ceph_conf = {Ceph configuration file path} libvirt_disk_cachemodes = "network=writeback" rbd_secret_uuid = {UUID}
 rbd_user = {ceph_userid} Restart Nova services

slide-33
SLIDE 33

RED HAT CEPH STORAGE VM TROUBLESHOOTING

On each compute node, make sure your /etc/ceph/ceph.conf file has [client.{user_name}] admin socket = /var/run/ceph/guests/$cluster-$type.$id.$pid.$cctid.asok log file = /var/log/qemu/qemu-guest-$pid.log VMs need restart for changes to take effect ceph –admin-daemon /var/run/ceph/guests/$cluster-$type.$id.$pid.$cctid.asok {command} Useful commands help <- List available commands perf dump <- Dump performance counters config show <- View all run time parameters config get {parameter} <- View the specific run time parameter config set {parameter} {value} <- Modify the specific run time parameter

slide-34
SLIDE 34

RED HAT CEPH STORAGE SWIFT

On OpenStack controler node, create a Swift service and endpoint

  • penstack service create --name=swift --description="Swift Service" object-store
  • penstack endpoint create --region RegionOne --publicurl "http://{rgw}:8080/swift/v1" \
  • -adminurl "http://{rgw}:8080/swift/v1” \
  • -internalurl "http://{rgw}:8080/swift/v1" swift

On your Keystone server, create an NSS database mkdir {certificate_directory}

  • penssl x509 -in /etc/keystone/ssl/cert/ca.pem -pubkey | certutil -d {certificate_directory} \
  • A -n ca -t "TCu,Cu,Tuw"

  • penssl x509 -in /etc/keystone/ssl/cert/signing_cert.pem -pubkey |certutil -d {certificate_directory} \
  • A -n signing_cert -t “TCu,Cu,Tuw”

On your Keystone server, copy the NSS database to the RADOS Gateway nodes scp –R {certificate_directory} {ceph_linux_user}@{rgw_node}:{certificate_directory}

slide-35
SLIDE 35

RED HAT CEPH STORAGE SWIFT

On your RADOS Gateway servers [{username}]
 rgw_keystone_url = http://a.b.c.d:{port} rgw_keystone_admin_user = {admin-user} rgw_keystone_admin_password = {admin-password} rgw_keystone_admin_tenant = {admin-tenant} rgw_keystone_accepted_roles = admin member swiftoperator rgw_keystone_token_cache_size = 200 rgw_keystone_revocation_interval = 300
 nss_db_path = {certificate_directory} Restart your RADOS Gateways

slide-36
SLIDE 36

CLOSING TIME

slide-37
SLIDE 37

WHERE TO GO TO ENGAGE

Red Hat Subscriptions: https://access.redhat.com/subscription-value

  • Evaluation, Pre-production, and Production contracts available through Red Hat sales

Red Hat Consulting: http://www.redhat.com/en/services/consulting/storage Red Hat Storage training: https://www.redhat.com/en/services/training Red Hat Storage test-drive: http://red.ht/cephtestdrive To engage a Territory Service Manager in your area, ask for a local Red Hat Storage sales professional at:

NORTH AMERICA: 1 (888) REDHAT-1; LATIN AMERICA: 54 (11) 4329-7300; EMEA: 00800 7334 2835 APJ: 65 6490 4200; Brazil: 55 (11) 3529-6000,; Australia: 1800 733 428; New Zealand: 0800 733 428

slide-38
SLIDE 38
  • Web: www.redhat.com/storage
  • Blog: http://redhatstorage.redhat.com/
  • Twitter: www.twitter.com/redhatstorage
  • Facebook: www.facebook.com/RedHatStorage/
  • YouTube: www.youtube.com/user/redhatstorage
  • Slideshare: www.slideshare.net/Red_Hat_Storage

WHERE TO GO ON SOCIAL MEDIA

slide-39
SLIDE 39

SERVICES AVAILABLE FROM RED HAT

for Red Hat Ceph Storage & Red Hat OpenStack Platform

  • Subscription Benefits – Streamlined product and resource access, lifecycle management,

Knowledgebase and deployment resources, security and accountability

  • Jumpstart - 3 day consulting for pre-production cluster implementation at customer site
  • Health Check - 3-5 day evaluation of total solution best practice engagement
  • Red Hat Storage Consulting Services - recommended for initial storage design and set-up
  • Global Services and Support - production subscriptions for smooth lifecycle deployments
  • Red Hat Global Learning Services – 2 and 5 day for storage or cloud admins
slide-40
SLIDE 40

THANK YOU