DSPACE CLUSTERING DSPACE CLUSTERING VIA PUPPET, HAPROXY AND CEPHFS - - PowerPoint PPT Presentation

▶

dspace clustering dspace clustering

DSPACE CLUSTERING DSPACE CLUSTERING VIA PUPPET, HAPROXY AND CEPHFS - - PowerPoint PPT Presentation

Mar 20, 2023 219 likes •485 views

DSPACE CLUSTERING DSPACE CLUSTERING VIA PUPPET, HAPROXY AND CEPHFS VIA PUPPET, HAPROXY AND CEPHFS DSpace-Clustering Bernd Nicklas, Paul Mnch OR2019 Kontakt: dspace@uni-marburg.de CC BY 4.0 TABLE OF CONTENTS TABLE OF CONTENTS 1. What we

slide-1

SLIDE 1

Bernd Nicklas, Paul Münch Kontakt: dspace@uni-marburg.de DSpace-Clustering OR2019 CC BY 4.0

DSPACE CLUSTERING DSPACE CLUSTERING

VIA PUPPET, HAPROXY AND CEPHFS VIA PUPPET, HAPROXY AND CEPHFS

slide-2

SLIDE 2

Bernd Nicklas, Paul Münch Kontakt: dspace@uni-marburg.de DSpace-Clustering OR2019 CC BY 4.0

TABLE OF CONTENTS TABLE OF CONTENTS

1. What we want
2. How we get what we

want

3. Code example

slide-3

SLIDE 3

Bernd Nicklas, Paul Münch Kontakt: dspace@uni-marburg.de DSpace-Clustering OR2019 CC BY 4.0

WHAT WE WANT WHAT WE WANT

slide-4

SLIDE 4

Bernd Nicklas, Paul Münch Kontakt: dspace@uni-marburg.de DSpace-Clustering OR2019 CC BY 4.0

Automatic, customizable and secure all-in-one conguration

slide-5

SLIDE 5

Bernd Nicklas, Paul Münch Kontakt: dspace@uni-marburg.de DSpace-Clustering OR2019 CC BY 4.0

Automatic, customizable cluster conguration which allows for Horizontal scaling, improved availability and improved maintainability

slide-6

SLIDE 6

Bernd Nicklas, Paul Münch Kontakt: dspace@uni-marburg.de DSpace-Clustering OR2019 CC BY 4.0

WHAT DSPACE NEEDS WHAT DSPACE NEEDS

slide-7

SLIDE 7

Bernd Nicklas, Paul Münch Kontakt: dspace@uni-marburg.de DSpace-Clustering OR2019 CC BY 4.0

SOFTWARE STACK SOFTWARE STACK

Apache web server Tomcat application server PostgreSQL database server Solr index server DSpace web application File system

slide-8

SLIDE 8

Bernd Nicklas, Paul Münch Kontakt: dspace@uni-marburg.de DSpace-Clustering OR2019 CC BY 4.0

HOW WE GET WHAT WE WANT HOW WE GET WHAT WE WANT

slide-9

SLIDE 9

Bernd Nicklas, Paul Münch Kontakt: dspace@uni-marburg.de DSpace-Clustering OR2019 CC BY 4.0

SOFTWARE STACK SOFTWARE STACK

GitLab: source code management & CI & CD Puppet: conguration management &

rchestration

Custom Puppet module for DSpace

slide-10

SLIDE 10

Bernd Nicklas, Paul Münch Kontakt: dspace@uni-marburg.de DSpace-Clustering OR2019 CC BY 4.0

CUSTOM PUPPET MODULE FOR DSPACE CUSTOM PUPPET MODULE FOR DSPACE

Handles installation & conguration of all DSpace components Handles rewall, orchestration and communication in cluster setting

slide-11

SLIDE 11

Bernd Nicklas, Paul Münch Kontakt: dspace@uni-marburg.de DSpace-Clustering OR2019 CC BY 4.0

GITLAB + PUPPET GITLAB + PUPPET

slide-12

SLIDE 12

Bernd Nicklas, Paul Münch Kontakt: dspace@uni-marburg.de DSpace-Clustering OR2019 CC BY 4.0

PUPPET AGENT ON A DSPACE APPLICATION NODE PUPPET AGENT ON A DSPACE APPLICATION NODE

Gets conguration from Puppet Master Clones/pulls the source code Fills and places template les Runs Maven and Ant Restarts Tomcat

slide-13

SLIDE 13

Bernd Nicklas, Paul Münch Kontakt: dspace@uni-marburg.de DSpace-Clustering OR2019 CC BY 4.0

DSPACE COMPONENTS DSPACE COMPONENTS

slide-14

SLIDE 14

Bernd Nicklas, Paul Münch Kontakt: dspace@uni-marburg.de DSpace-Clustering OR2019 CC BY 4.0

SHARED RESOURCES IN A CLUSTER SETTING SHARED RESOURCES IN A CLUSTER SETTING

Database Index AssetStore directory (we use CephFS) Other shared directories (e.g. exports or oai; we use CephFS) Session data? No, actually we just pin the client on a worker

slide-15

SLIDE 15

Bernd Nicklas, Paul Münch Kontakt: dspace@uni-marburg.de DSpace-Clustering OR2019 CC BY 4.0

CHARACTERISTICS OF OUR CLUSTERING CHARACTERISTICS OF OUR CLUSTERING

The components portal, application, database and index are installed and congured on designated nodes All DSpace-specic server components listen to local host Web server listens on the public interface In cluster mode all components are transparently connected via HAProxy Secure rewall and monitoring conguration

slide-16

SLIDE 16

Bernd Nicklas, Paul Münch Kontakt: dspace@uni-marburg.de DSpace-Clustering OR2019 CC BY 4.0

CONFIGURATION AND CUSTOMIZATION VIA PUPPET MODULE CONFIGURATION AND CUSTOMIZATION VIA PUPPET MODULE

General Conguration of System/Java/Tomcat environment variables Conguration of general DSpace-related settings Git repository with slightly modied basic DSpace source code, e.g. hot xes Git repository with different branches of customized code (maven

verlay mechanics)

slide-17

SLIDE 17

Bernd Nicklas, Paul Münch Kontakt: dspace@uni-marburg.de DSpace-Clustering OR2019 CC BY 4.0

CODE EXAMPLE CODE EXAMPLE

slide-18

SLIDE 18

Bernd Nicklas, Paul Münch Kontakt: dspace@uni-marburg.de DSpace-Clustering OR2019 CC BY 4.0

CONFIGURATION CLASS CONFIGURATION CLASS

slide-19

SLIDE 19

Bernd Nicklas, Paul Münch Kontakt: dspace@uni-marburg.de DSpace-Clustering OR2019 CC BY 4.0

ALL-IN-ONE INSTALLATION ALL-IN-ONE INSTALLATION

slide-20

SLIDE 20

Bernd Nicklas, Paul Münch Kontakt: dspace@uni-marburg.de DSpace-Clustering OR2019 CC BY 4.0

CLUSTER INSTALLATION CLUSTER INSTALLATION

slide-21

SLIDE 21

Bernd Nicklas, Paul Münch Kontakt: dspace@uni-marburg.de DSpace-Clustering OR2019 CC BY 4.0

CLUSTER INSTALLATION CLUSTER INSTALLATION

slide-22

SLIDE 22

Bernd Nicklas, Paul Münch Kontakt: dspace@uni-marburg.de DSpace-Clustering OR2019 CC BY 4.0

CLUSTER INSTALLATION CLUSTER INSTALLATION

slide-23

SLIDE 23

Bernd Nicklas, Paul Münch Kontakt: dspace@uni-marburg.de DSpace-Clustering OR2019 CC BY 4.0

CLUSTER INSTALLATION CLUSTER INSTALLATION

slide-24

SLIDE 24

Bernd Nicklas, Paul Münch Kontakt: dspace@uni-marburg.de DSpace-Clustering OR2019 CC BY 4.0

CLUSTER INSTALLATION CLUSTER INSTALLATION

slide-25

SLIDE 25

Bernd Nicklas, Paul Münch Kontakt: dspace@uni-marburg.de DSpace-Clustering OR2019 CC BY 4.0

THANK YOU! THANK YOU!

Visit us on . Soon on . GitHub Puppet Forge