dspace clustering dspace clustering
play

DSPACE CLUSTERING DSPACE CLUSTERING VIA PUPPET, HAPROXY AND CEPHFS - PowerPoint PPT Presentation

Mar 20, 2023 •219 likes •480 views

DSPACE CLUSTERING DSPACE CLUSTERING VIA PUPPET, HAPROXY AND CEPHFS VIA PUPPET, HAPROXY AND CEPHFS DSpace-Clustering Bernd Nicklas, Paul Mnch OR2019 Kontakt: dspace@uni-marburg.de CC BY 4.0 TABLE OF CONTENTS TABLE OF CONTENTS 1. What we

  1. DSPACE CLUSTERING DSPACE CLUSTERING VIA PUPPET, HAPROXY AND CEPHFS VIA PUPPET, HAPROXY AND CEPHFS DSpace-Clustering Bernd Nicklas, Paul Münch OR2019 Kontakt: dspace@uni-marburg.de CC BY 4.0

  2. TABLE OF CONTENTS TABLE OF CONTENTS 1. What we want 2. How we get what we want 3. Code example DSpace-Clustering Bernd Nicklas, Paul Münch OR2019 Kontakt: dspace@uni-marburg.de CC BY 4.0

  3. WHAT WE WANT WHAT WE WANT DSpace-Clustering Bernd Nicklas, Paul Münch OR2019 Kontakt: dspace@uni-marburg.de CC BY 4.0

  4. Automatic, customizable and secure all-in-one con�guration DSpace-Clustering Bernd Nicklas, Paul Münch OR2019 Kontakt: dspace@uni-marburg.de CC BY 4.0

  5. Automatic, customizable cluster con�guration which allows for Horizontal scaling, improved availability and improved maintainability DSpace-Clustering Bernd Nicklas, Paul Münch OR2019 Kontakt: dspace@uni-marburg.de CC BY 4.0

  6. WHAT DSPACE NEEDS WHAT DSPACE NEEDS DSpace-Clustering Bernd Nicklas, Paul Münch OR2019 Kontakt: dspace@uni-marburg.de CC BY 4.0

  7. SOFTWARE STACK SOFTWARE STACK Apache web server Tomcat application server PostgreSQL database server Solr index server DSpace web application File system DSpace-Clustering Bernd Nicklas, Paul Münch OR2019 Kontakt: dspace@uni-marburg.de CC BY 4.0

  8. HOW WE GET WHAT WE WANT HOW WE GET WHAT WE WANT DSpace-Clustering Bernd Nicklas, Paul Münch OR2019 Kontakt: dspace@uni-marburg.de CC BY 4.0

  9. SOFTWARE STACK SOFTWARE STACK GitLab: source code management & CI & CD Puppet: con�guration management & orchestration Custom Puppet module for DSpace DSpace-Clustering Bernd Nicklas, Paul Münch OR2019 Kontakt: dspace@uni-marburg.de CC BY 4.0

  10. CUSTOM PUPPET MODULE FOR DSPACE CUSTOM PUPPET MODULE FOR DSPACE Handles installation & con�guration of all DSpace components Handles �rewall, orchestration and communication in cluster setting DSpace-Clustering Bernd Nicklas, Paul Münch OR2019 Kontakt: dspace@uni-marburg.de CC BY 4.0

  11. GITLAB + PUPPET GITLAB + PUPPET DSpace-Clustering Bernd Nicklas, Paul Münch OR2019 Kontakt: dspace@uni-marburg.de CC BY 4.0

  12. PUPPET AGENT ON A DSPACE APPLICATION NODE PUPPET AGENT ON A DSPACE APPLICATION NODE Gets con�guration from Puppet Master Clones/pulls the source code Fills and places template �les Runs Maven and Ant Restarts Tomcat DSpace-Clustering Bernd Nicklas, Paul Münch OR2019 Kontakt: dspace@uni-marburg.de CC BY 4.0

  13. DSPACE COMPONENTS DSPACE COMPONENTS DSpace-Clustering Bernd Nicklas, Paul Münch OR2019 Kontakt: dspace@uni-marburg.de CC BY 4.0

  14. SHARED RESOURCES IN A CLUSTER SETTING SHARED RESOURCES IN A CLUSTER SETTING Database Index AssetStore directory (we use CephFS ) Other shared directories (e.g. exports or oai ; we use CephFS ) Session data? No, actually we just pin the client on a worker DSpace-Clustering Bernd Nicklas, Paul Münch OR2019 Kontakt: dspace@uni-marburg.de CC BY 4.0

  15. CHARACTERISTICS OF OUR CLUSTERING CHARACTERISTICS OF OUR CLUSTERING The components portal, application, database and index are installed and con�gured on designated nodes All DSpace-speci�c server components listen to local host Web server listens on the public interface In cluster mode all components are transparently connected via HAProxy Secure �rewall and monitoring con�guration DSpace-Clustering Bernd Nicklas, Paul Münch OR2019 Kontakt: dspace@uni-marburg.de CC BY 4.0

  16. CONFIGURATION AND CUSTOMIZATION VIA PUPPET MODULE CONFIGURATION AND CUSTOMIZATION VIA PUPPET MODULE General Con�guration of System/Java/Tomcat environment variables Con�guration of general DSpace-related settings Git repository with slightly modi�ed basic DSpace source code, e.g. hot �xes Git repository with different branches of customized code (maven overlay mechanics) DSpace-Clustering Bernd Nicklas, Paul Münch OR2019 Kontakt: dspace@uni-marburg.de CC BY 4.0

  17. CODE EXAMPLE CODE EXAMPLE DSpace-Clustering Bernd Nicklas, Paul Münch OR2019 Kontakt: dspace@uni-marburg.de CC BY 4.0

  18. CONFIGURATION CLASS CONFIGURATION CLASS DSpace-Clustering Bernd Nicklas, Paul Münch OR2019 Kontakt: dspace@uni-marburg.de CC BY 4.0

  19. ALL-IN-ONE INSTALLATION ALL-IN-ONE INSTALLATION DSpace-Clustering Bernd Nicklas, Paul Münch OR2019 Kontakt: dspace@uni-marburg.de CC BY 4.0

  20. CLUSTER INSTALLATION CLUSTER INSTALLATION DSpace-Clustering Bernd Nicklas, Paul Münch OR2019 Kontakt: dspace@uni-marburg.de CC BY 4.0

  21. CLUSTER INSTALLATION CLUSTER INSTALLATION DSpace-Clustering Bernd Nicklas, Paul Münch OR2019 Kontakt: dspace@uni-marburg.de CC BY 4.0

  22. CLUSTER INSTALLATION CLUSTER INSTALLATION DSpace-Clustering Bernd Nicklas, Paul Münch OR2019 Kontakt: dspace@uni-marburg.de CC BY 4.0

  23. CLUSTER INSTALLATION CLUSTER INSTALLATION DSpace-Clustering Bernd Nicklas, Paul Münch OR2019 Kontakt: dspace@uni-marburg.de CC BY 4.0

  24. CLUSTER INSTALLATION CLUSTER INSTALLATION DSpace-Clustering Bernd Nicklas, Paul Münch OR2019 Kontakt: dspace@uni-marburg.de CC BY 4.0

  25. THANK YOU! THANK YOU! Visit us on GitHub . Soon on Puppet Forge . DSpace-Clustering Bernd Nicklas, Paul Münch OR2019 Kontakt: dspace@uni-marburg.de CC BY 4.0

Download Presentation
Download Policy: The content available on the website is offered to you 'AS IS' for your personal information and use only. It cannot be commercialized, licensed, or distributed on other websites without prior consent from the author. To download a presentation, simply click this link. If you encounter any difficulties during the download process, it's possible that the publisher has removed the file from their server.

Recommend

Getting Started with DSpace 7: Advanced Training Andrea Bollini, 4Science Art Lowel, Atmire

Getting Started with DSpace 7: Advanced Training Andrea Bollini, 4Science Art Lowel, Atmire

Getting Started with DSpace 7: Advanced Training Andrea Bollini, 4Science Art Lowel, Atmire Tim Donohue, DuraSpace Workshop Schedule DSpace 7 UI deep dive (Angular) Customizing UI (beyond branding) DSpace 7 REST API deep dive

1.67k views • 156 slides
Into DSpace using SWORD & GLOBUS Lee Taylor, University of Exeter, UK 11 July 2013 Exeter

Into DSpace using SWORD & GLOBUS Lee Taylor, University of Exeter, UK 11 July 2013 Exeter

Moving BIG DATA Into DSpace using SWORD & GLOBUS Lee Taylor, University of Exeter, UK 11 July 2013 Exeter Landscape 2011 DSpace in use for three independent repositories A brand new ~1M Petabyte data store established to hold all

408 views • 12 slides
Lecture 23: Spectral clustering Hierarchical clustering What is a good clustering?

Lecture 23: Spectral clustering Hierarchical clustering What is a good clustering?

Lecture 23: Spectral clustering Hierarchical clustering What is a good clustering? Aykut Erdem May 2016 Hacettepe University Last time K-Means An iterative clustering algorithm - Initialize: Pick K random points as cluster

932 views • 64 slides
Clustering Hierarchical clustering, k-mean clustering Genome 559: Introduction to Statistical and

Clustering Hierarchical clustering, k-mean clustering Genome 559: Introduction to Statistical and

Clustering Hierarchical clustering, k-mean clustering Genome 559: Introduction to Statistical and Computational Genomics Elhanan Borenstein A quick review The clustering problem: partition genes into distinct sets with high homogeneity

772 views • 40 slides
Digital Libraries@UC with DSpace/GLAM Universidade de Coimbra (since 1290, Portugal) Ana Lusa

Digital Libraries@UC with DSpace/GLAM Universidade de Coimbra (since 1290, Portugal) Ana Lusa

Digital Libraries@UC with DSpace/GLAM Universidade de Coimbra (since 1290, Portugal) Ana Lusa Silva, Ana Maria Miguis, Bruno Neves, Mrio Bernardes Joint Library Service UC (SIBUC) 13.Jun.2019 Digital Libraries@UC with DSpace/GLAM UC

487 views • 14 slides
Graph Clustering Graph Clustering What is clustering? What is clustering? Finding patterns

Graph Clustering Graph Clustering What is clustering? What is clustering? Finding patterns

Graph Clustering Graph Clustering What is clustering? What is clustering? Finding patterns in data, or grouping similar groups of data-points together into clusters . Clustering algorithms for numeric data: Lloyds K-means, EM

688 views • 19 slides
Clustering Hierarchical clustering and k-mean clustering Genome 373 Genomic Informatics

Clustering Hierarchical clustering and k-mean clustering Genome 373 Genomic Informatics

Clustering Hierarchical clustering and k-mean clustering Genome 373 Genomic Informatics Elhanan Borenstein A quick review The clustering problem: partition genes into distinct sets with high homogeneity and high separation Many

768 views • 43 slides
Supervisor: Jonathan Emery PMSM driving and control 15 V Power dSPACE DS103 SLAVE I/O

Supervisor: Jonathan Emery PMSM driving and control 15 V Power dSPACE DS103 SLAVE I/O

Permanent Magnet Synchronous Motors (PMSM) DRIVING and CONTROL Supervisor: Jonathan Emery PMSM driving and control 15 V Power dSPACE DS103 SLAVE I/O Interface board (CP31) -15V Supply dSPACE to driver +15V P1 P2 P3 230 V 230 V

422 views • 6 slides
Clustering DSE 210 Clustering in R d Two common uses of clustering: Vector quantization Find

Clustering DSE 210 Clustering in R d Two common uses of clustering: Vector quantization Find

Clustering DSE 210 Clustering in R d Two common uses of clustering: Vector quantization Find a finite set of representatives that provides good coverage of a complex, possibly infinite, high-dimensional space. Finding meaningful

396 views • 36 slides
Clustering: Hierarchical Clustering and K- Means Clustering Machine

Clustering: Hierarchical Clustering and K- Means Clustering Machine

Clustering: Hierarchical Clustering and K- Means Clustering Machine Learning 10-601B Seyoung Kim Many of these slides are derived from William Cohen,

660 views • 46 slides
Clustering and Dimensionality Reduction Preview Clustering K -means clustering

Clustering and Dimensionality Reduction Preview Clustering K -means clustering

Clustering and Dimensionality Reduction Preview Clustering K -means clustering Mixture models Hierarchical clustering Dimensionality reduction Principal component analysis Multidimensional scaling Isomap

1.61k views • 17 slides
Sydney eScholarship Repository and DSpace Sten Christensen & Gary Browne Sydney eScholarship

Sydney eScholarship Repository and DSpace Sten Christensen & Gary Browne Sydney eScholarship

Sydney eScholarship Repository and DSpace Sten Christensen & Gary Browne Sydney eScholarship Repository Software: DSpace Version: 1.4.1 Standard customisations ses.library.usyd.edu.au Sydney eScholarship Repository An open access

363 views • 9 slides
Clustering A Categorization of Major Clustering Methods Partitioning Methods

Clustering A Categorization of Major Clustering Methods Partitioning Methods

Clustering What is Clustering? Types of Data in Cluster Analysis Clustering A Categorization of Major Clustering Methods Partitioning Methods Hierarchical Methods 1 2 What is Clustering? What is Clustering? Typical

647 views • 18 slides
Subspace Clustering Ensemble Clustering Subspace Clustering, Ensemble Clustering, Alternative

Subspace Clustering Ensemble Clustering Subspace Clustering, Ensemble Clustering, Alternative

LUDWIG- MAXIMILIANS- INSTITUTE DATABASE UNIVERSITT FOR SYSTEMS MNCHEN MNCHEN INFORMATICS INFORMATICS GROUP GROUP Subspace Clustering Ensemble Clustering Subspace Clustering, Ensemble Clustering, Alternative Clustering, Multiview

513 views • 9 slides
Reinventing the Wheel? - The Case for the Development of an alternative DSpace Submission

Reinventing the Wheel? - The Case for the Development of an alternative DSpace Submission

Reinventing the Wheel? - The Case for the Development of an alternative DSpace Submission Assistant for Psychological Science Open Repositories 2019 in Hamburg Peter Weiland, Christiane Baier, Roland Ramthun, Johannes Hhmann

593 views • 9 slides
Finding Clusters Types of Clustering Approaches: Linkage Based, e.g. Hierarchical Clustering

Finding Clusters Types of Clustering Approaches: Linkage Based, e.g. Hierarchical Clustering

Finding Clusters Types of Clustering Approaches: Linkage Based, e.g. Hierarchical Clustering Clustering by Partitioning, e.g. k-Means Density Based Clustering, e.g. DBScan Grid Based Clustering Compendium slides for Guide to Intelligent

1.32k views • 82 slides
INTENTIONAL COMMUNITY CLUSTER DEVELOPMENTS INTENT: To allow a con fi gura on of small

INTENTIONAL COMMUNITY CLUSTER DEVELOPMENTS INTENT: To allow a con fi gura on of small

Proposed Zoning Code Text Amendment: INTENTIONAL COMMUNITY CLUSTER DEVELOPMENTS INTENT: To allow a con fi gura on of small dwellings or rooming units and a common house on a lot. This amendment will provide innova ve and a ff ordable

866 views • 34 slides
Strengthening th the management capacity of f th the Energy and Biomass Clu luster fr from

Strengthening th the management capacity of f th the Energy and Biomass Clu luster fr from

Strengthening th the management capacity of f th the Energy and Biomass Clu luster fr from Republic of f Moldova based on positive experience of f Romanian Green Energy In Innovative Biomass Cluster Project information Implementation

358 views • 10 slides
IBM Platform Computing : infrastructure management for HPC solutions on OpenPOWER Jing Li,

IBM Platform Computing : infrastructure management for HPC solutions on OpenPOWER Jing Li,

IBM Platform Computing : infrastructure management for HPC solutions on OpenPOWER Jing Li, Software Development Manager IBM #OpenPOWERSummit Join the conversation at #OpenPOWERSummit 1 Scale-out and Cloud Infrastructure Management Needs

487 views • 20 slides
Processing Big Data with Pentaho Rakesh Saha Pentaho Senior Product Manager, Hitachi Vantara

Processing Big Data with Pentaho Rakesh Saha Pentaho Senior Product Manager, Hitachi Vantara

Processing Big Data with Pentaho Rakesh Saha Pentaho Senior Product Manager, Hitachi Vantara Agenda Pentahos Latest and Upcoming Features for Processing Big Data Batch or Real-time Process big data visually in future-proof way

258 views • 25 slides
WHO WE ARE The WIINTECH network links eight European leading world-class clusters belonging to the

WHO WE ARE The WIINTECH network links eight European leading world-class clusters belonging to the

W orldwide I ntercluster I nitiative for N ew materials and processes focused on clean TECH nologies WHO WE ARE The WIINTECH network links eight European leading world-class clusters belonging to the chemical and materials sector as listed -

406 views • 3 slides
Midlands aerospace cluster 2 1 Midlands aerospace systems: guts of the aircraft gas

Midlands aerospace cluster 2 1 Midlands aerospace systems: guts of the aircraft gas

Midlands Aerospace Cluster Midlands Aerospace Alliance Dr. Andrew Mair Chief Executive Midlands Aerospace Alliance 1 Midlands aerospace cluster 2 1 Midlands aerospace systems: guts of the aircraft gas turbine engines aircraft wing

267 views • 8 slides
with Cluster Development Ms. Atchaka Sibunruang Minister of Industry 23 November 2015 Concepts

with Cluster Development Ms. Atchaka Sibunruang Minister of Industry 23 November 2015 Concepts

Thailand Moving Ahead with Cluster Development Ms. Atchaka Sibunruang Minister of Industry 23 November 2015 Concepts and Directions of Cluster Development 2 CONCEPTS OF CLUSTER DEVELOPMENT Cluster Development Policy Develop potential and

551 views • 13 slides
Capacitated Problem Agenda: Definition Example Solution Techniques Implementation

Capacitated Problem Agenda: Definition Example Solution Techniques Implementation

Computational Complexity CSC 5802 Professor: Tom Altman Capacitated Problem Agenda: Definition Example Solution Techniques Implementation Capacitated VRP (CPRV) CVRP is a Vehicle Routing Problem (VRP) in which a fixed fleet of

671 views • 46 slides

More recommend