SVN Pilot: CVS Replacement Manuel Guijarro Jonatan Hugo Hugosson - - PowerPoint PPT Presentation

▶
svn pilot cvs replacement
SMART_READER_LITE
LIVE PREVIEW

SVN Pilot: CVS Replacement Manuel Guijarro Jonatan Hugo Hugosson - - PowerPoint PPT Presentation

Jul 31, 2023 217 likes •473 views

SVN Pilot: CVS Replacement Manuel Guijarro Jonatan Hugo Hugosson Artur Wiecek David Horat Jonathan Brugge Jonathan Brugge Michel Manent September 2008 CERN - IT Department CH-1211 Genve 23 Switzerland www.cern.ch/ i t Outline

slide-1
SLIDE 1

SVN Pilot: CVS Replacement

Manuel Guijarro Jonatan Hugo Hugosson Artur Wiecek David Horat Jonathan Brugge Jonathan Brugge Michel Manent September 2008

CERN - IT Department CH-1211 Genève 23 Switzerland

www.cern.ch/ it

slide-2
SLIDE 2

Outline

  • Introduction
  • Motivation
  • Subversion
  • Objectives
  • Performance Tests

Performance Tests

  • Security

I l t ti

  • Implementation
  • Questions

CERN - IT Department CH-1211 Genève 23 Switzerland

www.cern.ch/ it

2

slide-3
SLIDE 3

Version Control Systems

  • Maintain current and historical versions of

files and data (source code) files and data (source code)

  • There are many commercial and Open

S VC S Source VC Systems:

– (Centralised) CVS/SVN – (Distributed) GIT, Bazaar, Darcs, GNU arch, Mercurial, Monotone, etc – But subversion seems to be the most popular

  • ne (used by GCC, Phyton, PuTTY, Apache,

GNOME KDE etc) GNOME, KDE, etc)

  • Physics User Community: (IN2P3, ROOT,

Totem )

CERN - IT Department CH-1211 Genève 23 Switzerland

www.cern.ch/ it

Totem..)

3

slide-4
SLIDE 4

CERN Central CVS Service

  • Hosts over 330 Software Projects

29 f Atl – 29 for Atlas – 46 for CMS 8 f LHCb – 8 for LHCb,…..

  • Over 3000 developers registered
  • Over 90 GBytes of source code
  • Creates 250 Remedy tickets per year

Creates 250 Remedy tickets per year

  • Over 100000 commits per month

CERN - IT Department CH-1211 Genève 23 Switzerland

www.cern.ch/ it

4

slide-5
SLIDE 5

CERN Central CVS Service

CERN - IT Department CH-1211 Genève 23 Switzerland

www.cern.ch/ it

slide-6
SLIDE 6

Central CVS service features

  • High Availability and Load Balancing

f

  • Web interface to repositories
  • Usage Statistics
  • Repository Remote Replication + Mirroring
  • Daily archive of Repositories and DR

Daily archive of Repositories and DR

  • Developers Mailing list

P /P t C it A ti ( h il

  • Pre/Post Commit Actions (such us e-mail

notification, etc)

  • Various access method (ssh/kerberos)
  • Role split (CVS Admin/Librarian/Developer)

CERN - IT Department CH-1211 Genève 23 Switzerland

www.cern.ch/ it

p ( p )

6

slide-7
SLIDE 7

Motivation for SVN Pilot

  • Originally designed to host less than 100

projects projects

  • Requests to provide a central SVN service:

– From CMS – From ATLAS (case study in 2006) – And from many others

  • CVS is over 20 years old while SVN is this

y millennium technology

  • Requests for Read Access control

Requests for Read Access control

CERN - IT Department CH-1211 Genève 23 Switzerland

www.cern.ch/ it

slide-8
SLIDE 8

SVN vs. CVS

Feature SVN CVS Speed Faster Slower Speed Faster Slower Permission Full Limited File types All Limited Off line operations Yes No Repository format Database File system Locks No Yes Atomic commits Yes No

CERN - IT Department CH-1211 Genève 23 Switzerland

www.cern.ch/ it

8

slide-9
SLIDE 9

New Features (SVN 1.5)

  • Automatic update of working copy
  • Merge tracking

– Subversion keeps track of what changes have b d h been merged where

  • Sparse checkouts
  • Interactive conflict resolution

CERN - IT Department CH-1211 Genève 23 Switzerland

www.cern.ch/ it

9

slide-10
SLIDE 10

Pilot Objectives

  • Provide current CVS service features
  • Add new features (available with SVN)
  • Add new features (available with SVN)

– Control Read access per path (module) – Authenticated Web access Authenticated Web access – Binary files handling

  • Ease CVS to SVN migration

g

  • Improved usage statistics (SVN Stats)
  • Handling of first line support via the Help Desk

g pp p

  • Delegate administrative tasks to Software

Librarians of each project

  • Prevent uncontrolled setup of SVN servers
  • Manpower: 1.2 FTE project

CERN - IT Department CH-1211 Genève 23 Switzerland

www.cern.ch/ it

p p j

slide-11
SLIDE 11

Timetable

Feb 2008

  • Preliminary study

2008 May

  • CVS librarians feedback

y 2008 July

  • SVN Pilot

July 2008

SVN Pilot

  • SVN service in production

Dec 2008

p

  • CVS to SVN migration

Dec 2009

  • CVS service close down

CERN - IT Department CH-1211 Genève 23 Switzerland

www.cern.ch/ it

11

slide-12
SLIDE 12

SVN Pilot study

  • Access methods

htt – https – ssh

  • Shared storage

– NFS 3/4 – AFS

  • Securing service

g

– Restricted Shell – Chrooted hooks (commit scripts) ( p )

  • Infrastructure:

– Librarian tools Statistics Web Interface

CERN - IT Department CH-1211 Genève 23 Switzerland

www.cern.ch/ it

– Librarian tools, Statistics, Web Interface,…

12

slide-13
SLIDE 13

Performance Tests

  • SVN check out of a 110 Mb project
  • Parameters

– AFS/NFS3/NFS4 – HTTPS/SSH

CERN - IT Department CH-1211 Genève 23 Switzerland

www.cern.ch/ it

13

slide-14
SLIDE 14

AFS vs NFS3 (1 server)

CERN - IT Department CH-1211 Genève 23 Switzerland

www.cern.ch/ it

14

slide-15
SLIDE 15

AFS vs NFS4 (1 server)

CERN - IT Department CH-1211 Genève 23 Switzerland

www.cern.ch/ it

15

slide-16
SLIDE 16

AFS vs NFS4 (3 servers)

CERN - IT Department CH-1211 Genève 23 Switzerland

www.cern.ch/ it

16

slide-17
SLIDE 17

Preliminary Conclusions

  • AFS much faster than NFS

SS f

  • SSH much faster than https
  • SSH scales very well with high load
  • …
  • New tests ongoing (with mixture of read and

New tests ongoing (with mixture of read and write operations)

CERN - IT Department CH-1211 Genève 23 Switzerland

www.cern.ch/ it

17

slide-18
SLIDE 18

Security

  • Project Isolation

/

  • Windows/Linux clients
  • Worldwide access
  • Shared file system independent
  • Hooks executed on servers

Hooks executed on servers

  • Librarians may put any script into the hooks
  • Librarians might need file system level
  • Librarians might need file system level

access to repository – being studied

Security risk!! Security risk!!

CERN - IT Department CH-1211 Genève 23 Switzerland

www.cern.ch/ it

18

slide-19
SLIDE 19

Hooks (scripts)

Client Server

Svn commit Pre-commit hook Pre-commit hook is executed Post commit hook is executed SVN: Commit OK SVN: Commit OK Email notification recieved

CERN - IT Department CH-1211 Genève 23 Switzerland

www.cern.ch/ it

19

slide-20
SLIDE 20

Hook scripts chrooted:

Server

svnserer hooks/post-commit hook Librarian hooks: jailed

Usr-hooks/post- commit hook

Repositories

Repository (1)

System files

CERN - IT Department CH-1211 Genève 23 Switzerland

www.cern.ch/ it

20

slide-21
SLIDE 21

Architecture

  • svn.cern.ch (rw)

S d b i – Secured subversion server (only ssh) Read and write – Read and write access to repository

  • svnweb cern ch (ro)
  • svnweb.cern.ch (ro)

– User documentation P j t t – Project request – SVN web interface U t ti ti – Usage statistics

CERN - IT Department CH-1211 Genève 23 Switzerland

www.cern.ch/ it

21

slide-22
SLIDE 22

Pilot Implementation Summary

  • SSH access for SVN clients

R t i t d h ll f ll SVN li t – Restricted shell for all SVN clients – Hooks chrooted

  • SVN web (ro)

– Web interfaces: websvn, trac – SSO Authenticated access

  • Administration delegated to librarian

g

– Access rights – Hooks – Admin tools

CERN - IT Department CH-1211 Genève 23 Switzerland

www.cern.ch/ it

22

slide-23
SLIDE 23

Conclusions

  • Secure service

C S f

  • This will replace CVS by end of 2009
  • The service is supported (pre-production)
  • Pilot setup may differ from final setup

– Access method, Web interface, shared file Access method, Web interface, shared file system, etc. – Changes will be transparent to the users g p

CERN - IT Department CH-1211 Genève 23 Switzerland

www.cern.ch/ it

23

slide-24
SLIDE 24

Support

http://cern.ch/svn –Try the pilot –Documentation Svn support@cern ch Svn.support@cern.ch

CERN - IT Department CH-1211 Genève 23 Switzerland

www.cern.ch/ it

24

slide-25
SLIDE 25

Questions?

Th k F Li t i Thanks For Listening….

  • M. Guijarro, A. Wiecek, David Horat, Jonathan Bugge, M. Manent, H. Hugosson

CERN - IT Department CH-1211 Genève 23 Switzerland

www.cern.ch/ it

25