Automated OSCAR testing with Automated OSCAR testing with Linux- - - PowerPoint PPT Presentation
Automated OSCAR testing with Automated OSCAR testing with Linux- -VServers VServers Linux Fernando Laudares Camargos Fernando Laudares Camargos & & Beno t des Ligneris, Ph. D t des Ligneris, Ph. D Beno
Fernando Laudares Camargos Fernando Laudares Camargos & & Beno Benoî ît des Ligneris, Ph. D t des Ligneris, Ph. D
fernando@revolutionlinux.com fernando@revolutionlinux.com
Automated OSCAR testing with Automated OSCAR testing with Linux Linux-
VServers
➔ ➔ The testing process of a complex program managed
The testing process of a complex program managed with a version system and develop by a large team of with a version system and develop by a large team of international developers is a very lengthy and international developers is a very lengthy and demanding task. demanding task.
➔ ➔ The testing process of a complex program managed
The testing process of a complex program managed with a version system and develop by a large team of with a version system and develop by a large team of international developers is a very lengthy and international developers is a very lengthy and demanding task. demanding task.
➔ ➔ An automated testing infrastructure for OSCAR that rely
An automated testing infrastructure for OSCAR that rely
➔ ➔ The testing process of a complex program managed
The testing process of a complex program managed with a version system and develop by a large team of with a version system and develop by a large team of international developers is a very lengthy and international developers is a very lengthy and demanding task. demanding task.
➔ ➔ An automated testing infrastructure for OSCAR that rely
An automated testing infrastructure for OSCAR that rely
➔ ➔ The virtualization layer is provided by the Linux
The virtualization layer is provided by the Linux-
VServer project
➔ ➔ The testing process of a complex program managed
The testing process of a complex program managed with a version system and develop by a large team of with a version system and develop by a large team of international developers is a very lengthy and international developers is a very lengthy and demanding task. demanding task.
➔ ➔ An automated testing infrastructure for OSCAR that rely
An automated testing infrastructure for OSCAR that rely
➔ ➔ The virtualization layer is provided by the Linux
The virtualization layer is provided by the Linux-
VServer project
➔ ➔ Present the current design and implementation of the
Present the current design and implementation of the testing program from a developer's point of view. testing program from a developer's point of view.
Introduction
Introduction
System overview
Introduction
System overview
Actual implementation
Introduction
System overview
Actual implementation
Future contributions
Introduction
System overview
Actual implementation
Future contributions
Conclusion
The OSCAR project is maintained by a large group
geographically separated geographically separated
The OSCAR project is maintained by a large group
geographically separated geographically separated
All the development efforts are centralized with the Subversion version system Subversion version system
While OSCAR support multiple distributions and multiple architectures, most of the OSCAR multiple architectures, most of the OSCAR developers develop only on one one architecture developers develop only on one one architecture and distribution, sometimes two. and distribution, sometimes two.
While OSCAR support multiple distributions and multiple architectures, most of the OSCAR and multiple architectures, most of the OSCAR developers develop only on one one developers develop only on one one architecture and distribution, sometimes two. architecture and distribution, sometimes two.
This mean that a developer can not easily test the impact of his changes on another Linux the impact of his changes on another Linux distribution (architecture, version, variant) distribution (architecture, version, variant) directly. directly.
As a consequence, tests are delayed until the last stages of publication of a new OSCAR version, stages of publication of a new OSCAR version, when the subversion tree is frozen and OSCAR when the subversion tree is frozen and OSCAR reaches the beta quality stage. reaches the beta quality stage.
As a consequence, tests are delayed until the last stages of publication of a new OSCAR last stages of publication of a new OSCAR version, when the subversion tree is frozen and version, when the subversion tree is frozen and OSCAR reaches the beta quality stage. OSCAR reaches the beta quality stage.
Because of the relative inefficiency of this process, OSCAR releases has been less and less frequent: OSCAR releases has been less and less frequent: the burden of physically testing OSCAR against the burden of physically testing OSCAR against many architecture/distribution/version took a lot of many architecture/distribution/version took a lot of developer's time that could be better used to developer's time that could be better used to develop new features and functionalities. develop new features and functionalities.
For those reasons is extremely important, for the evolution and stability of the OSCAR project, that a evolution and stability of the OSCAR project, that a complete set of functional tests covering all the complete set of functional tests covering all the Linux distributions (supported) can be done Linux distributions (supported) can be done periodically in a fully automated way. periodically in a fully automated way.
For those reasons is extremely important, for the evolution and stability of the OSCAR project, that a evolution and stability of the OSCAR project, that a complete set of functional tests covering all the complete set of functional tests covering all the Linux distributions (supported) can be done Linux distributions (supported) can be done periodically in a fully automated way. periodically in a fully automated way.
However, such a procedure demands a complete assembling of an OSCAR cluster for each assembling of an OSCAR cluster for each supported distribution, which consumes a huge supported distribution, which consumes a huge amount of resources and time, making it a complex, amount of resources and time, making it a complex, and sometimes complicated, task. and sometimes complicated, task.
(finally)
The automated testing should not replace completely the quality insurance process of the completely the quality insurance process of the OSCAR project. OSCAR project.
(finally)
The automated testing should not replace completely the quality insurance process of the completely the quality insurance process of the OSCAR project. OSCAR project.
It has to be considered like a « «simulation simulation» » of a
cluster installation with OSCAR using only one cluster installation with OSCAR using only one physical computer. physical computer.
(finally)
The automated testing should not replace completely the quality insurance process of the completely the quality insurance process of the OSCAR project. OSCAR project.
This being said, we believe that this testing infrastructure will allow the OSCAR project to infrastructure will allow the OSCAR project to « «release early, release often release early, release often» » and, as a and, as a conseguence, to aument the functionality and conseguence, to aument the functionality and general quality of the project. general quality of the project.
It has to be considered like a « «simulation simulation» » of a
cluster installation with OSCAR using only one cluster installation with OSCAR using only one physical computer. physical computer.
(finally)
The objective of this article is to present an automated testing infrastructure that will ( automated testing infrastructure that will (can can) be ) be used for OSCAR development. used for OSCAR development.
(finally)
The objective of this article is to present an automated testing infrastructure that will ( automated testing infrastructure that will (can can) be ) be used for OSCAR development. used for OSCAR development.
The main goal of this testing infrastructure is to reduce the development time used for testing the reduce the development time used for testing the software. software.
(finally)
The objective of this article is to present an automated testing infrastructure that will (can) automated testing infrastructure that will (can) be used for OSCAR development. be used for OSCAR development.
The main goal of this testing infrastructure is to reduce the development time used for testing the reduce the development time used for testing the software. software.
We took advantage of a stable and mature virtualization technology that is distribution virtualization technology that is distribution independent and support several architectures. independent and support several architectures.
Before exposing the virtual testing Before exposing the virtual testing infrastructure, we'll briefly present the real infrastructure, we'll briefly present the real structure of the files and directories of the structure of the files and directories of the host system ( host system (Files organization Files organization), explain how ), explain how the sources of OSCAR are the sources of OSCAR are « «manipulated manipulated» » ( (From suversion to tarball From suversion to tarball) and, finally, ) and, finally, present the design utilized ( present the design utilized (Design overview Design overview) )
Files organization
/ / etc etc vservers vservers home home
HOST HOST
Files organization
/ /
vservers vservers
... ...
n
Files organization
/ / vservers vservers
var var
etc etc
Files organization
/ / vservers vservers / /
var var
etc etc
Files organization
/ / vservers vservers / /
var var
etc etc
Files organization
/ / vservers vservers / /
var var
etc etc home home mdk10.0 mdk10.0 rhel3.0 rhel3.0
Files organization
/ / vservers vservers / /
var var
etc etc home home mdk10.0 mdk10.0 rhel3.0 rhel3.0 tftpboot tftpboot rpm rpm
Files organization
/ / vservers vservers / /
var var
etc etc home home mdk10.0 mdk10.0 rhel3.0 rhel3.0 tftpboot tftpboot rpm rpm
Files organization
/ / vservers vservers / /
var var
etc etc home home mdk10.0 mdk10.0 rhel3.0 rhel3.0 tftpboot tftpboot rpm rpm
Files organization
/ / vservers vservers / /
var var
etc etc home home mdk10.0 mdk10.0 rhel3.0 rhel3.0 tftpboot tftpboot rpm rpm
Files organization
/ / vservers vservers
var var
var var lib lib sytemimager sytemimager
Files organization
/ / vservers vservers
var var
var var lib lib sytemimager sytemimager
Files organization
/ / vservers vservers
var var
var var lib lib sytemimager sytemimager
Files organization
/ / vservers vservers
var var
var var lib lib sytemimager sytemimager
Files organization
/ / vservers vservers
etc etc
Files organization
/ / vservers vservers
etc etc vservers vservers
Files organization
/ / vservers vservers
etc etc vservers vservers
From subversion to tarball
From subversion to tarball
➔ ➔ Once released, OSCAR is a
Once released, OSCAR is a « «tarball tarball» »: : a software file a software file containing all the necessary files and binaries to install an containing all the necessary files and binaries to install an OSCAR cluster OSCAR cluster
From subversion to tarball
➔ ➔ Once released, OSCAR is a
Once released, OSCAR is a « «tarball tarball» »: : a software file a software file containing all the necessary files and binaries to install an containing all the necessary files and binaries to install an OSCAR cluster OSCAR cluster
➔ ➔ Developers are using a subversion repository to track the
Developers are using a subversion repository to track the changes to the source code and an automated suite of changes to the source code and an automated suite of programs (autoconf, automake, make) is used to build the programs (autoconf, automake, make) is used to build the tarball tarball
From subversion to tarball
➔ ➔ While it's possible to test directly from the subversion
While it's possible to test directly from the subversion repository, this is not what a regular OSCAR user will use: he repository, this is not what a regular OSCAR user will use: he will use the tarball available will use the tarball available
From subversion to tarball
➔ ➔ While it's possible to test directly from the subversion
While it's possible to test directly from the subversion repository, this is not what a regular OSCAR user will use: he repository, this is not what a regular OSCAR user will use: he will use the tarball available will use the tarball available
➔ ➔ So, we build the tarball from the sources, copy it to the
So, we build the tarball from the sources, copy it to the main vserver and use it for install OSCAR main vserver and use it for install OSCAR
Design overview
Design overview
➔ ➔ A
A « «standard standard» » script ( script (script A script A) is used ) is used to create a vserver to create a vserver
Design overview
➔ ➔ A
A « «standard standard» » script ( script (script A script A) is used ) is used to create a vserver to create a vserver
➔ ➔ This script is generic and can be used to create any Linux
This script is generic and can be used to create any Linux-
VServer system (Fedora, RHAS, Mandrake, Debian)*
* At the time of this paper * At the time of this paper
Design overview
➔ ➔ A
A « «standard standard» » script ( script (script A script A) is used ) is used to create a vserver to create a vserver
➔ ➔ This script is generic and can be used to create any Linux
This script is generic and can be used to create any Linux-
VServer system (Fedora, RHAS, Mandrake, Debian)*
* At the time of this paper * At the time of this paper
➔ ➔ This is the starting point of the automated testing process:
This is the starting point of the automated testing process: the creation, from scratch, of a new Linux the creation, from scratch, of a new Linux-
VServer that will become the OSCAR master node will become the OSCAR master node
Design overview
➔ ➔ Three additional scripts will be used to test OSCAR:
Three additional scripts will be used to test OSCAR:
* At the time of this paper * At the time of this paper
Design overview
➔ ➔ Three additional scripts will be used to test OSCAR:
Three additional scripts will be used to test OSCAR:
script B: to adapt the original vserver configurations : to adapt the original vserver configurations to suit OSCAR and make a copy of the to suit OSCAR and make a copy of the subversion repository to the new vserver subversion repository to the new vserver
* At the time of this paper * At the time of this paper
Design overview
➔ ➔ Three additional scripts will be used to test OSCAR:
Three additional scripts will be used to test OSCAR:
script B: to adapt the original vserver configurations : to adapt the original vserver configurations to suit OSCAR and make a copy of the to suit OSCAR and make a copy of the subversion repository to the new vserver subversion repository to the new vserver
* At the time of this paper * At the time of this paper
script C: to initiate the installation process : to initiate the installation process
Design overview
➔ ➔ Three additional scripts will be used to test OSCAR:
Three additional scripts will be used to test OSCAR:
script B: to adapt the original vserver configurations : to adapt the original vserver configurations to suit OSCAR and make a copy of the to suit OSCAR and make a copy of the subversion repository to the new vserver subversion repository to the new vserver
* At the time of this paper * At the time of this paper
script C: to initiate the installation process : to initiate the installation process
script D: to create the client nodes : to create the client nodes
➔ ➔ The numbers of
The numbers of scripts as well as scripts as well as the user interaction the user interaction will be reduced in will be reduced in the future versions the future versions in order to achieve in order to achieve a better a better automatization of automatization of the testing process the testing process
1 1
➔ ➔ Create the vserver
Create the vserver ( (script A script A) with ) with OSCAR specifics OSCAR specifics ( (script B script B) )
2 2
➔ ➔ Start the vserver
Start the vserver master: master:
vserver oscarserver start vserver oscarserver start
3 3
➔ ➔ Change to the
Change to the vserver master: vserver master:
vserver oscarserver enter vserver oscarserver enter
4 4
➔ ➔ Launch
Launch install_cluster install_cluster
5 5
➔ ➔ Wizard: step 0
Wizard: step 0
6 6
➔ ➔ Wizard: step 1
Wizard: step 1
7 7
➔ ➔ Wizard: step 2
Wizard: step 2
8 8
➔ ➔ Wizard: step 3
Wizard: step 3
9 9
➔ ➔ Wizard: step 4
Wizard: step 4
10 10
➔ ➔ Wizard: step 5
Wizard: step 5
11 11
➔ ➔ Return to the host
Return to the host system system exit exit
12 12
➔ ➔ Create the vservers
Create the vservers clients from the clients from the
script D D) )
13 13
➔ ➔ Start each one of
Start each one of the clients the clients
14 14
➔ ➔ Return to the
Return to the vserver master vserver master (oscarserver) (oscarserver)
15 15
➔ ➔ Wizard: step 7
Wizard: step 7
16 16
➔ ➔ Wizard: step 8
Wizard: step 8
Actual implementation
Actual implementation
➔ ➔ Can be separated in two major steps:
Can be separated in two major steps:
Actual implementation
➔ ➔ Can be separated in two major steps:
Can be separated in two major steps:
OSCAR server ( OSCAR server (step 1 step 1) )
Actual implementation
➔ ➔ Can be separated in two major steps:
Can be separated in two major steps:
OSCAR server ( OSCAR server (step 1 step 1) )
a second to install OSCAR (step 2 step 2) )
Actual implementation
➔ ➔ There should not be big problems regarding the first one,
There should not be big problems regarding the first one,
location and location and the system is ready to host vservers the system is ready to host vservers
Actual implementation
➔ ➔ There should not be big problems regarding the first one,
There should not be big problems regarding the first one,
location and location and the system is ready to host vservers the system is ready to host vservers
➔ ➔ The testing will really begin in the second setp
The testing will really begin in the second setp
Generation of the OSCAR master node
Generation of the OSCAR master node
➔ ➔ The main
The main « «make make-
vserver» » script ( script (A A) can be considered as ) can be considered as a function requiring as input parameters a function requiring as input parameters hostname hostname, , domainname domainname and and ip address ip address of the OSCAR server you
want to generate. This want to generate. This « «function function» » will return a fully will return a fully functional Linux functional Linux-
VServer
Generation of the OSCAR master node
➔ ➔ The main
The main « «make make-
vserver» » script ( script (A A) can be considered as ) can be considered as a function requiring as input parameters a function requiring as input parameters hostname hostname, , domainname domainname and and ip address ip address of the OSCAR server you
want to generate. This want to generate. This « «function function» » will return a fully will return a fully functional Linux functional Linux-
VServer
➔ ➔ The option 'TYPE=OSCAR' will trigger additional
The option 'TYPE=OSCAR' will trigger additional threatment of the generated vserver. This will call script B threatment of the generated vserver. This will call script B to modify the original vserver so that it can be used to to modify the original vserver so that it can be used to build and install OSCAR (capabilities, ssh permissions, build and install OSCAR (capabilities, ssh permissions, /tftpboot, etc) /tftpboot, etc)
Generation of the OSCAR master node
➔ ➔ Those scripts also generate two configuration files for the
Those scripts also generate two configuration files for the vservers: vservers:
Generation of the OSCAR master node
➔ ➔ Those scripts also generate two configuration files for the
Those scripts also generate two configuration files for the vservers: vservers:
➔ ➔ «
«vserver_name vserver_name» ».conf .conf: data specific to the vserver (hostname, : data specific to the vserver (hostname, dommainame, ip address and capabilities) dommainame, ip address and capabilities)
Generation of the OSCAR master node
➔ ➔ Those scripts also generate two configuration files for the
Those scripts also generate two configuration files for the vservers: vservers:
➔ ➔ «
«vserver_name vserver_name» ».conf .conf: data specific to the vserver (hostname, : data specific to the vserver (hostname, dommainame, ip address and capabilities) dommainame, ip address and capabilities)
➔ ➔ «
«vserver_name vserver_name» ».sh .sh: contains : contains « «hooks hooks» » to scripts that should to scripts that should be executed pre or post be executed pre or post-
start of the Linux-
VServer and pre or post post-
stop
/ / vservers vservers
etc etc vservers vservers
IPROOT=192.168.17.1 IPROOT=192.168.17.1 #IPROOTMASK= #IPROOTMASK= #IPROOTBCAST= #IPROOTBCAST= IPROOTDEV=eth1 IPROOTDEV=eth1 ONBOOT=yes ONBOOT=yes S_HOSTNAME=oscarserver S_HOSTNAME=oscarserver S_DOMAINNAME=test S_DOMAINNAME=test S_NICE= S_NICE= S_FLAGS="lock nproc" S_FLAGS="lock nproc" ULIMIT=" ULIMIT="-
HS -
u 1000" S_CAPS="CAP_NET_RAW CAP_MKNOD CAP_CHOWN CAP_SYS_TIME CAP_SYS_RES S_CAPS="CAP_NET_RAW CAP_MKNOD CAP_CHOWN CAP_SYS_TIME CAP_SYS_RESOURCE" OURCE"
Generation of the OSCAR master node
Generation of the OSCAR master node
In order to allow OSCAR to be installed inside a Linux In order to allow OSCAR to be installed inside a Linux-
VServer, specific capabilities capabilities has to be given to this has to be given to this vserver: vserver:
Generation of the OSCAR master node
In order to allow OSCAR to be installed inside a Linux In order to allow OSCAR to be installed inside a Linux-
VServer, specific capabilities capabilities has to be given to this has to be given to this vserver: vserver:
CAP_MKNOD: : permit the privileged aspects of mknod permit the privileged aspects of mknod
Generation of the OSCAR master node
In order to allow OSCAR to be installed inside a Linux In order to allow OSCAR to be installed inside a Linux-
VServer, specific capabilities capabilities has to be given to this has to be given to this vserver: vserver:
CAP_MKNOD: : permit the privileged aspects of mknod permit the privileged aspects of mknod
CAP_CHOWN: : change the ownership and group change the ownership and group
Generation of the OSCAR master node
In order to allow OSCAR to be installed inside a Linux In order to allow OSCAR to be installed inside a Linux-
VServer, specific capabilities capabilities has to be given to this has to be given to this vserver: vserver:
CAP_MKNOD: : permit the privileged aspects of mknod permit the privileged aspects of mknod
CAP_CHOWN: : change the ownership and group change the ownership and group
CAP_SYS_TIME: : allow manipulation of the system clock allow manipulation of the system clock
Generation of the OSCAR master node
In order to allow OSCAR to be installed inside a Linux In order to allow OSCAR to be installed inside a Linux-
VServer, specific capabilities capabilities has to be given to this has to be given to this vserver: vserver:
CAP_MKNOD: : permit the privileged aspects of mknod permit the privileged aspects of mknod
CAP_CHOWN: : change the ownership and group change the ownership and group
CAP_SYS_TIME: : allow manipulation of the system clock allow manipulation of the system clock
CAP_SYS_RESOURCE: : override resource limits, quota,
reserved space on fs, ... reserved space on fs, ...
Files organizaton
/ / vservers vservers
etc etc vservers vservers
#!/bin/sh #!/bin/sh case $1 in case $1 in pre pre-
start) # Ajouter les actions de pr # Ajouter les actions de pré é-
dé émarrage ici marrage ici mount mount --
bind /home/mdk10.0 /vservers/oscarserver/tftpboot/rpm vproc vproc -
e /proc/net/unix vproc vproc -
e /proc/net vproc vproc -
e /proc/net/dev vproc vproc -
e /dev/null vproc vproc -
e /proc/net/route vproc vproc -
e /proc/sys vproc vproc -
e /proc/sys/* ;; ;; post post-
start) # Ajouter les actions de post # Ajouter les actions de post-
dé émarrage ici marrage ici ;; ;; pre pre-
stop) # Ajouter les actions de pr # Ajouter les actions de pré é-
arrêt ici ;; ;; post post-
stop) # Ajouter les actions de post # Ajouter les actions de post-
arrêt ici umount /vservers/oscarserver/tftpboot/rpm umount /vservers/oscarserver/tftpboot/rpm ;; ;; *) *) echo $0 pre echo $0 pre-
start echo $0 pre echo $0 pre-
stop echo $0 post echo $0 post-
start echo $0 post echo $0 post-
stop ;; ;; esac esac
Generation of the OSCAR master node
➔ ➔ The pre
The pre-
start actions include binding the distribution RPM's repository to /tftpboot/rpm and giving to the RPM's repository to /tftpboot/rpm and giving to the vserve access to several entries in the /proc vserve access to several entries in the /proc filesystem filesystem (using vproc) (using vproc)
Generation of the OSCAR master node
➔ ➔ The pre
The pre-
start actions include binding the distribution RPM's repository to /tftpboot/rpm and giving to the RPM's repository to /tftpboot/rpm and giving to the vserve access to several entries in the /proc vserve access to several entries in the /proc filesystem filesystem (using vproc) (using vproc)
➔ ➔ Other significant and necessary change is to add
Other significant and necessary change is to add localhost in the same line of the vserver ip address in localhost in the same line of the vserver ip address in /etc/hosts and delete the entry for 127.0.0.1 /etc/hosts and delete the entry for 127.0.0.1
Installing OSCAR
Installing OSCAR
Following, some final considerations about installing Following, some final considerations about installing
➔ ➔ The Linux
The Linux-
VServer use a virtual network interface name eth name ethx x: :« «vserver_name vserver_name» ». So, the command to . So, the command to launch the installation should be launch the installation should be install_cluster install_cluster eth ethx: x: « «vserver_name vserver_name» »
➔ ➔ Things must be in a well know state so that the OSCAR
Things must be in a well know state so that the OSCAR wizard can be loaded. Otherwise, errors will be wizard can be loaded. Otherwise, errors will be triggered and the installation will fail, as it would in a triggered and the installation will fail, as it would in a regular environment regular environment
➔ ➔ Things must be in a well know state so that the
Things must be in a well know state so that the OSCAR wizard can be loaded. Otherwise, errors OSCAR wizard can be loaded. Otherwise, errors will be triggered and the installation will fail, as it will be triggered and the installation will fail, as it would in a regular environment would in a regular environment
➔ ➔ When the wizard appears, this mean we have a
When the wizard appears, this mean we have a considerable part of the system operational for run considerable part of the system operational for run OSCAR (all the prerequisites packages are installed, so OSCAR (all the prerequisites packages are installed, so no package dependencies problem) no package dependencies problem)
➔ ➔ The step 6 of the Wizard (
The step 6 of the Wizard (Setup Networking Setup Networking) is not ) is not executed during the automated testing. executed during the automated testing.
➔ ➔ The step 6 of the Wizard (
The step 6 of the Wizard (Setup Networking Setup Networking) is not ) is not executed during the automated testing. executed during the automated testing.
➔ ➔ Instead of doing this step, the image created during step
Instead of doing this step, the image created during step 4 will be copied by the 4 will be copied by the script D script D and will be the root file and will be the root file system used by the newly created vservers that will act system used by the newly created vservers that will act as regular OSCAR nodes. as regular OSCAR nodes.
➔ ➔ The step 6 of the Wizard (
The step 6 of the Wizard (Setup Networking) is not Setup Networking) is not executed during the automated testing. executed during the automated testing.
➔ ➔ Not executing
Not executing Setup Networking Setup Networking means that we can not means that we can not test, with the virtualization technique we use, the test, with the virtualization technique we use, the following functionalities: setup DHCP, listening of MAC following functionalities: setup DHCP, listening of MAC addresses ( addresses (tcpdump tcpdump), TFTP server, rsync, remote ), TFTP server, rsync, remote installation, etc installation, etc
➔ ➔ Instead of doing this step, the image created during step
Instead of doing this step, the image created during step 4 will be copied by the 4 will be copied by the script D script D and will be the root file and will be the root file system used by the newly created vservers that will act system used by the newly created vservers that will act as regular OSCAR nodes. as regular OSCAR nodes.
➔ ➔ Because we are sharing the same hardware
Because we are sharing the same hardware resources between the server and the clients we resources between the server and the clients we can not use NFS in order to mount the server's can not use NFS in order to mount the server's home directory in each client home directory in each client
➔ ➔ Steps 7 (
Steps 7 (Complete Cluster Setup Complete Cluster Setup) and 8 ( ) and 8 (Test Cluster Test Cluster Setup Setup) be executed in order to test the functionality of ) be executed in order to test the functionality of the the « «brand new brand new» » cluster cluster
➔ ➔ Steps 7 (
Steps 7 (Complete Cluster Setup Complete Cluster Setup) and 8 ( ) and 8 (Test Cluster Test Cluster Setup Setup) be executed in order to test the functionality of ) be executed in order to test the functionality of the the « «brand new brand new» » cluster cluster
➔ ➔ In our tests, excepting for Ganglia, that rely on
In our tests, excepting for Ganglia, that rely on multicast (not yet supported in the Linux multicast (not yet supported in the Linux-
VServer project) all the final tests had positive results project) all the final tests had positive results
➔ ➔ Steps 7 (Complete Cluster Setup) and 8 (Test
Steps 7 (Complete Cluster Setup) and 8 (Test Cluster Setup) be executed in order to test the Cluster Setup) be executed in order to test the functionality of the functionality of the « «brand new brand new» » cluster cluster
➔ ➔ New tests should be added to exploit this testing
New tests should be added to exploit this testing structure. structure.
➔ ➔ In our tests, excepting for Ganglia, that rely on
In our tests, excepting for Ganglia, that rely on multicast (not yet supported in the Linux multicast (not yet supported in the Linux-
VServer project) all the final tests had positive results project) all the final tests had positive results
➔ ➔ The automated testing structure for OSCAR is a
The automated testing structure for OSCAR is a project that is still in development. project that is still in development.
➔ ➔ The automated testing structure for OSCAR
The automated testing structure for OSCAR is a project that is still in development. is a project that is still in development.
➔ ➔ The main objective of this article is to present it
The main objective of this article is to present it to the OSCAR developer community, pointing to the OSCAR developer community, pointing
system potentially has. system potentially has.
➔ ➔ There is certainly space for further development
There is certainly space for further development and improvements, both in its concept and actual and improvements, both in its concept and actual implementation implementation
➔ ➔ There is certainly space for further development
There is certainly space for further development and improvements, both in its concept and actual and improvements, both in its concept and actual implementation implementation
➔ ➔ Some of those will be presented in the following
Some of those will be presented in the following section section
➔ ➔ There is certainly space for further
There is certainly space for further development and improvements, both in its development and improvements, both in its concept and actual implementation concept and actual implementation
➔ ➔ Some of those will be presented in the following
Some of those will be presented in the following section section
➔ ➔ Ultimately, we hope that more ideas will come
Ultimately, we hope that more ideas will come from the feedback received from you and the from the feedback received from you and the automated testing results automated testing results
Automated testing without GUI
Automated testing without GUI
➔ ➔ Because the OSCAR Wizard is, in fact, a linear
Because the OSCAR Wizard is, in fact, a linear process consisting of 8 steps that have to be process consisting of 8 steps that have to be followed in order, we think that a very simple followed in order, we think that a very simple modification to the actual code base would allow modification to the actual code base would allow an automated testing, without wizard. an automated testing, without wizard.
Automated testing without GUI
➔ ➔ Before each step, we can look at some specific
Before each step, we can look at some specific place inside the /opt/oscar/automated_testing place inside the /opt/oscar/automated_testing directory and search for files named accordingly to directory and search for files named accordingly to the steps: step1.xml, step2.xml, ... , step3.xml the steps: step1.xml, step2.xml, ... , step3.xml
Automated testing without GUI
➔ ➔ Before each step, we can look at some specific
Before each step, we can look at some specific place inside the /opt/oscar/automated_testing place inside the /opt/oscar/automated_testing directory and search for files named accordingly to directory and search for files named accordingly to the steps: step1.xml, step2.xml, ... , step3.xml the steps: step1.xml, step2.xml, ... , step3.xml
➔ ➔ If a file is found, then the answer to the GUI step is
If a file is found, then the answer to the GUI step is read from this file instead of being obtained from read from this file instead of being obtained from the GUI the GUI
Automated testing without GUI
➔ ➔ With this system, we can very easily implement a
With this system, we can very easily implement a fully automated test plans that do not need the GUI fully automated test plans that do not need the GUI while being very close to the actual GUI while being very close to the actual GUI implementation implementation
Multiple distributions
Multiple distributions
➔ ➔ The structure presented here was validated using
The structure presented here was validated using the Mandrake Linux distribution, version 10.0, and the Mandrake Linux distribution, version 10.0, and experimentally with RedHat Advanced Server 3.0 experimentally with RedHat Advanced Server 3.0
Multiple distributions
➔ ➔ It can easily be extended to cover any Linux
It can easily be extended to cover any Linux distribution supported by OSCAR, needing only distribution supported by OSCAR, needing only some modifications to the vservers construction some modifications to the vservers construction scripts as well as the addition of specific rpmlists scripts as well as the addition of specific rpmlists for each new distribution added for each new distribution added
Reproducing real node installation (SIS testing) (SIS testing)
Reproducing real node installation (SIS testing) (SIS testing)
➔ ➔ With the objective of having a testing structure
With the objective of having a testing structure similar to the real one where clusters are installed similar to the real one where clusters are installed and used, the Linux and used, the Linux-
VServers can be used only for simulate the installation of the OSCAR server. simulate the installation of the OSCAR server.
Reproducing real node installation (SIS testing) (SIS testing)
➔ ➔ If needed, it is possible to use separate physical
If needed, it is possible to use separate physical machine(s) as OSCAR client nodes machine(s) as OSCAR client nodes
Reproducing real node installation (SIS testing) (SIS testing)
➔ ➔ If needed, it is possible to use separate physical
If needed, it is possible to use separate physical machine(s) as OSCAR client nodes machine(s) as OSCAR client nodes
➔ ➔ With this technique, one can still use most of the
With this technique, one can still use most of the automated testing structure but manually test the automated testing structure but manually test the physical installation process of the nodes physical installation process of the nodes
Reference platform for OSCAR development development
Reference platform for OSCAR development development
➔ ➔ OSCAR is a layer that is installed on the top of an
OSCAR is a layer that is installed on the top of an existing Linux installation existing Linux installation
Reference platform for OSCAR development development
➔ ➔ OSCAR is a layer that is installed on the top of an
OSCAR is a layer that is installed on the top of an existing Linux installation existing Linux installation
➔ ➔ Because of this, it is very difficult to have a similar
Because of this, it is very difficult to have a similar development platform common to every OSCAR development platform common to every OSCAR developers: each one installs it's own server, with developers: each one installs it's own server, with different options, partitioning, package selection, different options, partitioning, package selection, etc etc
Reference platform for OSCAR development development
➔ ➔ With the use of a virtualization technique it is
With the use of a virtualization technique it is possible to share a so called possible to share a so called « «reference platform reference platform» » for every supported architecture and Linux for every supported architecture and Linux distribution version distribution version
Reference platform for OSCAR development development
➔ ➔ With the use of a virtualization technique it is
With the use of a virtualization technique it is possible to share a so called possible to share a so called « «reference platform reference platform» » for every supported architecture and Linux for every supported architecture and Linux distribution version distribution version
➔ ➔ We believe that, with this setup, bug reproduction
We believe that, with this setup, bug reproduction will be much better performed will be much better performed
Nightly testing and publication of results
Nightly testing and publication of results results
➔ ➔ Once the tests are fully automatized, we intend to
Once the tests are fully automatized, we intend to test OSCAR functionality with the latest subversion test OSCAR functionality with the latest subversion version. version.
Nightly testing and publication of results
➔ ➔ Once the tests are fully automatized, we intend to
Once the tests are fully automatized, we intend to test OSCAR functionality with the latest subversion test OSCAR functionality with the latest subversion version. version.
➔ ➔ Those tests will be made against a so called
Those tests will be made against a so called « «OSCAR nightly build OSCAR nightly build» »
Nightly testing and publication of results results
➔ ➔ We think this could greatly reduce OSCAR
We think this could greatly reduce OSCAR development time as bugs will appear very rapidly development time as bugs will appear very rapidly and the faulty code could be easily removed or and the faulty code could be easily removed or fixed from the OSCAR subversion repository fixed from the OSCAR subversion repository
We presented a new and « «automated automated» » testing testing infrastrucutre using the virtualization technique from infrastrucutre using the virtualization technique from the Linux the Linux-
VServer project.
We presented a new and « «automated automated» » testing testing infrastrucutre using the virtualization technique from infrastrucutre using the virtualization technique from the Linux the Linux-
VServer project.
The main objective of the proposed testing infrastructure is to provide to the OSCAR developers infrastructure is to provide to the OSCAR developers the possibility of quickly test the development the possibility of quickly test the development versions of OSCAR for several sets of Linux versions of OSCAR for several sets of Linux distributions/ versions/architectures without the pain distributions/ versions/architectures without the pain and time consumption of effectively building a real and time consumption of effectively building a real cluster for this purpose. cluster for this purpose.
Due to the use of a virtualization technologie, the installation of various Linux distributions from a installation of various Linux distributions from a given host system is very easy and fully given host system is very easy and fully automated. automated.
Due to the use of a virtualization technologie, the installation of various Linux distributions from a given installation of various Linux distributions from a given host system is very easy and fully automated. host system is very easy and fully automated.
This, in turn, allow the simulation of an OSCAR cluster installation, for the server and nodes, as well cluster installation, for the server and nodes, as well as the functional test of multiple distributions using as the functional test of multiple distributions using several architectures several architectures
Even if this new system can not replace completely the conventional OSCAR testing, we completely the conventional OSCAR testing, we believe that it will contribute to the development believe that it will contribute to the development efforts of OSCAR by drastically reducing the efforts of OSCAR by drastically reducing the delay between the introduction of a new delay between the introduction of a new functionality and bug detection. functionality and bug detection.
This research has been funded by the This research has been funded by the National Research Council's Industrial National Research Council's Industrial Research Assistance Program (NRC Research Assistance Program (NRC-
IRAP), project number 547017 project number 547017