Outline The UNICORE Grid System 1. Introduction UNICORE – UNICORE Plus Project Tutorial The UNICORE Grid System – Software Status – Architecture [- Acronyms] Klaus-Dieter Oertel, Pallas GmbH 2. Client kdoertel@pallas.com – “My first job” Mathilde Romberg, Forschungszentrum Jülich – Installation – Configuration m.romberg@fz-juelich.de – Job Preparation – Job Monitoring – Application Specific Interfaces (Plugins) Euro-Par 2002, Paderborn Klaus-Dieter Oertel, Pallas GmbH 2 Mathilde Romberg, Forschungszentrum Jülich Outline (cont.) The UNICORE Grid System 3. Server Tutorial The UNICORE Grid System – Overview (Packages, Components, Prerequisites) The UNICORE Grid System – Gateway (Installation, Configuration, Maintenance) – Network Job Supervisor Part 1 • Installation, basic configuration Introduction • Incarnation Data Base • Maintenance • UNICORE User Data Base – Target System Interface (Installation, Configuration) 4. Discussion Klaus-Dieter Oertel, Pallas GmbH 3 Klaus-Dieter Oertel, Pallas GmbH 4 Mathilde Romberg, Forschungszentrum Jülich Mathilde Romberg, Forschungszentrum Jülich
UNICORE Plus Project Partner UN iform I nterface to CO mputing RE sources • Research Center Jülich (FZJ, Project Coordinator) • UNICORE Plus Project • UNICORE Plus Project • Follow-on of the UNICORE project • German Weather Service (DWD) • Funded by German Ministry of Education • Computer Center University of Stuttgart (RUS) Introduction UNICORE Introduction UNICORE and Research (bmb+f), Grant-id: 01 IR 001 • Pallas GmbH, Brühl • Grant period 1.1.2000 - 31.12.2002 • Center for High Performance Computing at TU Dresden (ZHR) • Development of a prototype • Computer Center University of Karlsruhe (RUKA) for the seamless, secure, and intuitive access • Konrad Zuse Center, Berlin (ZIB) to distributed supercomputer resources • Leibniz Computer Center, Munich (LRZ) • http://www.fz-juelich.de/unicoreplus • Paderborn Center for Parallel Computing (PC²) • http://www.unicore.de • Fujitsu Laboratory of Europe (former fecit) Klaus-Dieter Oertel, Pallas GmbH 5 Klaus-Dieter Oertel, Pallas GmbH 6 Mathilde Romberg, Forschungszentrum Jülich Mathilde Romberg, Forschungszentrum Jülich Project Plan Goals • Based on results of UNICORE project • Closer cooperation of HPC Centers • Resource Modelling (static) • UNICORE Plus Project • UNICORE Plus Project • Support for computational science • Application Specific Interfaces (CPMD, Introduction UNICORE Introduction UNICORE • Overcome the seams created by NASTRAN, ..., generic / services) – Incompatible system software • Extended Work Flow (repeat, if-then-else, ...) – Site policies and practices • Data Management Enhancements (high-speed, • Exploit existing and emerging technologies high-throughput, file transfer independent of job) – Java, Web-techniques – X.509 certificates • Metacomputing (Co-scheduling, • Minimal interference with local site policies MPI and PACX integration, Vampir extensions) and administration • Advanced administration Klaus-Dieter Oertel, Pallas GmbH 7 Klaus-Dieter Oertel, Pallas GmbH 8 Mathilde Romberg, Forschungszentrum Jülich Mathilde Romberg, Forschungszentrum Jülich
Software Status Software Status (cont.) • Current version 3.6 • UNICORE is deployed at the UNICORE • User Client available for Windows and Plus and the EUROGRID project partner Introduction UNICORE Introduction UNICORE Unix (Linux,…) , runs also on Macintosh centers • Software Status • Software Status • Servers to be run on Unix (Linux, …) • UNICORE Forum e.V. (www.unicore.org) • Target systems Unix (Linux, …) only distributes UNICORE under community • T3E, SP3, VPP, hpcLine, SR 8000, SX-5, source license PC-Clusters, … as targets • Public test system (www.fz-juelich.de/unicore-test) • Several NQS dialects, LL, LSF, PBS, CCS for testing client functions as target batch systems Klaus-Dieter Oertel, Pallas GmbH 9 Klaus-Dieter Oertel, Pallas GmbH 10 Mathilde Romberg, Forschungszentrum Jülich Mathilde Romberg, Forschungszentrum Jülich Download Page Deployment • UNICORE Software and Sources available at • European Projects use UNICORE http://www.unicore.org/downloads.htm: – EUROGRID (www.eurogrid.org) Introduction UNICORE Introduction UNICORE Application Testbed for European GRID computing • Software Status • Software Status • Introduction partly funded under EC grant IST-1999-20247 • Package bundles (Client, Server, full) – GRIP (www.grid-interoperability.org) • Abstract Job Object • Client Grid Interoperability Project • Plugins partly funded under EC grant IST-2001-32257 • Gateway – OpenMolGRID • Server Open Computing Grid for Molecular Science and • Contributions Engineering • License partly funded by EC grant IST-2001-37238 • Certificate Authority Klaus-Dieter Oertel, Pallas GmbH 11 Klaus-Dieter Oertel, Pallas GmbH 12 Mathilde Romberg, Forschungszentrum Jülich Mathilde Romberg, Forschungszentrum Jülich
Architecture Highlights User Workstation UNICORE GUI Site List • Support for Batch-Applications HTTP SSL • Heterogeneous metacomputing Introduction UNICORE Introduction UNICORE UNICORE Server UNICORE Server • Software Status • Transparent data staging / transfer • Uniform user authentication and security mechanisms • Architecture Gateway Gateway • Uniform GUI for job creation and monitoring SSL Network Job Supervisor Network Job Supervisor Network Job Supervisor Network Job Supervisor Network Job Supervisor Network Job Supervisor • Easy integration of new applications through plugins TCP/IP TCP/IP • Jobs in XML formats Batchsystem Target System Interface Target System Interface Batchsystem • Command line interface Batchsystem Batchsystem Batch SubSystem Batch SubSystem UNICORE Site 1 UNICORE Site n Klaus-Dieter Oertel, Pallas GmbH 13 Klaus-Dieter Oertel, Pallas GmbH 14 Mathilde Romberg, Forschungszentrum Jülich Mathilde Romberg, Forschungszentrum Jülich Abstract Job Object (AJO) UNICORE Job • Abstract representation of UNICORE job • Job contains • Recursive Java object – Sub-jobs and tasks Introduction UNICORE Introduction UNICORE • Specifies all actions to be performed by UNICORE – Dependency information • Architecture • Architecture – Execute task – Target system – File transfer task • Tasks are translated into batch jobs for – Control task • Contains dependency graph the destination system • Contains resource specification • Contains data set descriptions for data to be streamed Klaus-Dieter Oertel, Pallas GmbH 15 Klaus-Dieter Oertel, Pallas GmbH 16 Mathilde Romberg, Forschungszentrum Jülich Mathilde Romberg, Forschungszentrum Jülich
AJO - Example AJO – Class Hierarchy AbstractAction rootajo Introduction UNICORE Introduction UNICORE Import Import JobGroup AbstractTask • Architecture • Architecture (is DAG of AbstractActions) ajo 1 Execute Execute Transfer Transfer ajo 2 AbstractJob ExecuteTask FileTransfer Execute Export UserTask CompileTask LinkTask ImportTask ExportTask Klaus-Dieter Oertel, Pallas GmbH 17 Klaus-Dieter Oertel, Pallas GmbH 18 Mathilde Romberg, Forschungszentrum Jülich Mathilde Romberg, Forschungszentrum Jülich Data Model Security • Secure Socket Layer (SSL) • X.509v3 user and server certificates • UNICORE file space per UNICORE job • User’s secret key stored in pw protected keystore Introduction UNICORE Introduction UNICORE at client • Non-permanent • UNICORE Login ≡ User certificate • Architecture • Architecture • User has to specify remote data location • Authorization by mapping of certificate to local userid explicitly • AJOs signed with user certificate • Data import / export / transfer • Users may have multiple certs from different signers • Server components accept certs from multiple signers • Site Specific Security Objects (SSO) Klaus-Dieter Oertel, Pallas GmbH 19 Klaus-Dieter Oertel, Pallas GmbH 20 Mathilde Romberg, Forschungszentrum Jülich Mathilde Romberg, Forschungszentrum Jülich
Recommend
More recommend
