Parameter handling Parameter handling and the HADES Oracle database - - PowerPoint PPT Presentation

▶
parameter handling parameter handling
SMART_READER_LITE
LIVE PREVIEW

Parameter handling Parameter handling and the HADES Oracle database - - PowerPoint PPT Presentation

Mar 24, 2023 249 likes •495 views

Parameter handling Parameter handling and the HADES Oracle database and the HADES Oracle database Ilse Koenig, GSI HADES Analysis workshop, May 2017 Outline: Initialization concept Runtime database Parameter containers Example macros

slide-1
SLIDE 1

Ilse Koenig, GSI HADES Analysis workshop, May 2017

Parameter handling Parameter handling

and the HADES Oracle database and the HADES Oracle database

Outline: Initialization concept Runtime database Parameter containers Example macros Version management Parameter file generator Documentation on HADES computing page: HYDRA MANUALS: Parameter Initialization, Geometry

slide-2
SLIDE 2

Parameter handling - HADES Analysis Workshop at GSI, May 10-11 2017 2

The HADES Oracle database The HADES Oracle database

DAQ EPICS users

Oracle

related to runs by time stamps

  • nline

analysis

  • ffline analysis

simulation file catalog slow control data beam time logbook parameters geometry setup of electronics ROOT macros

Run: smallest data set individually treated several files, one per event builder

web interface

slide-3
SLIDE 3

Parameter handling - HADES Analysis Workshop at GSI, May 10-11 2017 3

Examples for parameters needed for a DST production Examples for parameters needed for a DST production

Unpacking Calibration Hit reconstruction Correlation, momentum reconstruction, ... Experiment Simulation Digitization Track candidates

thresholds channel masks lookup tables calibration parameters digitization parameters GEANT geometry geometry alignment parameters for hit finding field map spline parameters matching windows ... In total more than 100 parameter containers, about 70 needed for a DST production

slide-4
SLIDE 4

Parameter handling - HADES Analysis Workshop at GSI, May 10-11 2017 4

Analysis initialization concept Analysis initialization concept

HYDRA Designed for automatic initialization

Parameters (numbers, arrays, user defined or ROOT classes,…) are stored in parameter containers,

automatically in the init-functions of the unpackers and analysis tasks

and managed by the runtime database classes. User chooses one or two parameter sources in the analysis macro. The analyzed files define the parameter versions.

separate shared libraries Oracle Precompiler

ROOT TGeoManager Oracle ASCII Files ASCII, ROOT Files Web GUI Geometry Interface Runtime Database HGEANT HYDRA

slide-5
SLIDE 5

Parameter handling - HADES Analysis Workshop at GSI, May 10-11 2017 5

Runtime Database Runtime Database

Parameter containers:

derived from base class HParSet defines virtual functions init, write, clear, … data members: parameter context, IO versions, ... flags: static, changed created via container factories in a macro or the init function of a task (calls constructor if not existing) rtdb->getContainer(“ContainerName”); to search for a parameter container use rtdb->findContainer(“...”)

List of runs:

added during initialization rtdb->initContainers(runId);

Container factories:

loaded when Hydra2 is loaded

  • ne for each shared library with parameters

containers functions to create the parameter containers

slide-6
SLIDE 6

Parameter handling - HADES Analysis Workshop at GSI, May 10-11 2017 6

Parameter IO Parameter IO

Oracle HParOra2Io* ora = new HParOra2Io;

  • ra->open(); // input (default user HADES_ANA)
  • ra->open(“rich_oper”); // input/output (user with write access)

Root file HParRootFileIo* file = new HParRootFileIo;

file->open(“xxx.root”); // input (default “READ”) file

  • pen(“xxx.root”,”RECREATE”);

→ // output

ASCII file HParAsciiFileIo* file = new HParAsciiFileIo;

file->open(“xxx.txt”); // input (default “in”) file->open(“xxx.txt”,”out”); // output rtdb->setFirstInput(io); rtdb->setSecondInput(io); rtdb->setOutput(io);

slide-7
SLIDE 7

Parameter handling - HADES Analysis Workshop at GSI, May 10-11 2017 7

Flow of initialization Flow of initialization

calls rtdb->initContainers(runId) loops over all parameter containers tries to initialize a container from 1st input, if it fails from 2nd input

slide-8
SLIDE 8

Parameter handling - HADES Analysis Workshop at GSI, May 10-11 2017 8

Example initialization macro init_params.C Example initialization macro init_params.C

{ // create a Hades object and get the // pointers to the runtime database and // the spectrometer Hades* myHades = new Hades; HSpectrometer* spec = gHades->getSetup(); HRuntimeDb* rtdb = gHades->getRuntimeDb(); // create the detector and its setup // and add it in the spectrometer HWallDetector* wall = new HWallDetector; Int_t mods[] = {1}; wall->setModules(-1,mods); spec->addDetector(wall); // define the input HParOra2Io* ora = new HParOra2Io;

  • ra->open();
  • ra->setHistoryDate("now");

rtdb->setFirstInput(ora); // define the output HParRootFileIo* output = new HParRootFileIo;

  • utput->open("wallParams.root","RECREATE");

rtdb->setOutput(output); // create the parameter containers rtdb->getContainer("WallCalPar"); rtdb->getContainer("WallRefWinPar"); // initialize the parameter containers rtdb->initContainers(1620123104); // save the output // (writes the data to the file) rtdb->saveOutput(); // print content of the runtime // database on the screen rtdb->print(); // delete the Hades object delete myHades; }

slide-9
SLIDE 9

Parameter handling - HADES Analysis Workshop at GSI, May 10-11 2017 9

Layout of parameter containers Layout of parameter containers

Basically two groups of parameter containers:

1. “Tree-style”: array of objects each containing the same list of parameters

Data are stored in one or more individual tables in Oracle. Each parameter corresponds to a table column. Many records together form a version set of the parameter container. Example: Lookup table for an unpacker

(maps readout channels to detector cells) Constraints guarantee for each parameter container version that

  • ne readout channel maps to only one detector cell and both are unique.

Advantage: enforcement of data consistency via constraints and triggers Disadvantage: more dedicated C++ code needed, lack of flexibility Adding or discarding parameters needs changes in the tables and interfaces

have to be done by an expert Suitable only for stable code Used for low-level analysis parameters

slide-10
SLIDE 10

Parameter handling - HADES Analysis Workshop at GSI, May 10-11 2017 10

Generic parameter container HParCond Generic parameter container HParCond

All parameters are stored as name – object pairs in the same set of tables

parameter container parameter

  • bject

allows to add or discard (invalidate) a parameter makes it (almost) code independent

different versions

name type: Int_t, Float_t, Double_t, Char_t, Text_t, UChar_t, class type data stored as byte array (RAW or BLOB) number of values (single value or array) class version streamer info, root version for ROOT classes

  • wn version management

any class derived from TObject decoded in the analysis interface by ROOT streamer

slide-11
SLIDE 11

Parameter handling - HADES Analysis Workshop at GSI, May 10-11 2017 11

Implementation and interfaces Implementation and interfaces for generic parameter containers for generic parameter containers

Generic read and write interfaces for Oracle, ROOT and ASCII files

implemented in base classes

The Oracle interface stores a snapshot of parameters in a ROOT TList. The parameter container copies only parameters defined in the actual code version. The user stores the parameter in Oracle by a ROOT macro. At validation, new parameters are added automatically. Old parameters no longer needed stay valid until explicitly set invalid.

Requires only a few lines of dedicated C++ code in the container class implementation of virtual void putParams(HParamList*)=0;

virtual Bool_t getParams(HParamList*)=0; virtual void printParams(); (only needed for special layout)

No database expert needed! Data consistency must be checked by the user.

Web-based graphical interface to Oracle for validation, searches

and comparisons Suitable also for non-stable code Used for conditions and high-level analysis parameters

slide-12
SLIDE 12

Parameter handling - HADES Analysis Workshop at GSI, May 10-11 2017 12

Example: HRpcHitFPar (hrpchitfpar.h) Example: HRpcHitFPar (hrpchitfpar.h)

slide-13
SLIDE 13

Parameter handling - HADES Analysis Workshop at GSI, May 10-11 2017 13

hrpchitfpar.cc hrpchitfpar.cc (format changed slightly, comments added)

(format changed slightly, comments added)

slide-14
SLIDE 14

Parameter handling - HADES Analysis Workshop at GSI, May 10-11 2017 14

Write to Oracle (demo macro writeParToOra.C) Write to Oracle (demo macro writeParToOra.C)

Stored in intermediate „LOAD“ table WebDB GUI for validation

slide-15
SLIDE 15

Parameter handling - HADES Analysis Workshop at GSI, May 10-11 2017 15

Version management Version management

ASCII file: no version management ROOT file: local version management

may store different versions of the same parameter container (the variable fCycle in TKey ) version table stores the version number for each run and each parameter container stores the detector setup

parameter file for local access snapshots of versions in Oracle If the run id is not found in the ROOT file, the parameter containers cannot be initialized for

this run. But you may use an other run in the file as a reference run.

Oracle: complete history of all parameters since 2002 Storage of parameters: 1. Insert of parameters with a ROOT macro

  • 2. Validation with web GUI

!

slide-16
SLIDE 16

Parameter handling - HADES Analysis Workshop at GSI, May 10-11 2017 16

Version management in Oracle Version management in Oracle

  • 1. Parameters are not stable, but may change from run to run
  • 2. They may change for the same run over time (history)
  • 3. They may come in different flavors (depending on run type or an algorithm)

3-dimensional version management

history runs valid_since valid_until (default year 4000) date_create invalid_since

(year 4000 for actual version)

version 2 different flavors (contexts) For example: beam, simulation

defined by user automatically defined

slide-17
SLIDE 17

Parameter handling - HADES Analysis Workshop at GSI, May 10-11 2017 17

Version 1

Parameter versions Parameter versions

runs history

year 4000 year 4000 valid_since v1

During a beam time the user inserts a version 1. Each new run is initialized with version 1.

date_create v1 real time valid_until v1 valid_since v2 invalid_since v1 date_create v2

Later he adds a version 2. New runs are then initialized with version 2. run start The crossing point of run start and history date defines the parameter version version 1 version 1 version 2 history date

slide-18
SLIDE 18

Parameter handling - HADES Analysis Workshop at GSI, May 10-11 2017 18

Version 1

Parameter versions Parameter versions

runs history

year 4000 year 4000 valid_since v1

During a beam time the user inserts a version 1. Each new run is initialized with version 1.

date_create v1 real time valid_until v1 valid_since v2 invalid_since v1 date_create v2

Later he adds a version 2. New runs are then initialized with version 2. run start The crossing point of run start and history date defines the parameter version version 1 version 1 version 2 history date Later new versions 3 and 4 replace the old ones Before a DST production we do a beam time specific parameter release “history date with name” Old versions can be retrieved with an old history date version 4 version 3 parameter release

slide-19
SLIDE 19

Parameter handling - HADES Analysis Workshop at GSI, May 10-11 2017 19

allows to generate a parameter file from Oracle for a DST production Recipe (old-fashioned analysis macro):

1. add the following lines after definition of the Oracle input:

// generate a parameter file for beam time AUG14 in given time range if (!rtdb->makeParamFile("paramsAug14.root", "AUG14", "20-AUG-2014 00:00:00", "06-SEP-2014 10:00:00")) { delete gHades; return; }

Do not specify an output in the runtime database!

2. add at the end of the macro directly before the deletion of gHades: rtdb->saveOutput(); 3. Run one event

Parameter file generator Parameter file generator

!

Output: parameter ROOT file log file

List of runs, parameter containers, versions for each run List of runs not fully initialized (search for „Error“)

slide-20
SLIDE 20

Parameter handling - HADES Analysis Workshop at GSI, May 10-11 2017 20

Demo macro make_parameter_file.C Demo macro make_parameter_file.C

slide-21
SLIDE 21

Parameter handling - HADES Analysis Workshop at GSI, May 10-11 2017 21

Running the macro (status May 2016) Running the macro (status May 2016)

slide-22
SLIDE 22

Parameter handling - HADES Analysis Workshop at GSI, May 10-11 2017 22

Parameter Root file and log file Parameter Root file and log file

slide-23
SLIDE 23

Parameter handling - HADES Analysis Workshop at GSI, May 10-11 2017 23

Analysis on the batch farm Analysis on the batch farm Use parameter file created from Oracle!

Advantage: No load on the Oracle server Faster (no waiting for active Oracle session) No missing parameters Same history date for historyDate = „now“