McM Monte-Carlo Management Service Jean-Roch Vlimant for PdmV ( * ) & Generator Groups CMSDAS @ FNAL, January 2014 https://twiki.cern.ch/twiki/bin/view/CMS/SWGuideCMSDataAnalysisSchool2014GeneratorExerciseatFNAL https://twiki.cern.ch/twiki/bin/view/CMS/PdmVMcM https://cms-pdmv.cern.ch/mcm/ (production) https://cms-pdmv-dev.cern.ch/mcm/ (test instance) ( * ) PdmV : Physics Data-MC Validation
Global Picture Generator Contacts Production Manager Computing Operation Operate McM Prepare requests Guide preparation of Receive well formatted Report special needs requests requests Keep an eye on production Dispatch production to sites McM Operate at various steps Follow through with sites Provide validation issues Provide book-keeping Manage central space Guide production 2 McM @ CMSDAS, Jean-Roch Vlimant 1/5/14
The Path To An Analysis Dataset ● Figure out the relevant generator parameters ● Possible modifications in simulation itself ➢ for better data/MC agreement ● Figure out the requirements in terms of digitization and reconstruction to match the analyzed data ● All steps can be done in the same workflow, however ➢ Computing requirements Generation and simulation mostly done at T2s Digitization and reconstruction mostly done at T1s ➢ Chronology of production Generation and simulation may start before digi-reco Several digi-reco might be required for a given sample of generated events ➔ Steps are split in several processing workflows ➔ Each group of steps is organized in a campaign 3 McM @ CMSDAS, Jean-Roch Vlimant 1/5/14
The Concept of Chained Campaigns 4 McM @ CMSDAS, Jean-Roch Vlimant 1/5/14
Terminology ● Campaign Campaign is a group of requests that share the same of very similar configuration, energy, software release series, physics performance … ✔ Example : Summer12 is for all gen-sim at 8TeV using the 5.3 release serie ● Request Request is a specific member of a campaign for the production of a given dataset, being intermediate or for analysis (AOD) ✔ Example : TOP-Summer12-00177 is the request for production gen-sim dataset of top pair production and decay in leptons, with top mass hypothesis of 173.5 GeV ● Flow Flow is a connector between campaigns, it defines specific operation that needs to be done while using the output of one request as input to its next step ✔ Example : flowS12to53RD defines the necessary ingredients to have a “run dependent” digi-reco of Summer12 gen-sim samples within Summer12DR53X campaign ● Chained Campaign Chained Campaign is constructed from campaigns that are connected together with flows ✔ Example : chain_Summer12pLHE_flowLHE2S12_flowS12to53RD (aliased to pLHE12DR53RD) is the chain of steps required to produce a run-dependent analysis dataset from a privately produced set of .lhe files ● Chained Requests Chained Requests is a specific member of a chained campaign, made from existing requests. ✔ Example : HIG-chain_Summer12pLHE_flowLHE2S12_flowS12to53RD-00001 is the chain of request required to produced the analysis dataset for a Higgs+top production with decay in a pair of photons 5 McM @ CMSDAS, Jean-Roch Vlimant 1/5/14
Terminology ● Campaign Campaign ● Request Request ● Flow Flow ● Chained Campaign Chained Campaign ● Chained Requests Chained Requests 6 McM @ CMSDAS, Jean-Roch Vlimant 1/5/14
Why All This ● Flows and chained campaigns are overkills if there is only one way of producing samples ✔ It used to be the case ✗ It is far from being true anymore (RD digi-reco, several pile-up, …) ● In the context of knowing what should happen for the production of a given dataset, flows, chained campaigns and chained requests allow ✔ The production manager to know what is next the next step to be done ✔ The generator contact to keep an eye on the evolution of the production through the different steps ✔ The user to have at a glance the history for the production of an analysis sample ● Not only McM is a production management tool, it also allows for book-keeping and documentation ✔ Generator parameters can be updated at anytime ✔ Notes should contain details of the content of the sample if there are some specificity ✔ The configuration files used for production is accessible ✔ The “cmsDriver” command is accessible ● McM offers extra protection to computing operation and the user themselves ✔ Runtest of the request ensures run-ability ✔ Validation step allows to scrutinize the generator content of a request 7 McM @ CMSDAS, Jean-Roch Vlimant 1/5/14
Evolution of a Chain of Requests 8 McM @ CMSDAS, Jean-Roch Vlimant 1/5/14
Practical Exercise : Request 1/2 ● Required ingredients dsn : https://twiki.cern.ch/twiki/bin/view/CMS/ProductionDataSetNames ➔ Necessary to label the output datasets /<dsn>/<some string>/<data tier> Time/event ➔ Mandatory to be accurate so as to not exceed job time-out in production Size/event ➔ Required to be accurate to allocate enough space for the output dataset Efficiencies ➔ Mandatory to be accurate so that the exact number of events get produced Extension number ➔ Mandatory to guide the user in what need to be combined, and prevent dataset over-writting in production Process string ➔ Mandatory when the request is slightly changed from the campaign by the user, and need to be properly label /<dsn>/<some string containing the request process string>/<data tier> ● Insert a request in McM ➔ Create a new request and insert the gen-fragment from previous exercise ➔ Select to be provided with validation plots ➔ Then run the local test : to verify that everything is in order ● Verify requests values ➔ Edit with correct values : to have all ingredients accurately inserted ● Toggle “validation” (and move on to next step) : triggers the runtest to be performed under McM 9 McM @ CMSDAS, Jean-Roch Vlimant 1/5/14
Practical Exercise : Request 2/2 ● Prepare a request for processing the request towards an analysis dataset ➔ Insert an MccM document for the request : simplifies the report at MC coordination meeting with exact ingredients ➔ Do not select the block # until really sure of the rest : to prevent the chains to be generated automatically as part of the exercise ● When validation is finished (watch for notification and change of status) ➔ Inspect validation (requires cmsweb authentication) : verify that the generator fragment is creating what is expected ➔ Toggle “define” : status defined means that it's read and validated ● Watch the requests getting automatically processed as part of the exercise ✔ Approved : the generator conveners has looked at the request and things seem to be in order ✔ Chained : the production manager has acted on the MccM document to create the required chains, and reserved the necessary requests ✔ Submitted : the production manager manually submitted the request or McM submitted automatically once chained and approved ✔ Done : after processing by comp-ops, the output dataset is set to VALID and the request is “done”. At warp speed for the purpose of the exercise, there is no real processing 10 McM @ CMSDAS, Jean-Roch Vlimant 1/5/14
Practical Exercise : Find Information ● Use the production instance https://cms-pdmv.cern.ch/mcm/ ✔ Find the status of certain requests ➔ /TTJets_MSDecays_matchingdown_TuneZ2star_8TeV-madgraph- tauola/Summer12_DR53X-PU_S10_START53_V19-v2/AODSIM ➔ /TTJets_MSDecays_mass178_5_TuneZ2star_8TeV-madgraph- tauola/Summer12_DR53X-PU_S10_START53_V19-v1/AODSIM ✔ Find the Xsec of a given dataset ➔ /TTJets_SemiLeptDecays_8TeV-sherpa/Summer12_DR53X- PU_S10_START53_V19-v1/AODSIM ✔ Find the generator fragment used to produce a given dataset ➔ /TprimeTprimeToTHBW_HToGammaGamma_M-900_TuneZ2star_8TeV- madgraph/Summer12_DR53X-PU_S10_START53_V19-v1/AODSIM ✔ Find the filter efficiency of the physics process in a give dataset ➔ /Xibstar0ToXibPi_8TeV-pythia6-evtgen/Summer12_DR53X- PU_S10_START53_V19-v1/AODSIM ✔ Find the “cmsDriver” command used to produce a give dataset ➔ /TTToHcWb_HToGG_8TeV-madgraph-pythia6/Summer12_DR53X- PU_RD1_START53_V7N-v2/AODSIM ● Find the same info through DAS ➔ All but config file and cmsDriver can easily be obtained by having DAS show the information stored in McM 11 McM @ CMSDAS, Jean-Roch Vlimant 1/5/14
Documentation Monte Carlo Coordination Meeting https://twiki.cern.ch/twiki/bin/view/CMS/PdmVMccM Main Twiki https://twiki.cern.ch/twiki/bin/view/CMS/PdmVMcM Presentation and Tutorials https://twiki.cern.ch/twiki/bin/view/CMS/PdmVMcMTutorials 12 McM @ CMSDAS, Jean-Roch Vlimant 1/5/14
Summary of Exercise ● McM allows for submission of request to central production under central prioritization ● McM provides means of validation of the request content ● McM keeps track of the various required step of production ● McM provides book-keeping and documentation of the content of the analysis datasets ● McM information is aggregated and accessible through DAS 13 McM @ CMSDAS, Jean-Roch Vlimant 1/5/14
Recommend
More recommend
