PdmV activities and organization, Advancement since last PPD - - PowerPoint PPT Presentation

▶
pdmv activities and organization advancement since last
SMART_READER_LITE
LIVE PREVIEW

PdmV activities and organization, Advancement since last PPD - - PowerPoint PPT Presentation

Aug 19, 2023 323 likes •501 views

PdmV activities and organization, Advancement since last PPD workshop. G. Boudoul, G. Franzoni, J-R Vlimant P hysics D ata M ontecarlo V alidation https://twiki.cern.ch/twiki/bin/view/CMS/PdmV What ? Software, Conditions, HLT menu,

slide-1
SLIDE 1

PdmV activities and organization, Advancement since last PPD workshop.

  • G. Boudoul, G. Franzoni, J-R Vlimant
slide-2
SLIDE 2

7/10/13 PPD Coordination, Physics Week, J-R Vlimant 2

Physics Data Montecarlo Validation

  • What ?

✔ Software, Conditions, HLT menu, externals ✔ Analysis recipe is next ?

  • Who ?

✔ Good coverage from the DPGs ➔ On the other hand they know what they have changed ➔ Need validation coverage on the HLT aspect ✔ Good coverage from the POGs ➔ On the other hand they know what they have changed ✗ Fair coverage from the PAGs ➔ On the other hand they need to know what has changed

  • How ?

✔ Pre-defined set of MC and data samples ✔ Targeted dedicated samples on case by case basis ✔ Use of the DQM infrastructure as much as possible ✔ Getting more people involved

Volunteer to your PAG / POG / DPG conveners or contacts https://twiki.cern.ch/twiki/bin/view/CMS/PdmVGroupDescription#Contacts

  • Why ?

➔ Protect everyone from everyone else on possible mistakes ➔ Prevent complicated analysis procedures due to mistakes ➔ Get everyone to know what is coming up

https://twiki.cern.ch/twiki/bin/view/CMS/PdmV

slide-3
SLIDE 3

7/10/13 PPD Coordination, Physics Week, J-R Vlimant 3

We Need You

  • Please join the effort of physics validation from the PAG perspective

➔ Will get you to know how things work.

  • Please continue and renew the effort of physics validation from the DPG & POG

perspective

➔ Mandatory in the scope of upgrade development and preparation.

  • Please implement software in DQM that scales in time, maintainability and manpower

➔ Will reduce the load on the validation task. ➔ We need help on the validation task so as to re-route more attention to other important

community tasks (object ID, analysis recipes, DATA/MC validation, ...)

Many Thanks to all current validators

https://twiki.cern.ch/twiki/bin/view/CMS/PdmVGroupDescription#Contacts

for their precious and continuous effort to maintain data quality ! Many Thanks to the L3 who joined PdmV !

slide-4
SLIDE 4

7/10/13 PPD Coordination, Physics Week, J-R Vlimant 4

Wish list in Nov/Dec 2012

https://indico.cern.ch/conferenceDisplay.py?confId=215512

  • Upgrade software validation with standard tools
  • Meeting dedicated to validation
  • Validation in analysis phase-space
  • Validation documentation
  • Data.vs.MC validation
  • Improve communication on HLT validation
  • Tie trigger development with trigger validation tools
  • Extend samples for HLT validation
  • Develop HLT validation on data samples
  • Involve DPG in HLT validation
  • Collect special procedure to be applied in analysis
  • Collect POG-blessed object identification in PdmV twikis
  • Transform object id recipe into python/c++ centralized tools
  • Organize validation for event interpretation (EI)
  • Re-new request management tools

Pretty large wish list and goals

slide-5
SLIDE 5

7/10/13 PPD Coordination, Physics Week, J-R Vlimant 5

Status at first Glance 1/2

  • Upgrade software validation with standard tools

✔ 6.1.SLHC cycle fully validated http://cern.ch/go/9flJ

  • Meeting dedicated to validation

✔ https://twiki.cern.ch/twiki/bin/view/CMS/PdmVDQMJointGroupMeeting

  • Validation in analysis phase-space

✗ Strengthen PAG validation on-going

  • Validation documentation

✗ Stalled due to over-load, but still planned

  • Data.vs.MC validation

✔ On-going : https://twiki.cern.ch/twiki/bin/view/CMS/PdmVDataMCvalidation

  • Improve communication on HLT validation

✔ Re-newed e-group, documentation and improved interaction

  • Tie trigger development with trigger validation tools

✗ Much like “releases notes”, not a popular topic

slide-6
SLIDE 6

7/10/13 PPD Coordination, Physics Week, J-R Vlimant 6

Status at first Glance 2/2

  • Extend/modify samples for HLT validation

✔ New requests handled. Workflow modifications to be discussed

  • Develop HLT validation on data samples

✗ Lost interest with the end of data-taking. To be continued

  • Involve DPG in HLT validation

➔ Call were made. Collecting applications with the Steam group

  • Collect special procedure to be applied in analysis

✔ https://twiki.cern.ch/twiki/bin/view/CMS/PdmVKnowFeatures

  • Collect POG-blessed object identification in PdmV twikis

✗ Stalled due to over-load, but still planned

  • Transform object id recipe into python/c++ centralized tools

 Longer term

  • Organize validation for event interpretation (EI)

✔ On-going : https://twiki.cern.ch/twiki/bin/view/CMS/PdmVEIValidation

  • Re-new request management tools

✔ Imminent deployment : https://twiki.cern.ch/twiki/bin/view/CMS/PdmVMcM

slide-7
SLIDE 7

7/10/13 PPD Coordination, Physics Week, J-R Vlimant 7

DPG/POG Validation L3

Many thanks to Nancy Marinelli (University of Notre Dame, US) for agreeing on taking the position

➔Charge

  • Will assist validators in their task
  • Should facilitate communication between groups
  • Will pursue de DQM-ification of the validation procedures

Thanks for the work already done !

https://twiki.cern.ch/twiki/bin/view/CMS/PdmVGroupDescription#POG_and_DPG

Upgrade Validation

  • 6 releases of the 6.1.x.SLHCy cycle were validated

➔ Several rounds of validation samples produced ➔ Validation funneled in valDB http://cern.ch/go/9flJ ✔ Allowed to spot issues before production ➔ Very good and continued integration of validation with the validators

  • Continuing effort with the participation of the validation L3s
slide-8
SLIDE 8

7/10/13 PPD Coordination, Physics Week, J-R Vlimant 8

PdmV/DQM Joint Meeting

After discussion with the DQM Group we have agreed to transform 1 out of 4 DQM meeting on Tuesday 14:00 to 16:00 in a joint PdmV/DQM meeting April 23, May 21, June 18 Next meeting on July 16th https://indico.cern.ch/conferenceDisplay.py?confId=226661 https://twiki.cern.ch/twiki/bin/view/CMS/PdmVDQMJointGroupMeeting

➔Charge:

  • Review existing validation procedures

We'll be making calls for reviews. Please volunteer

  • Bridge between validators

Bring together validators from correlated area of validation

  • Scout for new ideas on samples and variables

Create a knowledge database for developers to pick from

  • Development of validation software for data, MC and data/MC validation

Consolidate the validation software integration communication

Very useful and good attendance so far. “One meeting per month” schedule allows for preparation.

slide-9
SLIDE 9

7/10/13 PPD Coordination, Physics Week, J-R Vlimant 9

Items of Discussion

➢ Review : we will invite and accommodate for volunteers to come and present the current

status of the validation strategy in their group or area of expertise. This will serve as a knowledge database for groups still in the process of implementing or looking for solutions for validation.

➢ Bridge and hand-shaking : harmonize the scrutiny and understanding of modifications

to RECO software or Alca which affect performance or physics variables across multiple DPG/POG/PAG/HLT groups. Promote exchange of information and complementarity among different validators. Encourage the software developers within DPG/POG to notify changes and interact with DQM developers and validators to assure that validation can follow the evolution closely.

➢ Talent scouting : we invite analyzer and collaborators to come and present control

variables and samples which are worth monitoring and having available systematically in release/conditions/trigger validation. Strategies to develop data-VS-Monte Carlo validation and customize to the needs of each group will be discussed. The presentation would serve to fill a pool of ideas for developer to implement later on.

➢ Validation Development : we invite developers of dqm packages for validation (as

  • pposite to developments for certification ) being for Monte-Carlo comparison or purely

based on reconstructed data to come and present their software development and how this cover the task of validation.

slide-10
SLIDE 10

7/10/13 PPD Coordination, Physics Week, J-R Vlimant 10

PAG Validation L3

Many thanks to Cristina Ferro (National Central University, TW) Francesco Fabozzi (Universita della Basilicata e INFN, IT) for agreeing on taking the position

➔Charge

  • Will assist validators in their task
  • Should facilitate communication between groups
  • Will scout for new validators
  • Should collect ideas for validation
  • Cristina could specialize on Data/MC validation
  • Francesco could specialize on event-interpretation validation

Thanks for the work already done !

https://twiki.cern.ch/twiki/bin/view/CMS/PdmVGroupDescription#PAG

✔ Several areas already been tackled ✔ The scouting functioned as we're getting more HI contribution

slide-11
SLIDE 11

7/10/13 PPD Coordination, Physics Week, J-R Vlimant 11

Data.vs.MC Validation

https://twiki.cern.ch/twiki/bin/view/CMS/PdmVDataMCvalidation

  • The DQM team has provided the required infrastructure and wf to

perform:

✔ Weighted harvesting : done ➔ Provide histograms, efficiencies, already re-weighted ✔ Harvesting on multiple samples : done ➔ Provide stacked histograms & efficiencies, already re-weighted

  • Request manager changes

✔ For multiple harvesting : done ➔ Allows a request to produce more than one /DQM harvested in its

  • wn way

✗ Harvesting with multiple /DQM in input : to be implemented  Need to provide a concrete example

  • Application to physics validation

 Pilot example foreseen for next week  Call made to groups in prevision of the GED/PPD workshop

https://indico.cern.ch/conferenceDisplay.py?confId=251030

 To be continued ...

slide-12
SLIDE 12

7/10/13 PPD Coordination, Physics Week, J-R Vlimant 12

HLT Validation

https://twiki.cern.ch/twiki/bin/view/CMS/PdmVTriggerValidation

  • Fully integrated to validation procedures

✔ Good coverage from POGs. ✗ Limited coverage from PAGs. ➔ Call to validators from DPG still on-going

  • Improved documentation from validators

➔ PdmVTriggerValidation<group> all under PdmV* twiki tree

  • Suggested modifications to workflows to be discussed

➔ Event-to-event comparison tools ➔ Factor out noise in digitization ➔ ...

slide-13
SLIDE 13

7/10/13 PPD Coordination, Physics Week, J-R Vlimant 13

Object Identification and Recipes

  • Recipes for know features:

https://twiki.cern.ch/twiki/bin/view/CMS/PdmVKnowFeatures

➢ Documentation as features and recipe arise

  • Object Identification

https://twiki.cern.ch/twiki/bin/view/CMS/PdmVObjectIdentification

✗ Several tentative starting points in the past, failed out of steam ✗ Probably lack of understanding of what is the mandate ➔ Could resume the effort with added man-power ➔ Brain-storming on the needed tools for such organisation

slide-14
SLIDE 14

7/10/13 PPD Coordination, Physics Week, J-R Vlimant 14

Event Interpretation (EI) Validation

  • Jargon for those who know about it

➔ EI = PFBRECO

  • TOP flavored EI integrated to 6.1 by the GED group

 Part of thje standard reconstruction sequence

  • Factorized out of reconstruction standard sequence in 6.2

✔ Simple commands ✔ In standard processing cmsDriver.py ... -s RAW2DIGI,RECO,EI:top ✔ Can be run stand-alone from /AOD : cmsDriver.py ... -s EI:top ➔ Always included in RECO&AOD if ran ➔ One relval workflow can serve the validation of more than one EI  DIGI⇨RECO ⇨

EI1 ⇨ EI2 ...

➔ Allows for more than one EI to be implemented & validated ✔ Implementation to be supervised by GED/RECO/AT ➔ Sorting out how to transparently validate EI and non-EI collections  Call made to groups in prevision of the GED/PPD workshop

https://indico.cern.ch/conferenceDisplay.py?confId=251030 https://twiki.cern.ch/twiki/bin/view/CMS/PdmVEIValidation

slide-15
SLIDE 15

7/10/13 PPD Coordination, Physics Week, J-R Vlimant 15

McM (p.k.a PREP2)

  • Development instance extensively alpha tested

https://cms-pdmv-dev.cern.ch/mcm/

  • Production instance ready

https://cms-pdmv.cern.ch/mcm/

  • Instructions available and

https://twiki.cern.ch/twiki/bin/view/CMS/PdmVMcM#Instructions

  • Beta testers for production

➔ TSG https://cms-pdmv.cern.ch/mcm/requests?page=-1&pwg=TSG ✔ Went through all requests types ✗ Only performance issues faced, inherent to HighPU@13TeV@5.3

  • Several groups have decided to utilize McM only

➔ TSG, TOP, B2G, BPH ✔ Migration done or on-going

  • Support for the infamous “step0”

✔ Will remove the book-keeping hole we have dug ourselves in

  • Full integration of all experience gained in 2012/3 operation

✔ Should allow even better traceability of requests through the maze

  • Technical support available

✔ Technical student from July 2013 to July(++) 2014 ✔ Antanas as PdmV/DQM CatA

  • Full deployment is imminent

➔ This needs to happened in tight connection with all parties : gen contact, prep

support, gen conveners, prod managers, computing operation to avoid disruptions

slide-16
SLIDE 16

7/10/13 PPD Coordination, Physics Week, J-R Vlimant 16

Generator Validation

https://twiki.cern.ch/twiki/bin/view/CMS/PdmVTasks#Generators

Ian Nugent for the PdmV/Generator joint effort of generator validation.

  • Please contribute to validation software

https://hypernews.cern.ch/HyperNews/CMS/get/generators/1796.html

  • Please provide them with feedback on generator validation
  • The scope is two-fold

➔ Release validation : need handles (variables and samples) to perform the

validation of new features of generators

 Many different aspects need to be discussed on how to address this ➔ Monte-Carlo production : need handles (distributions in DQM) to perform

the sanity check as part of a request approval in McM :

✔ Procedure available ✔ To be commissioned once migrated to McM

slide-17
SLIDE 17

7/10/13 PPD Coordination, Physics Week, J-R Vlimant 17

Summary

Many thanks to Nancy, Cristina and Francesco for joining as validation L3 Many thanks to all validators for their renewed and valuable contributions Items announced at the WS in Nov 2013 addressed in order of priority and based on availability New features for validation will increase the coverage of scrutiny Manpower is needed within DPGs, POGs and PAGs Potential calls for new L3s to scout and coordinate un-covered activities