PdmV activities and organization, Advancement since last PPD - PowerPoint PPT Presentation

PdmV activities and organization, Advancement since last PPD workshop. G. Boudoul, G. Franzoni, J-R Vlimant

P hysics D ata M ontecarlo V alidation https://twiki.cern.ch/twiki/bin/view/CMS/PdmV ● What ? ✔ Software, Conditions, HLT menu, externals ✔ Analysis recipe is next ? ● Who ? ✔ Good coverage from the DPGs ➔ On the other hand they know what they have changed ➔ Need validation coverage on the HLT aspect ✔ Good coverage from the POGs ➔ On the other hand they know what they have changed ✗ Fair coverage from the PAGs ➔ On the other hand they need to know what has changed ● How ? ✔ Pre-defined set of MC and data samples ✔ Targeted dedicated samples on case by case basis ✔ Use of the DQM infrastructure as much as possible ✔ Getting more people involved Volunteer to your PAG / POG / DPG conveners or contacts  https://twiki.cern.ch/twiki/bin/view/CMS/PdmVGroupDescription#Contacts ● Why ? ➔ Protect everyone from everyone else on possible mistakes ➔ Prevent complicated analysis procedures due to mistakes ➔ Get everyone to know what is coming up 2 PPD Coordination, Physics Week, J-R Vlimant 7/10/13

We Need You ● Please join the effort of physics validation from the PAG perspective ➔ Will get you to know how things work. ● Please continue and renew the effort of physics validation from the DPG & POG perspective ➔ Mandatory in the scope of upgrade development and preparation. ● Please implement software in DQM that scales in time, maintainability and manpower ➔ Will reduce the load on the validation task. ➔ We need help on the validation task so as to re-route more attention to other important community tasks (object ID, analysis recipes, DATA/MC validation, ...) Many Thanks to all current validators https://twiki.cern.ch/twiki/bin/view/CMS/PdmVGroupDescription#Contacts for their precious and continuous effort to maintain data quality ! Many Thanks to the L3 who joined PdmV ! 3 PPD Coordination, Physics Week, J-R Vlimant 7/10/13

Wish list in Nov/Dec 2012 https://indico.cern.ch/conferenceDisplay.py?confId=215512 ● Upgrade software validation with standard tools ● Meeting dedicated to validation ● Validation in analysis phase-space ● Validation documentation ● Data.vs.MC validation ● Improve communication on HLT validation ● Tie trigger development with trigger validation tools ● Extend samples for HLT validation ● Develop HLT validation on data samples ● Involve DPG in HLT validation ● Collect special procedure to be applied in analysis ● Collect POG-blessed object identification in PdmV twikis ● Transform object id recipe into python/c++ centralized tools ● Organize validation for event interpretation (EI) ● Re-new request management tools Pretty large wish list and goals 4 PPD Coordination, Physics Week, J-R Vlimant 7/10/13

Status at first Glance 1/2 ● Upgrade software validation with standard tools ✔ 6.1.SLHC cycle fully validated http://cern.ch/go/9flJ ● Meeting dedicated to validation ✔ https://twiki.cern.ch/twiki/bin/view/CMS/PdmVDQMJointGroupMeeting ● Validation in analysis phase-space ✗ Strengthen PAG validation on-going ● Validation documentation ✗ Stalled due to over-load, but still planned ● Data.vs.MC validation ✔ On-going : https://twiki.cern.ch/twiki/bin/view/CMS/PdmVDataMCvalidation ● Improve communication on HLT validation ✔ Re-newed e-group, documentation and improved interaction ● Tie trigger development with trigger validation tools ✗ Much like “releases notes”, not a popular topic 5 PPD Coordination, Physics Week, J-R Vlimant 7/10/13

Status at first Glance 2/2 ● Extend/modify samples for HLT validation ✔ New requests handled. Workflow modifications to be discussed ● Develop HLT validation on data samples ✗ Lost interest with the end of data-taking. To be continued ● Involve DPG in HLT validation ➔ Call were made. Collecting applications with the Steam group ● Collect special procedure to be applied in analysis ✔ https://twiki.cern.ch/twiki/bin/view/CMS/PdmVKnowFeatures ● Collect POG-blessed object identification in PdmV twikis ✗ Stalled due to over-load, but still planned ● Transform object id recipe into python/c++ centralized tools  Longer term ● Organize validation for event interpretation (EI) ✔ On-going : https://twiki.cern.ch/twiki/bin/view/CMS/PdmVEIValidation ● Re-new request management tools ✔ Imminent deployment : https://twiki.cern.ch/twiki/bin/view/CMS/PdmVMcM 6 PPD Coordination, Physics Week, J-R Vlimant 7/10/13

DPG/POG Validation L3 https://twiki.cern.ch/twiki/bin/view/CMS/PdmVGroupDescription#POG_and_DPG Many thanks to Nancy Marinelli (University of Notre Dame, US) for agreeing on taking the position ➔ Charge ● Will assist validators in their task ● Should facilitate communication between groups ● Will pursue de DQM-ification of the validation procedures Thanks for the work already done ! Upgrade Validation ● 6 releases of the 6.1. x .SLHC y cycle were validated ➔ Several rounds of validation samples produced ➔ Validation funneled in valDB http://cern.ch/go/9flJ ✔ Allowed to spot issues before production ➔ Very good and continued integration of validation with the validators ● Continuing effort with the participation of the validation L3s 7 PPD Coordination, Physics Week, J-R Vlimant 7/10/13

PdmV/DQM Joint Meeting https://twiki.cern.ch/twiki/bin/view/CMS/PdmVDQMJointGroupMeeting After discussion with the DQM Group we have agreed to transform 1 out of 4 DQM meeting on Tuesday 14:00 to 16:00 in a joint PdmV/DQM meeting April 23, May 21, June 18 Next meeting on July 16 th https://indico.cern.ch/conferenceDisplay.py?confId=226661 ➔ Charge : ● Review existing validation procedures We'll be making calls for reviews. Please volunteer ● Bridge between validators Bring together validators from correlated area of validation ● Scout for new ideas on samples and variables Create a knowledge database for developers to pick from ● Development of validation software for data, MC and data/MC validation Consolidate the validation software integration communication Very useful and good attendance so far. “One meeting per month” schedule allows for preparation. 8 PPD Coordination, Physics Week, J-R Vlimant 7/10/13

Items of Discussion ➢ Review : we will invite and accommodate for volunteers to come and present the current status of the validation strategy in their group or area of expertise. This will serve as a knowledge database for groups still in the process of implementing or looking for solutions for validation. ➢ Bridge and hand-shaking : harmonize the scrutiny and understanding of modifications to RECO software or Alca which affect performance or physics variables across multiple DPG/POG/PAG/HLT groups. Promote exchange of information and complementarity among different validators. Encourage the software developers within DPG/POG to notify changes and interact with DQM developers and validators to assure that validation can follow the evolution closely. ➢ Talent scouting : we invite analyzer and collaborators to come and present control variables and samples which are worth monitoring and having available systematically in release/conditions/trigger validation. Strategies to develop data-VS-Monte Carlo validation and customize to the needs of each group will be discussed. The presentation would serve to fill a pool of ideas for developer to implement later on. ➢ Validation Development : we invite developers of dqm packages for validation (as opposite to developments for certification ) being for Monte-Carlo comparison or purely based on reconstructed data to come and present their software development and how this cover the task of validation. 9 PPD Coordination, Physics Week, J-R Vlimant 7/10/13

PAG Validation L3 https://twiki.cern.ch/twiki/bin/view/CMS/PdmVGroupDescription#PAG Many thanks to Cristina Ferro (National Central University, TW) Francesco Fabozzi (Universita della Basilicata e INFN, IT) for agreeing on taking the position ➔ Charge ● Will assist validators in their task ● Should facilitate communication between groups ● Will scout for new validators ● Should collect ideas for validation ● Cristina could specialize on Data/MC validation ● Francesco could specialize on event-interpretation validation Thanks for the work already done ! ✔ Several areas already been tackled ✔ The scouting functioned as we're getting more HI contribution 10 PPD Coordination, Physics Week, J-R Vlimant 7/10/13

Data.vs.MC Validation https://twiki.cern.ch/twiki/bin/view/CMS/PdmVDataMCvalidation ● The DQM team has provided the required infrastructure and wf to perform: ✔ Weighted harvesting : done ➔ Provide histograms, efficiencies, already re-weighted ✔ Harvesting on multiple samples : done ➔ Provide stacked histograms & efficiencies, already re-weighted ● Request manager changes ✔ For multiple harvesting : done ➔ Allows a request to produce more than one /DQM harvested in its own way ✗ Harvesting with multiple /DQM in input : to be implemented  Need to provide a concrete example ● Application to physics validation  Pilot example foreseen for next week  Call made to groups in prevision of the GED/PPD workshop https://indico.cern.ch/conferenceDisplay.py?confId=251030  To be continued ... 11 PPD Coordination, Physics Week, J-R Vlimant 7/10/13