Requirements Verification in Industry gauthier.fanmuy@adn.fr +33 6 - PowerPoint PPT Presentation

Requirements Verification in Industry gauthier.fanmuy@adn.fr +33 6 10 76 29 06 March 29, 2011 Ce document est la propriété de la société ADN et ne peut être reproduit et communiqué sans son autorisation

Ce document est la propriété de la société ADN et ne peut être reproduit et communiqué sans son autorisation 70% of defects are introduced during requirements phases Source: National Institute of Standards and Technology (NIST) - 2002 RTI Project 7007.011 2

Ce document est la propriété de la société ADN et ne peut être reproduit et communiqué sans son autorisation Project Performance versus Project performances and Requirements Engineering Effort (source: SEI et NDIA) -2008 Requirements Project Performance 18 % 21 % CMU/SEI-2008-SR-034 55 % 38 %  Strong correlation between 53 % Requirements Engineering and 18 % 44 % projects’ performances 26 % 27 %  Native instability of Lower Moderate Higher requirements of complex Capabilities Capabilities Capabilities Requirements engineering systems (source: Defense Management College)  Convergence between requirements and complex solutions not always mastered  Few actors are trained in Requirements Engineering  Need for greater efficiency at lower costs in projects: « Lean » Cost Impact of Poor Requirements Engineering 3

Requirement Analysis and Modeling Process (RAMP) Ce document est la propriété de la société ADN et ne peut être reproduit et communiqué sans son autorisation  3 industrial companies from different domains :  2 smaller companies :  4 Academics : Université Paris 1 Panthéon-Sorbonne  To improve the efficiency and quality of requirements expressed in natural language during the development of complex systems. 4

Ce document est la propriété de la société ADN et ne peut être reproduit et communiqué sans son autorisation RAMP rationale Methods & tools helping to : • Eliminate wording defects through syntactic analysis of Master complex systems development requirements expressed in natural language, • Automatically identify redundant textual Reinforce quality of Requirements requirements , baselines • Extract data from scenarios allowing to formalize Requirements and their Assist Engineers in writing & analyzing environment, requirements • Create, use, re-use Business allowing them to concentrate on content know-how through ontologies rather than wording and format and other models , • … 5

Ce document est la propriété de la société ADN et ne peut être reproduit et communiqué sans son autorisation RAMP project: Survey on Industrial Requirements Engineering practices  Study made end 2010 over 22 industrials: interviews and questionnaires  Several domains:  Aeronautics,  Automotive,  Energy,  Space,  Defense,  Medical Devices. 6

Survey on Industrial Requirements Engineering Ce document est la propriété de la société ADN et ne peut être reproduit et communiqué sans son autorisation practices: Main results  Requirements are mainly written in natural language  Most common defects on requirements :  Not solution free,  Ambiguity, inconsistency, incompleteness, non verifiability, imprecision  Several requirements in a single requirement,  …  Review process is costly and time consuming  Is effective process with the intervention of the good specialists / experts. It can take 2-3 months in some cases.  Remains difficult since the final customer does not participate but is represented.  in SW: 10% of the global development Reviews are not performing as expected 7

Ce document est la propriété de la société ADN et ne peut être reproduit et communiqué sans son autorisation Current practices on requirements verification  Type of reviews  Inspections : QA reviews with pre-defined criteria's  Company rules (requirements quality rules e.g. SMART)  Check lists …  Peer reviews : reviews on the substance  Feedback from previous projects or similar projects  Business expertise  Actors  Quality Assurance, subject matter experts, systems engineers, designers, testers…  Statistics  More than 70% of organizations have requirements quality rules policies (at corporate or project level)  Quality rules policies are :  Correctly applied: 15%  Not applied : 35%  Variably applied: 50% 8

Current practices on requirements verification Ce document est la propriété de la société ADN et ne peut être reproduit et communiqué sans son autorisation by reviews Most organizations do not use dedicated tools to support Requirements Verification Naval systems Specialized tools (RQA, Lexior …) Critical Aerospace systems systems Automotive Defense systems RMS tools systems (DOORS, Life Science Reqtify …) systems Energy Office tools systems (Review Item Hi-Tech Discrepancy Information systems (RID ), Reports…) systems Empirical representation of the use of tools for Requirements Verification 9

Ce document est la propriété de la société ADN et ne peut être reproduit et communiqué sans son autorisation Current practices on requirements verification by reviews: RMS tools  Verification of traceability, requirements quality (e.g. forbidden words, naming conventions), documents properties (e.g. tables), changes impact analysis,… Process Rules Checker & Naming conventions Testing tools Office Modeling tools RMS tools Example of Requirement Report Generation verification with an RMS tool XML ASCII 10

Current practices on requirements verification Ce document est la propriété de la société ADN et ne peut être reproduit et communiqué sans son autorisation by reviews: specialized tools  Issue: Requirements are grammatically incorrect, words are ambiguous … Engineers waste time on wording and format rather than to concentrate on the content  Specialized tools: Lexical and syntactical analysis The application shall be able to notify periodically offers and/or rebates to our customers application shall be able to periodically and/or synonym ambiguity imprecisions system requirement Subject Verb Direct object Direct object Indirect object adverbial phrase of time active voice Detection of : ambiguous sentences, complex sentence structures, multiple requirements, imprecise requirements… 11

Ce document est la propriété de la société ADN et ne peut être reproduit et communiqué sans son autorisation Survey on Industrial Requirements Engineering practices: Main results  Requirements verification by models  Not as commonly used as requirements reviews  Practice judged as providing a real value in improving the engineering of the project 12

Ce document est la propriété de la société ADN et ne peut être reproduit et communiqué sans son autorisation Current practices on requirements verification by models Moteur Moteur tournant tournant Contact On Sélection RVV (CRUISE, OFF) RVV sélectionné R0 V>Vmin Frein off Embrayage off Accélération État pédale accélération Rapport BVM > Rmin R4 Mode drive BVA ESP off RVV en reprise pédale Activation RVV set +/- R1 Cancel/RESUME V véhicule courante R2 Défaut RVV V régulation set +/- long R3 RVV sélectionné RVV actif RVV en reprise pédale R6 V régulation set +/- court Défaut RVV R5 Disabled Enabled Absence CRUISE Non active défaut RVV Active Non activable RESUME SET +/- Action Absence pédale action pédale utilisateur utilisateur Back pedal CANCEL Activable OFF Absence Absence Défaut RVV Défaut RVV défaut RVV défaut RVV Défaut Default RVV Example of Requirement verification by modeling and simulation 13

Ce document est la propriété de la société ADN et ne peut être reproduit et communiqué sans son autorisation Survey on Industrial Requirements Engineering practices: Industry Challenges  Requirements Engineering: Main benefits  Increased legibility of project information  Improved communication between stakeholders  Industry Challenges  Explain and make understandable the fundamentals of Requirements Engineering  Be more efficient in Requirements authoring: reach a Lean Requirements Engineering  Verification of requirements based on knowledge management  Extensive use of modeling techniques (e.g. ontologies) 14

Industry Challenges : Efficiency in Requirements Ce document est la propriété de la société ADN et ne peut être reproduit et communiqué sans son autorisation Authoring  Issue: Assist System Engineers to write from the beginning well formed requirements with controlled vocabulary , rather than verify requirements afterwards  Requirement modeling using Requirements boiler plates UR044 : The Radar shall be able to detect hits at a minimum rate of 10 units per second THE <OBJECT DETECTION> SHALL <DETECT> <ITEMS> AT <MINIMUM> <RATE VALUE> Other possibilities: <<Minimum Identify <<Detect>> Value>> Recognize Other possibilities: Other possibilities: 10 units second Doppler Radar Radar Hits Targets Sonar Echoes (Source: RQA – The Reuse Company) 15