Requirements Verification in Industry gauthier.fanmuy@adn.fr +33 6 - - PowerPoint PPT Presentation

Requirements Verification in Industry

Ce document est la propriété de la société ADN et ne peut être reproduit et communiqué sans son autorisation

gauthier.fanmuy@adn.fr +33 6 10 76 29 06 March 29, 2011

Ce document est la propriété de la société ADN et ne peut être reproduit et communiqué sans son autorisation

70% of defects are introduced during requirements phases

2

Source: National Institute of Standards and Technology (NIST) - 2002 RTI Project 7007.011

Ce document est la propriété de la société ADN et ne peut être reproduit et communiqué sans son autorisation

Project performances and Requirements

• Strong correlation between

Requirements Engineering and projects’ performances

• Native instability of

requirements of complex systems

• Convergence between

requirements and complex solutions not always mastered

• Few actors are trained in

Requirements Engineering

• Need for greater efficiency at

lower costs in projects: « Lean »

(source: SEI et NDIA) -2008 CMU/SEI-2008-SR-034 (source: Defense Management College)

18 % 38 % 44 % 21 % 53 % 26 % 55 % 18 % 27 %

Project Performance Requirements engineering

Lower Capabilities Moderate Capabilities Higher Capabilities

Project Performance versus Requirements Engineering Effort Cost Impact of Poor Requirements Engineering 3

Ce document est la propriété de la société ADN et ne peut être reproduit et communiqué sans son autorisation

Requirement Analysis and Modeling Process (RAMP)

• 3 industrial companies from

different domains :

• 2 smaller companies :
• 4 Academics :

 To improve the efficiency and quality of requirements expressed in natural language during the development of complex systems.

Université Paris 1 Panthéon-Sorbonne

4

Ce document est la propriété de la société ADN et ne peut être reproduit et communiqué sans son autorisation

RAMP rationale

Methods & tools helping to :

• Eliminate wording defects

through syntactic analysis of requirements expressed in natural language,

• Automatically identify

redundant textual requirements,

• Extract data from scenarios

allowing to formalize Requirements and their environment,

• Create, use, re-use Business

know-how through ontologies and other models,

• …

Reinforce quality of Requirements baselines Assist Engineers in writing & analyzing requirements

allowing them to concentrate on content rather than wording and format

Master complex systems development

5

Ce document est la propriété de la société ADN et ne peut être reproduit et communiqué sans son autorisation

RAMP project: Survey on Industrial Requirements Engineering practices

• Study made end 2010 over 22 industrials:

interviews and questionnaires

• Several domains:
• Aeronautics,
• Automotive,
• Energy,
• Space,
• Defense,
• Medical Devices.

6

Ce document est la propriété de la société ADN et ne peut être reproduit et communiqué sans son autorisation

Survey on Industrial Requirements Engineering practices: Main results

• Requirements are mainly written in natural language
• Most common defects on requirements :
• Not solution free,
• Ambiguity, inconsistency, incompleteness, non verifiability, imprecision
• Several requirements in a single requirement,
• …
• Review process is costly and time consuming
• Is effective process with the intervention of the good specialists / experts. It can take 2-3

months in some cases.

• Remains difficult since the final customer does not participate but is represented.
• in SW: 10% of the global development

7

Reviews are not performing as expected

Ce document est la propriété de la société ADN et ne peut être reproduit et communiqué sans son autorisation

Current practices on requirements verification

• Type of reviews
• Inspections: QA reviews with pre-defined criteria's

 Company rules (requirements quality rules e.g. SMART)  Check lists…

• Peer reviews: reviews on the substance

 Feedback from previous projects or similar projects  Business expertise

• Actors
• Quality Assurance, subject matter experts, systems engineers,

designers, testers…

• Statistics
• More than 70% of organizations have requirements quality rules

policies (at corporate or project level)

• Quality rules policies are :

 Correctly applied: 15%  Not applied : 35%  Variably applied: 50%

8

Ce document est la propriété de la société ADN et ne peut être reproduit et communiqué sans son autorisation

Current practices on requirements verification by reviews

9

Most organizations do not use dedicated tools to support Requirements Verification Information systems Hi-Tech systems Naval systems Automotive systems Aerospace systems Defense systems Energy systems Life Science systems Empirical representation of the use of tools for Requirements Verification RMS tools (DOORS, Reqtify…) Office tools (Review Item Discrepancy (RID), Reports…) Specialized tools (RQA, Lexior…) Critical systems

Ce document est la propriété de la société ADN et ne peut être reproduit et communiqué sans son autorisation

Current practices on requirements verification by reviews: RMS tools

• Verification of traceability, requirements quality (e.g.

forbidden words, naming conventions), documents properties (e.g. tables), changes impact analysis,…

10

Testing tools Office RMS tools Modeling tools

Process Rules Checker & Naming conventions

XML ASCII

Report Generation Example of Requirement verification with an RMS tool

Ce document est la propriété de la société ADN et ne peut être reproduit et communiqué sans son autorisation

active voice Subject Verb adverbial phrase of time Indirect object Direct object Direct object

Current practices on requirements verification by reviews: specialized tools

• Issue: Requirements are grammatically incorrect, words

are ambiguous … Engineers waste time on wording and format rather than to concentrate on the content

• Specialized tools: Lexical and syntactical analysis

11

The application shall be able to notify periodically offers and/or rebates to our customers synonym system imprecisions application be able to periodically and/or requirement shall ambiguity Detection of : ambiguous sentences, complex sentence structures, multiple requirements, imprecise requirements…

Ce document est la propriété de la société ADN et ne peut être reproduit et communiqué sans son autorisation

Survey on Industrial Requirements Engineering practices: Main results

• Requirements verification by models
• Not as commonly used as requirements reviews
• Practice judged as providing a real value in improving

the engineering of the project

12

Ce document est la propriété de la société ADN et ne peut être reproduit et communiqué sans son autorisation

Current practices on requirements verification by models

13

R0

R1

R2

R3

R5

R4

R6

Enabled Non active Non activable

Default

Disabled Active Back pedal

Activable

Example of Requirement verification by modeling and simulation

Ce document est la propriété de la société ADN et ne peut être reproduit et communiqué sans son autorisation

Survey on Industrial Requirements Engineering practices: Industry Challenges

• Requirements Engineering: Main benefits
• Increased legibility of project information
• Improved communication between stakeholders
• Industry Challenges
• Explain and make understandable the fundamentals of

Requirements Engineering

• Be more efficient in Requirements authoring: reach a

Lean Requirements Engineering

• Verification of requirements based on knowledge

management

 Extensive use of modeling techniques (e.g. ontologies)

14

Ce document est la propriété de la société ADN et ne peut être reproduit et communiqué sans son autorisation

Industry Challenges : Efficiency in Requirements Authoring

• Issue: Assist System Engineers to write from the

beginning well formed requirements with controlled vocabulary, rather than verify requirements afterwards

• Requirement modeling using Requirements boiler plates

15

THE <OBJECT DETECTION> SHALL <DETECT> <ITEMS> AT <MINIMUM> <RATE VALUE>

<<Detect>> Radar Hits

<<Minimum Value>> 10 units second

UR044 : The Radar shall be able to detect hits at a minimum rate of 10 units per second

Other possibilities: Doppler Radar Sonar Other possibilities: Identify Recognize Other possibilities: Targets Echoes

(Source: RQA – The Reuse Company)

Ce document est la propriété de la société ADN et ne peut être reproduit et communiqué sans son autorisation

Industry Challenges : Verification of requirements based on knowledge management

• Issue: Assist System Engineers

to get a set of requirements complete and consistent (e.g. compliance to regulation, business rules, non redundancy

• f requirements…)
• Requirements transformation

and comparison to formal representations

• Ontologies
• Natural language processing

techniques: enhancing transformation, semantic analysis

16

(Source: RQA – The Reuse Company)

Ce document est la propriété de la société ADN et ne peut être reproduit et communiqué sans son autorisation

Knowledge representation: Ontologies

Controlled vocabulary Taxonomy Thesaurus Light Ontology Ontology Vocabulary analysis Terminology analysis and concepts expansion. Inference rules analysis Concepts coupling

Assessment of the quality of a set

• f Requirements

Assessment of the quality of single Requirements

(Source: RQA – The Reuse Company)

Ce document est la propriété de la société ADN et ne peut être reproduit et communiqué sans son autorisation

Industry Challenges : Verification of requirements based on knowledge management

• Semantic graphs for Requirements formal

representation: an example

UR001: …. UR023: The system shall send weekly notifications to the customers including our

• ffers

URxxx: … UR842: The application shall be able to notify periodically all of our offers to our clients UR999: …

UR023 UR842 <<Notify>> System Customer Offer

18

Ce document est la propriété de la société ADN et ne peut être reproduit et communiqué sans son autorisation

Tooling

• EADS IW assessment over 15 tools (source EADS IW, June

2010, RAMP project)

• RQA (Requirements Quality Analyzer) by TRC (The

Reuse Company) is the Leading tool

• Lexical and syntactical analysis, Requirements writing rules
• Semantic analysis, ontologies
• RQA Products Line
• DOORS Edition (DQA)
• IRQA Edition (IQA)
• Excel Edition (EQA)
• Challenger: LEXIOR by CORTIM
• Lexical and syntactical analysis, Requirements writing rules

19

Ce document est la propriété de la société ADN et ne peut être reproduit et communiqué sans son autorisation

Conclusion

• Typical assessments results have shown that about

25% of requirements are critical and can grammatically be improved

• Absence of shall : 8 to 10%
• Forbidden words : 10 to 15%
• Subject, multiple objects, design : 15%
• Incorrect grammar : 50%
• Paradox
• Active research in Requirements ambiguity (D. M.

Berry, D. Zowghi, S. Boyd, …)

• “Classical” review process is not performing as

expected and is costly and time consuming

20

Ce document est la propriété de la société ADN et ne peut être reproduit et communiqué sans son autorisation

Conclusion

• Industry challenges
• Technologies
• Lexical and syntactic analysis
• Requirements modeling
• Model transformation and comparison
• Knowledge modeling: ontologies
• NLP can help
• Formalizing or verifying requirements (best practices )
• Semantic analysis based on ontologies

21

Efficiency in Requirements Authoring Enhanced Requirements Verification

S M A R T equirements

Ce document est la propriété de la société ADN et ne peut être reproduit et communiqué sans son autorisation

Contact

Gauthier Fanmuy gauthier.fanmuy@adn.fr Phone: +33 1 72 03 23 92 Mobile: +33 6 10 76 29 06 ADN 17 rue Louise Michel 92300 Levallois Perret France http://www.adneurope.com

22