SLIDE 1 BIO PRESENTATION International Conference On Software Testing Analysis & Review October 27-31, 2003 San Jose, CA USA
T16
Thursday, October 30, 2003 3:00 PM
ENSURING REQUIREMENTS TRACEABILITY IN FUNCTIONAL
AND PERFORMANCE TESTING
Marc Bloom Capital One Financial Corp
SLIDE 2 Marc S. Bloom
Marc S. Bloom heads up End to End Systems Integration Testing for Capital One. He is an experienced IT Project Manager and Quality Assurance professional with expertise designing, implementing, as well as testing enterprise-level network environments. Accomplishments include integrating MVS, Unisys, UNIX, Oracle, and NT solutions for core financial services and E-Commerce systems. Before joining Capital One in 1998, Marc spent 10 years as a certified systems engineering consultant deploying and supporting large-scale network solutions for Fortune 500 corporations. Marc holds a Bachelor’s degree from James Madison University and a Master’s certificate in Project Management from George Washington.
SLIDE 3
Requirements Traceability in Functional and Performance Testing
Marc S. Bloom QA Test Manager marc.bloom@capitalone.com
SLIDE 4
2
Agenda
Requirements Definition Requirements Traceability Requirements Management Functional vs. Performance Requirements Key Similarities Key Differences
SLIDE 5 3
Capital One at a Glance
A leading financial services
company
issuer in the U.S.
managed loans
accounts
- Located in 8 U.S. cities,
Canada, U.K., France and South Africa
#191!
Numerous awards including:
- Top 100 training
- rganization – Training
magazine
magazine
- One of the “Best Companies
to Work for” in the U.K.-The Sunday Times
Customer Relationship Management”
SLIDE 6
Requirements Definition
SLIDE 7 5
What are Requirements?
“Business requirements represent high-level objectives of the
- rganization or customer requesting the system or product. They
are captured in a document describing the project’s vision and scope.” “User requirements describe tasks the user must be able to accomplish with the product. These are captured in use cases or scenario descriptions.” “Functional requirements define the software functionality the developers must build into the product to enable users to accomplish their tasks, thereby satisfying the business
- requirements. Functional requirements are documented in a
software requirements specification (SRS).”
- Karl Wiegers 1999. Software Requirements
SLIDE 8
6
When are Requirements Defined? By whom?
Analysis Phase of the Software Development Life Cycle (SDLC). IT Business Analysts meet with various stakeholders and document needs. All stakeholders verify requirements documentation as correct and unambiguous before Development begins.
SLIDE 9
7
What are the Sources of Requirements Documentation?
Business Rules Business Models System Process Flow Use Cases System Specifications User Interface Specifications Standards – Internal & External
SLIDE 10 8
What are Some Characteristics
Complete Consistent Correct Unambiguous Non-compound Feasible Necessary Prioritized Be SMART S imple M easurable A ccurate R easonable T estable
SLIDE 11
9
What are Testable Requirements?
To be most useful in providing a solid basis for a Test Plan, requirements must be verifiable. A "Testable Requirement" is one that can be verified through the execution of a defined test case.
SLIDE 12
Requirements Traceability
SLIDE 13 11
Requirements Traceability Defined
The degree to which a relationship or link can be established between two or more products
- f the development process – particularly
when having a predecessor-successor or master-subordinate relationship to one
SLIDE 14
12
What are Some Traceability Benefits?
Certification of requirements coverage Change impact analysis Maintenance throughout SDLC Project tracking and metrics Re-engineering support Reuse of code in other projects Testing linkage
SLIDE 15 13
What is a Requirements Traceability Process?
The process that ensures Development has designed the right software product in the right way, i.e. that it meets the stated needs
T E S T C A S E S CODE MODULES PRIORITY USE CASES C U S T O M E R R E Q U I R E M E N T S
1. Requirements 2. Use Cases 3. Prioritization 4. Code 5. Test Cases
SLIDE 16 14
Requirements Traceability Model
Legend: Gray – Artifacts Green – Actors Red - Processes
SLIDE 17
15
Requirements Traceability Matrix
Key method for associating individual requirements to test cases Dynamic artifact in Requirements Process Test cases can represent single or multiple requirements
SLIDE 18 16
Requirements Traceability Sample Matrix
CAA3-001 Add_CAA3 DBM Add CAA23 CAA3 CAA2-001 ABCD_CAA1 EJB Connect to PCAAS CAA2 P3-002 Find_stat.44 AccountStat Servelet Return COAR Status P3 CAA1-003 Query.code2 3 JAVA Q Query Transfer Acct# CAA1 P3-002 Query.code2 AccountLookup Servlet Query Credit Acct# P3 P3-001 FindCOAR AccountLookup Servlet Query COAR Acct# P3 Test Case Code (Lines) Design Element Functional Requirement Use Case
SLIDE 19
Requirements Management
SLIDE 20 18
Requirements Management Defined
The purpose of Requirements Management is to establish a common understanding and agreement between the customer and the software project of the customer’s requirements that will be addressed by the software.
* CMM L2-1
SLIDE 21
19
Requirements Management Process
It's rarely the case that the set of requirements with which a project originally starts is the same set of requirements that the final product fulfills. The Requirements Management Process insures the accurate traceability of individual requirements throughout the SDLC.
SLIDE 22
20
Requirements Management Process (cont.)
Formal Change Control and Configuration Management are essential to documenting and tracking system or application requirements. A Change Control Board, with a defined CCB process, should approve all changes, additions or modifications.
SLIDE 23
21
Downstream Impact of Requirements Changes
Architecture Design Code Development Use Cases User Documentation & Manuals Help Screens/Error Messages Test Cases
SLIDE 24
Functional vs. Performance Testing Requirements
SLIDE 25
23
Stakeholder Identification
Ask your team the question, “Who are we testing for?” The number of answers received may surprise you. Understanding the differences and overlaps in the beginning of the process insures that testing is properly targeted to each stakeholder’s needs.
SLIDE 26 24
Stakeholders (cont.)
Functional:
Business Customer
Marketing Operations Customer Relations
IT Development Internal Audit
Performance:
Business Customer IT Platform Management Systems Engineering Capacity Planning IT Infrastructure Mgt. Future Technologies R&D Internal Audit
SLIDE 27
25
Focus of Functional Requirements
Positive-based Process Flow Alternate or Exception Flows Data Integrity Verification GUI Interface Validation Information Security Risk Mitigation Legal and Compliance Verification
SLIDE 28
26
Focus of Performance Requirements
Increased Priority on IT-defined Requirements Positive-based Process Flow
Excludes Boundary and Exception Tests System Monitoring Resource Utilization Platform Capacity/Scalability Infrastructure Stability
SLIDE 29
27
Key to Breaking out Performance Requirements
Knowing the Demographics of the users What are their high percentage activities? How many total users in the system at any given time? Concurrent Load Tests What are the peaks during the day? Elasticity (Spike/Bounce) Tests What are acceptable response times? Stress and Endurance Tests
SLIDE 30
28
Summary
Requirements drive the entire testing process. Use the Traceability Process to provide the necessary linkage to your Testing Process. Testing Tools provide excellent support for Requirements Management and Traceability. Ensure that you know your stakeholders to properly define the Functional and Performance testing needs.
SLIDE 31
29
Bibliography
Rodger Drabick, “On-Track Requirements”, STQE Magazine, 1999 Johanna Rothman, Brian Lawrence, “Testing in the Dark”, STQE Magazine, 1999 Paul Gerrard, “Testing Requirements”, EuroSTAR ’94 Bill and Carol Councill, “Automating Requirements Traceability”, STQE Magazine, 2000 Karl E. Wiegers, “Requirements When the Field Isn’t Green”, STQE Magazine, 2001 Software Requirements, Second Edition, Karl E. Wiegers, 2003.
SLIDE 32
30
Questions?
