M9 Thursday, March 8, 2001 4:30 PM R EQUIREMENTS -D RIVEN A - PDF document

P R E S E N T A T I O N Presentation Paper Bio M9 Thursday, March 8, 2001 4:30 PM R EQUIREMENTS -D RIVEN A UTOMATED T ESTING Jeff Tatelman Spherion Technology Architects International Conference On Software Test Automation March 5-8, 2001 San Jose, CA, USA

Requirements Driven A utomated Testing Do you have your license to drive? Jeff Tatelman SQuAD November 14 th , 2000

A genda - Business challenges - Business solution - Building the requirements driven testing process - Building the automated regression testing process - Q uestions & discussion

Iterative Development Challenge Testing can no longer be the last step...it Testing can no longer be the last step...it must be an on-going part of the must be an on-going part of the development process. development process. App Build 1 Build 2 Build 3 Build 4 Build 1 Build 2 Build 3 Build 4 spec Test Test Test Test Test Test Test Test

Challenge O f L inking Requirements To Test Cases Requirements Changes Management Development Life Cycle Test Test Management Automation Test Automation

Challenge O f How To G et Started? - How many test cases do I need? - W hat type of test cases should I automate ? - W hat process should I use? - How can I take advantage of reusability?

Business Solution Building the Requirements Driven Testing Process

Iterative T esting Analysis Design Test Test Scope & Strategy Planning Test Execution/ Test Case Evaluation Construction Implement Construction

O bject O riented T esting Approach Unit Integration Testing Testing Requirements Systems Regression Testing Testing

Build The Testing A pproach Business Traceablility Events Matrix Requirements Develop Develop Test Plan Test Cases

1. D eterm ine Business Events Customer Order Service Processing Requirements Inventory Shipping Maintenance Example of an Inventory Control System

2. Develop Traceability M atrix Test Planning Function Coverage Matrix Requirements Repository Test Case Run 1 Run 2 Run 3 R 1 Customer Service Repository TC1 Add customer R 2 Order Processing TC 1 Add customer TC2 add order TC 2 Add Order R 3 Order Change TC3 Mod Order TC 3 Modify Order TC 4 Change Address TC4 Chg Addr R 4 Shipping TC 5 Ship Order TC5 Ship TC 6 Approve Order TC 7 Bill Order TC6 Approve order Example of an Inventory Control System

Traceability M atrix A dvantages - Validation of requirements - Ensure coverage efficiency - Knowledge of what is being testing - Knowledge of what is not being tested - Reduce the number of test cases - Track changes: - requirements to testing - testing to requirements - Reusability for regression/automation

3a. Develop Test Plan (Runs) Test Runs: Run Date: Assigned: Test Run 1 Description: Add a customer and regular order. Do not approve the order. Assumptions Customer service, products, prices, records predefined. Order number will be systems generated. Inputs UI objects- Order Service, Customer Accounts Customer John Smith, Products X23, AR232 Output Reports - Orders by Customer, Billing Signoff Example of an Inventory Control System

3b. Develop the Test Plan (C ycles) Day 1 (test cycle) Objective: To add initial orders and new customers Test Run 1 Description: Add a customer and regular order. Do not approve the order. Test Run 3 Description: Add an inter-company order with five items. Approve the order and ship in the same day. Example of an Inventory Control System

4. Develop T est C ases Detail Test Test Data Assumptions Design Test Expected Flow Results

4. Develop T est C ases TEST CASE SPECIFICATION FORM Project No: 1965 Project Name: ABC Inventory Ordering System Page 1 of 2 Test Case Description: Add a customer. Build No: 1 Case No: 2 Execution Retry No: Written By: Ruby Tuesday Requirement No: 1, 6, Executed By: Amanda Jones Date: 7/25/00 Date: 7/30/00 Setup For Test: Access to ABC Inventory ordering system website Step Action Expected Results Pass / Action Taken Fail if Step Fails 2.1 Click on ‘click here to continue’ ABC Search Customer Page becomes active page. 2.2 Click text field to enter Customer name Cursor appears and text box is enabled

Business Solution Building t he Aut om at ed Reg ression Testing Pr ocess

Regression T esting Definition - The testing of software after a modification has been made to ensure the reliability of each software release - The objective is to prevent defects in How Test production by building and executing an to Test automated testing process that covers the critical business events.

How To Develop A utomated Regression T esting - Build a defined and repeatable process - U se the matrix to help control the scope - Se t up reusable production test data - Se t up a separate test environment

How To Develop A utomated Regression T esting (cont.) - Create an automation test plan - use a subset of systems test cases - review systems test cases with users - separate test cases into cycles - Document screens and test files

Run Run Run

W hy Is The Event/Scenario A pproach So Important For T est A utomation? - Managing your tests - Business events to test automation - Ease of maintenance - Supports data driven approach - Reusability (& modularization) - Same test event “add customer” is reused in several automated test scenarios

K ey Components To Receiving Y our License To D rive - Requirements management process – treating requirements as objects using a repository - Test management process – testing repository – events/ scenarios - Automated testing standards – test plan – reusability - Resources/ Training

Summary - Business challenges - Iterative development - Changes in life cycle - How to get started - Business solution - Traceability matrix - Business events/ scenarios - Reuse test cases for automated regression testing

Process Summ ary Gather business functions Develop test case matrix Develop manual test using business/functions/events from requirement repository, cases and plan to determine what to test Documents or application. 1. Gather requirements 3. Develop manual process 2. Develop trace Develop automation Determine which test Enter test cases into test environment cases to automate a repository 6. Develop environment 5. Brainstorm 4. Manage test cases Set up data files in Formulate design approach Create data source test environment & identify required routines and test scripts 7. Analyze 8. Design 9. Develop

Test Automation Testing Conference Requirements Driven Automated Testing Inventory Control System Traceability Matrix Example Function Name Run1 Run2 Run3 Run4 Run5 Run6 Run7 Run8 Run9 Customer Service New X X X Existing X X X X X Delete X Order Processing order type phone X X X X X X X mail X X order status complete X X X X X X incomplete X X X pricing taxable X X X X X X tax exempt X X X Inventory Maint. instock X X X X X X X out of stock X X replenish inventory X X Shipping bill customer approved X X X X X X X past due X X

Test Automation Testing Conference Requirements Driven Automated Testing Inventory Control System Test Run Example Test Run Description: (Test Run #1) Add a new customer, with an order by phone with in stock inventory for all items. The order is completed and billing is approved. Assumptions: 1. Use default values unless specified here. Input: 1. Customer number: D109166 2. ITEM ID QTY P30KS80 10 P30KB51 20 3UA5400-2D 10 Q220 2 NR421 2 SA1E025 2 LN1E100 5 3. Attach input screens for verification. OUTPUT: 1. Use Account Summary Inquiry and Item Inquiry for on-line validation. 2. Print screen data before ENTER of data. Attach batch reports printout, and screen data printout from inquiry screens. SIGN OFF: The above test case is: _______________ Approved _______________ Need to re-run ______________________ ____________________ _______________ (USER) (ANALYST) (DATE)

Test Automation Testing Conference Requirements Driven Automated Testing Inventory Control System Test Run Example Test Run Description: (Test Run #2) Use an existing customer, with an order by phone with in stock inventory for all items. The order is completed and billing is approved. Assumptions: 1. Customer must exist on the test database. Input: 1. Customer number: S122906 2. ITEM ID QTY P30KS80 10 P30KB51 20 LN1E100 5 3.Attach input screens for verification OUTPUT: 1. Use Account Summary Inquiry for on-line validation 2. Use Past Due Report for printed validation. 3. Print screen data before ENTER of data. Attach batch reports printout, and screen data printout from inquiry screens. SIGN OFF: The above test case is: _______________ Approved _______________ Need to re-run ______________________ ____________________ _______________ (USER) (ANALYST)