The Change Agent: Leveraging the Testing Role David Capocci CQA, - - PDF document

▶
the change agent
SMART_READER_LITE
LIVE PREVIEW

The Change Agent: Leveraging the Testing Role David Capocci CQA, - - PDF document

May 18, 2023 136 likes •351 views

P R E S E N T A T I O N Presentation Bio W11 Wednesday, October 31, 2001 3:15 PM T HE C HANGE A GENT : L EVERAGING THE T ESTING R OLE David Capocci SAFECO Insurance International Conference On Software Testing Analysis & Review October

slide-1
SLIDE 1

P R E S E N T A T I O N International Conference On Software Testing Analysis & Review October 29-November 2, 2001 San Jose, CA USA Wednesday, October 31, 2001 3:15 PM

THE CHANGE AGENT: LEVERAGING THE TESTING ROLE

David Capocci

SAFECO Insurance

W11

Presentation Bio

slide-2
SLIDE 2

1

The Change Agent:

“ Leveraging the Testing Role ”

David Capocci CQA, CSTE Senior Quality Assurance Consultant SAFECO Insurance Davcap@SAFECO.com 206-545-6187

slide-3
SLIDE 3

2

Agenda

  • SAFECO Past & Present
  • Background for a Change Agent
  • People Perspective: Talk the Talk
  • Process Perspective: Set Forth a Model
  • Project Perspective: Test Requirements
  • Summary, Q&A
slide-4
SLIDE 4

3

SAFECO… Where we were

  • “Thrown over the wall” Testing was norm
  • Firefighters
  • Everyone was their own island
  • Tribal knowledge
  • Expert in systems vs. independent testers
  • Testing was an entry level position
  • Attrition of QC skills to Developer role
  • Hunger for early involvement in projects
  • Called QA, but doing QC
slide-5
SLIDE 5

4

SAFECO… Where we are today

  • An established QC Model
  • Baselined understanding of QA & QC
  • Development of QC training based on model
  • Identifying Test Requirements is providing

leverage early on in the project life cycle

  • QC community better networked
  • Prior workbenches understanding the QC role
slide-6
SLIDE 6

5

Background for a QA/QC Change Agent…

  • Passion for QA/QC
  • Ability to simplify the use of

tools/models/industry practices

  • Familiar with the CMM
  • Familiar with a tool for modeling a process
slide-7
SLIDE 7

6

Background… Tool: The 3 P’s Model

People Process Project

Finger on the pulse Work Samples Find Common Denominators Informal Surveys Steering Committee Industry Practices Workbench Concept Standards, Institutionalize Define, Make Visible Automation Pilot Accountability Successful case studies Best Practices Process & People “Testing Ground”

slide-8
SLIDE 8

7

People…

  • Understand Organizational Change…

Awareness Understanding Definition Installation Adoption

Adapted from CMM Implementation Guide by Kim Caputo

Chasm

  • f Adoption

Institutionalization

Crack of

Institutionalization

Crack of Installation Crack of Definition Crack of Understanding

Time Commitment

slide-9
SLIDE 9

8

People…

  • Find the Common Denominator:

– Know where our culture is at with QA & QC – Work with assumptions others make

  • f our role

– Stick with Back to Basics mentality – Here’s the technique we used…

slide-10
SLIDE 10

9

People… The SDLC & The

Requirements Analogy

Concept Requirements Design Code Test Operation & Maintenance

BA PA QA QC

Generating Requirements:

  • Functional Specs
  • Documents
  • Diagrams

(generated formally & informally) Reviewing Requirements:

  • in the context of

technology Refining Requirements:

  • limitations of technology

Introducing Changed Requirements

Detailing functional/business requirements into lower level technical requirements Detailing technical requirements into code Responding to Mismatched Requirements (Defect fixing)

Generating TEST REQUIREMENTS

  • from Business, Functional, & Technical Requirements

Test Planning, Test Scenarios, Automation Strategy Planning

After Code

Validation Activities:

  • Executing Tests
  • Retesting
  • Measuring Stability &

Test Coverage FOCUS: Defect Detection Verification Activities: Testing the Requirements

  • Inspections, Walkthroughs, Reviews, Deskchecks

(while officially true, many times it is not reality) FOCUS: Defect Prevention

Before Code

Handout!

slide-11
SLIDE 11

10

People… Watch your Tongue!

  • Baseline your vocabulary:

– Establish a glossary of terms, online or intranet – Don’t allow jargon assumptions! – Breakdown and differentiate between QA & QC – Also, Verification (Defect Prevention) & Validation (Defect Detection) – And… Requirements: Functional, Non-Functional, Technical, and Test. – Define each of the deliverables from the test process.

slide-12
SLIDE 12

11

Process… The Key “Approach” Points

  • A. Create a high level Process Map of your

work.

  • B. Identify Customers & Suppliers.
  • C. Make your Entrance & Exit Criteria known.
  • D. Decompose the granular processes that

comprise your process map.

  • E. Implement in a way that fits your culture

(change agent).

slide-13
SLIDE 13

12

Execute

Do

Rework

Tools Standards

Check

Test Execution Wor kbench

Review & Report

Do

Rework

Tools Standards

Check

Test Results Workbench

Track

Do

Rework

Tools Standards

Check

Defect Management Workbench

Plan, Gather, Assess

Do

Rework

Tools Standards

Check

Test Planning Workbench

Prepare

Do

Rework

Tools Standards

Check

Test Preparation Workbench

Test Plan Test Requirements Hierarchy Risk Assessment

(on Test Requirements)

Traceability Matrix Test Environment- Data Issues Test Environment- Architecture Issues Test Environment- Training Issues

Test Case Design Running the Test Reviewing Results Reporting Test Status

Defect Audit Trail Defect Tracking Standards Defect Summary Reporting

Quality Control

Process… The QC

Framework Model Handout!

slide-14
SLIDE 14

13

Process… Implement in a way that fits your culture

  • Example: The “In-Service” Approach

– Learned from a non-profit, volunteer org. – Short talk (<3 hours) on a specific topic – Includes short exercises – Taken directly to the team or unit meeting

slide-15
SLIDE 15

14

Process… Implement in a way that fits your culture

  • Example: The “Pilot” Approach

– Pilot personally with your own work – Market success to your manager – Show time savings, reusability, efficiency, portability to others, etc. – Share with co-workers – Put together a reference or tutorial – Invite interested people to a “demo” meeting

slide-16
SLIDE 16

15

Project… Standardizing Test Requirements

High level Functional Areas: usually from “Functional Spec” type docs, or BA work Lower level Functional Areas: usually from “Technical Spec” type docs regarding internal logic,

  • r PA work

Business Function Tasks within the Function Data Entry Types for transactions Transactions to perform a task Field Validation

slide-17
SLIDE 17

16

Summary

SAFECO Past & Present Background for a Change Agent People Perspective: Talk the Talk Process Perspective: Set Forth a Model Project Perspective: Test Requirements Q&A

slide-18
SLIDE 18

Components & Roles during SDLC

Concept Requirements Design Code Test Operation & Maintenance

BA PA QA QC

Generating Requirements:

  • Functional Specs
  • Documents
  • Diagrams

(generated formally & informally) Refining Requirements:

  • limitations of technology

Introducing Changed Requirements Reviewing Requirements:

  • in the context of technology

Detailing functional/ business requirements into lower level technical requirements Detailing technical requirements into code Responding to mismatched requirements (Defect fixing) Verification Activities: Testing the Requirements

  • Inspections, Reviews, Walkthroughs, Desk Checks

(while officially true, many times it is not reality) Focus: DEFECT PREVENTION

Before Code After Code

Generating TEST REQUIREMENTS:

  • from Business, Functional, & Technical Requirements

Test Planning, Test Scenarios, Automation Strategy Planning Validation Activities:

  • Executing Tests
  • Retesting
  • Measuring Stability &

Test Coverage Focus: DEFECT DETECTION BA strength: Workflows & Modeling. PA strength: knowledge of technology. QA strength: Process & Traceability. QC strength: investigative, problem solving, heuristics (should be Testing & Measuring )

11/23/99

David Capocci, CQA, CSTE Sr. QA/QC Consultant SAFECO

slide-19
SLIDE 19

David Capocci, CQA, CSTE Sr. QA/QC Consultant SAFECO

Quality Control Validation Activities Focus: Defect Detection

09/20/00 DO Check REWORK Standards Tools

Test Planning Workbench

DO Check REWORK Standards Tools

Test Preparation Workbench

DO Check REWORK Standards Tools

Test Preparation Workbench

DO Check REWORK Standards Tools

Test Execution Workbench

DO Check REWORK Standards Tools

Test Results Workbench

DO Check REWORK Standards Tools

Defect Management Workbench

Plan, Gather, Assess Prepare Execute Review & Report Track

Test Plan -

  • What is the QC process for

this project?

  • What will the deliverables

be from testing?

  • Estimations & Milestones
  • What types of testing will be

done? (i.e. function-based, integration, systems-level)

  • What is the strategy/

approach for each of the areas of the QC process?

Test Requirements Hierarchy -

  • Gathering all info to generate

Test Requirements

  • Organized in a measurable

format

  • Sources identified
  • Test Coverage established
  • Verified & Signed off by

Project Team

Risk Assessment (on Test Requirements)-

  • Identifying risk factors
  • Probability of faults
  • Consequence of failures
  • Assessed as part of the Test

Requirements Hierarchy

Test Requirements/ Test Case

Traceability Matrix -

  • Make Test Coverage visible
  • Linking the test cases back to

the test requirements they are covering

  • Assists in test case design

status reporting

Test Environment

Data issues -

  • Identify sources for data
  • Identify strategies for

Gernerating & Maintaining data

Defect Audit Trail-

  • Tracking status of a defect

throughout its life cycle (New, Assigned, Postponed, Investigated, Resolved, Validated, Failed, or Closed)

  • Clearly established roles for who

can alter a defect's status

  • Traceability of a defect back to

the gerenating test case and covered test requirement

Reporting Test Status-

  • Use of measurements with in

metrics

  • Plotting planned/actual/

executed tests against schedule

  • Extrapolating Trends
  • Reporting Test Coverage

(Test Requirements)

  • Reporting Test Stability

(Defects)

  • Assessing schedule impacts,

risks, and maintenance costs

Reviewing Test Results-

  • Comparing Actual against

Expected Results

  • Measuring "Passed" Test

Coverage

  • Identifying Defects
  • Maintaining Test cases and

automation

Test Environment

Training issues -

  • Identify subject matter experts
  • Lining up training

Test Environment

Architecture issues -

  • Managing databases
  • Designing, Procuring, Installing,

Administering

  • Configuring: processing schedules,

communications, hardware

  • Consideration of interface with

automated tools.

Defect Tracking Standards-

  • Defect Type Category or problem

type classification

  • Area of problem occurence
  • Severity: impact on the system
  • Priority: order in which to address
  • Persons assigned
  • Test case / Test Requirement
  • Estimated/Actual hours

Defect Summary Reporting-

  • DEFECTS BY PRIORITY

REPORT: defect numbers tallied by Priority

  • DEFECTS BY SEVERITY: defect

numbers tallied by Severity

  • MEAN TIME TO RESOLVE: The

average time between discovery

  • f a defect and its resolution to be

incorporated into a new build or fix

  • MEAN TIME TO CLOSE: The

average time between discovery

  • f a defect and its close after

retesting.

Test Case Design-

  • Test Design Specs, or what to track
  • Black & White box techniques
  • Case Scenarios
  • Mapping out data & workflows
  • Create reference sheets

Running the Test -

  • Piloting automation
  • Recording raw/actual results
  • Updating test cases
slide-20
SLIDE 20

David Capocci, CQA, CSTE

  • Advisory Board for the University of Washington's Software Testing Certificate

program.

  • Instructor for the UW program during its first couple of quarters.
  • Member, Speaker, & Committee Chair, Seattle Area Software Quality Assurance

Group (www.SASQAG.org).

  • Seattle coordinator for offering the certification exams through SASQAG/QAI.
  • Certified Quality Analyst (CQA)
  • Certified Software Test Engineer (CSTE)
  • Senior QA/QC Consultant at SAFECO

Senior QA/QC Consultant SAFECO Insurance (206) 545-6187 DAVCAP@SAFECO.COM