1 Quality is ... I know it when I see it it suits the - - PDF document

1
SMART_READER_LITE
LIVE PREVIEW

1 Quality is ... I know it when I see it it suits the - - PDF document

Ume University Department of Computing Science Programvaruteknik VT17 Implement Quality and Quality Assurance Jonny Pettersson http://www.cs.umu.se/kurser/5DV151 Last Time Short on implementation Systematic testing 24/4 - 17


slide-1
SLIDE 1

1

Umeå University Department of Computing Science

Programvaruteknik VT17

Implement – Quality and Quality Assurance

Jonny Pettersson http://www.cs.umu.se/kurser/5DV151

24/4 - 17 Programvaruteknik - Jonny Pettersson, UmU

Last Time

  • Short on implementation
  • Systematic testing

24/4 - 17 Programvaruteknik - Jonny Pettersson, UmU

Today

  • What is Quality?
  • What is Quality Assurance?
  • Planning Quality
  • Inspections
slide-2
SLIDE 2

2

24/4 - 17 Programvaruteknik - Jonny Pettersson, UmU

Quality is ...

  • … I know it when I see it
  • … it suits the client/user
  • … it conforms to the specification
  • … it has some inherent quality
  • … it depends on the price
  • … it depends on the delivery date

24/4 - 17 Programvaruteknik - Jonny Pettersson, UmU

And Quality is …

Efficiency Reliability Easy to learn Functionality Flexibility Increased productivity Low costs Easy to use Readable code Good design Good documentation Few errors

Sponsor User Maintainer/modifier

Short time of delivery Easy to remember

24/4 - 17 Programvaruteknik - Jonny Pettersson, UmU

Quality Factors (ISO 9126)

Functionality

Replaceability Suitability Accuracy Interoperability Security Maturity Fault tolerance Recoverability Understandability Learnability Operability Time behavior Resource behavior Analyzability Changeability Stability Testability Adaptability Installability Conformance

Reliability Efficiency Usability Maintainability Portability

slide-3
SLIDE 3

3

24/4 - 17 Programvaruteknik - Jonny Pettersson, UmU

The Quality Triangle

  • Different goals for different

projects and organizations

− Medical equipment − Text editor − Game app ➨ ”Good Enough” Software ➨ Quality must be measured

goal-oriented

Lead time (close delivery time) Quality (absense of errors) Functionality (number of “features”)

24/4 - 17 Programvaruteknik - Jonny Pettersson, UmU

What is Quality Assurance?

  • Constructive vs. analytic approaches to QA
  • Qualitative vs. quantitative quality goals
  • Measurement

− Derive qualitative factors from measurable quantitative factors − Software measures (metrics) QA is the combination of planned and unplanned activities to ensure the fulfillment of predefined quality standards.

24/4 - 17

Approaches to QA

  • Constructive Approaches

− Syntax-directed editors − Enforced coding guidelines − Type systems − ...

  • Software Process Improvement
  • Analytic approaches

− Inspections − Static analysis tools (e.g., lint) − Testing − ...

The usage of methods, languages, and tools that ensure the fulfillment of some quality factors. The usage of methods, languages, and tools to observe the current quality level. The improvement of development processes to yield better products.

Programvaruteknik - Jonny Pettersson, UmU

slide-4
SLIDE 4

4

24/4 - 17 Programvaruteknik - Jonny Pettersson, UmU

How To Measure Quality?

Quality Factor

Property/ Criteria Property/ Criteria Property/ Criteria

Measure Measure Measure

depends

  • n

determined by

Efficiency

Speed Size

Response time in s LOC ...

subjective properties

  • bjective

measurements

24/4 - 17 Programvaruteknik - Jonny Pettersson, UmU

Purpose of Measurement

  • Analysis: Determine current quality
  • Prediction: Predict future quality
  • Not only code can be measured, but also

− Other work products

  • Vagueness in the requirements
  • Amount of documentation

− Processes

  • Schedule performance
  • Defect removal efficiency

− Resources

  • Effort
  • Expenses

24/4 - 17

Some Example Measures

  • To measure efficiency

− Time behaviour

  • Transactions per second
  • Response time
  • Screen refresh time

− Resource behaviour

  • KBytes of executables
  • LOC
  • Number of processors
  • To measure usability

− Training time − Number of help frames

  • To measure reliability

− MTTF (Mean Time To Failure) − Availability

  • To measure robustness

− Time to restart after a failure − Probability of data corruption on failure

Æ Measurement is necessary

Some measures might be indicators for several quality factors

Programvaruteknik - Jonny Pettersson, UmU

slide-5
SLIDE 5

5

24/4 - 17

  • Code
  • Design
  • Requirements
  • ...

Improving Quality

  • Personnel
  • Tools
  • Budget
  • ...
  • Methods
  • Management
  • Accounting
  • ...

Resources Products Processes

Ordinary testing Easy to measure

Programvaruteknik - Jonny Pettersson, UmU 24/4 - 17 Programvaruteknik - Jonny Pettersson, UmU

The Message Quality is relative Quality covers more than just code

24/4 - 17 Programvaruteknik - Jonny Pettersson, UmU

Quality Assurance

  • What is Quality Assurance?
  • What is Quality?
  • Planning Quality
  • Inspections
slide-6
SLIDE 6

6

24/4 - 17 Programvaruteknik - Jonny Pettersson, UmU

Quality Planning

  • Quality planning is the process of developing a quality

plan for a project

− Sets out the desired software qualities − How these are to be assessed

  • Outline for a quality plan (Humphrey, 1989)

− Product introduction − Product plans − Process descriptions − Quality goals − Risk and risk management

24/4 - 17 Programvaruteknik - Jonny Pettersson, UmU

Software Quality Attributes

  • Potential software quality attributes

− Safety, security, reliability, resilience, robustness, understandability, testability, adaptability, modularity, complexity, portability, usability, reusability, efficiency, learnability

  • Not possible to optimise for all attributes
  • The quality plan should define the most important quality

attributes

24/4 - 17 Programvaruteknik - Jonny Pettersson, UmU

Costs from

  • prev. activity

Defects from

  • prev. activity

Quality Levels Can be Planned

Effectivity Costs

50 50 80% 2

units defect

200 160 80 100

20

Defects to next activity

360

Costs to next activity Defects introduced in current activity Defects ”inherited” from previous activity Defects deleted Total cost of quality in current activity Accumulated costs of quality for all activities

slide-7
SLIDE 7

7

24/4 - 17 Programvaruteknik - Jonny Pettersson, UmU

An Example

50% 15 8 7,5 7,5 1

Requirements inspection

50% 35 7,5 72 64 21,3 3

Design inspection

50% 30 21,2 328 256 25,6 10

Unit test

50% 10 25,6 862 534 17,8 30

Integration test

50% 10 17,8 1279 417 13,9 30

System test

100% 0 13,9 4059 2780 13,9 200

Operation Review, individual (40%,2) + Formal inspection, team (70%,5)

24/4 - 17 Programvaruteknik - Jonny Pettersson, UmU

An Example (cont.)

50% 15 8 7,5 7,5 1

Requirements inspection

40% 35 7,5 42 34 17 2

Design reviews

70% 25,5 132 90 17,9 5

Formal design inspection

50% 10 18,8 752 432 14,4 30

Integration test

50% 10 14,4 1118 366 12,2 30

System test

100% 0 12,2 3558 2440 12,2 200

Operation

50% 30 7,6 320 188 18,8 10

Unit test 13,9 4059

Fewer defects to lower costs!!!

Old QA plan

24/4 - 17 Programvaruteknik - Jonny Pettersson, UmU

High Variation in Defect Removal Effectivity

  • Individuals testing their own code

− Usually < 50%

  • Normal testing activities

− Often < 70%

  • Design- and code inspections

− Often > 65%, up to 85%

  • Formal inspections plus formal testing activities

− Often > 96%, up to 99%

  • Formal inspections can lead to up to 30% decreases in

costs and lead times

slide-8
SLIDE 8

8

24/4 - 17 Programvaruteknik - Jonny Pettersson, UmU

Formal Inspections

g distribute materials (documents, process, checklists) g select participants (4–6)

Report

pass?

Document

Next development phase

Y N Y partial re-inspection

pass?

Fix problems Preparation Individual review Inspection meeting Fix problems Review changes

24/4 - 17 Programvaruteknik - Jonny Pettersson, UmU

Formal Inspections (cont.)

  • Effective QA method
  • Applicable to all types of work products

− Requirements documents − Design documents − Documentation − Code − ...

  • Expensive
  • Positive side-effects

Æ Most effective in early development phases

24/4 - 17 Programvaruteknik - Jonny Pettersson, UmU

Optimization of Inspections

  • Right mix of participants (number, experience, error

profile)

  • Right duration
  • Right supporting materials (checklists, reading

techniques, ...)

  • Evaluation of effectivity

− Defects found per unit of time − Defect profiles of participants

Goal: Stop when desired quality level is reached Problem: How can we know the quality level (defect density) in the input?

slide-9
SLIDE 9

9

24/4 - 17 Programvaruteknik - Jonny Pettersson, UmU

Re-Inspections Are Expensive And Must be Avoided

  • Compare to benchmarks and historical data

− Fewer defects detected ⇒ bad inspection ⇒ re-inspection − More defects detected ⇒ bad input ⇒ re-inspection

  • Problem

− Very good input ⇒ few defects to detect ⇒ re-inspection − Bad input & bad inspection ⇒ normal ⇒ no re-inspection

Æ Develop fault profiles Æ Subjective evaluation by participants Æ Error seeding or capture-recapture

24/4 - 17 Programvaruteknik - Jonny Pettersson, UmU

Capture-Recapture

  • Used to estimate the size of animal populations

− Catch and mark animals − Release marked animals − Catch animals again under similar conditions Æ Estimate population size by means of the relationship between marked and unmarked animals

inspector 1 inspector 2

24/4 - 17 Programvaruteknik - Jonny Pettersson, UmU

The Message There is more to quality assurance than just testing

slide-10
SLIDE 10

10

24/4 - 17 Programvaruteknik - Jonny Pettersson, UmU

Summary

  • What is quality
  • What is quality assurance
  • Planning quality
  • Inspections