1 Quality is ... I know it when I see it it suits the - PDF document

Umeå University Department of Computing Science Programvaruteknik VT17 Implement – Quality and Quality Assurance Jonny Pettersson http://www.cs.umu.se/kurser/5DV151 Last Time ● Short on implementation ● Systematic testing 24/4 - 17 Programvaruteknik - Jonny Pettersson, UmU Today ● What is Quality? ● What is Quality Assurance? ● Planning Quality ● Inspections 24/4 - 17 Programvaruteknik - Jonny Pettersson, UmU 1

Quality is ... ● … I know it when I see it ● … it suits the client/user ● … it conforms to the specification ● … it has some inherent quality ● … it depends on the price ● … it depends on the delivery date 24/4 - 17 Programvaruteknik - Jonny Pettersson, UmU And Quality is … Sponsor User Low costs Functionality Efficiency Easy to learn Increased productivity Easy to remember Short time of delivery Easy to use Reliability Flexibility Few errors Good design Readable code Good documentation Maintainer/modifier 24/4 - 17 Programvaruteknik - Jonny Pettersson, UmU Quality Factors (ISO 9126) Suitability Accuracy Functionality Interoperability Security Maturity Reliability Fault tolerance Recoverability Understandability Usability Learnability Operability Time behavior Efficiency Resource behavior Analyzability Changeability Maintainability Stability Testability Adaptability Installability Portability Conformance Replaceability 24/4 - 17 Programvaruteknik - Jonny Pettersson, UmU 2

The Quality Triangle ● Different goals for different Lead time projects and organizations (close delivery time) − Medical equipment − Text editor − Game app ➨ ”Good Enough” Software ➨ Quality must be measured goal-oriented Functionality Quality (number of “features”) (absense of errors) Programvaruteknik - Jonny Pettersson, 24/4 - 17 UmU What is Quality Assurance? QA is the combination of planned and unplanned activities to ensure the fulfillment of predefined quality standards. ● Constructive vs. analytic approaches to QA ● Qualitative vs. quantitative quality goals ● Measurement − Derive qualitative factors from measurable quantitative factors − Software measures (metrics) 24/4 - 17 Programvaruteknik - Jonny Pettersson, UmU Approaches to QA ● Constructive Approaches ● Analytic approaches The usage of methods, languages, The usage of methods, languages, and tools that ensure the and tools to observe the current fulfillment of some quality factors. quality level. − Inspections − Syntax-directed editors − Static analysis tools (e.g., − Enforced coding guidelines lint) − Type systems − Testing − ... − ... ● Software Process Improvement The improvement of development processes to yield better products. 24/4 - 17 Programvaruteknik - Jonny Pettersson, UmU 3

How To Measure Quality? subjective Quality Factor Efficiency properties depends on Property/ Property/ Property/ Speed Criteria Criteria Criteria Size determined by objective Response time in s Measure Measure Measure LOC measurements ... 24/4 - 17 Programvaruteknik - Jonny Pettersson, UmU Purpose of Measurement ● Analysis: Determine current quality ● Prediction: Predict future quality ● Not only code can be measured, but also − Other work products ● Vagueness in the requirements ● Amount of documentation − Processes ● Schedule performance ● Defect removal efficiency − Resources ● Effort ● Expenses 24/4 - 17 Programvaruteknik - Jonny Pettersson, UmU Some Example Measures ● To measure efficiency ● To measure reliability − Time behaviour − MTTF (Mean Time To Failure) ● Transactions per second − Availability ● Response time ● Screen refresh time ● To measure robustness − Resource behaviour − Time to restart after a ● KBytes of executables failure ● LOC − Probability of data ● Number of processors corruption on failure ● To measure usability − Training time Some measures might be − Number of help frames indicators for several quality factors Æ Measurement is necessary 24/4 - 17 Programvaruteknik - Jonny Pettersson, UmU 4

Improving Quality Resources Products Processes ● Personnel ● Code ● Methods ● Tools ● Design ● Management ● Budget ● Requirements ● Accounting ● ... ● ... ● ... Ordinary testing Easy to measure 24/4 - 17 Programvaruteknik - Jonny Pettersson, UmU The Message Quality is relative Quality covers more than just code 24/4 - 17 Programvaruteknik - Jonny Pettersson, UmU Quality Assurance ● What is Quality Assurance? ● What is Quality? ● Planning Quality ● Inspections 24/4 - 17 Programvaruteknik - Jonny Pettersson, UmU 5

Quality Planning ● Quality planning is the process of developing a quality plan for a project − Sets out the desired software qualities − How these are to be assessed ● Outline for a quality plan (Humphrey, 1989) − Product introduction − Product plans − Process descriptions − Quality goals − Risk and risk management 24/4 - 17 Programvaruteknik - Jonny Pettersson, UmU Software Quality Attributes ● Potential software quality attributes − Safety, security, reliability, resilience, robustness, understandability, testability, adaptability, modularity, complexity, portability, usability, reusability, efficiency, learnability ● Not possible to optimise for all attributes ● The quality plan should define the most important quality attributes 24/4 - 17 Programvaruteknik - Jonny Pettersson, UmU Quality Levels Can be Planned 50 200 units Defects from 80% 2 Costs from defect prev. activity prev. activity Effectivity Costs 50 Defects Defects Total cost of Accumulated introduced in ”inherited” Defects quality in costs of current from previous deleted current quality for all 100 80 activity activity activity activities 160 Defects to Costs to next next activity activity 20 360 24/4 - 17 Programvaruteknik - Jonny Pettersson, UmU 6

An Example 50% 7,5 1 Requirements inspection 15 0 7,5 8 Review, individual (40%,2) 50% 21,3 3 + Design inspection 35 7,5 64 72 Formal inspection, team (70%,5) 50% 25,6 10 Unit test 30 21,2 256 328 50% 17,8 30 Integration test 10 25,6 534 862 50% 13,9 30 10 17,8 417 1279 System test 100% 13,9 200 Operation 0 13,9 2780 4059 24/4 - 17 Programvaruteknik - Jonny Pettersson, UmU An Example (cont.) 50% 7,5 1 15 0 7,5 8 Requirements inspection 40% 2 Design reviews 35 7,5 17 34 42 70% 17,9 5 Formal design inspection 0 25,5 90 132 50% 18,8 10 Unit test 30 7,6 188 320 50% 14,4 30 Integration test 10 18,8 432 752 50% 12,2 30 10 14,4 366 1118 System test 100% 12,2 200 Operation 0 12,2 2440 3558 Fewer defects to lower 13,9 4059 costs!!! Old QA plan 24/4 - 17 Programvaruteknik - Jonny Pettersson, UmU High Variation in Defect Removal Effectivity ● Individuals testing their own code − Usually < 50% ● Normal testing activities − Often < 70% ● Design- and code inspections − Often > 65% , up to 85% ● Formal inspections plus formal testing activities − Often > 96% , up to 99% ● Formal inspections can lead to up to 30% decreases in costs and lead times 24/4 - 17 Programvaruteknik - Jonny Pettersson, UmU 7

Formal Inspections g distribute materials (documents, process, checklists) Document Preparation g select participants (4–6) Individual review Fix Inspection Report problems meeting re-inspection pass? Y partial Fix Review problems changes Next development phase pass? N Y 24/4 - 17 Programvaruteknik - Jonny Pettersson, UmU Formal Inspections (cont.) ● Effective QA method ● Applicable to all types of work products − Requirements documents − Design documents − Documentation − Code − ... ● Expensive ● Positive side-effects Æ Most effective in early development phases 24/4 - 17 Programvaruteknik - Jonny Pettersson, UmU Optimization of Inspections ● Right mix of participants (number, experience, error profile) ● Right duration ● Right supporting materials (checklists, reading techniques, ...) ● Evaluation of effectivity − Defects found per unit of time − Defect profiles of participants Goal: Stop when desired quality level is reached Problem: How can we know the quality level (defect density) in the input? 24/4 - 17 Programvaruteknik - Jonny Pettersson, UmU 8

Re-Inspections Are Expensive And Must be Avoided ● Compare to benchmarks and historical data − Fewer defects detected ⇒ bad inspection ⇒ re-inspection − More defects detected ⇒ bad input ⇒ re-inspection ● Problem − Very good input ⇒ few defects to detect ⇒ re-inspection − Bad input & bad inspection ⇒ normal ⇒ no re-inspection Æ Develop fault profiles Æ Subjective evaluation by participants Æ Error seeding or capture-recapture 24/4 - 17 Programvaruteknik - Jonny Pettersson, UmU Capture-Recapture ● Used to estimate the size of animal populations − Catch and mark animals − Release marked animals − Catch animals again under similar conditions Æ Estimate population size by means of the relationship between marked and unmarked animals inspector 1 inspector 2 24/4 - 17 Programvaruteknik - Jonny Pettersson, UmU The Message There is more to quality assurance than just testing 24/4 - 17 Programvaruteknik - Jonny Pettersson, UmU 9

Summary ● What is quality ● What is quality assurance ● Planning quality ● Inspections 24/4 - 17 Programvaruteknik - Jonny Pettersson, UmU 10