1 QUALITY ASSURANCE - PROCESS QUALITY METRICS PREREQUISITES Top - - PDF document

1

24.3.2004 Software Engineering 2004 Jyrki Nummenmaa 1

SOFTWARE ENGINEERING SOFTWARE QUALITY

• Today we talk about software process quality and

certification

24.3.2004 Software Engineering 2004 Jyrki Nummenmaa 2

SOFTWARE QUALITY - QUALITY COMPONENTS

• Objective quality component: properties that can

be measured or approximated objectively

• Subjective quality component: customer

satisfaction (”What does the product feel like?”)

• Other: features which can not be (even

subjectively) evaluated at the time. This is related with future events which can not be predicted - unexpected circumstances, changes, etc.

24.3.2004 Software Engineering 2004 Jyrki Nummenmaa 3

THE QUALITY SYSTEM

• Quality control - controlling the way things are

done

• Quality assurance - making sure quality is

achieved

• Quality policy
• Quality planning
• Quality improvement
• These terms come from the standard ISO 8402

24.3.2004 Software Engineering 2004 Jyrki Nummenmaa 4

THE QUALITY SYSTEM (cont’d)

• The quality system of a company is simply the way

the company works and it covers all areas of activity.

• Therefore, a quality system always exists. It may

be documented or not.

• However, in the long run in a big company it

makes a major difference, if the management takes quality seriously and knowingly emphasizes it in all activities.

24.3.2004 Software Engineering 2004 Jyrki Nummenmaa 5

QUALITY CONTROL

• Controlling the software development process
• Standards for the development process, e.g.
• well-defined phases
• checklists
• reviews: what and when
• organisational standards
• Visibility and bookkeeping of the development

process

• Standards for software code and documentation, e.g.
• naming and style
• document skeletons and formats
• different kinds of review and feedback forms

24.3.2004 Software Engineering 2004 Jyrki Nummenmaa 6

QUALITY ASSURANCE

• Improving software quality by monitoring the

products (software) and process

• Ensuring full compliance with the standards for

products and process

• Ensuring that any inadequacies in the product and

process (and standards) are brought to management’s attention.

2

24.3.2004 Software Engineering 2004 Jyrki Nummenmaa 7

QUALITY ASSURANCE - PREREQUISITES

• Top management commitment
• The quality assurance organisation must be

independent from the development organisation.

• The quality assurance organisation must be

properly staffed.

• The quality assurance organisation must co-
• perate properly with the development
• rganisation. Quality must be seen as a common

goal.

24.3.2004 Software Engineering 2004 Jyrki Nummenmaa 8

PROCESS QUALITY METRICS

• Actual vs. estimated time and cost
• Number of faults detected
• Phase of the project when faults were found
• Productivity (e.g. lines of code / man-month)
• Quality expenses (e.g. repairs required after

delivery)

24.3.2004 Software Engineering 2004 Jyrki Nummenmaa 9

QUALITY REVIEW TYPES

• Design or program inspections/audits. These are

commonly driven by a checklist of possible errors.

• Management reviews. These are intended to

provide information for the management about the

• verall progress of the software project.
• Quality reviews. The work of an individual or a

team is reviewed by a panel made up of project members and technical management.

24.3.2004 Software Engineering 2004 Jyrki Nummenmaa 10

A GENERAL SCHEME FOR REVIEWS

• Select the review team.
• Arrange the place and time.
• Distributed the documents.
• Hold the review.
• Note the actions and complete the review forms.

24.3.2004 Software Engineering 2004 Jyrki Nummenmaa 11

POSSIBLE ACTIONS FOR FINDINGS IN REVIEWS

• No action. Some kind of an anomaly was found,

but it was not cost-effective to fix it.

• Refer for repair. The team or individual in charge
• f the product has to correct the fault.
• Reconsider the overall design. It may be more

reasonable to fix other components of the system to solve the problem which was found.

24.3.2004 Software Engineering 2004 Jyrki Nummenmaa 12

INSPECTIONS / AUDIT SESSIONS

• The ”Fagan method” - named after an IBM

employee who pioneered the technique

• All types of defects are noted - not just logic or

function errors.

• Inspections can be carried out by collagues at all

levels except the very top.

• Inspections are carried out using a predefined set
• f steps.
• Inspection meetings do not last for more than two

hours.

• The inspection is lead by a moderator who has a

specific trainging in the technique.

3

24.3.2004 Software Engineering 2004 Jyrki Nummenmaa 13

INSPECTIONS / AUDIT SESSIONS (continued)

• The other participants have defined roles. For

example, one person will act as a recorder and note all defects found and another will act as a reader who takes the other participants through the document under inspection.

• Checklists are used to assist the fault-finding

process.

• Material should be inspected at an optimal rate of

about 100 lines per hour.

• Statistics are maintained so that the effectiveness
• f the inspection process can be monitored.

24.3.2004 Software Engineering 2004 Jyrki Nummenmaa 14

A CHECKLIST FOR REVIEW EVALUATIONS

• Is material complete (and does it meet the

standards)?

• Was material distributed on time?
• Are applicable standards referenced and available?
• Were attendees prepared to contributed?
• Did evaluation start on time?
• Were number of defects identified?
• Was review conducted per standard protocols?
• Were entrance criteria met?
• Were exit criteria met?

24.3.2004 Software Engineering 2004 Jyrki Nummenmaa 15

A CHECKLIST FOR REVIEW EVALUATIONS (cont’d)

• Are all issues scheduled for resolution?
• Are interface issues coordinated?
• Is disposition of all defects complete?
• Were HCI, testing, maintenance, and tools

considered?

• Were quality attributes reported?
• Has trace of defects been initiated?

24.3.2004 Software Engineering 2004 Jyrki Nummenmaa 16

SOFTWARE QUALITY CIRCLES

• A Japanese practice.
• A quality circle is a group of four to ten volunteers

working in the same area who meet for, say, an hour a week to identify, analyse and solve their work-related problems.

• One of the group is a group leader and one

(maybe from outside) is a facilitator giving advice.

• Training is required, and so is full support from the

management.

24.3.2004 Software Engineering 2004 Jyrki Nummenmaa 17

PROBLEM SOLVING BY SOFTWARE QUALITY CIRCLES

• Identify a list or problems and choose one.
• Clarify the problem.
• Identify and evaluate the causes.
• Identify and evaluate the solutions.
• Decide on the solution.
• Develop an implementation plan.
• Present the plan to the management.
• Implement the plan.
• Monitor the plan.
• Consider wider apllicability of solution.
• Restart from choosing another problem.

24.3.2004 Software Engineering 2004 Jyrki Nummenmaa 18

ISO 9000

• Series of standards for general use in different

fields of industry.

• ISO 9001 is for companies with product

development and production.

• Separate certification organisations give

certification for companies which fulfill the requirements of the standards.

4

24.3.2004 Software Engineering 2004 Jyrki Nummenmaa 19

INGREDIENTS OF ISO 9001

• Management responsibility
• Quality system
• Contract review
• Design control
• Document control
• Purchasing
• Purchaser supplied control
• Product identification adn traceability
• Process control
• Inspection and testing

24.3.2004 Software Engineering 2004 Jyrki Nummenmaa 20

INGREDIENTS OF ISO 9001 (continued)

• Inspection, measuring and test equipment
• Inspection and test status
• Control of nonconforming product
• Corrective action
• Handling, storage, packaging and delivery
• Quality records
• Internal quality audits
• Training
• Servicing
• Statistical techniques

24.3.2004 Software Engineering 2004 Jyrki Nummenmaa 21

AN OVERVIEW OF ISO 9001 REQUIREMENTS

• The management must define and document the

policy concerning quality and must ensure that this policy is communicated to all levels of the

• rganisation.
• All quality control procedures must be

documented.

• All contracts to supply goods or services should

contain mutually agreed requirements that the developer is capable of delivering.

• There should be procedures to approve design and
• ther documentation.

24.3.2004 Software Engineering 2004 Jyrki Nummenmaa 22

AN OVERVIEW OF ISO 9001 REQUIREMENTS (continued)

• Where components of the system to be supplied to

the client are obtained from third parties there must be procedures to ensure, check, and maintain the quality of these components.

• Individual products should be identifiable as

should their components.

• The process by which the final product is created

should be planned and monitored.

• Inspection and testing should take place during

the development phase, at its completion and before delivery. Tests and inspections should also be carried out on components obtained from third parties.

24.3.2004 Software Engineering 2004 Jyrki Nummenmaa 23

AN OVERVIEW OF ISO 9001 REQUIREMENTS (continued)

• The equipment used in the production process

itself should be properly controlled with respect to quality.

• The testing status of all components and systems

should be clearly recorded all times.

• Care must be taken to ensure that items which are

known to be defective are not inadvertently used.

• When a defect is detected, measures must be

undertaken to remove the defective part and to ensure that the defect does not occur again.

• Satisfactory procedures must be in place to deal

with correct handling, storage, packaging, and delivery of the product.

24.3.2004 Software Engineering 2004 Jyrki Nummenmaa 24

AN OVERVIEW OF ISO 9001 REQUIREMENTS (continued)

• Sufficient records must be maintaned to

demonstrate that the quality system is working satisfactorily.

• The software quality management system should

be audited on a regular basis.

• Servicing and supprt activiies must be subject to

the quality management system.

• The developer must establish appropriate

statistical techniques to verify the acceptability of the final product.

5

24.3.2004 Software Engineering 2004 Jyrki Nummenmaa 25

CERTIFICATION (for standards like ISO 9001)

• The company familiarises itself with the standards.
• The company applies for the certification.
• The certification organization inspects the

company (based on documents which it requires from the company).

• An evaluation meeting is held with representatives

from both the company and the certification

• rganisation.
• If necessary, some corrective actions are

performed.

• A certification is given to an accepted company.
• Re-evaluations will be performed.

24.3.2004 Software Engineering 2004 Jyrki Nummenmaa 26

GENERAL CONCLUSIONS

• The development of a quality system requires

changes and changes create resistance.

• Be realistic.
• Quality depends a lot on attitudes.
• Internal and management motivation is essential.
• Small steps can be the best way forward.