[PPT] - Quality Assurance & Standards PSD3 Nov 2013 PSD3 QA Lecture PowerPoint Presentation

SLIDE 1

PSD3

1

QA Lecture

Quality Assurance & Standards

PSD3 Nov 2013

SLIDE 2

PSD3

2

QA Lecture

Overview

 Quality Assurance (QA)

» checking what we are producing

 QA standards

» checking the checking

SLIDE 3

PSD3

3

QA Lecture

Capability Maturity Model

 framework for improving software

engineering processes

 Recommended set of best practices  Organizations are appraised

SLIDE 5

PSD3

5

QA Lecture

SLIDE 6

PSD3

6

QA Lecture

CMM and Quality

 Key feature of CMM is centrality of

process and product quality

 How is this achieved?  Answer: Quality Assurance or QA

SLIDE 7

PSD3

7

QA Lecture

What is Quality Assurance?

 Quality Assurance is “the process of

defining how software quality can be achieved and how the development

rganisation knows that the software

has the required level of quality.” [Sommerville]

SLIDE 8

PSD3

8

QA Lecture

What Kinds of Problem

 defect – failure to meet requirements  fault – condition that lead to a system

failure

 bug – error, mistake or omission in the

software which are faults (lead to failure

r unexpected behaviour)

 Note: quality is not simply the absence

f faults

SLIDE 9

PSD3

9

QA Lecture

Sommerville on Quality Assurance

 for Sommerville:

» Quality Assurance – framework of procedures and standards » Quality Plan – selection and adaptation of procedures and standards for a project » Quality Control – carrying out processes that ensure procedures and standards are followed

 the term ‘Quality Assurance’

sometimes refers to all of the above

SLIDE 10

PSD3

10

QA Lecture

What’s involved in software quality assurance?

 At the level of an institution, company or

standards body

» Develop procedures and standards » Perform certification

– to prove that the QA mechanism used is acceptable and effective

SLIDE 11

PSD3

11

QA Lecture

What’s involved in software quality assurance?

 At the level of a particular project

» Prepare a quality plan – specifying processes, deliverables, measures of quality (metrics, standards) » Carry out quality control – Collect data

 Metrics  Compared to standards

– Conducting reviews

 checking reality against plan and against standards

SLIDE 12

PSD3

12

QA Lecture

What’s involved in software quality assurance?

 At all levels

» Change attitudes – convince staff that quality is important – develop a “quality culture”

SLIDE 13

PSD3

13

QA Lecture

Techniques for Assessing Quality

 progress review

» examination of progress with respect to plans

 quality review

» examination of project artefacts with respect to attributes of quality » more or less structured

 Inspection

» a (technical) review that tries to identify faults and lack of conformity to standards » not led by author » often applies to code

 code walkthrough

» less formal than inspection » author leads

 pair programming  deskcheck or

passaround

 ad hoc review

SLIDE 14

PSD3

14

QA Lecture

Review Dos and Don’ts

 DO make it peer group review, applicable to all stages

f software development

» method of finding faults cheaply » method of training and learning » method of control » method of encouraging “egoless teamwork”

 IT IS NOT

» a problem-solving session – faults should be identified, but solutions should not be patched together ‘in committee’ » a managerial appraisal of personnel

SLIDE 15

PSD3

15

QA Lecture

Code Review

SLIDE 16

PSD3

16

QA Lecture

Standards for Assessment

 Lots of sources of standards

» Standards Bodies » Professional Associations » Technology and Service Providers » Company Standards (e.g., “House Style”)

 Can be based on

» Objective Software Metrics » Conventions and Agreements » Informal Professional Judgements

SLIDE 17

PSD3

17

QA Lecture

reviewing the inspection process

 statistics collected on

» details of items inspected » list of faults found & classification » resources required for re-working » number of people involved & time

 analysis provides

» fault checklists » management reports on effectiveness of inspections

 It’s the process being assessed, not the authors or

inspectors

SLIDE 18

PSD3

18

QA Lecture

What is a Standard?

"A standard is a document approved by a recognized body, that provides, for common and repeated use, rules, guidelines, or characteristics for products, processes or services with which compliance is not mandatory.” 

A Guide to Project Management Body of Knowledge, 1996

SLIDE 19

PSD3

19

QA Lecture

Standards Organisations

 International Organization for

Standardization (ISO)

» non-governmental » develops standards for various technical

fields (more than 11000)

» 120 national members, which are

themselves standards organisations

SLIDE 20

PSD3

20

QA Lecture

Standards Organisations

 BSI (UK)  ANSI (USA)  DIN (Germany)  IEC (International)  Other standard setting bodies include

» IET » BCS » EU » W3C » OMG

SLIDE 21

PSD3

21

QA Lecture

Software Engineering/IT Standards

 ISO/IEEE/IET/W3C/EU have specific standards that

can be used for measuring product and process quality

 E.g.,

» ISO/IEC TR 14471:1999 : Information Technology – Software Engineering – Guidelines for the adoption of CASE tools » ISO 9241: Ergonomics of Human System Interaction

– Includes requirements for keyboard and non-keyboard input devices, workstation layout, displayed colours, menu/command/ direct manipulation and form-filling dialogues

» W3C

– Standard = “recommendation” – currently includes: XML 1.1 (plus XQuery, XPath), XSLT, HTML, XHTML, SOAP, SSML, VoiceXML, OWL, RDF, P3P, SMIL,

SLIDE 22

PSD3

22

QA Lecture

ISO 9000

 international standard for quality management

and quality assurance

» states what must be in a quality management

system

» first established in 1987 » derived from BS 5750, a British standard

 ISO 9001

» applies to products involving design » Latest version is ISO 9001:2008 » 90003 interprets 9001 for software development

SLIDE 23

PSD3

23

QA Lecture

ISO 9000 (cont’d)

 certification is not carried out by ISO

» carried out by independent certification

bodies

» organisation is awarded a Certificate of

Conformity

SLIDE 24

PSD3

24

QA Lecture

What’s in ISO 9000?

 covers 20 topics, including

» Quality System » Design Control » Process Control » Inspection & Testing » Contract Review » Quality Records » Internal Quality Audits » Training

SLIDE 25

PSD3

25

QA Lecture

What’s in ISO 9000? (cont’d)

 standard is abstract; it sets ends not

means

 for example, “The supplier shall establish and maintain a documented quality system as a means of ensuring that product confirms to specified requirements.”   [from Section 4.2 of ISO 9001]

SLIDE 26

PSD3

26

QA Lecture

TickItPlus

 developed by UK and Sweden  provides a nationally accredited

software quality certification scheme

 interpretation of ISO 9000 and related

standards

SLIDE 27

PSD3

27

QA Lecture

www.tickitplus.org

SLIDE 28

PSD3

28

QA Lecture

TickItPlus (cont’d)

 gives concrete guidelines on how

software development should conform to the standard

» based on developing a scheme of internal audits » audits related to standards compliance » auditors require training » audits involve document reviews and staff

interviews

SLIDE 29

PSD3

29

QA Lecture

QA Standards: A Good Thing?

 pro

» makes quality assurance assessable » as of the end of 2009, 1,064,785

rganisations in 161 countries held ISO

9001:2000 certificates

» US survey

– 89% reported greater operational efficiency – 48% reported increased profitability

SLIDE 30

PSD3

30

QA Lecture

QA Standards: A Good Thing?

 con

» fosters “command & control” style of

management

– emphasises inflexible compliance with a set of

rigid written rules

» standards rely heavily on assessors’

judgements

– standards are not completely objective

SLIDE 31

PSD3

31

QA Lecture

QA Standards: A Good Thing?

» staff will pay attention to controls, not the

things affected by the controls

– attention to quality inspection and monitoring

can deflect from attention to quality itself

– like the problem of exams distorting education

Quality Assurance & Standards

PSD3 Nov 2013

Overview

 Quality Assurance (QA)

» checking what we are producing

 QA standards

» checking the checking

Further Reading

 Sommerville

» 9th Edition, Chap. 24 » 7th & 8th Editions, Chap. 27

Capability Maturity Model

 framework for improving software

engineering processes

 Recommended set of best practices  Organizations are appraised

CMM and Quality

 Key feature of CMM is centrality of

process and product quality

 How is this achieved?  Answer: Quality Assurance or QA

What is Quality Assurance?

 Quality Assurance is “the process of

defining how software quality can be achieved and how the development

has the required level of quality.” [Sommerville]

What Kinds of Problem

 defect – failure to meet requirements  fault – condition that lead to a system

failure

 bug – error, mistake or omission in the

software which are faults (lead to failure

 Note: quality is not simply the absence

Sommerville on Quality Assurance

 for Sommerville:

» Quality Assurance – framework of procedures and standards » Quality Plan – selection and adaptation of procedures and standards for a project » Quality Control – carrying out processes that ensure procedures and standards are followed

 the term ‘Quality Assurance’

sometimes refers to all of the above

What’s involved in software quality assurance?

 At the level of an institution, company or

standards body

» Develop procedures and standards » Perform certification

– to prove that the QA mechanism used is acceptable and effective

What’s involved in software quality assurance?

» Prepare a quality plan – specifying processes, deliverables, measures of quality (metrics, standards) » Carry out quality control – Collect data

– Conducting reviews

What’s involved in software quality assurance?

 At all levels

» Change attitudes – convince staff that quality is important – develop a “quality culture”

Techniques for Assessing Quality

» less formal than inspection » author leads

passaround

Review Dos and Don’ts

Code Review

Standards for Assessment

 Lots of sources of standards

 Can be based on

reviewing the inspection process

inspectors

What is a Standard?

"A standard is a document approved by a recognized body, that provides, for common and repeated use, rules, guidelines, or characteristics for products, processes or services with which compliance is not mandatory.”

Standards Organisations

 International Organization for

Standardization (ISO)

fields (more than 11000)

themselves standards organisations

Standards Organisations

» IET » BCS » EU » W3C » OMG

Software Engineering/IT Standards

can be used for measuring product and process quality

ISO 9000

and quality assurance

system

 ISO 9001

ISO 9000 (cont’d)

 certification is not carried out by ISO

bodies

Conformity

What’s in ISO 9000?

 covers 20 topics, including

What’s in ISO 9000? (cont’d)

 standard is abstract; it sets ends not

means

 for example, “The supplier shall establish and maintain a documented quality system as a means of ensuring that product confirms to specified requirements.” [from Section 4.2 of ISO 9001]

TickItPlus

"A standard is a document approved by a recognized body, that provides, for common and repeated use, rules, guidelines, or characteristics for products, processes or services with which compliance is not mandatory.” 

 for example, “The supplier shall establish and maintain a documented quality system as a means of ensuring that product confirms to specified requirements.”   [from Section 4.2 of ISO 9001]