Quality Assurance & Standards PSD3 Nov 2013 PSD3 QA Lecture - PowerPoint PPT Presentation

Quality Assurance & Standards PSD3 Nov 2013 PSD3 QA Lecture 1

Overview  Quality Assurance (QA) » checking what we are producing  QA standards » checking the checking PSD3 QA Lecture 2

Further Reading  Sommerville » 9 th Edition, Chap. 24 » 7 th & 8 th Editions, Chap. 27 PSD3 QA Lecture 3

Capability Maturity Model  framework for improving software engineering processes  Recommended set of best practices  Organizations are appraised PSD3 QA Lecture 4

PSD3 QA Lecture 5

CMM and Quality  Key feature of CMM is centrality of process and product quality �  How is this achieved? �  Answer: Quality Assurance or QA � PSD3 QA Lecture 6

What is Quality Assurance?  Quality Assurance is “ the process of defining how software quality can be achieved and how the development organisation knows that the software has the required level of quality. ” [Sommerville] � PSD3 QA Lecture 7

What Kinds of Problem  defect – failure to meet requirements  fault – condition that lead to a system failure  bug – error, mistake or omission in the software which are faults (lead to failure or unexpected behaviour)  Note: quality is not simply the absence of faults PSD3 QA Lecture 8

Sommerville on Quality Assurance  for Sommerville: � » Quality Assurance – framework of procedures and standards � » Quality Plan – selection and adaptation of procedures and standards for a project � » Quality Control – carrying out processes that ensure procedures and standards are followed �  the term ‘ Quality Assurance ’ sometimes refers to all of the above � PSD3 QA Lecture 9

What ’ s involved in software quality assurance?  At the level of an institution, company or standards body � » Develop procedures and standards � » Perform certification � – to prove that the QA mechanism used is acceptable and effective PSD3 QA Lecture 10

What ’ s involved in software quality assurance?  At the level of a particular project � » Prepare a quality plan � – specifying processes, deliverables, measures of quality (metrics, standards) � » Carry out quality control � – Collect data �  Metrics �  Compared to standards � – Conducting reviews �  checking reality against plan and against standards � PSD3 QA Lecture 11

What ’ s involved in software quality assurance?  At all levels � » Change attitudes � – convince staff that quality is important � – develop a “ quality culture ” � PSD3 QA Lecture 12

Techniques for Assessing Quality  progress review  code walkthrough » examination of progress with » less formal than respect to plans  quality review inspection » examination of project » author leads artefacts with respect to attributes of quality  pair programming » more or less structured  Inspection  deskcheck or » a (technical) review that tries passaround to identify faults and lack of conformity to standards  ad hoc review » not led by author » often applies to code PSD3 QA Lecture 13

Review Dos and Don’ts  DO make it peer group review, applicable to all stages of software development � » method of finding faults cheaply � » method of training and learning � » method of control � » method of encouraging “ egoless teamwork ” �  IT IS NOT � » a problem- solving session – faults should be identified, but solutions should not be patched together ‘ in committee ’ � » a managerial appraisal of personnel � PSD3 QA Lecture 14

Code Review PSD3 QA Lecture 15

Standards for Assessment  Lots of sources of standards � » Standards Bodies � » Professional Associations � » Technology and Service Providers � » Company Standards (e.g., “ House Style ” ) �  Can be based on � » Objective Software Metrics � » Conventions and Agreements � » Informal Professional Judgements � PSD3 QA Lecture 16

reviewing the inspection process  statistics collected on � » details of items inspected � » list of faults found & classification � » resources required for re-working � » number of people involved & time �  analysis provides � » fault checklists � » management reports on effectiveness of inspections �  It’s the process being assessed, not the authors or inspectors � PSD3 QA Lecture 17

What is a Standard? "A standard is a document approved by a recognized body, that provides, for common and repeated use, rules, guidelines, or characteristics for products, processes or services with which compliance is not mandatory. ” 
 � � A Guide to Project Management Body of Knowledge, 1996 � PSD3 QA Lecture 18

Standards Organisations  International Organization for Standardization (ISO) � » non-governmental � » develops standards for various technical fields (more than 11000) � » 120 national members, which are themselves standards organisations � PSD3 QA Lecture 19

Standards Organisations  BSI (UK) �  ANSI (USA) �  DIN (Germany) �  IEC (International) �  Other standard setting bodies include � » IET � » BCS � » EU � » W3C � » OMG � PSD3 QA Lecture 20

Software Engineering/IT Standards  ISO/IEEE/IET/W3C/EU have specific standards that can be used for measuring product and process quality  E.g., » ISO/IEC TR 14471:1999 : Information Technology – Software Engineering – Guidelines for the adoption of CASE tools » ISO 9241: Ergonomics of Human System Interaction – Includes requirements for keyboard and non-keyboard input devices, workstation layout, displayed colours, menu/command/ direct manipulation and form-filling dialogues » W3C – Standard = “recommendation” – currently includes: XML 1.1 (plus XQuery, XPath), XSLT, HTML, XHTML, SOAP, SSML, VoiceXML, OWL, RDF, P3P, SMIL, PSD3 QA Lecture 21

ISO 9000  international standard for quality management and quality assurance � » states what must be in a quality management system � » first established in 1987 � » derived from BS 5750, a British standard �  ISO 9001 � » applies to products involving design � » Latest version is ISO 9001:2008 � » 90003 interprets 9001 for software development � PSD3 QA Lecture 22

ISO 9000 (cont ’ d)  certification is not carried out by ISO � » carried out by independent certification bodies � » organisation is awarded a Certificate of Conformity � PSD3 QA Lecture 23

What ’ s in ISO 9000?  covers 20 topics, including � » Quality System � » Design Control � » Process Control � » Inspection & Testing � » Contract Review � » Quality Records � » Internal Quality Audits � » Training � PSD3 QA Lecture 24

What ’ s in ISO 9000? (cont ’ d)  standard is abstract; it sets ends not means �  for example, � � “ The supplier shall establish and maintain a documented quality system as a means of ensuring that product confirms to specified requirements. ” 
 [from Section 4.2 of ISO 9001] � PSD3 QA Lecture 25

TickItPlus  developed by UK and Sweden �  provides a nationally accredited software quality certification scheme �  interpretation of ISO 9000 and related standards � PSD3 QA Lecture 26

www.tickitplus.org PSD3 QA Lecture 27

TickItPlus (cont ’ d)  gives concrete guidelines on how software development should conform to the standard � » based on developing a scheme of internal audits � » audits related to standards compliance � » auditors require training � » audits involve document reviews and staff interviews � PSD3 QA Lecture 28

QA Standards: A Good Thing?  pro � » makes quality assurance assessable � » as of the end of 2009, 1,064,785 organisations in 161 countries held ISO 9001:2000 certificates � » US survey � – 89% reported greater operational efficiency � – 48% reported increased profitability � PSD3 QA Lecture 29

QA Standards: A Good Thing?  con � » fosters “ command & control ” style of management � – emphasises inflexible compliance with a set of rigid written rules � » standards rely heavily on assessors ’ judgements � – standards are not completely objective � PSD3 QA Lecture 30

QA Standards: A Good Thing? » staff will pay attention to controls, not the things affected by the controls � – attention to quality inspection and monitoring can deflect from attention to quality itself � – like the problem of exams distorting education � PSD3 QA Lecture 31

Tutorial this afternoon  Please sit in teams  Each team will split in half – the inspectors and the inspected  Inspect outcomes from other team’s prototyping sprint last week  Inspection checklist will be available PSD3 QA Lecture 32