Learning objectives Understand the purposes of planning and - PowerPoint PPT Presentation

Learning objectives • Understand the purposes of planning and monitoring • Distinguish strategies from plans, and Planning and Monitoring the Process understand their relation • Understand the role of risks in planning • Understand the potential role of tools in monitoring a quality process • Understand team organization as an integral part of planning (c) 2007 Mauro Pezzè & Michal Young Ch 20, slide 1 (c) 2007 Mauro Pezzè & Michal Young Ch 20, slide 2 What are Planning and Monitoring? Quality and Process • Planning: • Quality process: Set of activities and responsibilities – Scheduling activities (what steps? in what order?) – focused primarily on ensuring adequate dependability – concerned with project schedule or with product usability – Allocating resources (who will do it?) • A framework for – Devising unambiguous milestones for monitoring – selecting and arranging activities • Monitoring: Judging progress against the plan – considering interactions and trade-offs – How are we doing? • Follows the overall software process in which it is embedded • A good plan must have visibility : – Example: waterfall software process ––> “V model”: unit testing – Ability to monitor each step, and to make objective starts with implementation and finishes before integration judgments of progress – Example: XP and agile methods ––> emphasis on unit testing – Counter wishful thinking and denial and rapid iteration for acceptance testing by customers (c) 2007 Mauro Pezzè & Michal Young Ch 20, slide 3 (c) 2007 Mauro Pezzè & Michal Young Ch 20, slide 4

Example Process: Cleanroom Example Process: Cleanroom Customer Requirements Customer Requirements Integrated into an Activities and overall development responsibilities Specification Specification process focused on quality Function Usage Function Usage Incremental Incremental Functional specifications Usage specifications Functional specifications Usage specifications Development Development Planning Planning Formal Design Statistical test case Formal Design Statistical test case Correctness Verification generation Correctness Verification generation Source Source code Test cases Test cases code Statistical testing Statistical testing Interfail times Interfail times Improvement Feedback Improvement Feedback Quality Certification Model Quality Certification Model MTTF statistics MTTF statistics (c) 2007 Mauro Pezzè & Michal Young Ch 20, slide 5 (c) 2007 Mauro Pezzè & Michal Young Ch 20, slide 6 Example Process: Software Reliability Software Reliability Engineering Testing Engineering Testing (SRET) (SRET) Define “Necessary” Define “Necessary” Activities and Reliability Reliability responsibilities focused on quality Development Development Operational Profiles Operational Profiles Prepare Prepare Prepare Prepare Integrated into an for Testing for Testing for Testing for Testing overall development Interpret Failure Interpret Failure process Execute Execute Data Data tests tests Requirements and Design and System Test and Requirements and Design and System Test and Architecture Implementation Acceptance Test Architecture Implementation Acceptance Test (c) 2007 Mauro Pezzè & Michal Young Ch 20, slide 7 (c) 2007 Mauro Pezzè & Michal Young Ch 20, slide 8

Example Process: Extreme Programming Extreme Programming (XP) (XP) Activities and Incremental responsibilities Incremental Next version Next version Release Release focused on quality Review, Review, Refine, Refine, Passed all unit tests Passed all unit tests prioritize pass prioritize pass Pair Pair Generate User Create Unit Passed all Acceptance Create Unit Passed all Acceptance Generate User Programming Programming unit tests unit tests Stories Tests Testing Tests Testing Stories + unit testing + unit testing Failed acceptance test Failed acceptance test Create Create Integrated into an Acceptance Acceptance overall development Tests Tests process (c) 2007 Mauro Pezzè & Michal Young Ch 20, slide 9 (c) 2007 Mauro Pezzè & Michal Young Ch 20, slide 10 Overall Organization Verification Steps of a Quality Process for Intermediate Artifacts • Key principle of quality planning • Internal consistency checks – the cost of detecting and repairing a fault increases – compliance with structuring rules that define “well-formed” artifacts of that type as a function of time between committing an error and detecting the resultant faults – a point of leverage: define syntactic and semantic rules thoroughly and precisely enough that many common errors • therefore ... result in detectable violations – an efficient quality plan includes matched sets of • External consistency checks intermediate validation and verification activities – consistency with related artifacts that detect most faults within a short time of their – Often: conformance to a “prior” or “higher-level” specification introduction • Generation of correctness conjectures • and ... – Correctness conjectures: lay the groundwork for external – V&V steps depend on the intermediate work consistency checks of other work products products and on their anticipated defects – Often: motivate refinement of the current product (c) 2007 Mauro Pezzè & Michal Young Ch 20, slide 11 (c) 2007 Mauro Pezzè & Michal Young Ch 20, slide 12

Test and Analysis Strategy Strategies vs Plans • Lessons of past experience Strategy Plan – an organizational asset built and refined over time Scope Organization Project • Body of explicit knowledge – more valuable than islands of individual competence Structure Organization structure, Standard structure – amenable to improvement and content experience and policy prescribed in based on over several projects strategy – reduces vulnerability to organizational change (e.g., loss of key individuals) Evolves Slowly, with Quickly, adapting to • Essential for organization and policy project needs – avoiding recurring errors changes – maintaining consistency of the process – increasing development efficiency (c) 2007 Mauro Pezzè & Michal Young Ch 20, slide 13 (c) 2007 Mauro Pezzè & Michal Young Ch 20, slide 14 Considerations in Fitting a Strategy to Elements of a Strategy an Organization • Structure and size • Common quality requirements that apply to all – example or most products • Distinct quality groups in large organizations, overlapping of roles – unambiguous definition and measures in smaller organizations • greater reliance on documents in large than small organizations • Set of documents normally produced during the • Overall process quality process – example • Cleanroom requires statistical testing and forbids unit testing – contents and relationships – fits with tight, formal specs and emphasis on reliability • Activities prescribed by the overall process • XP prescribes “test first” and pair programming – fits with fluid specifications and rapid evolution – standard tools and practices • Application domain • Guidelines for project staffing and assignment – example of roles and responsibilities • Safety critical domains may impose particular quality objectives and require documentation for certification (e.g,RTCA/DO-178B standard requires MC/DC coverage) (c) 2007 Mauro Pezzè & Michal Young Ch 20, slide 15 (c) 2007 Mauro Pezzè & Michal Young Ch 20, slide 16

Test and Analysis Plan Main Elements of a Plan Answer the following questions: • Items and features to be verified • What quality activities will be carried out? – Scope and target of the plan • Activities and resources • What are the dependencies among the quality activities and between quality and other – Constraints imposed by resources on activities development activities? • Approaches to be followed • What resources are needed and how will they – Methods and tools be allocated? • Criteria for evaluating results • How will both the process and the product be monitored? (c) 2007 Mauro Pezzè & Michal Young Ch 20, slide 17 (c) 2007 Mauro Pezzè & Michal Young Ch 20, slide 18 Quality Goals Task Schedule • Initially based on • Expressed as properties satisfied by the product – quality strategy – must include metrics to be monitored during the – past experience project • Breaks large tasks into subtasks – example: before entering acceptance testing, the – refine as process advances product must pass comprehensive system testing • Includes dependencies with no critical or severe failures – among quality activities – not all details are available in the early stages of – between quality and development activities development • Guidelines and objectives: – schedule activities for steady effort and continuous progress • Initial plan and evaluation without delaying development activities – based on incomplete information – schedule activities as early as possible – increase process visibility (how do we know we’re on track?) – incrementally refined (c) 2007 Mauro Pezzè & Michal Young Ch 20, slide 19 (c) 2007 Mauro Pezzè & Michal Young Ch 20, slide 20