BIO PRESENTATION F1 Friday, May 19, 2006 10:00AM T HE L AST P RESENTATION ON T EST E STIMATION Y OU W ILL E VER N EED TO A TTEND Geoff Horne Geoff Horne Testing International Conference On Software Testing Analysis and Review May 15-19, 2006 Orlando, Florida USA
Geoff Horne Geoff Horne is based in New Zealand and has founded and run two testing consultancy companies which grew to enjoy an international clientele. He has over 28 years experience in IT including software development, sales and marketing and IT and project management. In 1994, almost by accident, he found himself involved in testing a complex fault management system, which led to further testing and QA assignments covering a wide range of applications and tools. Geoff’s companies were subsequently founded to bring a full range of testing consultancy services to the IT industry. Latterly, Geoff has focused on a few select clients running complex test projects in a programme test management capacity. Geoff has written a variety of white papers on the subject of software testing and has been a regular speaker at the Star testing conferences. He is married with four children and in his spare time (which there is not a lot of) enjoys writing and recording original contemporary Christian music.
www.ghtest.com
www.ghtest.com The Last Presentation on Test Estimation You Will Ever Need to Attend (I think!) Geoff Horne - Principal
www.ghtest.com Why estimate testing? To endeavour to quantify: • Timetable • Resourcing • Budget • Environments
www.ghtest.com Why is estimating so hard? So many variables… • Product - complexity, availability, stability • Test environment - availability, stability • Issues - volume, severities, time to fix, regression • Personnel - suitability, experience • Test stages - analysis, design, development, execution • Evolving project – re/descoping, priority shifts
www.ghtest.com And what’s the usual response? Something like, and I quote... • “We don’t have that luxury” • “We don’t have time for best practice” • “We’re building a Holden not a Rolls Royce” • “I think you’re over-complicating the exercise” • “We brought you in to save time” • “Ha ha ha” (sic: maniacal laughter)
www.ghtest.com Test estimation Two types: • “Guesstimation” • “Testimation”
www.ghtest.com Examples of “Guesstimation” Based on previous exercises and prior knowledge: • Percentages - of development efforts • Comparative size - to previous or similar exercises • Application metrics - no. of screens, functions etc. • And the one we all love….. you’ve got n people and n weeks!
www.ghtest.com “Guesstimation:” Good starting point however…
www.ghtest.com “Guesstimation:” Only a starting point! • Educated “finger in the wind” • Beware of getting locked into a “guesstimation” • You don’t know what you don’t know • Watch out for the monkey on your back!
www.ghtest.com “Guesstimation” to “Testimation” Applying some science • Start with assumptions • Add what you do know • Allow for what you don’t • Refine as “unknowns” become “knowns” • Develop “testimation” models
www.ghtest.com “Guesstimation” to “Testimation” Key: Gaining buy-in to the process, from: • Project managers • Test team • Other project team leaders • Sponsors • Developers • Business
www.ghtest.com “Testimation” models: Simple model • Based on averages • Assumes some sort of previous metrics to work from • Provides “lump sum” result • Good for structured testing initiatives
www.ghtest.com “Testimation” models: Simple model
www.ghtest.com “Testimation” models: Simple model OK however… • Usually not 8 productive work hours in a day • Test effort cannot always be divided evenly • Test team productivity not always linear • No allowance for complexity or priority • Tester productivity differences • Time estimates are flat
www.ghtest.com “Testimation” models: Simple model OK however… • Time models need to allow for: • Development • Requirements analysis • Test case design • Test script development • Reviews and other overheads
www.ghtest.com “Testimation” models: Simple model OK however… • Time models need to allow for: • Execution • Pretest – test data setup, prerequisites etc. • Execution of script • Retesting • Wait time for issue repair • Documentation and other overheads
www.ghtest.com “Testimation” models: More comprehensive model… • Still based on averages • Still requires some sort of metrics to start, however… • Allows for: different levels of tester multiple complexities multiple time models productivity diminishing returns variables work hours
www.ghtest.com “Testimation” models: More comprehensive model…
www.ghtest.com Sou rce D ocu m e n t s Te st St ra t e g y Test Lifecycle: A N A Te st Pla n Re q u ire m e n t s L D Y E S V E E AN ALYSE L O R P Te st Re q u ire m e n t s E ( re vie w ) V E I X E E D ESI GN W C U T Te st Ca se s M E ( re vie w ) E A D EVELO P U R S E U Te st Scrip t s V R ( re vie w ) I E E W R EPAI R EX ECU TE R E I n cid e n t Te st Log s R P ( re vie w ) E Re p ort s O V R I SI GN O FF T S E
www.ghtest.com Test estimation; “Testimation” An evolutionary process: Stage View Method Activity Viscosity Rough estimate based on Estimate Strategy 40,000’ % of development effort Better estimate based on Estimate Plan 20,000’ expected no. of test cases and modelling Better estimate still, based on Estimate Design 10,000’ test case design and schedules Monitor and Development 5,000’ manage Monitor and Execution sea level manage
www.ghtest.com “Guesstimation” vs “Testimation” Spot the difference: • “Guesstimation” is a good starting point however… • Not what you want to get locked in to • “Testimation” provides quantifiable estimates • Can be used to create buy-in • Refining the “testimation” provides basis for test plan
www.ghtest.com “Testimation” Feedback loop: • Capture test metrics as plan is executed • Regularly measure against estimates • Full analysis at test completion • Use to measure test efficiency • Document for future projects • Learn!
www.ghtest.com Summary • Estimating test effort is an evolutionary process • Modelling provides tangible estimates • Gaining buy-in allows estimate to be refined • Reviewing estimates vs actuals measures accuracy • Feeding back lessons learned improves process
www.ghtest.com
Recommend
More recommend
