T12 November 7, 2002 1:30 PM M AKING A D IFFERENCE WITH T EST A - PDF document

BIO PRESENTATION PAPER T12 November 7, 2002 1:30 PM M AKING A D IFFERENCE WITH T EST A SSESSMENTS Sigrid Eldh Ericsson AB International Conference On Software Testing Analysis & Review November 4-8, 2002 Anaheim, CA USA

Sigrid Eldh Sigrid Eldh has over 20 years experience in the software industry working with a wide range of software in different companies. She is a co-founder of SAST, the Swedish Association for Software Testing and also started ASTA, the Australian Software Testing Association. She has a Masters degree in Computer Science and a degree in Psychotherapy and Organisational behaviour. Sigrid is also on the examination board for ISEB certification of testers, and the chair for the Swedish National Board for the of ISEB certification. She is an author of books in software, testing and management. She currently works as a Test Expert and Manager for Ericsson AB. Sigrid was the program chair of EuroStar’99. Sigrid is a frequent speaker at international testing conferences and has published many papers throughout the world on software testing and related subjects.

Making a Difference with Test Assessments Sigrid Eldh Ericsson AB Sigrid.Eldh@uab.ericsson.se  Sigrid Eldh 1 (24)

Who am I? Test Expert Frequent speaker at test, quality and measurement conferences Author of book(s) ISEB chair in Sweden Starter of Swedsh SAST (and ASTA) Former program chair EuroStar (99) Flower & Garden enthusiast (Takes photos too) Disclaimer • I’m from another part of the world, and I’m not perfect in English (American) or in you culture, thus some advice might not fit you ☺ • I assume (maybe wrongly?) you know testing enough (at least you will after this conference ☺ ) which is a bases for this talk! • Anything I say represent my views not my companies or anyone else´s.  Sigrid Eldh 2 (24)

Overview – The assessments strength • Why Assess? When? How? For what? • Independent vs self-assessments • Improvements • Reach your testing goals • Process, Management, Automation and Test Deliverables • Powerful feedback – and what to do with it • Implementation • Benefits and Side Benefits • How to use the result for yourself and your organisation  Sigrid Eldh 3 (24)

Why Assess? When? For what? • You are biased! You need an ”independent” view – Set of questions, some ”model” to base yourself on • You need confirmation on good and bad, and see possibilities to move forward • Assessing your test will clarify your goals and give you that ”push” for moving forward • An assessment can be done at ”anytime” – Totally new projects/organisations can use it as a “checklist” – More value if organisation is “stable” in some sense • There is time, money and motivation to gain • You can be successful you will improve!  Sigrid Eldh 4 (24)

What to select for improvement • Quick winners – Low cost – Easy actions first – Ongoing actions first – Degree of acceptance – Cost/benefits ratio – Reduce highest risk – Most visible • In synchronization with goals  Sigrid Eldh 5 (24)

Self-Assessment vs Independent Assessor • There are totally different approaches (but same underlying method) • Self- Assessment has some drawbacks – You will be biased – differentiate your view from others – People have pre-conceptual ideas about you (depending on things beyond you, such as organisation, history.....) • .. And some advantages – You know what to aim for – You can talk to ”the others” with support from the assessment – You might get change faster • You show interest and you know where to look  Sigrid Eldh 6 (24)

Conducting an assessment • Define goal for assessment – Scope, when, for whom • Define structure and method – Choose ”model” to use – ”Kick-off meeting” – Document checking – Combined interviews (talk also to non-testers) & dependent organisation, design… – Self assessment of several persons (roles) – Improvements that exists within organisation – Compare Questionnaires – “Assess” • Result Presentation and Report  Sigrid Eldh 7 (24)

How? Step-by-Step • Kick-off meeting • Documentation • Interviews • Questionnaire • Report • Seminar & follow up Improvement Goal, scope Assessment actions &Approach Planning Evaluation Implementation Awareness  Sigrid Eldh 8 (24)

Finding a Model that suits! • CMMI (are great for projects and indirectly support testing, by order and control, but will not focus particularly on testing aspects) – Process area ”Verification & Validation” • TPI – Test Process Improvement - Book by T. Koomen and M. Pol (Addison-Wesley, ISBN 0-201-59624-5) – http://www.iquip.nl/tpi/ has a questionnaire to support the book’s model (My recommendation!) • Any good ”complete” test book (!!!!) – Use content and make a question out of the statements: Example: What Test Techniques are you familiar with and use? Much more work- but is a great competence build-up.....  Sigrid Eldh 9 (24)

What to investigate Scope of methodology Test strategy Communication Life-cycle model • Areas (from TPI): Reporting Moment of involvement – Life-cycle model Defect management Estimating and planning Testware management – Techniques Test specification techniques Test process management Static test techniques – Infra-structure Evaluation Metrics – Organisation Low-level testing Test automation Test environment Office environment Commitment and motivation Test functions and training  Sigrid Eldh 10 (24)

Your role as test assessor • Do the Questions yourself (update after interview) • Understand what they are doing, ask questions on documentation (control-check) – Can use interviews for checking too… • What is NOT talked about (Model in back of your head!) • Ask about improvements each person sees – Open ended question – What is not said (understood, end by confirm?) – Talk about areas: “but what about xxx?” How do you do that? (relevant to your opinion of level) – THEN – hand out self-assessment questions  Sigrid Eldh 11 (24)

The “hidden” ladder of becoming better - Where are you? • Non-existent, ignored, avoided • Awareness (of existence) but have not yet done anything • Tried (but failed) to put practice in place (limited knowledge or not fitted corrected) • Practice or area exists and is used. Basic knowledge in place, just by some projects and people • Improved area, there are small things that have been adjusted, and improvements are made More than one project is using it • There is quality in the practise or area, and other alternatives are evaluated, many projects and people are using it. • The area or practice is optimised and used by all projects (and people)  Sigrid Eldh 12 (24)

More about how to ask X Questions! • Existing? Is there a X? Knowledge about it? • Quality of X? • Do people use X? • What is lacking for X to become better, more useful etc? Example X= Test Strategy! • Do you have a test strategy? Written down? Show me! • Do you believe the test strategy i useful? Does in include all levels of testing? • Are all familiar with the strategy (ex. designers for unit test) • What is lacking for the test strategy to become better?  Sigrid Eldh 13 (24)

Look for the signs of: • Lack of management support? • Lack of cooperation between organisations? • Lack of quality in the result? • Lack of competence in testing? – Areas of training (Particular persons?) • Lack of coherence? • Efficiency? • Cost effectiveness vs. Over-administration?  Sigrid Eldh 14 (24)

Important Findings • Collaborative evidence (non-collaborative) • You will get a summary of improvements • You will ”see” what the organisation do NOT focus on (but use value judgement on why!) • Assess = put together all info in the report based on areas: – Explain where they are – What is missing for next level – Suggestions of improvement (for next level OR?)  Sigrid Eldh 15 (24)

Important findings • Many knows what is important, yet they don’t do it: WHY? – The assessment role – Find out what ”stops them” • Other organisation issues? (”blaming”, ”non-listening” ”management hostage” ”resistance to organisation...”) – Must be from the ”outside” to get a grip of this (or dare say it!) • Technical/Judgemental feedback – Your experience with domain & similarities • Very different views: Yes we do it vs. I didn’t know we did it? Is it a real problem? – Knowledge/Information  Sigrid Eldh 16 (24)

What do a finding really mean? • ”Model” 2 collaborative, one non-collaborative? Philosophical! • Differ YOUR opinion & suggestion vs. Organisations own summary • Who ”owns” the value judgement? – Model – Assessor Decide who knows – Organisation best....  Sigrid Eldh 17 (24)

Follow-up & next step • What will be done with this information? • Why were you doing it in the first place? • How is the organisation to proceed? • Who will be the owner of the next step? • Re-assessment?  Sigrid Eldh 18 (24)