"Test Automation Success: Choosing the Right People and - - PDF document
W8 Track 4/28/2010 1:45 PM "Test Automation Success: Choosing the Right People and Process" Presented by: Kiran Pyneni Aetna, Inc. Brought to you by: 330 Corporate Way, Suite 300, Orange Park, FL 32073 888 268 8770 904
330 Corporate Way, Suite 300, Orange Park, FL 32073 888‐268‐8770 ∙ 904‐278‐0524 ∙ sqeinfo@sqe.com ∙ www.sqe.com
As senior QA manager at Aetna, Kiran Pyneni is responsible for improving test organization efficiencies through automation. Kiran leads a team of innovative automation enablers responsible for driving automation across Aetna. Prior to joining Aetna, Kiran had several roles: software developer, senior tester, testing architect, QA lead, and delivery manager for Fortune 500 organizations. Kiran was an invited speaker at the HP Software Universe 2009 USA Conference speaking about automation testing.
Automation Team Company Background Automation Objectives Automation Lifecycle Strategy Team Delivery Team Benefits of Teams Strategy Our Journey… Automation Objectives Automation Lifecycle Metrics Of Our Success Questions Automation Coverage, Efforts Automated, Utilization & Cost Avoidance Benefits of Process Integration
companies, serving members with information and resources to help them make better informed decisions about their health care.
P d t d t P
– Aetna offers a broad range of insurance and employee benefits products. – The first national, full-service health insurer to offer a consumer-directed health plan, Aetna continues to lead the way with its Aetna HealthFund2 line of products, including HSA, HRA and RRA options. – Aetna offers a wide array of programs and services that help control rising employee benefits costs while striving to improve the quality of health care, such as case management; disease and patient safety programs; integrated medical, dental, pharmaceutical, behavioral health and disability information. – Aetna provides members with access to convenient tools and easy-to-understand information that can help them make better-informed decisions about their health information that can help them make better informed decisions about their health and financial wellbeing.
3
Improves efficiencies and quality at our organization by automating consistent and repeatable manual tasks using industry standard technologies / tools / frameworks.
Automation Team
Assessments & Designs Reports & Metrics Standards & Best Practices Automation Development & Maintenance
Automating Right Tasks Sustaining and Growing Automation Suite Utilization of existing automation
Decentralized Approach Centralized Approach
– Easy identification of automation opportunities
– Standardization of automation tool set at enterprise level automation opportunities – Higher collaboration between application & automation teams
– Automation development has become expensive – Usage of different tools & solutions – Standards & Best practices are not shared across applications tool set at enterprise level – Sharing of coding best practices, templates and standards – Cost of automation development is lowered
– Slower automation coverage progression – Lower automated scripts utilization – Funding Challenges
Challenges
Challenges
based on ROI
automation scripts
business teams
Solution
delivery team
Solution
integrate with company's overall software development life cycle
Automation Team
Automation St t T Automation D li T Strategy Team Delivery Team
– To provide automation strategic direction and identify potential new automation opportunities
– Project management experience – Various automation frameworks/tools knowledge – Ability to manage multiple projects in parallel – Very strong written and verbal communication skills – Very strong leadership and analytical skills
– Process & Best Practices – Assessment & Strategies – Design Solutions & Techniques g q – Estimations & Planning – ROI Cost/Benefit Analysis – Reports & Metrics – Tool Technical Support & Analysis
– To develop, deliver, and maintain automation solutions
– Understanding of programming principles and best U de sta d g o p og a g p c p es a d best practices – Knowledge of various automation tools – Knowledge of multiple programming languages – Strong communication skills
– Scripts Development – Scripts Maintenance – Coding Standards – Scripts Planning & Execution p g – Day to Day interactions w/End User
Automation Evaluation
the project / application
Phases
Automation Preparation
needed.
requirements with the help
Automation Generation
Test (BPT) components
BPT components
automation
Execution & Acceptance
G1 G2 G3
processes
environments
details
compatibility
Activities
requirements with the help application demos.
framework / design
milestones
automation tools
Setup BPT components
quality and correctness
components together to create a test script
for BPT components
conduct unit & acceptance testing automation
application team
application team
Re-usable components, Portability, test data grid design, etc.
(BPT) components
scripts
execution results
G1 G2 G3
Deliverables
Automation Strategy Team Automation Delivery Team
Automation Evaluation Automation Preparation Automation Generation Execution & Acceptance
Automation Evaluation Feasibility/Analysis Automation Preparation Design Automation Generation Implementation Execution & Acceptance Testing & Maintenance
35%
Year-2008
42%
Q1-2009
45%
Q2-2009
47%
Q3-2009
35%
Year 2008
52%
Q4-2009
11,750 hrs 13,803 hrs 7,047 hrs 8,570 hrs 10,200 hrs Year As of 2008 As of 2009 2009 only efforts Total Efforts Automated 7047 hrs 13803 hrs 6756 hrs Year-2008 Q1-2009 Q2-2009 Q3-2009 Q4-2009 MR1
Utilization
87% 150% 134% 130%
Cost Avoidance
10,810 hrs Q1-2009 Q2-2009 Q3-2009 Q4-2009 45% Year-2008 Q1-2009 Q2-2009 Q3-2009 Q4-2009 7,200 hrs , 9,618 hrs 9,867 hrs
Slide 17 MR1 Deleted "I&T" from chart
Matt Rosenfeld, 2/1/2010