W15 Track 5/6/2009 3:00:00 PM "End-to-End Testing - PDF document

W15 Track 5/6/2009 3:00:00 PM "End-to-End Testing "End-to-End Testing in an Enterprise Agile Environment" in an Enterprise Agile Environment" Pr Prese esented by d by: : Billie Bell Billie Bell Intuit, Inc. Intuit, Inc. Presented at: 330 Corporate Way, Suite 300, Orange Park, FL 32073 888-268-8770 · 904-278-0524 · sqeinfo@sqe.com · www.sqe.com

Billie Bell Billie J. Bell , E2E QA Program Manager for Intuit, developed the end-to-end (E2E) QA Program Manager role after recognizing the benefits of engaging QA teams across Intuit's primarily agile project teams in understanding how cross-project integration testing influences achievement of overall business objectives. Using the rollout of specific company-wide initiatives provides a pragmatic context and motivation for development of new processes and promotes an E2E quality mindset beyond QA. Billie’s recent accomplishments include coordinating the integration of 30+ applications with a PCI compliance platform, a consolidated entitlement and software license management platform infrastructure integrating desktop and Web service applications, and a 10-layer subscription and billing SAAS initiative integrating 3rd party and internally-developed applications. Including her eight years at Intuit, Billie brings more than 20 years of experience in enterprise financial software development in QA management, product management, and program management roles in both Fortune 500 and start-up software development companies.

� 2/16/2009 Business Dem ands End-to-End Testing in an Enterprise Agile Environment Billie Bell Com plex Intuit, Inc. Technology Technology A il Agile Developm ent 1 This presentation discusses the theory and practice of large scale integration testing An enterprise working to reach middle An enterprise working to reach middle ground between agile and process- Context based methodologies Some practical examples of Best mechanisms Intuit has found to be mechanisms Intuit has found to be practice practice tip effective appear throughout the presentation 2 � 1

� 2/16/2009 Three forces in today’s business environm ent interact to create challenges for quality leaders Accelerated Business Dem ands Agile Developm ent Developm ent Com plex Com plex Technology 3 Despite our best efforts, results can som etim es com e across as grim headlines Accelerated Business v Dem ands Agile Com plex Developm ent Technology 4 � 2

� 2/16/2009 W e expect headlines that reflect our intentions to deliver a high quality release on tim e Accelerated Business v Dem ands Agile Com plex Developm ent Technology 5 Today’s challenges bring opportunities to take a fresh look at quality leadership 6 � 3

� 2/16/2009 Quality leaders are uniquely capable to set the course for success in today’s business setting Community Community organizers for quality Visibility Visibility promoters QA leadership role Expand quality definition Enlist all stakeholders 7 As a com m unity organizer for quality you can build the foundation for end-to-end m indset Expand quality definition Enlist all stakeholders QA leadership role 8 � 4

� 2/16/2009 Start w ith your ow n vision QA leadership role Expand quality definition Enlist all stakeholders 9 Develop shared vision Expand quality definition Enlist all stakeholders QA leadership role 10 � 5

� 2/16/2009 Lead process change across the organization QA leadership role Expand quality definition Enlist all stakeholders 11 A quality leader’s enhanced role should also include prom oting visibility √ Help QA and other teams see their applications in p Q pp the context of the enterprise √ Help the organization stay one step ahead by acknowledging and managing risk Expand quality definition Enlist all stakeholders QA leadership role 12 � 6

� 2/16/2009 Prom oting visibility into business rationale is an elem ent of the new role QA leaders play Business context and relative priority result in more effective testing More of the business transaction is outside a t ti i t id Accelerated single application Business v Dem ands Agile Developm ent Com plex Technology Agile best practices emphasize connection with the customer QA is a business discipline, not just a set of tasks that occur in serial fashion after developm ent 13 Best practice Best practices to prom ote visibility bring tip practicality to the quality com m unity Some examples of ways to promote visibility p y p y √ Weekly 1-hour Spotlight Sessions for overview presentation by √ business process experts √ Enterprise and system architects √ 2-day workshops for more complex, new systems √ Cross-project test case reviews Expand quality definition Enlist all stakeholders QA leadership role 14 � 7

� 2/16/2009 Quality leaders have the perspective to lead the organization in seeing the risks ahead QA leadership role Expand quality definition Enlist all stakeholders 15 Risk assessm ent and m anagem ent by a cross-team netw ork prom otes accountability Use risk planning sessions to 1. Identify risks 2. Identify indicators 3. Plan mitigation 4. Identify response 5. Monitor risks k Expand quality definition Enlist all stakeholders QA leadership role 16 � 8

� 2/16/2009 Best Best practices used to m anage risk include practice tip collaboration and analysis Ways to uncover risks y √ Pre-assessment questionnaire √ Defect analysis from previous projects √ Risk workshops QA leadership role Expand quality definition Enlist all stakeholders 17 W ith a collective understanding of w here w e are, w e can exam ine how w e m easure quality 18 � 9

� 2/16/2009 Extend definition of quality Beyond a single Beyond a single application Across release cycles cycles QA leadership role Expand quality definition Enlist all stakeholders 19 Order System High Level Architectural Diagram Order Sources Paym ent blank blank blank Processing and and W eb Accounting applications Call Center App Call center Processor Legacy applications 20 � 10

� 2/16/2009 Order System High Level Architectural Diagram blank blank blank W eb applications Call Center App Call center Processor Legacy applications 21 The Purchase use case is architecturally nine layers deep Call Center App Processor 22 � 11

� 2/16/2009 The Update custom er inform ation use case spans five layers Business Use Case Update Web App Purchase customer Info A Bill Bill Web App Web App Cancel subscription B C Process Process 1 2 Web Order Processing 3 rd Party Credit Card CC Call Center Authorization Processor C a l l C e n t e r s , App M a i l a n d F a x o r d e r s Order C Engine Engine Settlement Order Processor Source Financial IVR Accounting Customer Information 23 The Cancel use case is 5 layers deep Call Center App C Processor 24 � 12

� 2/16/2009 Bill Subscription use case is executed outside the application Call Center App C Processor 25 Realities of today’s enterprise technology can leave our traditional quality m onitors lacking 26 � 13

� 2/16/2009 W hen it com es to code coverage, the w hole is not the sum of its parts Com ponent Coverage Com ponent 1 Com ponent 2 Com ponent 3 8 0 % 8 0 % 8 0 % 8 0 % x 8 0 % 6 4 % x 8 0 % 8 0 % 6 4 % 5 1 % Process Coverage QA leadership role Expand quality definition Enlist all stakeholders 27 Best practice Payoff for integration analysis from business tip and technical view is m ore robust test cases Integration analysis � Identify systems and system-to-system interfaces � Identify and high level business transactions that are dependent � Identify and high-level business transactions that are dependent on these interfaces or trigger their execution � Analyze the use of data across multiple systems to track data flow and reconciliation � Simplify and communicate all of the above � Use UML diagrams to illustrate functional touchpoints � Summarize and conceptualize the wealth of information � Communicate the big picture, provide maps to details Reporting test execution results by functional use case Expand quality definition Enlist all stakeholders QA leadership role 28 � 14

� 2/16/2009 Onboarding applications post release extend accountability for quality End-to-end testing becomes more like no-end-in-sight testing QA leadership role Expand quality definition Enlist all stakeholders 29 Onboarding a new applications to our m odel m ay appear as a new box on the diagram Web App A Web App New Web App B Process Process 2 Process 2 1 1 Web Order Processing 3 rd Party Credit Card CC Call Center Authorization Processor s C a l l C e n t e r , App M a i l a n d F a x o r d e r s Order Engine Settlement Order sor Source Source Proces Financial IVR Accounting Customer Information Expand quality definition Enlist all stakeholders QA leadership role 30 � 15