Apollo Group, Inc is the parent organization of University of - PDF document

APOLLO BACKGROUND • Apollo Group, Inc is the parent organization of University of Phoenix and several other educational institutions • Apollo IT develops and tests both internal- and external-facing software for staff, students, and faculty • Focus today is on the methodologies of the I HAVE TWO M ANAGERS?! : ONE COM PANY’S M ODEL FOR A CONSULTATIVE Business Systems Division (BSD) TESTING TEAM AND M ATRIX M ANAGEM ENT • BSD develops and tests software used by staff who Amy Yosowitz support students (admissions, academic counseling, Senior Information Technology M anager, Quality Assurance student records, etc) Apollo Group, Inc. amy.yosowitz@apollogrp.edu TESTING ORGANIZATION CONSULTATIVE M ODEL • Follows consultative model and matrix • QA management assigns testers to a particular management reporting structure development team permanently • Senior QA M anager with one QA M anager and total • Assignment takes into account multiple items: of 25 Software Quality Analysts (SQAs) on team – communication style Common M odel Apollo M odel – business knowledge Pool of resources with generally similar product and Pool of resources, each with varying testing experience testing knowledge and each with specialized product/ business knowledge – technical skill Resources are assigned relatively short-term tasks, then Resources are each dedicated to a development team – SQA’s ability to work independently when complete, are assigned to new testing tasks and perform all testing that team needs on an ongoing basis – style of development team QA management distributes tasks among SQAs QA management mainly serves as a mentor and advocate for quality. SQAs, along wit h their respective dev leads, delineate the testing tasks that need to be done for a particular release cycle of each product. SQAs all sit together in a central area Each SQA sits embedded within the development team they serve Some SQAs create tests, others execute those tests Each SQA is responsible for both test case creation and execution for their product M ATRIX M ANAGEM ENT M ATRIX M ANAGEM ENT • Technically, SQA reports to a QA manager but has a • QA management has more general oversight dotted line to a development team lead • QA managers ensure SQAs are doing the right • Dev lead has overall responsibility for their things to test their applications respective application(s) • QA management mentors SQAs on: • Day-to-day interaction is mainly between SQA and – areas of testing (including design and execution) dev lead – technical skills – understanding requirements – communication skills – creating estimates – system integration points – defining the testing approach – time management – test cases – troubleshooting – raising system issues

M ATRIX M ANAGEM ENT BENEFITS ROLES • Best of both worlds: • There are three primary roles that are the players in this model: – dev lead who knows ins and outs of their particular product – Software Quality Analyst (SQA) – QA manager with QA process expertise and a – Quality Assurance M anager (QA manager) higher level view of all of our systems – Development Team Lead (dev lead) • M ultiple escalation paths for issues, ensuring quality • Ever heard “ I know you haven’t had enough time to test, but we need to send this to production anyway” ? – QA manager and dev lead examine these situations together and work to assess risk and arrive at the best possible outcome SQA ROLE SQA ROLE (CONTINUED) • Often functions as a “ jack of all trades” – part • Logs and helps the dev lead to prioritize bug reports tester, part business analyst, part developer • M akes go/ no-go release recommendations to the • Helps the dev team to understand the user dev lead, but does not have the final decision perspective and business reasoning • Generally has responsibility to protect the • Decides what testing is necessary for new changes production environment and bug fixes • Often has responsibility to organize, script, and/ or • Compiles all test cases and test data perform product demonstrations for business users • M aintains and executes manual and automated • Calculates Defect Detection Rate (DDR) metrics on a regression test scripts regular basis • Creates and maintains automated smoke test • Directly observes end users at least a few hours per scripts, used both in the QA environment as well as month for production release validation QA M ANAGER ROLE QA M ANAGER ROLE (CONTINUED) • M entors SQAs in various areas • Coordinates testing large IT projects • Helps SQAs reach beyond the everyday to provide • Spreads knowledge between development teams, the most value to the organization including: • M ediates conflicts between SQAs and development – best practices • Serves as the advocate for the SQA and for quality – process improvements software in general – problem solutions • Performs all administrative tasks for SQAs • Understands the high-level system architecture, • Addresses any testing or SQA concerns the dev lead data sharing between applications, and raises dependencies between products • Organizes training and mentoring opportunities • M anages the SQA team to work toward larger initiatives like quarterly goals

DEV LEAD ROLE DEV LEAD ROLE (CONTINUED) • Leader and single point of contact for an application • Works with SQA to prioritize testing: • Has ultimate accountability for the quality and – new features success of their application(s) – bug fixes • Represents the application to the corresponding – regression tests business unit(s) • Regularly reviews regression test cases for coverage • M anages and organizes developers and their work and depth • Ensures adherence to team’s chosen processes and • M akes the final go/ no go decision on production standards releases, using information provided by the SQA • Interacts with the SQA on day-to-day activities CORE ELEM ENTS - M EETINGS CORE ELEM ENTS – OTHER PRACTICES • M onthly one hour SQA/ QA M anager meetings • Two-week release cycles (overlapping or consecutive) • Reviews • M onthly ½ hour QA manager/ dev lead meetings – Semi-annual and Annual • SQAs and dev leads encouraged to have monthly – Peer feedback one-on-ones – Collaboration between QA manager and dev lead • Dev team 15-minute daily stand-up (including SQA) • M etrics (Defect Detection Ratio and Day-by-day) • Weekly 90-minute BSD SQA team meeting, led by • SQAs sit among development team QA managers • SQAs attend most meetings with business – Announcements • SQAs do not make final go/no-go decision on releases – “ Bug of the Week” • SQAs move between projects every few years • Quarterly goals – General Topic • Ratio of one SQA for every three developers • M onthly one hour all SQA/ SQE meeting for all of IT M ISCELLANEOUS ELEM ENTS OVERALL M ODEL ADVANTAGES • Application Software Quality Engineer (SQE) team • Each SQA has a great sense of ownership, loyalty, and belonging to their development team and their product. • Database SQE team • SQAs are the subject matter experts and are very much go- • Database developer team separate from application to people on their respective development teams. development team • SQAs are typically the “ right hand man” for their • Human-Computer Interaction (HCI) specialist team development lead. • The dev leads have a high sense of trust in their S QA. • No traditional business analysts • QA is an essential depended-upon part of the organization. • Follow a mix of various agile methodologies • The SQAs get a great variety in their jobs. • There is little to no “ throw the code over the wall” mentality. • We have very few examples of contentious relationships between QA and development.

M ODEL CHALLENGES CONCLUS ION • Difficulty in setting and ensuring adherence to • Consultative model and matrix management standards, such as: reporting have been extremely successful for the Business Systems Division of Apollo IT – test case documentation • SQAs feel very valued and fulfilled in their jobs – automation scripting – very little turnover – telecommuting policy • Organization values quality from the top down • Difficulty in objectively monitoring productivity of SQAs • We enjoy taking part in creating innovative software that fulfills our business goal of serving • SQAs’ focus on one application at the expense of students well understanding multiple applications, their dependencies, and the entire system • Vacations • Keeping up with automated regression QUESTIONS ?