F1 5/20/2005 10:00 AM L EGAL C OMPLIANCE IN Q UALITY A SSURANCE - - PDF document

International Conference On Software Testing Analysis & Review May 16-20, 2005 Orlando, FL USA

F1

5/20/2005 10:00 AM

LEGAL COMPLIANCE IN QUALITY ASSURANCE

Elle Ringham Fidelity National Financial

BIO PRESENTATION PAPER

Elle Ringham, J.D.

Elle Ringham has been involved in Quality Assurance and Quality Management since

• 1990. Ms. Ringham graduated Law School in 2002, and has since incorporated

compliance, auditability, SLA enforcement/measurement, etc. into her Quality Assurance

• practice. Elle considers education of all groups involved, coupled with a structured

process improvement, to be the most effective way to introduce true Quality Assurance/Quality Management. Her approach ensures buy-in and support from everyone… stakeholders, executives, corporate counsel, developers, and QA resources.

Welcome! Welcome! Welcome! Welcome! Legal Compliance in Quality Assurance

Agenda Agenda Agenda Agenda

What this lecture covers… what it doesn’t What is Legal Compliance How QA Fits In Where Do You Start What Do You Ask How Do You Facilitate Compliance and Auditability Templates and Artifacts Contact Information

What This Lecture Covers… What it Doesn’t What This Lecture Covers… What it Doesn’t What This Lecture Covers… What it Doesn’t What This Lecture Covers… What it Doesn’t Will Cover

Determine if compliance issues apply Asking the right questions How to capture and measure auditability

Won’t Cover

HOW to test various legal issues IF a legal issue applies to your application Specific legal advice Specific compliance issues

What is Legal Compliance? What is Legal Compliance? What is Legal Compliance? What is Legal Compliance?

Legal issues: State and Federal Accountability Auditability Legal Counsel Due Diligence Contracts, standards, expectations

Standards

Requirements

Process Other SLA’s Client needs Audit Budget Federal Statutes

How It Fits In How It Fits In How It Fits In How It Fits In

User Acceptance Testing Testing Lab, Multiple platform Standards, Process Improvement Functional and Negative testing System Integration Testing Requirements/Use Cases to Test Cases Defect Tracking Automation and Regression Load/Performance Testing Test Planning

• 1. Bringing QA from a testing group

into (true) Quality Assurance

• 2. Quality Management
• 3. Higher skill set required
• 4. Requires education with

stakeholder and early introduction into project

Where Do You Start Where Do You Start Where Do You Start Where Do You Start

Assess your business need Assess how your application addresses the need Review information with PMO and Stakeholders Research legal issues Discuss findings with counsel Research audit guidelines Assess appropriate QA efforts

Industry, Guidelines Business

Data, Communication, Commerce

Ensure coverage and Understanding

State, Federal, Etc.

Elements of audit

Merge Technology with Audit Process, Metrics, Reporting

What Do You Ask What Do You Ask What Do You Ask What Do You Ask

Stakeholders

What are the known

compliance concerns?

Expectations How are these issues

addressed in other function?

Define known and

foreseeable risks

Mitigation plan for risk(s) Define resources,

locations, tasks and utilization

PMO

What other functional teams work

with compliance

Add tasks into project plan Ensure time added to project plan for

research

Deviations expressed as impacts and

risks; also noted within SQA Test Plan and Testing Report

Ensure time added to project plan for

corporate counsel

Other

How will I add this to the Test Plan How will I audit the elements of the statues

(guidelines, laws, etc.)

What type/form of results will I need to compile Where must the information be stored Must the information be published Is anyone required to review the results; who? Keep Risks and Issues open for upcoming or similar

projects

Corporate Counsel

What are the known compliance concerns? Do we have SLA’s or other contracts to

audit?

What are the elements of the statute, law,

• etc. that we need to audit?

Explain some current case precedent of

these compliance issues

What do you require from other areas of

the company?

Are you familiar with how technology

handles data?

Developers/DBAs

How does the design handle

process/business flow

How is data captured What standards are used for security Ask about design patterns Ask to see all models There will be specific questions

associated with your compliance issues too!

QA Group

Where in the Process does this fit? Who owns this area? How to we capture metrics? Note impacts, risks, mitigation steps taken

How Do You Facilitate Compliance and How Do You Facilitate Compliance and Auditability Auditability How Do You Facilitate Compliance and How Do You Facilitate Compliance and Auditability Auditability

45 45% 25% 25% 20% 20% 10% 10%

Test Cases to Elements of Audit UAT Development Testing Reporting

Templates and Artifacts Templates and Artifacts Templates and Artifacts Templates and Artifacts

Mapped areas of coverage Metrics of coverage per release (functional) Load/Performance Data pools, and negative efforts Standards, Best Practice and Due Diligence

Templates (Cont.) Templates (Cont.) Templates (Cont.) Templates (Cont.)

Sarbanes-Oxley Template

Microsoft Excel Worksheet

Artifacts (Cont.) Artifacts (Cont.) Artifacts (Cont.) Artifacts (Cont.) Example of Sarbanes-Oxley Document

Microsoft Word Document

Contact Information Contact Information Contact Information Contact Information www.SANS.com http://www.developer.com/java/ent/print. php/3320861 http://www.softlanding.com/sox/docs/wo rkingguide.pdf http://www.gain2.org/sox404toolsum.ht m www.FindLaw.com Elle.Ringham@fnf.com

Questions? Questions? Questions? Questions? Thank you for your time! I cannot answer your legal questions. Please seek counsel for your specific needs. Elle Ringham, J.D.

Quality Assurance Office

Legal Compliance in Quality Assurance

Elle Ringham, J.D. Spring 2005 Biography: Elle Ringham has been involved in Quality Assurance and Quality Management since 1990. Ms. Ringham graduated Law School in 2002, and has since incorporated compliance, auditability, SLA enforcement/measurement, etc. into her Quality Assurance practice. Elle considers education of all groups involved, coupled with a structured process improvement, to be the most effective way to introduce true Quality Assurance/Quality Management. Her approach ensures buy-in and support from everyone… stakeholders, executives, corporate counsel, developers, and QA resources. The law and Quality Assurance has been a misunderstood marriage. Using the definitions and practices of law within the detailed, methodical, approached of Quality Assurance, organizations can increase effectiveness of production. It takes a holistic approach to understanding expectations in order to increase the actual (and perceived) level of quality. This is especially true when you marry technology and the law. In the last few decades, The Department of Defense and the Department of Justice have understood the need for this marriage. However, their approach was to find technology issues (be it in the form of risks or dependencies) and adjust our legal system (and responses) accordingly. We in the civilian field aren’t blessed with such a luxury; thus, we educated ourselves on the legal issues and add this information to our process. The following follows the “Who, What, Where, When, How” approach. When one is entering into an unknown domain, “where to start” is often the most difficult question to answer. Use the screen shots as a reference to the class taken. Additional information follows each slide.

Determining whether legal issues apply to your development efforts isn’t always simple. There may be obvious factors: Your efforts are within a well regulated industry, you are aware of Service Level Agreements, you are aware of state or federal agencies which oversee an aspect of your industry… etc. However, it may not be so obvious… you may have an eCommerce site, your portal collects information, you produce propriety software only, etc. Asking the right questions will certainly help, but what you do what the answers is equally

• important. The QA group will now take these answers and create templates for measurement and

metrics, auditability metrics, and reports. Only your corporate counsel will know for sure whether a particular legal issue applies to your

• rganization. Detailed legal advice needs to come from within, not a class or lecture like this.

Although your research should be thorough, and your incorporation of legal elements into the QA process well defined, the actual legal elements are determined by legal counsel and state/federal agencies.

Legal compliance is the taking of a law, statute, etc. and mapping the elements of that law to areas

• f our technology. This can include, but is not limited to, individual elements of development, as

well as the overall architecture, data acquisition, data repositories, security, and archiving. You will be mapping the accountability of various functions to their legal counterparts. These mapped elements allow for one level of auditability. It is with “first pass” that you begin to add depth to the auditability of compliance and software engineering. Planning with your company’s legal counsel involved will become standard practice. Due diligence will mean more than a phase within vendor selection! Legalese is your new second language (third, fourth… fifth). Although this may be new to your group, it will become second nature.

Adding Compliance and Audit brings QA from a Testing group to a true Software Quality Assurance/Quality Management group. It allows a greater degree of mapping and coverage from the standard Requirements Based approach. Your team will require a different skill set than you may have required before the introduction of

• audit. Understanding the law, dissecting elements, and mapping to the technology aren’t easy. Like

most detailed tasks, they require experience and education to perform them effectively. The initial skills you should look for include an advanced reading and comprehension level, advanced degree preferred, and patience. Team members involved in this task should enjoy research, reading, writing, and multiple conversations about the same subject. Although with some internal education, the QA team will also need to educate external functional groups (including stakeholders). Process change is necessary for compliance efforts to be added to the SDLC, and education of all parties makes buy-in and support easier. Begin with the basics: your QA Process, what compliance issues pertain to your development efforts or product, how you can measure compliance, how will you report (metrics), and what their involvement is. Prepare to take the “steps” approach. Small, manageable steps… then onto to larger steps.

Your business (Industry) has areas of compliance or audit in which they must conduct their efforts. Find what these are. Once known, research and discussion with counsel will help identify how this pertains to the development efforts.

Record the answers given. They need to be objective measurable results. Look for “pass/fail” values.

Like all requirements, you will map your test cases to the elements of the law. As an example, the elements of a valid contract include offer and acceptance, consideration, competent parties, proper subject matter, mutual right to remedy, and mutual obligation to perform. Once you know what the elements are, you map you test cases and determine pass/fail criteria. Users, which may include corporate counsel, will perform the User Acceptance Testing aspect. They should validate expectations and auditability. Development efforts may be required in some areas of testing (SOX and Security is one example). Ensure that the testing is documented, including expected and actual results. Finally, it is the reporting aspect that allows for audit control and the element of true compliance. It’s not what you do, or say that you do, but what you can prove you did!

Remember that compliance may reach beyond just a requirements validation. Your Load and Stress efforts may be required; negative testing may be required. This relates to the practice of Due Diligence. Due diligence is used most often in connection with the performance of a professional or fiduciary duty, or with regard to proceeding with a court action. Due care is used more often in connection with general tort actions. Such diligence as a reasonable person under the same circumstances would use. Use of reasonable but not necessarily exhaustive efforts called also reasonable diligence The care that a prudent person might be expected to exercise in the examination and evaluation of risks affecting a business transaction The process of investigation carried on usually by a disinterested third party (as an accounting or law firm) on behalf of a party contemplating a business transaction (as a corporate acquisition or merger, loan of finances, or esp. purchase of securities) for the purpose of providing information with which to evaluate the advantages and risks involved (the greatest exposure…for failure to conduct adequate due diligence arises in the context of public offerings of securities - G.

• M. Lawrence)

The defense (as to a lawsuit) that due diligence was conducted