Evidence in Class Actions Lessons From Recent Cases on the Use of - - PowerPoint PPT Presentation

Statistics in Class Certification and at Trial: Leveraging and Attacking Statistical Evidence in Class Actions

Lessons From Recent Cases on the Use of Statistics to Prove Classwide Liability and Damages

Today’s faculty features:

1pm Eastern | 12pm Central | 11am Mountain | 10am Pacific

The audio portion of the conference may be accessed via the telephone or by using your computer's

• speakers. Please refer to the instructions emailed to registrants for additional information. If you

have any questions, please contact Customer Service at 1-800-926-7926 ext. 10.

TUESDAY, MAY 12, 2015

Presenting a live 90-minute webinar with interactive Q&A Paul G. Karlsgodt, Partner, Baker Hostetler, Denver Brian A. Troyer, Partner, Thompson Hine, Cleveland Edward J. Wynne, Esq., Wynne Law Firm, Greenbrae, Calif.

Tips for Optimal Quality

Sound Quality If you are listening via your computer speakers, please note that the quality

• f your sound will vary depending on the speed and quality of your internet

connection. If the sound quality is not satisfactory, you may listen via the phone: dial 1-866-869-6667 and enter your PIN when prompted. Otherwise, please send us a chat or e-mail sound@straffordpub.com immediately so we can address the problem. If you dialed in and have any difficulties during the call, press *0 for assistance. Viewing Quality To maximize your screen, press the F11 key on your keyboard. To exit full screen, press the F11 key again.

FOR LIVE EVENT ONLY

Continuing Education Credits

For CLE purposes, please let us know how many people are listening at your location by completing each of the following steps:

• In the chat box, type (1) your company name and (2) the number of

attendees at your location

• Click the SEND button beside the box

In order for us to process your CLE, you must confirm your participation by completing and submitting an Official Record of Attendance (CLE Form) to Strafford within 10 days following the program. The CLE form is included in your dial in instructions email and in a thank you email that you will receive at the end of this program. Strafford will send your CLE credit confirmation within approximately 30 days of receiving the completed CLE form. For additional information about CLE credit processing call us at 1-800-926-7926

• ext. 35.

FOR LIVE EVENT ONLY

Program Materials

If you have not printed the conference materials for this program, please complete the following steps:

• Click on the ^ symbol next to “Conference Materials” in the middle of the left-

hand column on your screen.

• Click on the tab labeled “Handouts” that appears, and there you will see a

PDF of the slides for today's program.

• Double click on the PDF and a separate page will open.
• Print the slides by clicking on the printer icon.

FOR LIVE EVENT ONLY

“Conceptual Gaps” and Related Problems: Understanding and Challenging Statistical Evidence in Class Actions

Brian Troyer

Brian.Troyer@ThompsonHine.com

Two Contrasting Uses of “Representative”

• In statistics, a representative sample is one from which population characteristics can be

inferred.

– Representative sample: random selection, sufficient size, population characteristics – Population statistics, including those calculated from samples, support only predictions or estimates about any individual class member or other unit of analysis.

• Under Rule 23, a representative plaintiff must “possess the same interest and share the

same injury.” Wal-Mart Stores Inc. v. Dukes, 131 S. Ct. 2541 (2011).

– In a class action, it must be possible to draw conclusions about individuals. The theory of a class action is that it is fair to draw conclusions because the evidence in the plaintiff’s individual case is the same evidence that would be used in other class member’s cases. Under Rule 23, evidence is representative because it is the same evidence for each class member.

• What evidence would be used in the plaintiff’s individual case (or another’s)?
• Would the evidence in each class members’ case consist of that same evidence?
• Is statistical evidence being offered only to justify class treatment, or would the named plaintiff use that

evidence?

– Aggregate or composite proof is not the same as representative proof, nor is a mere sample of aggregate proof. The important question is whether there is different evidence, not whether there is any similar evidence.

Ecological Fallacy

• This type of fallacy occurs when a conclusion is drawn about an individual from analysis of

group data. Population statistics do not permit such conclusions.

– If seniors at Smart University have a mean LSAT score 5 points above the national average, we cannot conclude that Joe’s score is 5 points above it. Even with additional statistics like distribution and standard deviation calculations, we can only obtain probabilities or estimates about individuals.

• Ecological fallacies are prevalent in many kinds of class actions when statistics are offered

as common proof of conduct, impact, or damages, and population characteristics are imputed to each individual member of a class or other individual unit of analysis.

– If women on average were less likely to be promoted or given raises, Jane Doe suffered discrimination (or there is a common question of discrimination). – If X% of promotion decisions were discriminatory, Jane Doe suffered X such acts of discrimination. – If employees on average worked 2 hours per week overtime, John Doe was denied overtime pay. – If employees worked on average more than 50% in the office, John Doe worked more than 50% in the office and was non-exempt. – If 75% of widgets are 10% out of tolerance, Joe’s widget is 10% out of tolerance. – If ABC product contained on average more than 50% ingredient A, and it should contain 50% or less, there is a common question of whether ABC product is defective.

Statistics and The Illusion of Commonality

• Statistics often show only an illusory form of commonality. An

inference that 51% of a population is affected by a practice

• disproves rather than supports a finding of commonality;
• leaves unanswered which individuals within the population were affected.
• Statistical probabilities cannot supplant direct, individual proof

and do not establish commonality (differences matter).

– How would we ascertain Joe’s actual LSAT score if it was important to know? Look at the score.

• Practice pointer: plaintiffs’ use of statistics or aggregate

evidence cannot properly deprive a defendant of the right to discover and use individual evidence.

Additional Conceptual and Technical Issues

• Erroneous or unsupported assumptions

– All off-label marketing is fraudulent (i.e., off-label use is ineffective). – All class members were unaware the drug was unapproved.

• Statistical measures without reference to any objective standard

– Standard deviation without reference to magnitude of permissible deviation – No objective standards of value similarity or correspondence – No objective definitions of product or performance characteristics or requirements underlying calculation

• Misapplication or misuse of statistical measures

– Standard deviations and/or averages used to portray uniformity or consistency

• Averages by their nature conceal variations.
• Standard deviations do not account for all cases.

Additional Conceptual and Technical Issues

• Correlations that depend upon arbitrary scales

– Data/variables placed on a scale from 0-100% in arbitrary percentage ranges may exhibit correlations dependent upon how the scale is divided.

• Correlations and trends that are artifacts of arbitrary data aggregation

and averaging, concealing individual variables and confounding effects

– In re Graphics Processing Units Antitrust Litig., 253 F.R.D. 478 (N.D. Cal. July 18, 2008) (false correlations created by averaging price changes across products, purchasers, and channels). – Simpson’s Paradox (trend or correlation reverses with combination or separation

• f data sets)
• Implicit statistical inferences denied to be subject to rules of statistics

– Testing a handful out of millions of specimens and drawing conclusions as to all – Selection of “representative” plaintiffs or cases based upon expert judgment (purposive sampling), or improper extrapolation by “subject matter expert”

Additional Conceptual and Technical Issues

• Data collected for different purposes and without accounting for

critical variables

– Were the necessary parameters properly measured or counted? – Double counting, miscounting? – Do the data sources reflect different population subsets or assumptions?

• Sampling from the dependent variable (or no true independent

variable), but extrapolation of inferences to all

– Statistical analyses only of defective product units – Measurement of injury rate only among those known to have actual exposure – Comparison of extent of damage to extent of exposure only in damaged units

Wal-Mart Stores Inc. v. Dukes, 131 S. Ct. 2541 (2011)

• Plaintiffs proposed to prove a general policy of sex discrimination, consisting of

a “common mode” of exercising discretion, through:

– Statistical comparison, by region, of eligible and promoted women (Dr. Drogin); and – Statistical comparison to promotions of women by competitors (Dr. Bendlich).

• These analyses failed to bridge the “conceptual gap” (ecological fallacy):

– Regional and national disparities would not establish uniform, store-by-store discrimination. – Even if they did, they would not demonstrate discrimination at an individual level.

• Plan for “trial by formula” violated the Rules Enabling Act and Due Process

Clause by depriving Wal-Mart of the right to litigate individual defenses.

– District Court proposed summary proceedings on a sample of class members’ claims, followed by extrapolation of aggregate results to the entire class. – Averaging winners and losers? Extrapolating to injured and uninjured without regard to individual facts? – Limits of “trial by formula” are being tested in lower courts.

Comcast Corp. v. Behrend, 133 S. Ct. 24 (2013)

• Econometric analysis of class-wide damages must be consistent with

plaintiffs’ liability and causation theory.

• District Court accepted only one of four theories of antitrust impact

(injury) proffered by plaintiffs’ expert.

• Refused to consider failure of his damage model to distinguish between

impact theories, on grounds that this challenge went to the merits.

• Supreme Court reversed class certification, holding that the lower

courts were required to evaluate whether plaintiffs’ damage model was “consistent with their liability case.”

• Courts must resolve issues necessary to determine whether Rule 23 is

satisfied, regardless of overlap with the merits.

• In the absence of a model “establishing that damages are capable of

measurement on a class-wide basis,” individual damage questions would “overwhelm” the litigation (predominance lacking).

McLaughlin v. American Co., 522 F.3d 215 (2d Cir. 2008)

• Plaintiffs alleged implicit representation that light cigarettes are healthier; sought

$800 billion under RICO and certification of a nationwide class of all purchasers.

• Plaintiffs relied upon sixteen experts, including economists who proposed

statistical and econometric analyses, under a “price impact” theory of reliance similar to the fraud-on-the-market theory.

• Second Circuit reversed certification, finding that individual proof of reliance,

loss causation, injury, damages (and limitations) was required:

– Market for light cigarettes is not efficient (econometric method inappropriate). – Individual evidence presented to show lack of reliance by customers. – Expert’s survey evidence regarding consumer reliance “pure speculation.” – Statistical analysis did not prove the relevant facts but actually assumed reliance. – Rejected “fluid recovery” approach of awarding aggregate “class” damages followed by “simplified proof of claim procedure” and cy pres.

In re Neurontin Sales & Mkt’g Practices Litig.

• Alleged off-label promotion, RICO and NJCFA claims
• Problem: commonality of question of causation by allegedly

fraudulent marketing

• In the first opinion denying class certification, 244 F.R.D. 89

(D. Mass. 2007), plaintiffs were given the opportunity to show through “statistical proof” that essentially all prescriptions in each prescription category were caused by fraud.

• Second class certification motion also denied, 257 F.R.D. 315

(D. Mass. 2009), vacated and remanded, 712 F.3d 60 (1st Cir. 2013).

– Not an efficient market – Defendant’s right to present evidence defeats predominance

In re Neurontin Sales & Mkt’g Practices Litig.

• District Court’s denial of consumer class

– Certification denied for indications with less than substantially all sales caused by fraud.

• Impossible to identify affected sales

– Certification denied for indications allegedly > 99% caused by fraud, because of faulty methodology.

• False assumption that all “detailing” was off-label and fraudulent
• Plaintiffs’ physicians were not even detailed
• Failure/inability to account for factors other than fraud
• District Court’s denial of TPP class

– Certification denied because differences between TPP knowledge and reimbursement decisions required individual inquiries to determine the percentage of payments by each TPP that were fraud-induced.

In re Neurontin Sales & Mkt’g Practices Litig.

 After trial of individual TPP claims, the Third Circuit vacated and remanded for consideration whether its holding that statistical proof was sufficient to avoid summary judgment on proximate and but-for causation on an individual claim would change the class certification

• result. Harden Mf’g Corp. v. Pfizer, Inc., 712 F.3d 60 (1st Cir. 2013).

 Held economist’s statistical analysis purporting to show percentages of prescriptions induced by fraud sufficient to show causation.  Credited economist’s opinion that physician testimony about factors in prescribing decisions is unreliable.  Most courts have reached contrary conclusions about whether similar statistical analyses can be used to prove causation and reliance in pharmaceutical marketing cases.

UFCW Local 1776 & Participating Health & Welfare Fund v. Eli Lilly & Co., 620 F.3d 121 (2d Cir. 2010) (In re Zyprexa)

• “Excess price” analysis could not provide common proof of

– but-for (transactional) causation, because drug pricing is inelastic; – proximate (direct) causation,

• faulty assumption that physicians prescribe based on TPP negotiated prices
• variations in TPP price negotiations showed the causal chain was incomplete.
• “Excess sales” theory could not provide common proof of causation

because, e.g.,

– it assumed away all other factors affecting prescriptions; – it ignored individualized evidence of non-reliance; – TPPs likely paid for different percentages of off-label prescriptions; and, – it ignored alternative prescriptions and costs, some of which could even have cost more than Zyprexa.

Rhodes v. E.I. Du Pont de Nemours and Co., 253 F.R.D. 365 (S.D.W.V. 2008)

• The court denied certification of medical monitoring claims based on

contamination of drinking water.

• Selected deficiencies in toxicologist’s and epidemiologist’s quantitative opinions
• ffered to establish common proof:

– Significant Exposure

• The background exposure level remained unknown (no reference standard).
• Analysis instead was focused on risk level (conflating legal elements).

– Significantly Increased Risk of Disease

• They relied on data showing a conservative, safe level of exposure rather than a level causing

significantly increased risk.

• They could not show that any individual suffered a significantly increased risk, only some

higher population risk.

• They relied upon preliminary research data and data collected for regulatory purposes.
• They failed to rule out other variables affecting the claimed health effects.

Duran v. U.S. Bank N.A., 325 P.3d 916, 172

• Cal. Rptr. 3d 371(Cal. 2014)
• Plaintiffs claimed unpaid overtime (misclassification).
• Defendant classified them as outside salespersons (who customarily

and regularly work more than half their time away from office).

• This exemption defense is based “first and forest on how employee

actually spends his or her time.”

• The trial court certified a class of 260 and selected its own

nonrandom sample of 20 for trial, excluding all other evidence, and extrapolated liability and damages to the entire class.

• It concluded that entire class was misclassified, without input from

experts, and when plaintiffs’ expert calculated a 13% margin of error in extrapolation to all class members (i.e., a minimum 87% misclassified).

Duran v. U.S. Bank N.A., 325 P.3d 916, 172

• Cal. Rptr. 3d 371(Cal. 2014)
• The California Supreme Court reversed, finding that the trial plan improperly extrapolated

liability and damages to all class members from a small, biased sample.

– Too small – Not random – Large margins of error (in calculation of averages)

• The trial plan also failed to allow an opportunity to litigate affirmative defense.
• “Statistical methods cannot entirely substitute for common proof ….”
• “Class actions do not create a ‘requirement of common evidence.’ Instead, class litigation

may be appropriate if the circumstances of a particular case demonstrate that there is common evidence.”

• While statistical proof can be useful to prove wrongdoing in pattern and practice, securities,
• r mass tort cases, “[t]his rationale for aggregate proof simply has no application in wage

and hour litigation alleging misclassification.”

• Statistical methods must be compatible with the nature of the claims and defenses.

PART II: PLAINTIFF STRATEGIES FOR LEVERAGING STATISTICAL SAMPLING

Duran v. U.S. Bank, N.A. (2014) 59 Cal.4th 1 “We encounter here an exceedingly rare beast: a wage and hour class action that proceeded through trial to verdict.”

Edward J. Wynne, Esq. Wynne Law Firm ewynne@wynnelawfirm.com

Plaintiffs’ claim and Defendant’s affirmative defense

• Plaintiffs claimed Defendant’s employees with the title

Business Banking Officer (BBO) were misclassified as exempt and therefore owed overtime compensation. Defendant claimed the employees were exempt from

• vertime compensation under California’s outside sales

exemption.

• California’s outside sales exemption: “any person…who

customarily and regularly works more than half the working time away from the employer's place of business selling tangible or intangible items or obtaining orders or contracts for products, services or use of facilities.” California Industrial Welfare Commission Wage Order No. 4–2001,

• subd. 2(M).

23

Procedural History

• Class certified and summary adjudication of Defendant’s defenses of the

administrative and commission sales exemptions.

• Trial planning:

– Plaintiffs: bifurcated trial based on a random sample of class members. Size of random sample determined from survey of entire class. – Defendant: every class member must testify, non-random selection of any sample, or mini-trials of all before a special master. Defendant claimed it had a due process right examine every class member. – Ruling: bifurcated trial based on a random sample but size of sample determined by Court. Court decided a sample of 20. Court rejected Defendant’s claim that it had a due process right to examine every class member and restricted evidence to the trial sample.

24

Procedural History

• Plaintiffs dismissed their legal claims and went to trial
• nly on their equitable claims. California Business &

Professions Code § 17200.

• Notice to class of dismissal and opportunity to opt-out

per California Rule of Court 3.770. Nine people opted-

• ut including four who were randomly selected.
• One of the randomly selected class members had been

incorrectly designated in Defendant’s HR records as a BBO when he was not. Plaintiffs’ motion in limine to exclude was granted.

25

Trial: liability and damages phases

• 19 of the 20 randomly selected class members testified

at trial including the two named plaintiffs. All 21 were found to have been misclassified and found to have worked overtime. Some of the findings were in the form of a range of hours.

• Liability: Plaintiffs’ expert testified that at a 95%

confidence interval, the point estimate of the balance

• f the class being misclassified was 100% with a 13%

margin of error.

• Damages: Plaintiffs’ expert testified that the weighted

average overtime hours was 11.86 per week with a margin of error of 5.14 hours or 43.3%.

26

Supreme Court opinion

Endorsing the use of statistics to prove liability and damages.

• Liability: “[I]f sufficient common questions exist to support class

certification, it may be possible to manage individual issues through the use of surveys and statistical sampling.” Duran at 31.

– Rejecting Defendant’s argument that statistics cannot be used to prove liability but cautioned that (1) any plan must allow a defendant to litigate their affirmative defenses, and (2) the model must accommodate case-specific

• deviations. Duran at 40.
• Damages: “The use of statistical sampling to prove damages in overtime

class actions is less controversial.” Duran at 40 citing Anderson v. Mt. Clemens Pottery Co. (1946) 328 U.S. 680 [relaxed standard of proof for damages that need only be established by a “just and reasonable inference”].

27

Principles For Use of Representative Evidence Per Duran

• Expert testimony. The court must make a

“reasoned, informed decision about manageability at the certification stage…” Duran at 29, emph. added.

• Rigorous – not just a proposal but a plan. A

“statistical plan for managing individual issues must be conducted with sufficient rigor.” Duran at 31.

• Presented at certification. “[T]rial courts would

be well advised to obtain such a plan before deciding to certify a class action.” Duran at 32.

28

Principles

• Variability. “[A] preliminary assessment should be done

to determine the level of variability in the class.” Duran at 33.

– Random sampling of data before sample chosen. – Discovery of relevant data.

• Sample size. “A sample must be sufficiently large to

provide reliable information about the larger group.” Duran at 42.

– Level of variability will be quantified by the margin of error. – Cost and benefit of sample size. Federal Judicial Center, Reference Manual on Scientific Evidence, 3d. ed., p. 246.

29

Principles

• Random. “A sample must be randomly

selected for its results to be fairly extrapolated to the entire class.” Duran at 43.

– Select after class is set. – Vet the class list prior to selection. – Ensure everyone testifies. – Only randomly drawn evidence.

30

Principles

• Margin of error. Cannot be “intolerably high.”

Duran at 46.

– Duran rejected 43% margin of error on damages. – Bell v. Farmers Ins. Exchange (2004) 115 Cal.App.4th 715 approved a 10% margin of error and rejected a 32% margin of error on damages. – Bolstering factors per Bell: high response rate, probable distribution within the margin of error, absence of measurement error, consideration of other alternatives.

31

Post-Duran Opinions

• Martinez v. Joe’s Crab Shack Holdings (2014) 231 Cal.App.4th 362.

Denial of certification reversed in misclassification case finding that statistical sampling is relevant to the liability determination but rejecting plaintiffs’ plan because it “lacked the specificity contemplated by Duran.”

• Cochran v. Schwan’s Home Service, Inc. (2014) 228 Cal.App.4th 1137.

Reversing denial of certification in Calif. Labor Code sec. 2802 case and directing trial court to apply the principles in Duran regarding statistical sampling.

• Jimenez v. Allstate Ins. Co. (9th Cir. 2014) 765 F.3d 1161. Affirming

grant of class certification in off-the-clock case. Representative evidence to determine defendant’s liability was consistent with due process but not with respect to damages. Bifurcation “preserved both Allstate's due process right to present individualized defenses to damages claims and the plaintiffs' ability to pursue class certification on liability issues.”

32

Post-Duran Opinions

• Mies v. Sephora U.S.A., Inc. (2015) 234 Cal.App.4th 967. Denial of

certification affirmed in misclassification case. Plaintiff’s “undeveloped and unsubstantiated” proposal for statistical evidence not sufficient.

• Hale v. Sharp Healthcare (2014) 232 Cal.App.4th 50. Decertification

affirmed in consumer class action. Proposal to use statistical sampling not “an adequate evidentiary substitute for establishing commonality or entitlement to damages…”

• Trahan v. U.S. Bank N.A. (N.D Cal. July 29, 2014) 2014 WL 3750053. Order

denying motion to approve survey in related case to Duran. Finding that if the survey is to be done anonymously, court could not reconcile with a defendant’s right to discovery.

– State of Oklahoma v. Tyson Foods, Inc. (N.D. Okla., March 11, 2009) 2009 WL

• 10271835. Granting protective order and denying motion to compel

identifying information on survey respondents. – Federal Judicial Center, Reference Manual on Scientific Evidence, 3d. ed., p.

• 417. Affirming that survey respondents are unavailable for cross-examination.

33

Duran Post-Script

• Case has been remanded to the Superior Court
• Plaintiffs conducted a survey on the class related

to the key issues in the case.

– expert calculated averages and confidence intervals for each of three issues. – Expert created a “menu” of various sample sizes for each issue based on confidence intervals from survey.

• Class certification determination due this

summer.

34

STATISTICAL EVIDENCE IN CLASS ACTIONS: PRACTICE STRATEGIES

AND TACTICS

Paul G. Karlsgodt, Partner Denver, CO PKarlsgodt@bakerlaw.com

Part 1: Justifying or Challenging the Applicability of Statistical Evidence to Support or Defend Class Certification

36

How Are Statistics Used to Support Class Certification?

• The existence of a common practice.
• A relationship between the defendant’s conduct

and some injury to class members.

• The total damages or other impact caused by a

practice.

• The percentage of people impacted by a

practice.

• Given a set of characteristics, the probability

that a person was impacted by a practice.

• Common reliance:

– Uniform common reliance, e.g. “fraud on the market.” – Reliance by “most” of the class.

37

Evaluating Common Impact

• Is there truly a common impact?

– Recurring theme: Aggregate impact is mistaken for uniform impact.

• Dukes: Fewer promotions doesn’t mean that all women suffered

discrimination.

• McLaughlin: Individual facts (descriptive statistics) presented to show

non-reliance by customers.

• Rhodes: Did not provide any common proof that any given individual

suffered a significantly increased risk of the exposure.

• Did the challenged practice cause the common impact?

– Recurring Theme: Inability to specify root causes and predictors that are common to all class members and rule out other potential causes.

• Dukes: Conclusions don’t go further than showing that disparities exist
• Rhodes: Did not address the question of the relationship between

exposure and a significantly increased risk of health problems

• Comcast: Analysis took into account the combined value of four different

potential impacts, only one of which the trial court had found could be a common impact.

38

Common Impact Fact Patterns

• Trial by Formula (Duran, Dukes)

– Statistics used to estimate percentage of class members to whom the defendant may be liable –

• Creates the practical problem of not being able to identify which particular class

members suffered injury.

• Due process issues.
• Winners and losers – Some class members may be better off as a result of a

practice.

– Statistics used to apportion damages

• Might be acceptable if all class members suffered some damages.
• May violate due process rights of absent class members if the formula does not fairly

apportion.

• Root Cause Analysis (Comcast, Dukes)

– Have to be able to rule out causes not consistent with liability. – Has to be able to establish cause as opposed to mere correlation. – Theory of damages has to align with theory of common impact.

39

Part 2: Using or Challenging Expert Witnesses in Class Certification Proceedings

40

41

Tips for Dealing With Experts

41

Draw Inferences (optional) Analyze Collect Data Considerations



How collected? Trusted source?



Is the method / measurement process reliable (consistent performance with repetition)?



Valid?



Recorded properly?



Categories appropriate?



What is the non-response rate (survey)? Why?



Has the methodology been peer reviewed? Discredited?



Can the results be generalized?



How are charts/graphs presented?



What method is used to select the units (or scale)?



Do analyses reach different

• pinions?



What variables were omitted?



Did the expert answer the right question?



How do I estimate whatever is missing?



Is the sample size big enough to be predictive?



How accurate are the predictions?



Does the analysis prove a fact to be true, or does it assume the fact is true?

Ask, “What is missing? Who would know?”

Challenging Expert Credentials

• Does the statistics expert have expertise in the subject area s/he is analyzing?

– Or is the expert simply a statistician with no particular understanding of the subject at issue? If so, is there a relevant supporting expert?

• In what ways should the statistician’s testimony be supplemented by:

– Other experts with subject area expertise. – Data quality knowledge?

• What are the strengths and weaknesses of the foundational philosophies and

historical tendencies of this expert’s approach?

– Consider an expert’s subfield emphasis—e.g. econometrics, biostats, product testing. – Consider the expert’s attention to the entire lifecycle of a statistic—e.g. initial data profiling, study design, collection method.

• What evidence of error exists in the expert’s published materials? Commentary

by other statisticians?

– What evidence of error consideration is given in the expert’s own published materials? Are none, some, or all potential deficiencies noted by the expert? – How extensive and meaningful were peer reviews for published materials, assuming they exist at all?

42

Evaluating Expert Methodologies

• Take a hard look at the statistician’s methodology. It can have a big

impact on the case outcome. – Dukes: Trial by formula not allowed. – McLaughlin: The expert’s survey methodology deemed “pure speculation.” – Neurontin: Inability to ID root cause 1) Where expert’s opinion was that less than substantially all (>99%) of prescriptions were caused by fraud, individual inquiry required; 2) Where expert’s opinion was that substantially all prescriptions were caused by fraud, the expert analysis was flawed. – Zyprexa: Root cause not pinpointed by expert. – Rhodes: 1) Preliminary and insufficient data was used. 2) Failed to rule out other variables. – Duran: 1) Methodology was flawed because sample was arbitrary; 2) Sampling would have been improper even if used to calculate damages due to the high margin of error. – Facebook: there was no way to conduct this type of highly specialized and individualized analysis for each of the thousands of advertisers in the proposed class.

43

Understanding kinds of statistics

“Statistics is the science and art of describing data and drawing inferences from them”*

44

*(Finkelstein and Levin, p. 1)

Describes relationships, correlations, events Pay for women in company X is 15% lower than it is for men.

Statistics Inferential Statistics Descriptive Statistics

Makes inferences, generalizations, estimates, predictions Company X has a discriminatory pay policy.

Terminology of “Statistics”?

• Probability

– How likely is something to be true?

• Regression analysis

– Discussed in Wal-Mart Stores, Inc. v. Dukes and Comcast – Examines the relationship between variables.

• Surveys

– Of X population, Y are likely to respond this way.

• Econometrics

– E.g., “but for the misrepresentation, the price would have been X dollars lower”

• Compilations of Data

– Not “statistics” per se, but may raise some of the same issues.

45

46

Visualizing Root Causes

• Is there a common “answer” for all class members—i.e. did the same set of

circumstances apply to each class member; “Yes” in Halliburton; “No” in Dukes

• Is there some other possible explanation (other than gender bias)?
• Root Cause (Ishikawa) Diagram

46

Class Definition “’[A]ll women [w]ho have been or may be subjected to Wal-Mart’s challenged pay and management track promotions policies and practices.” Personal Traits Employment Status Management Behaviors Policies & Procedures

Gender Personal decisions Age Family Situation Full-/part-time Tenure Role Previous job Performance Biased Discretion Non-biased discretion Mobility Workload Expectations

What Else?

Paraphrasing: While disparity may exist, the underlying root causes are likely to be different among class members

Region

Dept Store

Challenging Validity of Data

– Is there an actively-engaged, senior executive-sponsored data governance body in place? – What evidence is available to demonstrate that a robust data quality management program is used? – Are data stewardship roles & responsibilities well-defined? – Is a standards-based data management process and procedure framework in place? – Does the company have day-to-day reliance on purpose-built data governance tools and performance metrics? – Does a robust and active master data management program exist? – Is a financial information management program in place? – How well and how often are errors identified, analyzed for root cause, and corrected? – How well-managed is the quality of data coming into the system, both manually and in an automated fashion? – What data in the system can’t be considered the source of truth?

47

For Further Study

• David H. Kaye & David A. Freedman, Reference Guide on Statistics,

Reference Manual on Scientific Evidence 2d Ed. (Federal Judicial Center 1981) (http://www.fjc.gov/public/pdf.nsf/lookup/sciman02.pdf/$file/sciman02.pdf)

• Robert Ambrogi, Statistics Surge as Evidence in Trials, IMS Newsletter,

BullsEye: August 2009, (http://www.ims- expertservices.com/newsletters/aug/statistics-surge-as-evidence-in-trials- 081409.asp)

• Edward K. Cheng, A Practical Solution to the Reference Class Problem, 109
• Colum. L. Rev. 2081 (2009)

(http://www.columbialawreview.org/assets/pdfs/109/8/Cheng.pdf)

• Denise Martin, Stephanie Plancich, and Mary Elizabeth Stern, Class

Certification in Wage and Hour Litigation: What Can We Learn from Statistics? (Nera Economic Consulting 2009) (http://www.nera.com/extImage/PUB_Wage_Hour_Litigation_1109_final.pdf)

• Dukes, plaintiff’s Expert Dr. Richard Drogin’s Statistical Report

(http://www.walmartclass.com/all_reports.html)

• Michael O. Finkelstein and Bruce Levin, Statistics for Lawyers: Second Edition

(Springer, 2001)

• Finkelstein, Michael O., Basic Concepts of Probability and Statistics in the Law

(Springer, 2009)

• Olive Jean Dunn and Virginia A. Clark, Applied Statistics: Analysis of Variance

and Regression, Second Edition (John Wiley & Sons, 1987)

48

Atlanta Chicago Cincinnati Cleveland Columbus Costa Mesa Denver Houston Los Angeles New York Orlando Philadelphia Seattle Washington, DC www.bakerlaw.com

These materials have been prepared by Baker & Hostetler LLP for informational purposes only and are not legal advice. The information is not intended to create, and receipt of it does not constitute, a lawyer-client relationship. Readers should not act upon this information without seeking professional counsel. You should consult a lawyer for individual advice regarding your own situation.