How to Recognize and Avoid False Violations When Using Whole - - PDF document

tmoore@risk-sciences.com 8/21/2013 (c) 2013, Risk Sciences 1

How to Recognize and Avoid False Violations When Using Whole Effluent Toxicity (WET) Test Methods to Evaluate Stormwater Samples

Timothy F. Moore Risk Sciences Rockvale, TN

No Toxics… …In Toxic Amounts

tmoore@risk-sciences.com 8/21/2013 (c) 2013, Risk Sciences 2

tmoore@risk-sciences.com 8/21/2013 (c) 2013, Risk Sciences 3

tmoore@risk-sciences.com 8/21/2013 (c) 2013, Risk Sciences 4

tmoore@risk-sciences.com 8/21/2013 (c) 2013, Risk Sciences 5

tmoore@risk-sciences.com 8/21/2013 (c) 2013, Risk Sciences 6

130 120 110 100 90 80 70 60 50 40 30 20 10

Number of Tests Performed

100% 90% 80% 70% 60% 50% 40% 30% 20% 10% 0%

Probability of Passing All Tests

tmoore@risk-sciences.com 8/21/2013 (c) 2013, Risk Sciences 7

1) Random, Unusual, & Rare 3) NOEC Not Consistent w/ IC25

tmoore@risk-sciences.com 8/21/2013 (c) 2013, Risk Sciences 8

tmoore@risk-sciences.com 8/21/2013 (c) 2013, Risk Sciences 9

tmoore@risk-sciences.com 8/21/2013 (c) 2013, Risk Sciences 10

tmoore@risk-sciences.com 8/21/2013 (c) 2013, Risk Sciences 11

“I certify under penalty of law that this document and all attachments were prepared under my direction or supervision in accordance with a system designed to assure that qualified personnel properly gather and evaluate the information

• submitted. Based on my inquiry of the person or persons who

manage the system, or those persons directly responsible for gathering the information, the information is, to the best of my knowledge and belief, true, accurate, and complete. I am aware that there are significant penalties for submitting false information, including the possibility of fine and imprisonment for knowing violations.” 40 CFR 122.22(d)

DMR Certification Statement Systematic WET Test Review

1) Know the Expected Total Number of False Positives 2) Run a Full Dilution Series, Not a Screening Test 3) Use Multiple Control Waters for Ionic Imbalance 4) Calculate and Compare Both the NOEC and the IC25 5) Demand Comprehensive Laboratory Control Charts 6) Perform Stats Using 95% and 99% Confidence Levels 7) Analyze Identical Split Samples in Accelerated Testing 8) Consider Using Alternate Test Species 9) Develop Written Data Quality Objectives and Checklist

HOW TO RECOGNIZE AND AVOID FALSE VIOLATIONS WHEN USING WHOLE EFFLUENT TOXICITY (WET) TEST METHODS TO EVALUATE STORMWATER SAMPLES. Timothy F. Moore1 INTRODUCTION Federal and state law universally prohibits the discharge of "toxic substances in toxic amounts." For the first twenty years following adoption of the Clean Water Act compliance was determined using chemical analyses to demonstrate that pollutant concentrations were within allowable limits. However, it was impractical to develop numeric water quality criteria for the many thousands of different chemicals in routine use. Nor would individual water quality criteria address the synergistic effects that may occur when aquatic organisms are exposed to a complex mixture of several different chemical pollutants simultaneously. Whole Effluent Toxicity ("WET") testing was developed to address these concerns. In the mid-1990's EPA promulgated standard methods for conducting toxicity tests using sensitive aquatic organisms. Since then, these methods have come into widespread use - particularly for municipal and industrial wastewaters. It is only recently that this trend has expanded to include stormwater discharges. Many MS4 permittees are now required to perform routine WET tests on samples of stormwater and receiving water. Most are only required to monitor and report the results

• f such testing. But, a few are now required to rely on WET testing to demonstrate compliance with

narrative or numeric effluent limits prohibiting toxic discharges. And, that number is expected to increase with time. As the very name suggests, WET test methods were originally designed to evaluate municipal and industrial effluents. Using the standard test procedures and organisms to assess potential toxicity in stormwater samples poses unique challenges. Chief among these is the elevated risk of false test failures caused by various interference factors such as the low conductivity and pH levels found in natural rainfall. When negotiating monitoring requirements or discharge limitations in an MS4 permit, or contracting with a laboratory to perform WET tests, great care must be taken to consider the special demands associated with evaluating potential toxicity in stormwater samples. EPA guidance provides considerable flexibility to address such concerns but, because most MS4 agencies are relatively new to WET testing, few permittees are fully aware of their implementation options. The purpose of this paper is to describe some of the most useful alternatives.

1 Risk Sciences, 125 New Dawn Rd., Rockvale, TN 37153; (615) 274-2745; tmoore@risk-sciences.com

WHAT IS A "FALSE" TEST FAILURE? Most toxicity tests proceed by exposing standardized organisms (primarily fish and macroinvertebrates) to an effluent or receiving water sample and comparing the rate of survival, reproduction and/or growth to that observed for identical organisms that have been exposed to non- toxic laboratory control water. Any statistically-significant reduction in survival, growth or reproduction is deemed to be evidence that the sample water is probably "toxic." Chemical toxicity can, indeed, inhibit reproduction or growth and increase the risk of mortality. But, statistically-significant differences can also occur due to other factors outside a dischargers control. When a test fails, it is often difficult to distinguish between genuine toxicity and artifactual causes. Consequently, preferring to err on the side of caution, regulators are initially predisposed to consider all such failures as potential exceedances of state water quality standards. Even where a WET test failure is not automatically deemed to be a permit violation, it will often trigger expensive and time-consuming follow-on testing (called a "Toxicity Identification Evaluation") to determine what is causing the toxicity or to demonstrate the initial test failure was an anomaly. And, for MS4 agencies that are in the early phases of WET monitoring where they are only required to "Monitor-and-Report" results, false test failures may lead to an incorrect conclusion that there is Reasonable Potential for toxicity at their outfalls and, eventually, actual effluent limits for WET. WHAT CAUSES FALSE TEST FAILURES? As with any other form of chemical analysis, toxicity testing is vulnerable to sample contamination, laboratory error, and similar phenomenon that can produce incorrect data. However, because WET testing relies on living organisms as the primary instrument of detection, natural biological variability is also an important source of potential error. By definition, when one uses a 95% confidence level to identify statistically-significant changes in survival, reproduction or growth, one is also accepting the small (5%) chance that some large differences may be due to nothing more than random chance. A 5% risk of error is relatively low for any individual test. But, the risk adds up over time. If a discharger is required to perform quarterly chronic toxicity tests then a total of 20 samples will be analyzed during the course of a normal 5-year permit term. Most toxicity tests will evaluate two species (a fish and an invertebrate) and two endpoints for each species (mortality and a sublethal metric such as reproduction or growth). Consequently, the testing laboratory will perform a total of 80 statistical analyses (20 samples x 2 species x 2 endpoints) during the 5-year period. Since 5% of all tests are expected to fail due to random chance alone, then it is likely that most dischargers engaged in a similar monitoring program will observe and report 1 false failure in every 20 tests. It is easy to see why such a record might lead state authorities to conclude that there is Reasonable Potential for toxicity and include a WET limit in subsequent NPDES permits.

In addition to the inevitable and unavoidable risk of statistical error that results from natural biological variability, false violations can also be caused by test interference. This is a particular problem for MS4 permittees because rainfall is naturally low in pH, conductivity, alkalinity and

• hardness. Samples of pristine rainfall, collected in elevated sterile containers, will easily and routinely

fail a toxicity test. The standard organisms (particularly Ceriodaphnia dubia) are unusually sensitive to changes in pH or conductivity. It takes a while for rainfall to become freshwater through the runoff process. And, until that equilibrium is reached, there is a very real possibility of WET test failure caused by ionic interference. HOW CAN FALSE TEST FAILURES BE IDENTIFIED? There are numerous tell-tale signs that a particular test failure may reflect anomalies in the data rather than actual toxicity in the effluent: First, isolated test failures that occur at a discharge location with a long history of consistently passing prior tests are more likely to be false. This is particularly true when one or more of the other tell-tale signs is also present. Second, statistically-significant differences caused by relatively small (<12%) reductions in growth or reproduction are generally considered anomalous. In fact, EPA guidance explicitly recommends that such small differences should not be considered evidence of actual toxicity. Third, when the IC25 and NOEC provide inconsistent conclusions regarding the presence or absence

• f toxicity, this is a strong indication that the test results may be false. Some states require both

endpoints to fail before the discharger is required to certify the final result. Fourth, abnormally high reproduction or growth among the control organisms is a common cause of false test failures. This is particularly true where the average performance among effluent-exposed

• rganisms is at or near its historic norm. It is important to track control and effluent data over the

long-term (≈20 tests) to be able to accurately identify the aberrant test results. Fifth, false failures are frequently associated with sudden and dramatic shifts in the sensitivity of standard test organisms. All biomonitoring laboratories perform routine monthly reference toxicant tests to assess the health and sensitivity of their stock cultures. Standard organisms often become hyper-sensitive just prior to a culture crash. Therefore, it is important to review both the reference toxicant data conducted before and after any WET test that is suspected to be a false failure. Sixth, large differences in the conductivity of the sample water compared to the conductivity of the control water are a reasonably reliable indicator of potential ionic interference. Hardness values lower than 25 mg/L may cause increased mortality among Ceriodaphnia dubia. Hardness values less than 50 mg/L will also inhibit invertebrate reproduction sufficiently to be mistaken for toxicity. Finally, an unstable or inconsistent dose-response relationship provides strong evidence that the failed WET test may be anomalous. An unreliable dose-response curve usually indicates that the test itself is also unreliable and should be repeated at the earliest available opportunity.

HOW CAN THE RISK OF FALSE VIOLATIONS BE MINIMIZED? While it is impossible to prevent all false test failures, there are a number of steps that MS4 permittees can take to reduce their risk: First, the risk of such false failures is much lower for acute tests (which rely exclusively on a mortality endpoint) than for chronic methods. Survival data shows far less natural variability than sublethal

• endpoints. Since most rain events are relatively short, there is little justification for using the 7-day

chronic methods, and the resulting sublethal data, to evaluate potential toxicity in storm water

• runoff. Where chronic methods are mandated, storm water dischargers should prefer to use the

Fathead minnow growth test which exhibits only half the natural biological variability of the Ceriodaphnia dubia reproduction test. Second, even where the permit allows a discharger to perform toxicity tests using a simplified screening procedure (e.g. only one control and one sample concentration), it is advisable to always perform toxicity tests using a full dilution series. This is the only way to produce the data needed to confirm the presence or absence of a valid dose-response relationship. Third, where there is a substantial difference between the conductivity of the sample water and the conductivity of the control water, the test should be performed using EPA's "dual control" procedure. A standard laboratory control is run, usually using moderately-hard synthetic freshwater, to demonstrate the standard test organisms are healthy. In addition, a second control is run using synthetic freshwater that more closely matches the ionic composition of stormwater runoff (e.g. low conductivity, low hardness, low alkalinity). The latter control water also provides the baseline condition for all statistical analyses and is used as the diluent throughout the test. Fourth, discharger should request that their laboratory graph the long-term trends in survival, reproduction and growth for control organisms. The easiest way to do this is to add such information when reporting the results of regular reference toxicant tests. However, it would also be advisable for the discharger to maintain similar trend charts for the performance of control organisms in all of their own WET tests. Fifth, dischargers who are required to perform chronic tests using the Ceriodaphnia dubia should ask the lab to continue collecting data for a full 8-days even when the test would normally be concluded after just 6 or 7 days. Ionic interference is often manifest by delaying, not precluding, reproduction. This condition can only be detected by demonstrating that effluent-exposed organisms reproduced normally after a short period of reacclimation to the new ionic matrix. Sixth, in some cases, and particularly following an initial instance of suspected false failure, it is well worth the extra cost incurred to analyze identical split samples in two different laboratories. This is especially appropriate when a discharger is required to initiate accelerated WET monitoring when the culture organisms at the primary testing laboratory may still be responding abnormally (as indicated by the reference toxicant and/or control performance charts).

CONCLUSION Like all other forms of laboratory analysis, toxicity testing is subject to a degree of error and

• uncertainty. However, unlike chemical analyses, there is no procedure for establishing a Method

Detection Level (MDL) or Practical Quantitation Level (PQL) for WET testing. Consequently, dischargers and laboratories must exert greater diligence when interpreting the biological response data used to infer the presence or absence of potential toxicity in any given sample. It is strongly recommended that dischargers develop formal, written Data Quality Objectives (DQOs) describing the rules and checklist procedures that will be used to identify anomalous WET test data before certifying results on a Discharge Monitoring Report (DMR). It is essential to use the same procedures to evaluate all WET data, regardless of a test initially passes or fails, in order to demonstrate the objectivity and credibility of such a review process. In addition, it is not enough to merely disqualify suspect tests. Dischargers must also be prepared to collect and analyze new samples to replace any anomalous data. All WET test data, including that which has been determined to be anomalous and subsequently disqualified, must still be reported to state authorities. Finally, stormwater dischargers should become familiar with various supplemental guidance documents (beyond the method manuals themselves) that EPA has produced regarding the issue of biological variability in WET testing. Essential reading includes: 1) U.S. EPA. Understanding and Accounting for Method Variability in WET Applications Under the NPDES Program; EPA-833-R-00-003; June, 2000 2) U.S. EPA. Method Guidance and Recommendations for Whole Effluent Toxicity (WET) Testing (40 CFR Part 136). EPA-821-B-00-004; July, 2000. 3) U.S. EPA. Certification of “Accuracy” of Information Submissions of Test Results Measuring Whole Effluent Toxicity. EPA Memorandum to Regional Water Management Division Directors and Regional Enforcement Division Directors. Signed by Charles S. Sutfin, Director Water Permits Division, EPA Office of Wastewater Management, and Sheila E. Frace, Director Engineering and Analysis Division, EPA Office of Science and Technology, and Brian J. Maas, Director Water Enforcement Division, EPA Office of Regulatory Enforcement. March 3, 2000. 4) U.S. EPA. Clarifications Regarding Whole Effluent Toxicity Test Methods Recently Published at 40 CFR 136 and Guidance on Implementation of Whole Effluent Toxicity in

• Permits. Memorandum to Water Management Division Directors, Regions I-X and

Environmental Services Division Directors, Regions I-X from Tudor T. Davies, Director Office of Science and Technology, and Michael B. Cook, Director Office of Wastewater Management; July 21, 1997. 5) U.S. EPA. NPDES Permit Writer's Guide to Data Quality Objectives; Nov., 1990. All of these, and numerous other useful documents, can be downloaded at www.toxicity.com or from EPA's official website.