Designed Experiments, One Factor with Two levels The simplest - - PowerPoint PPT Presentation

ST 370 Probability and Statistics for Engineers

Designed Experiments, One Factor with Two levels

The simplest experimental design problem is where only a single factor is being studied. When the factor has only two levels, interpreting the data is straightforward. Experiments with more than one factor, or a single factor with more than two levels, are more complex.

1 / 22 Statistical Inference for Two Samples Designed Experiments, One Factor with Two levels

ST 370 Probability and Statistics for Engineers

The pull-off forces for two different designs of nylon connector are an example of this design: Response: pull-off force; Factor: wall thickness; Levels:

3 32 inch, 1 8 inch.

Note Other factors could also affect the pull-off force: Nylon formulation (6, 66, 510, etc.); Additives to the nylon; Other dimensions of the connector. These were all held constant.

2 / 22 Statistical Inference for Two Samples Designed Experiments, One Factor with Two levels

ST 370 Probability and Statistics for Engineers

Randomization All 16 measurements should be made in random order: Label the sample connectors from 1 to 16;

sample(1:16) gives a random order for the measurements.

Randomization guards against possible biases in the measurements. For instance, if the equipment temperature changes during the measurements, the later measurements could be systematically different from the earlier measurements. If, say, all the smaller connectors were tested before the larger connectors, the temperature effect would be confounded with the wall-thickness effect.

3 / 22 Statistical Inference for Two Samples Designed Experiments, One Factor with Two levels

ST 370 Probability and Statistics for Engineers

Data Analysis The R tools that we shall use to analyze data collected in factorial experiments are generally lm() and aov(). Example: Nylon connector In R:

summary(lm(Force ~ factor(Thickness), nylonConnector))

The output shows that the larger connectors had an average pull-off force 0.4125 lbf higher than the smaller connectors, in this set of data.

4 / 22 Statistical Inference for Two Samples Designed Experiments, One Factor with Two levels

ST 370 Probability and Statistics for Engineers

Inference to the population But we know that other runs of the whole experiment would give different results. We assume that if we tested enough connectors, we would find a unique difference for the population. The population difference is called a parameter, which we will denote δ. The observed difference 0.4125 lbf is an estimate of δ, written ˆ δ. The standard error of ˆ δ (“Std. Error” in the output) is a measure of the uncertainty of ˆ δ as an estimate of δ.

5 / 22 Statistical Inference for Two Samples Designed Experiments, One Factor with Two levels

ST 370 Probability and Statistics for Engineers

Confidence Interval Suppose I assert that δ lies between ˆ δ ± 2 × standard error = 0.4125 ± 2 × 0.2438 = (−0.0751, 0.9001). Some assumptions and some theory show that an assertion made in this way has around a 95% chance of being correct, so we say that we have 95% confidence in the assertion. The range of values ˆ δ ± 2 × standard error is called a 95% confidence interval for δ. Tables of Student’s t-distribution show that to achieve 95% confidence, the multiplier 2 should actually be 2.145.

6 / 22 Statistical Inference for Two Samples Designed Experiments, One Factor with Two levels

ST 370 Probability and Statistics for Engineers

Note on interpretation It is not true that: The probability that δ lies between −0.0751 and 0.9001 is 95%. The correct interpretation is: When an interval is constructed in this way, the probability that it contains the true value is 95%. The 95% probability associated with the way the interval is constructed gives us 95% confidence in the assertion.

7 / 22 Statistical Inference for Two Samples Designed Experiments, One Factor with Two levels

ST 370 Probability and Statistics for Engineers

Statistical Hypotheses Note that the range of values in the confidence interval includes zero, which would mean that there is no difference in pull-off force between the two wall thicknesses. Often the first question we ask is “Did the factor have any effect on the response?” We frame the question in terms of statistical hypotheses: The null hypothesis is that the factor had no effect; H0 : δ = 0. The alternate hypothesis is that the factor had a non-zero effect; H1 : δ = 0.

8 / 22 Statistical Inference for Two Samples Designed Experiments, One Factor with Two levels

ST 370 Probability and Statistics for Engineers

Based on the data, we may reject H0 in favor of H1, or we may fail to reject H0. Often we know that the null hypothesis cannot be exactly true. It is inconceivable that connectors with thicker walls have exactly the same mean pull-off force as those with thinner walls. Failing to reject a null hypothesis does not mean that we believe it to be true, just that we do not have enough evidence to reject it.

9 / 22 Statistical Inference for Two Samples Designed Experiments, One Factor with Two levels

ST 370 Probability and Statistics for Engineers

Testing the Hypotheses To decide whether to reject the null hypothesis, we ask: If the null hypothesis were true, what is the probability of finding as large an effect as is shown in the data? If that probability (the P-value) is low, the data are not consistent with the null hypothesis, and we reject the null hypothesis. A common threshold is 5% (P < 0.05), but a smaller P-value is stronger evidence against the null hypothesis.

10 / 22 Statistical Inference for Two Samples Designed Experiments, One Factor with Two levels

ST 370 Probability and Statistics for Engineers

Finding the P-value We could carry out similar experiments in which we know that the null hypothesis is true. For example: Make two sets of 8 nylon connectors with the same wall thickness, and compare pull-off forces; Repeat say 100 times; the number of times that |ˆ δ| ≥ (|ˆ δobserved| = 0.4125) is an estimate of the P-value (as a percentage). Never feasible in practice.

11 / 22 Statistical Inference for Two Samples Designed Experiments, One Factor with Two levels

ST 370 Probability and Statistics for Engineers

Student’s t-test The most widely used test is Student’s t-test. We assume that the measurements follow the normal distribution, and then it can be shown that, when the null hypothesis is true, T = ˆ δ standard error follows a related distribution called Student’s t-distribution. The probability that |T| ≥ |tobs| can be calculated as an area under the corresponding density function.

12 / 22 Statistical Inference for Two Samples Designed Experiments, One Factor with Two levels

ST 370 Probability and Statistics for Engineers

In R Recall the model output:

summary(lm(Force ~ factor(Thickness), nylonConnector))

The output includes the t-statistic 1.692, and the P-value 0.113. This means that if the null hypothesis were true, the probability that |T| ≥ (|tobs| = 1.692) would be 0.113, or around 11%.

13 / 22 Statistical Inference for Two Samples Designed Experiments, One Factor with Two levels

ST 370 Probability and Statistics for Engineers

Note on interpretation of the P-value It is not true that: The probability that the null hypothesis is true is 0.113. The correct interpretation is: If the null hypothesis were true, the probability of seeing as large a difference between groups would be 0.113.

14 / 22 Statistical Inference for Two Samples Designed Experiments, One Factor with Two levels

ST 370 Probability and Statistics for Engineers

Notes The P-value calculation in the t-test depends on the assumption that dispersion is the same in both groups. A modified t-test may be used when that cannot be assumed. In R For the particular case of a single factor with 2 levels, R provides

t.test() as well as the more general lm(). t.test() by default does

not assume that the variances are equal:

t.test(Force ~ Thickness, data = nylonConnector) # compare with: t.test(Force ~ Thickness, nylonConnector, var.equal = TRUE)

15 / 22 Statistical Inference for Two Samples Designed Experiments, One Factor with Two levels

ST 370 Probability and Statistics for Engineers

Hand calculation The results from lm() are easily reproduced by hand. Write yi,j for the jth measurement of the ith level of the factor (j = 1, 2, . . . , ni; i = 1, 2, . . . , a), where a = 2 is the number of levels

• f the factor, and n1 = n2 = 8 are the numbers of measurements

made at the two levels. Then ¯ yi· = 1 ni

ni

• j=1

yi,j is the sample mean of the measurements at the ith level.

16 / 22 Statistical Inference for Two Samples Designed Experiments, One Factor with Two levels

ST 370 Probability and Statistics for Engineers

For these data, we have ¯ y1· = 13.0, ¯ y2· = 13.4125, and ˆ δ = ¯ y2· − ¯ y1· = 0.4125.

17 / 22 Statistical Inference for Two Samples Designed Experiments, One Factor with Two levels

ST 370 Probability and Statistics for Engineers

The standard error calculation is based on the sample variances s2

i =

1 ni − 1

ni

• j=1

(yi,j − ¯ yi·)2, i = 1, 2. We pool them (a weighted average): s2

pooled = (n1 − 1)s2 1 + (n2 − 1)s2 2

n1 + n2 − 2 . The standard error of ˆ δ is

• s2

pooled

1 n1 + 1 n2

• .

18 / 22 Statistical Inference for Two Samples Designed Experiments, One Factor with Two levels

ST 370 Probability and Statistics for Engineers

For these data, we have s2

1 = 0.2285714,

s2

2 = 0.2469643,

s2

pooled = 0.2377679,

and standard error of ˆ δ = 0.2438072.

19 / 22 Statistical Inference for Two Samples Designed Experiments, One Factor with Two levels

ST 370 Probability and Statistics for Engineers

Degrees of Freedom Student’s t-distribution is not a single distribution, but a family of distributions, distinguished by their degrees of freedom. When we need the multiplier to construct a confidence interval, or calculate a P-value as the area under a curve, we must use the degrees of freedom to identify the appropriate member of the family. The degrees of freedom describe how much information was used to estimate the sample variance, and hence the standard error of the estimate: A single sample of size n provides n − 1 degrees of freedom; A pair of samples of sizes n1 and n2 provide (n1 − 1) + (n2 − 1) = n1 + n2 − 2 degrees of freedom.

20 / 22 Statistical Inference for Two Samples Designed Experiments, One Factor with Two levels

ST 370 Probability and Statistics for Engineers

Example: Nylon connectors The 95% confidence interval for the effect of wall thickness is ˆ δ ± t.025,14 × standard error where tα/2,ν is the percentage point that cuts off a tail area α/2 in the t-distribution with ν degrees of freedom. We use α/2 = .05/2, because the confidence interval may be wrong in two ways: either entirely too high, or entirely too low. But α/2 measures the probability of either error, so the total error probability is α. Table V in Appendix A gives tα,ν for various values of α and ν. In R, use qt(alpha, nu, lower.tail = FALSE).

21 / 22 Statistical Inference for Two Samples Designed Experiments, One Factor with Two levels

ST 370 Probability and Statistics for Engineers

In testing the null hypothesis δ = 0, we found |tobs| = 1.692. In Table V, we see that this falls between t.10,14 and t.05,14, so we can state that 2 × .10 = .20 > P > 2 × .05 = .10 but we cannot find P more precisely. In R, we compute the P-value as

2 * pt(1.692, 14, lower.tail = FALSE), giving

P = .1128 as in the output from lm() or t.test().

22 / 22 Statistical Inference for Two Samples Designed Experiments, One Factor with Two levels