and
with two levels each as given in the following table of population means:
Before giving details, it is helpful to introduce some basic terms: factor, level, and design. An experiment is usually carried out to study one or more variables. These variables are called the factors in the experiment. Each factor may be investigated at several different settings. These settings are called the levels of the factor. An experiment consists of a number of separate ``runs'' or measurements. The manner in which levels of all the factors are changed from one run to the next is the experimental design. There are two more terms that are often used to describe designs. Replications are observations made under identical experimental conditions. If every level of every factor appears an equal number of times, the design is said to be balanced.
In this lab we will use a subset of the actuator experiment named jet as an illustration (see Lab 2 or type actuator to see the full data set).
jet
act press force order 1 A2 30psi 0.0992 11 2 A1 30psi 0.0708 5 3 A2 30psi 0.0754 12 4 A1 30psi 0.0690 8 5 A2 100psi 0.4875 3 6 A1 100psi 0.4428 2 7 A2 100psi 0.5039 9 8 A1 100psi 0.3700 6 9 A2 30psi 0.0830 7 10 A1 30psi 0.0581 15 11 A2 30psi 0.0677 1 12 A1 30psi 0.0498 4 13 A2 100psi 0.4864 10 14 A1 100psi 0.3908 16 15 A2 100psi 0.4867 14 16 A1 100psi 0.3726 13For readability the force values have been rounded by typing
options(digits=3)
which sets the number of significant digits printed. If you want to see the options in force, just type options().
You might recall that actuators are small jets using compressed air that are designed for positioning and stabilizing a space platform. For this experiment the factors are 1) act, the particular actuator and 2) press, the amount of air pressure supplied to the actuator. There were two actuators in the experiment, and so this first factor has two levels: using the first actuator (A1) or the second (A2). The second factor, pressure, also has two levels: 30 psi and 100 psi. By looking at the data, it is clear that the experiment consists of 16 ``runs,'' and the design is to consider all different combinations of the levels for the two factors with 4 replications of each of the combinations. Also note that every level appears exactly 8 times in the design, so it is balanced. This is a factorial design because all different combinations of the factors levels have been included.
This experimental design has the desirable feature that both factors are varied simultaneously. At first this may seem to just scramble up the variables and make the results useless. However, this feature is actually very beneficial and gives information about how the two factors interact with each other. With the correct statistical approach the results will be easy to interpret. In Lab 2 on data analysis and plotting, an interaction plot was given for these data which showed that the amount of force depended not only on the actuator but that the differences were more apparent at the higher level of the line pressure. Our goal here is to be able to give a quantitative description of this interaction.
For the jet data one would like to estimate the effect on the force as the pressure supply is changed from 30 psi to 100 psi. One way to do this is to find the statistics for all the measurements taken at the same level of pressure.
stats(jet$force,by=jet$press) (too much output)
stats(jet$force,by=jet$press)[1:3,] (prints only first 3 rows)
30psi 100psi
N 8.0000 8.0000
mean 0.0716 0.4426
Std.Dev. 0.0151 0.0567
From this output we see that the mean difference between these two
settings is 0.0716 - 0.4426 = -.3710. We could also estimate the effect
due to the difference between the two actuators by computing the means for
the two levels of the act factor:
stats(jet$force,by=jet$act)[1:3,]
A2 A1
N 8.000 8.000
mean 0.286 0.228
Std.Dev. 0.219 0.179
yielding 0.286 - 0.228= .058.
We should also look at the set of means from the four groups obtained at the different combinations of factors. To make this easy we have written a function means which will print out the above means as well as the two-way table of means:
means(jet$force,jet$act,jet$press)
Mean of Y variable jet$force by X variable jet$act
A1 A2
N 8.000 8.000
mean 0.228 0.286
Std.Dev. 0.179 0.219
******************************
Mean of Y variable jet$force by X variable jet$press
100psi 30psi
N 8.000 8.000
mean 0.443 0.072
Std.Dev. 0.057 0.015
******************************
Note: in the following, if sample sizes are not equal
row and col means will not be the average of cell means.
Mean of Y variable jet$force by X variables jet$act and jet$press
100psi 30psi Row Mean
A1 0.394 0.062 0.228
A2 0.491 0.081 0.286
Col Mean 0.443 0.072 0.257
The means function can handle as many factors as needed
(e.g., means(y,x1,x2,x3,x4)) and will
print out the mean of the first variable (the response variable)
broken down by the values of
each of the factors as well as by all combinations of two factors.
In the above two-way table of means it helps to add a 4th column of mean differences (and note that we have rounded to two decimals):
30psi 100psi mean diff.
----- ------ ---- -----
A1 .06 .39 .23 .33
Actuator
A2 .08 .49 .29 .41
--- --- --- ---
column mean .07 .44 .26 .37
Some conclusions:
1) Looking at the row and column means, we can see that press is a
much more important variable than act because .44 - .07=.37, the mean
change in force due to changing pressure levels, is large compared to
.29 - .23=.06, the mean change in force due to changing actuators.
2) The change in mean force is not uniform: the row difference column shows that a change in pressure from 30 to 100 psi for actuator 1 (A1) is .33 compared to a change of .41 for actuator 2 (A2). This nonuniformity is called an interaction between the factors act and press.
In general experiments without interactions are easier to analyze
and allow a much simpler description of how the factors
affect the response variable. In the absence of interaction we can express
the dependence of Y on factors
and
with two levels each as given in the following table of population means:
| X1 | |||||
| Level 2 | |||||
| Level 1 | Level 2 | - Level 1 | |||
| X2 | Level 1 | µ | µ + A | A | |
| Level 2 | µ + B | µ + A + B | A | ||
| Level 2 - Level 1 | B | B | |||
This is called an additive model because the affect of changing a factor level is to add a fixed amount (possibly negative) to the mean of Y regardless of the values of the other factors. The key point is that the difference in means for the first factor is A for both rows (levels of the second factor). Similarly, the difference in means for the second factor is B for both levels of the first factor. If these differences were not the same, then an interaction would be present. (Remember that the sample estimates of the population means would rarely exhibit the perfect additivity of the above population means because of random variation.)
As a graphical companion to these summary statistics, one could make up a panel of boxplots where the data are divided by the different factors. However, an alternative plot can be more informative for revealing information about interactions between plots. The function mplot(y,x1,x2) will plot the means of y against the levels of x1 and labeled with the levels of x2. Here is an example:
mplot(jet$force,jet$act,jet$press,both=T)
(If both=T is left off, then only the first plot is given.) In this type plot, parallel lines are evidence of no interaction (also called additivity), whereas nonparallel lines suggest an interaction between the factors. Thus, the plot is here suggestive of an interaction similar to the row difference column of the above means table. Also, horizontal lines are evidence that the factor plotted on the x axis is unimportant. If the lines are close to each other, then the factor represented by different lines is unimportant.þ
To further iluustrate the use of mplot try the following with the full actuator data set:
actuator -> z
mplot(z$force,z$act,z$press,z$nozzle,z$line)
So far we have used fairly simple descriptive statistics to analyze the results of a factorial experiment. In the next section we give a more computationally complicated procedure which is used regularly by experimenters to decide what factors are important and whether interaction is present.
The acronym ANOVA stands for ANalysis Of VAriance. This term is somewhat
misleading because our goal is to compare means. However, it turns out
that in the process of deciding whether two means
are different we end up comparing measures of variability (variances).
The procedure is a bit complicated, but remember that S will
do all the routine calculations for you. The most important part is
that you know how to interpret the results. An ANOVA table is
obtained by using two functions, lm which makes the
computations, and anova which prints the table. Here is the
ANOVA table for the data set jet with a model which includes
the factors act and press and their interaction, which is
denoted in the formula by act:press.
In the formula below the ~ is like an equal sign where the response
variable is on the left side, and the model is specified on the right.
lm(force ~ act + press +act:press, data=jet) -> out
anova(out)
Analysis of Variance Table
Response: force
Terms added sequentially (first to last)
Df Sum of Sq Mean Sq F Value Pr(F)
act 1 0.0136 0.0136 36 0.000059
press 1 0.5505 0.5505 1475 0.000000
act:press 1 0.0060 0.0060 16 0.001696
Residuals 12 0.0045 0.0004
The Analysis of Variance Table organizes the results
for the two factors and their interaction in terms of
Df (degrees of freedom), Sum of Sq (sums of squares),
Mean Sq (mean squares), F Value (F values), and
Pr(F) (p-values).
Here we will give a very brief description of a few entries from this
table.
The Mean Sq results for act and press
are like estimated variances among the means of the response
variable force as the levels of act and press
are changed.
Thus, they measure the variation in the means of the
response variable force for changes in the factor levels. The
Mean Sq on the Residuals line is another estimated
variance for comparison purposes. The F values
are just the ratio of the Mean Sq for the factors to the Mean Sq
of the error. For example, in the first row
F=36=0.0136/0.0004 (not exactly since we have rounded).
When a factor is not important, then the Mean Sq in the numerator
and denominator of the F should be of similar size. Thus,
if the F value is small (near 1), then we know that changing
the levels of a factor has little effect on the response variable.
But how large does the F have to be for us to be sure that
a factor is important?
The last column of this table, Pr(F), gives a
quick way to decide if the factors (rows) are statistically
significant. The elements in this column are usually called p-values. If
the p-value is less than .05, then we typically declare that the effect of
the factor associated with that row can be confidently declared to be not
equal to zero. If it is greater than .05, then there is not strong evidence
from the experiment that the particular factor or interaction has an
effect. Since all three p-values in the above table are small compared to
.05, we can be confident that both factors have a nonzero effect as well as
an interaction.
One confusing point about p-values is that they can be small if the sample size is large even when the true effect of the factor is quite small. We will discuss later the important distinction between statistically signicant results and practically significant results.þ
The ANOVA method of comparing means is very useful for deciding which variables are most important in their effect on a response variable. Let us go back to the full actuator data set and compute an ANOVA table for all four factor variables and their interactions with press.
lm(force~act+press+nozzle+line+act:press+press:nozzle
+press:line,data=actuator)->out2
anova(out2)
Analysis of Variance Table
Response: force
Terms added sequentially (first to last)
Df Sum of Sq Mean Sq F Value Pr(F)
act 1 0.0136 0.0136 43 0.0002
press 1 0.5505 0.5505 1744 0.0000
nozzle 1 0.0010 0.0010 3 0.1204
line 1 0.0010 0.0010 3 0.1206
act:press 1 0.0060 0.0060 19 0.0024
press:nozzle 1 0.0000 0.0000 0 0.8709
press:line 1 0.0000 0.0000 0 0.7334
Residuals 8 0.0025 0.0003
Looking at the last column we are confirmed again that
act,
press,
and their interaction are the most important factor effects.