10  Factorial ANOVA

In science, outcomes rarely depend on just one thing. Climate, soil type, and water availability all act together to influence ecosystems in ways a single-factor design can’t capture. Factorial designs let us study several factors, and how they interact with each other, at once.

This chapter covers both halves of that. First, how to compute a factorial ANOVA, including a full by-hand walkthrough of the sums-of-squares arithmetic. Second, how to interpret the result. This can be tricky with multiple factors. When an interaction is present, it changes what every other number in your results table means.

10.1 Factorial basics

Imagine investigating the growth rate of a plant species, with independent variables like soil pH or water variability. Each added IV multiplies the number of conditions you can observe, and with it, the questions you can ask about how those factors work together. Consider the following example:

1. 1 IV (with two levels)

How does soil pH affect plant growth?

• Independent Variable: Soil pH (Acidic vs. Neutral)

We have one IV (soil pH), with two levels (Acidic vs. Neutral). A t-test is appropriate here, since we are dealing with just two levels of our independent variable.

2. 1 IV (three levels):

How does varying water availability impact plant growth?

• Independent Variable: Water Availability (Low, Medium, High)

Our single IV (water availability) has three levels (Low, Medium, High). You can’t run a t-test anymore. An ANOVA is appropriate here, given the variable has more than two levels.

3. 2 IVs: IV1 (two levels), IV2 (two levels)

How do soil pH and water availability together affect plant growth?

• IV1 (Soil pH: Acidic vs. Neutral);
• IV2 (Water Availability: Low vs. High)

Now we have two IVs, soil pH and water availability, each with two levels. This is called a factorial design.

10.1.1 2x2 Designs

Each combination of soil type and water availability creates a unique environment for the plants, giving four distinct scenarios to measure growth rates: 2 IVs, each with 2 levels, for 2x2 = 4 conditions total. The notation directly tells you the number of conditions.

10.1.2 Factorial Notation

Anytime all of the levels of each IV in a design are fully crossed, so that they all occur for each level of every other IV, we can say the design is a fully factorial design.

The notation system indicates the structure of the factorial design. Each IV gets a number representing its levels. Here are a few examples:

• 2x2: Two IVs, each with two levels (yields four unique conditions).

• 2x3: Two IVs, with the first having two levels and the second three, (yields six unique conditions).

• 3x2: Similar to the 2x3, but the first IV has three levels, and the second has two, also giving us six unique conditions.

• 4x4: Two IVs, each with four levels, resulting in sixteen conditions.

10.2 Pollution, flow, and stream life

The rest of this chapter uses one example throughout.

The question. Streams are affected by pollution, but also by how fast the water moves. Fast-moving water flushes contaminants downstream and re-oxygenates the channel; slow water lets them sit. So pollution might do more damage in a sluggish stream than a fast one. To measure that damage, we need to operationalize stream health.

The dependent variable is macroinvertebrate species count: how many distinct kinds of aquatic insects, snails, worms, and crustaceans live in a stream. This is a standard measure of stream health. These animals sit in the middle of the food web, many are pollution-sensitive, and they can’t flee a degraded stream the way fish can. More species present means a healthier stream.

The design. We run an experiment in artificial stream channels, each assigned one pollution condition and one flow rate. Figure 10.1 summarizes it.

Figure 10.1: A fully crossed factorial design, read as arithmetic: two pollution levels times two flow levels gives four conditions. Every combination is measured, which is what ‘fully crossed’ means.

Two IVs, two levels each, so you should already be able to name this design: it’s a 2x2. Reading the schematic left to right also shows why the notation works the way it does. Crossing two levels with two levels is the multiplication that gives four conditions, and “fully crossed” means no cell is left empty.

That four-cell grid resembles the chi-square table from Chapter 7, but the cells mean different things. There, the cells held counts of stream sites that already had a given land use and E. coli level; nobody put them there. Here, the cells are conditions we assign channels to, and within each one we then measure something else entirely, the species count.

Predictions. No pollution plus high flow should support the most species; pollution plus low flow should support the fewest. The other two combinations are the interesting ones, and a one-factor study could never distinguish them: does pollution matter only when flow is low, with fast water washing its effect away entirely? Or does pollution hurt a stream no matter how fast the water moves? An interaction test in a factorial experiment answers that question.

10.2.1 Cell means and marginal means

Each combination of conditions, such as no pollution with low flow, is called a cell. The cell mean is the mean species count across the five stream channels assigned to that combination. A marginal mean is the mean for one level of a factor, averaged across the levels of the other factor: the no-pollution marginal mean averages the two no-pollution cells, one at each flow rate. Table 10.1 shows the means from the experiment this chapter analyzes.

Table 10.1: Mean macroinvertebrate species count. The four inner numbers are cell means; the last row and column hold the marginal means, and the corner is the grand mean.

	Low Flow	High Flow	Marginal mean
No Pollution	9.6	12.6	11.1
Pollution	3.6	10.0	6.8
Marginal mean	6.6	11.3	8.95

The pollution marginal means (11.1 and 6.8) average each row across both flow rates. The flow marginal means (6.6 and 11.3) average each column across both pollution conditions. The corner, 8.95, is the grand mean of all 20 channels.

10.3 Why factorial designs matter

Factorial designs let us ask more nuanced questions than a single-factor study can. Suppose we ignored flow entirely and just compared polluted to unpolluted channels. That compares the two pollution marginal means, 11.1 vs. 6.8 species, shown in Figure 10.2.

Figure 10.2: Mean macroinvertebrate species count under no pollution vs. pollution, averaged across flow rate (the pollution marginal means).

A factorial design lets us ask the follow-up question a one-factor design can’t: does the effect of pollution itself depend on flow? With flow rate added as a second IV, there are four cell means instead of two, so we can compare polluted and unpolluted channels within each flow rate, shown in Figure 10.3.

Figure 10.3: The effect of pollution within each flow rate (the gap between the two bars) is larger under low flow than under high flow.

Under low flow, species count drops sharply with pollution (9.6 to 3.6, a difference of 6.0). Under high flow, the drop is smaller (12.6 to 10.0, a difference of 2.6). The effect of pollution depends on flow rate.

A factorial design can show what changes the size of an effect, in addition to whether the effect exists.

10.3.1 Main effects and interactions

A factorial design gives you two kinds of results: main effects and interactions. How many of each depends on the number of IVs.

A main effect is the average difference associated with a single IV, the difference between its marginal means, one per IV. A 2x2 design has two IVs, so two main effects: one for Pollution and one for Flow. A 2x2x2 design has three main effects, one per IV; a 3x3x3 design also has three, since the number of main effects tracks the number of IVs, not their levels.

The effect of one IV at a single level of the other, such as the pollution effect under low flow, is a simple effect. Figure 10.3 shows two of them, one per flow rate, and the main effect of pollution is their average: \((6.0 + 2.6)/2 = 4.3\), the same 4.3 you get by subtracting the pollution marginal means.

An interaction occurs when the effect of one IV depends on the level of another, which is exactly what Figure 10.3 shows: the simple effect of pollution changed with flow.

Equivalently, an interaction is a difference between differences, which is what the arithmetic computes. Each simple effect of pollution is a difference (unpolluted minus polluted). If that difference itself changes across flow rates, that’s an interaction.

The number of possible interactions scales with the number of IVs. A 2x2 design has exactly one interaction, the combined effect of the two IVs. A design with three IVs (A, B, C) has three two-way interactions (AB, AC, BC) and one three-way interaction (ABC).

10.4 What a 2x2 actually tests

A two-way ANOVA tests three effects and therefore has three null hypotheses:

1. a main effect of pollution,
2. a main effect of flow, and
3. a pollution × flow interaction.

We can represent the four combinations of conditions and their mean species counts as:

Pollution	Flow	Mean species count
No Pollution	Low Flow	\(\mu_{\text{NoPoll,Low}}\)
No Pollution	High Flow	\(\mu_{\text{NoPoll,High}}\)
Pollution	Low Flow	\(\mu_{\text{Poll,Low}}\)
Pollution	High Flow	\(\mu_{\text{Poll,High}}\)

10.4.1 Hypotheses

Main effect of pollution

• Null (\(H_0\)): There is no effect of pollution on mean species count.
• Alternative (\(H_a\)): Pollution affects mean species count.

Main effect of flow

• Null (\(H_0\)): There is no effect of flow rate on mean species count.
• Alternative (\(H_a\)): Flow rate affects mean species count.

Pollution × flow interaction

• Null (\(H_0\)): The effect of pollution is the same at both flow rates. (In symbols: \((\mu_{\text{NoPoll,Low}}-\mu_{\text{Poll,Low}})=(\mu_{\text{NoPoll,High}}-\mu_{\text{Poll,High}})\).)
• Alternative (\(H_a\)): The effect of pollution differs across flow rates.

Plain English: Under the interaction null, the gap between polluted and unpolluted channels is the same whether the water is moving fast or slow. If that gap changes, for example if pollution does far more damage under low flow, you’ve found an interaction.

NoteNote: General form

You’ll sometimes see this written in abstract terms, with “Factor A” and “Factor B.” For example: \(H_0: (\mu_{A1,B1}-\mu_{A2,B1})=(\mu_{A1,B2}-\mu_{A2,B2})\). It’s the same logic: just replace A and B with your own variables.

10.5 What the ANOVA compares

Recall from earlier: an interaction is a difference between differences. Here’s that idea with the four cell means from Table 10.1, before we touch a single sum of squares. Writing a simple effect as \(\Delta\), with the level it is computed at after the bar:

\[\Delta_\text{Pollution|Low} = \bar{x}_\text{NoPoll,Low} - \bar{x}_\text{Poll,Low} = 9.6 - 3.6 = 6.0\]

\[\Delta_\text{Pollution|High} = \bar{x}_\text{NoPoll,High} - \bar{x}_\text{Poll,High} = 12.6 - 10.0 = 2.6\]

Those two numbers aren’t the same, so the effect of pollution isn’t constant across flow rates, which is exactly what an interaction means. The gap between them, \(6.0 - 2.6 = 3.4\), is a first, informal look at the interaction itself. The formal test below tells us whether a gap that size is larger than chance would produce.

10.6 Computing a factorial ANOVA by hand

In our between-subjects 2x2 design, different stream channels appear in each condition: 5 stream channels assigned to each combination of conditions, 20 channels total, no repeats. The logic is the same as the one-way ANOVA in Chapter 8: partition the total variance into parts. For a 2x2 design, there’s now one more part, the interaction:

\[SS_\text{Total} = SS_\text{Pollution} + SS_\text{Flow} + SS_\text{Interaction} + SS_\text{Error}\]

The tables below build up each piece the same way the one-way ANOVA chapter did: one row per channel, showing what that channel’s score gets replaced with, its distance from the grand mean, and that distance squared.

10.6.1 SS Total

\(SS_\text{Total}\) captures all of the variation in the data: the difference between each channel’s score and the grand mean, squared and summed. The table shows the first few channels and the last, since the pattern is the same all the way down.

Pollution	Flow	Species	Diff	Diff Squared
No Pollution	Low Flow	10	1.05	1.1
No Pollution	Low Flow	8	-0.95	0.9
No Pollution	Low Flow	11	2.05	4.2
...	...	...	...	...
Pollution	High Flow	12	3.05	9.3
Sums		179	0	242.95
Means		8.95

The mean of all 20 scores is the grand mean, 8.95. The Diff column holds every channel’s distance from it; Diff Squared squares those distances so they don’t cancel (the same reason we squared deviations for the one-way ANOVA). Summing the squared column gives \(SS_\text{Total} = 242.95\).

10.6.2 SS Pollution

\(SS_\text{Pollution}\) isolates the variation we can explain by pollution condition alone: replace every score with its pollution marginal mean, then find that mean’s distance from the grand mean.

Pollution	Flow	Species	Marginal Mean	Diff	Diff Squared
No Pollution	Low Flow	10	11.1	2.15	4.62
No Pollution	Low Flow	8	11.1	2.15	4.62
No Pollution	Low Flow	11	11.1	2.15	4.62
...	...	...	...	...	...
Pollution	High Flow	12	6.8	-2.15	4.62
Sums		179		0	92.45

Every no-pollution channel is replaced by the no-pollution marginal mean, 11.1; every pollution channel by the pollution marginal mean, 6.8. Squaring and summing those means’ distances from the grand mean gives \(SS_\text{Pollution} = 92.45\): the variation explained by pollution condition alone.

10.6.3 SS Flow

\(SS_\text{Flow}\) works the same way, grouping by flow rate instead of pollution condition.

Pollution	Flow	Species	Marginal Mean	Diff	Diff Squared
No Pollution	Low Flow	10	6.6	-2.35	5.52
No Pollution	Low Flow	8	6.6	-2.35	5.52
No Pollution	Low Flow	11	6.6	-2.35	5.52
...	...	...	...	...	...
Pollution	High Flow	12	11.3	2.35	5.52
Sums		179		0	110.45

Every low-flow channel is replaced by the low-flow marginal mean, 6.6; every high-flow channel by the high-flow marginal mean, 11.3. This gives \(SS_\text{Flow} = 110.45\).

10.6.4 SS Interaction

The interaction is the one new piece a factorial ANOVA adds. It asks: do the four cell means behave differently than the two main effects alone would predict?

If the main effects told the whole story, each cell mean would be the grand mean, shifted by that cell’s pollution effect and its flow effect. Take the no-pollution, low-flow cell. The no-pollution marginal mean is \(11.1 - 8.95 = 2.15\) above the grand mean, and the low-flow marginal mean is \(8.95 - 6.6 = 2.35\) below it, so the two main effects predict \(8.95 + 2.15 - 2.35 = 8.75\). The observed cell mean is 9.6. The leftover, 0.85, is the part of that cell the two main effects can’t explain. Those leftovers, one per cell, are what the interaction captures.

To compute it, replace each score with its own cell mean, then subtract that channel’s pollution marginal mean and flow marginal mean, and add back the grand mean once (to avoid subtracting it twice):

\[\bar{x}_\text{cell} - \bar{x}_\text{Pollution marginal} - \bar{x}_\text{Flow marginal} + \bar{x}_\text{grand}\]

Pollution	Flow	Species	Cell Mean	Diff	Diff Squared
No Pollution	Low Flow	10	9.6	0.85	0.72
No Pollution	Low Flow	8	9.6	0.85	0.72
No Pollution	Low Flow	11	9.6	0.85	0.72
...	...	...	...	...	...
Pollution	High Flow	12	10	0.85	0.72
Sums		179		0	14.45

Squaring and summing gives \(SS_\text{Interaction} = 14.45\).

10.6.5 SS Error

The last piece follows by subtraction, since every other term in the identity is already known:

\[SS_\text{Error} = SS_\text{Total} - SS_\text{Pollution} - SS_\text{Flow} - SS_\text{Interaction}\]

\[SS_\text{Error} = 242.95 - 92.45 - 110.45 - 14.45 = 25.6\]

\(SS_\text{Error}\) is the same quantity Chapter 8 called \(SS_\text{Within}\): the spread of channels around their own cell mean, the variation none of the three effects can explain.

10.6.6 Degrees of freedom, mean squares, and F

Each sum of squares becomes a mean square the same way it did in Chapter 8, by dividing by its degrees of freedom. With \(a\) levels of pollution, \(b\) levels of flow, and \(N\) channels in all:

• \(df_\text{Pollution} = a-1 = 1\)
• \(df_\text{Flow} = b-1 = 1\)
• \(df_\text{Interaction} = (a-1)(b-1) = 1\)
• \(df_\text{Error} = N-ab = 20-4 = 16\), one lost for each of the four cell means
• \(df_\text{Total} = N-1 = 19\), and \(1+1+1+16 = 19\)

Each effect gets its own \(F\), and all three share the same denominator:

\[F_\text{Pollution} = \frac{MS_\text{Pollution}}{MS_\text{Error}}, \qquad F_\text{Flow} = \frac{MS_\text{Flow}}{MS_\text{Error}}, \qquad F_\text{Interaction} = \frac{MS_\text{Interaction}}{MS_\text{Error}}\]

Here \(MS_\text{Error} = 25.60/16 = 1.60\). Every effect has 1 degree of freedom, so each effect’s \(MS\) equals its \(SS\), and for pollution \(F = 92.45/1.60 = 57.78\).

10.6.7 Check your work

R’s aov() builds the same table directly. The * in the formula tells R to include both main effects and their interaction; R writes the interaction row as Pollution:Flow and the error row as Residuals.

Source	df	SS	MS	F	p
Pollution	1	92.45	92.45	57.78	< .001
Flow	1	110.45	110.45	69.03	< .001
Pollution:Flow	1	14.45	14.45	9.03	0.008
Residuals	16	25.60	1.60

The SS column matches the hand-calculated values above: 92.45 for Pollution, 110.45 for Flow, 14.45 for the interaction, 25.6 for Error. All three effects are significant here, including the interaction we previewed informally: the difference in the pollution effect between the two flow rates is larger than chance would produce.

This example is balanced: every combination of conditions has the same number of stream channels. With unequal numbers, two things change. The sums of squares depend on the order of terms in aov(), because R computes them sequentially, and the marginal means become model-based estimates (estimated marginal means) rather than simple averages of the scores. Balanced designs avoid both complications, so plan equal sample sizes when you can.

10.6.8 Effect size in a factorial design

The one-way ANOVA in Chapter 8 had one effect and so needed one effect size, \(\eta^2 = SS_\text{Between}/SS_\text{Total}\). A 2x2 runs three tests, so it needs three effect sizes, and the sums of squares we just computed by hand are all the ingredients required.

The direct extension keeps \(SS_\text{Total}\) in the denominator for every effect:

\[ \eta^2 = \frac{SS_\text{Effect}}{SS_\text{Total}} \]

Each effect then reports its share of all the variation in the study, so the three values plus error’s share add to 1. That is easy to interpret, but it has an awkward property: adding a second factor to a study inflates \(SS_\text{Total}\), which shrinks the \(\eta^2\) of a factor whose behavior did not change at all. The same effect looks smaller purely because you studied something else alongside it.

Partial eta-squared avoids that by setting the other factors’ variation aside instead of counting it:

\[ \eta_p^2 = \frac{SS_\text{Effect}}{SS_\text{Effect} + SS_\text{Error}} \]

The question it answers is narrower: of the variation this effect could plausibly explain, once the other factors are held out, how much does it explain? Because each effect gets its own denominator, partial values do not sum to 1 and are always at least as large as the corresponding \(\eta^2\).

This is also why the distinction never came up in Chapter 8. A one-way ANOVA has only one effect, so there are no other factors to set aside, and \(SS_\text{Effect} + SS_\text{Error}\) is \(SS_\text{Total}\). The two formulas give the same number there. They only diverge once a design has more than one factor.

Table 10.2: Eta-squared and partial eta-squared for each effect in the 2x2 design.

Effect	SS	Eta-squared	Partial eta-squared
Pollution	92.45	0.381	0.783
Flow	110.45	0.455	0.812
Pollution x Flow	14.45	0.059	0.361

Flow explains the largest share of the total variation, pollution is close behind, and the interaction is much smaller at \(\eta^2 = 0.06\). The interaction is significant, and it is also the smallest of the three effects, which is a common combination and not a contradiction: significance and magnitude answer different questions. The interaction explains little of the total variation, but it changes what the pollution main effect means.

Which one to report. Most statistical software, including SPSS and R’s effectsize package, gives you partial \(\eta_p^2\) by default, and it is the more common choice in published work. Either is defensible, but say which you used, because the two can differ substantially: pollution is \(\eta^2 = 0.38\) but \(\eta_p^2 = 0.78\) here. Reporting the larger number without labeling it overstates the effect. Never compare a partial value from one study against an unpartialed value from another.

10.7 Interpreting main effects and interactions

With the ANOVA table computed, the next step is to interpret it.

It’s convenient to think of a main effect as one manipulation’s consistent influence. An interaction means the effect of one IV depends on the level of another. So when there is an interaction, at least one main effect is not consistent across conditions.

10.7.1 Reading patterns in a 2x2

In any 2x2 ANOVA, there are eight general outcome patterns, different combinations of main effects and interactions. You don’t need to memorize all eight, but you should learn to read them in a line plot:

Pattern	Main Effect: Pollution	Main Effect: Flow	Interaction
1
2	X
3	X		X
4	X	X
5	X	X	X
6		X
7		X	X
8			X

We’ll create bar and line graphs to illustrate these patterns. Bar graphs are great for seeing differences in means directly, while line graphs help us spot interactions. Figure 10.4 shows the possible patterns in bar graph form. These are abstract illustrations of every combination that can occur, so a few of them describe patterns you would not expect ecologically. Use them to learn to recognize the shapes.

Every figure in this chapter puts flow rate on the x-axis and draws one line (or bar) per pollution condition: blue for no pollution, red for pollution. Here’s what to look for:

• Parallel lines → no interaction.
• Non-parallel or crossing lines → interaction likely.
• Vertical separation between marginal means → main effect.

For example:

• If the no-pollution line sits above the pollution line on average, there’s a main effect of pollution.
• If the lines converge, diverge, or cross, the pollution effect changes across flow rates, showing an interaction.
• If the lines rise or fall on average from low to high flow (say, high flow supports more species in both pollution conditions), that’s a main effect of flow.

The eight panels below are numbered to match the table above, so panel “Pattern 3” in the figure is row 3 in the table, and so on.

Figure 10.4: Eight example patterns of main effects and interactions in a pollution × flow design, numbered to match the pattern table above.

Figure 10.5 shows the same patterns in line graph form:

Figure 10.5: Line graphs showing eight possible outcomes for a pollution × flow design.

In the line graphs, interactions are easiest to see:

• Parallel lines mean no interaction (the pollution effect is the same at both flow rates).
• Crossing lines mean an interaction (the pollution effect differs by flow). The lines don’t have to cross inside the plot: converging or diverging lines would cross if you extended them, so they count too.
• The average gap between the two lines is the main effect of pollution; the lines’ average rise or fall from low to high flow is the main effect of flow.

TipTip: Which factor goes on the x-axis?

The statistics are identical either way, since the same cell means get plotted regardless. Two rules of thumb make the picture easier to read. Put the factor with more levels on the x-axis, since one long line is easier to follow than several short, crowded ones. And draw the factor whose effect you’re asking about as the lines, so its simple effect at each level of the other factor is the gap between the lines. That’s why this chapter puts flow on the x-axis: the question is how the effect of pollution changes with flow.

10.7.2 Simple effects

When an interaction is present, the natural follow-up is to describe each simple effect: the effect of one factor at a single level of the other. Start with the \(\Delta\) values from the cell means, as we did earlier:

\[\Delta_\text{Pollution|Low} = 9.6 - 3.6 = 6.0, \qquad \Delta_\text{Pollution|High} = 12.6 - 10.0 = 2.6\]

Those two numbers describe the interaction: pollution costs 6.0 species under low flow and 2.6 under high flow. But saying “pollution reduced species count under low flow” is still a claim, and like any claim in this book, it needs its own test, not just a look at the plot.

Testing a simple effect asks whether pollution’s effect holds at one specific level of flow, so it looks like a two-sample comparison: no pollution vs. pollution, at low flow only. The temptation is to pull out just the low-flow rows and run a fresh two-sample test on them. Don’t. That throws away the high-flow channels when estimating the error variance, so the estimate uses half the data and half the degrees of freedom. You already have a better estimate of that error variance, from all 20 channels: the \(SS_\text{Error}\) you computed for the omnibus ANOVA above.

The fix is to reuse that pooled estimate. \(MS_\text{Error}\) and its 16 degrees of freedom already describe the noise in this design better than any subset could, so the simple-effects test just plugs them into a two-sample \(t\):

\[t = \frac{\bar{x}_1 - \bar{x}_2}{\sqrt{MS_\text{Error}\left(\dfrac{1}{n_1}+\dfrac{1}{n_2}\right)}}, \qquad df = df_\text{Error}\]

Using \(MS_\text{Error}=1.60\) and \(df=16\) from the omnibus table computed earlier:

Table 10.3: Simple effects of pollution, tested separately within each flow rate, using the pooled error term from the full model.

Simple effect	Difference	t	df	p
Pollution, within Low Flow	6.0	7.50	16	< .001
Pollution, within High Flow	2.6	3.25	16	0.005

Both comparisons clear the critical value, \(t_\text{crit}(16) = 2.12\) at \(\alpha=.05\), two-tailed. So the interaction here does not mean “pollution matters at low flow but not at all at high flow.” That pattern would show up as one significant row and one not. The interaction means “pollution matters at both flow rates, but far more at low flow.”

When a factor has only two levels, as Pollution does here, the simple-effects test is the full answer; there’s nothing left to compare.

10.7.3 Order of interpretation

When we conduct a two-way ANOVA, we always begin by reading the interaction term, because it decides what the main effects mean.

• If the interaction is not significant, examine the two main effects separately, interpreting each as an overall difference averaged across the other factor. If a significant main effect has three or more levels, follow it with Tukey HSD on that factor, exactly as in Chapter 8.

• If the interaction is significant, lead with the simple effects: how one factor behaves at each level of the other. “Pollution reduced species count sharply under low flow, and less under high flow” is a simple-effects sentence; “there was a main effect of pollution” on its own is not enough. Whether the main effect is still worth reporting depends on the direction of the simple effects:

  • If the simple effects all point the same way and differ only in size, the main effect still describes something real. Report it, qualified by the interaction: “There was a main effect of pollution, qualified by a Pollution × Flow interaction.”
  • If one simple effect is near zero, or they point in opposite directions, the main effect is an average that describes no actual condition. Don’t report it as a general effect of that factor.

• If neither the interaction nor either main effect is significant, there is nothing to follow up on. Do not run, or interpret, post-hoc tests on a result that never cleared the omnibus test in the first place.

The three examples below show each case of that rule.

10.7.4 A consistent main effect and an interaction

Figure 10.6: Pollution shows a general (main) effect, but the size of that effect differs by flow rate (interaction).

Figure 10.6 shows both a main effect and an interaction. Unpolluted channels are above polluted ones at both flow rates. That is the main effect. But the size of that gap is larger under low flow (12 vs. 5) than under high flow (14 vs. 11), so there’s an interaction too.

There was a main effect of pollution, qualified by a Pollution × Flow interaction: the size of the pollution effect changed with flow rate, larger under low flow and smaller under high flow. That qualification doesn’t mean the main effect isn’t real. Pollution is associated with lower species counts at both flow rates. Flow changes how much lower, not the direction.

10.7.5 An “averaged” main effect driven by an interaction

Figure 10.7: An apparent main effect of pollution arises from averaging over an interaction.

Figure 10.7 shows a starker 2x2 pattern: a large difference between polluted and unpolluted channels under low flow, and no difference at all under high flow, an interaction visible at a glance.

Yet an ANOVA run on this data can still report a significant main effect of pollution. Averaged across flow rates, unpolluted channels give \((10+5)/2=7.5\) species and polluted channels give \(5\), a main-effect difference of 2.5. That number is real, but it doesn’t describe what pollution actually does: it removes 5 species under low flow and none under high flow, not 2.5 everywhere. The “main effect” here is an artifact of averaging over a strong interaction.

The ANOVA shows a significant main effect of pollution, but it should not be reported as a general effect: the pollution effect was large under low flow and zero under high flow. In a case like this, it’s fair to doubt whether pollution has a general effect on species count at all. It may only do damage under specific conditions, in combination with other factors, rather than on its own.

10.7.6 A main effect masked by an interaction

Figure 10.8: Pollution’s effect reverses direction between flow rates, so the main effect averages to zero even though pollution matters at both.

Figure 10.8 shows the most misleading pattern of the three. Under low flow, pollution reduces species count (8 vs. 4). Under high flow, pollution increases it (8 vs. 12). Both are real, sizable effects, just pointed in opposite directions.

Average across flow rates to get a main effect, and they cancel: unpolluted channels give \((8+8)/2=8\), polluted channels give \((4+12)/2=8\), a main-effect difference of exactly \(0\). An ANOVA on this data would report no main effect of pollution at all, not because pollution doesn’t matter, but because its two real, opposite effects average each other out.

This is the case the “read the interaction first” rule exists for. A non-significant main effect here would be true and misleading: pollution has no effect on average, and a large effect in each condition. Reporting only the main effect would suggest pollution is irrelevant, when it has a large effect at both flow rates, in opposite directions.

10.7.7 Reporting the stream result

Back to the stream experiment. Figure 10.9 plots its four cell means. Pollution lowers species count at both flow rates, so the lines never cross; they converge, because the gap under high flow is smaller. That is the first case of the rule: report the simple effects, and report the main effect of pollution with its qualification attached.

Figure 10.9: Cell means from the stream experiment. Pollution lowers species count at both flow rates, by more under low flow.

ImportantResults reporting: factorial ANOVA

Lead with the interaction, then give the simple effects. Report a main effect only when the simple effects point the same way, and then with its qualification attached. Every effect gets its own effect size, labeled so the reader knows whether it is partialed.

The effect of pollution on macroinvertebrate species count depended on flow rate, F(1, 16) = 9.03, p = .008, \(\eta_p^2 = 0.36\). Under low flow, polluted channels supported 6.0 fewer species than unpolluted ones (3.6 vs. 9.6); under high flow that gap narrowed to 2.6 species (10.0 vs. 12.6). Both main effects were also significant, pollution F(1, 16) = 57.78, p < .001, \(\eta_p^2 = 0.78\), and flow F(1, 16) = 69.03, p < .001, \(\eta_p^2 = 0.81\), but each should be read in light of the interaction: pollution costs a stream far more species when the water is moving slowly.

10.8 More complicated designs

Up until now we have focused on the simplest case for factorial designs, the 2x2 design, with two IVs, each with 2 levels. The same logic extends to larger designs.

10.8.1 3x2 design

This design works like the 2x2, with one more level added to one factor. The test structure is the same (two main effects and one interaction). The interaction is harder to interpret.

Let’s expand our stream example. Suppose we examine macroinvertebrate species count (DV) under three flow rates (low, medium, high) crossed with two pollution conditions (no pollution vs. pollution).

• The main effect of pollution tests whether, on average, polluted and unpolluted channels differ in species count across all flow rates.
• The main effect of flow tests whether mean species count differs among flow rates.
• The interaction tests whether the pollution effect (unpolluted − polluted) changes with flow rate.

In a 2x2 design, you can easily spot interactions by looking for non-parallel lines. In a 3x2, the idea is the same, but with three points per line the patterns are harder to read.

You have three hypotheses in a 3x2 factorial design:

Hypothesis 1: Main effect of flow

• Null (\(H_{0,1}\)): \(\mu_{\text{Low}} = \mu_{\text{Medium}} = \mu_{\text{High}}\)
• Alternative (\(H_{a,1}\)): At least one flow rate mean differs from the others.

Hypothesis 2: Main effect of pollution

• Null (\(H_{0,2}\)): \(\mu_{\text{NoPoll}} = \mu_{\text{Poll}}\)
• Alternative (\(H_{a,2}\)): Mean species count differs between polluted and unpolluted channels.

Hypothesis 3: Pollution × flow interaction

• Null (\(H_{0,3}\)): The pollution effect is the same across all flow rates. \([ (\mu_{\text{NoPoll,Low}}-\mu_{\text{Poll,Low}}) = (\mu_{\text{NoPoll,Med}}-\mu_{\text{Poll,Med}}) = (\mu_{\text{NoPoll,High}}-\mu_{\text{Poll,High}}) ]\)
• Alternative (\(H_{a,3}\)): The pollution effect differs across flow rates (i.e., at least one difference between differences is not equal).

Remember, this is far better than running a separate \(t\)-test of polluted vs. unpolluted channels at each of the three flow rates, because you have more statistical power (higher degrees of freedom) for the tests of interest, and you get a formal test of the interaction between factors which is often scientifically interesting.

We might expect data like shown in Figure 10.10:

Figure 10.10: Example means for a 3x2 factorial design in environmental science

The figure shows made-up means for all six conditions.

1. Main Effect of Pollution: The main effect of pollution is evident. Averaged across flow rates, unpolluted channels support higher species counts (10 vs. 7.3). The “No Pollution” line sits above the “Pollution” line at low and medium flow, and the two coincide at high flow.

2. Main Effect of Flow: There is a main effect of flow as well. Averaged over pollution condition, species counts are lowest at low flow (8) and higher at medium and high flow (9 each).

3. Interaction Between Pollution and Flow: There is an interaction. Remember, an interaction occurs when the effect of one IV depends on the levels of another. The pollution gap is 6 species at low flow, 2 at medium, and 0 at high. So the size of the pollution effect changes with flow rate, which is also why the pollution main effect above describes no single flow condition well. You would have to conduct an inferential test on the interaction term to see if these differences were likely or unlikely to be due to sampling error.

If there was no interaction and no main effect of flow, we would see something like the pattern in Figure 10.11.

Figure 10.11: Example means for a 3x2 design with only one main effect

What would you say about the interaction if you saw the pattern in Figure 10.12?

Figure 10.12: Example means for a 3x2 design showing a different interaction pattern

The correct answer is that there is evidence in the means for an interaction. Remember, we are measuring the pollution effect three times, once at each flow rate. The pollution effect is the same at low and medium flow, but it is much smaller at high flow. The size of the pollution effect depends on the level of the flow IV, so here again there is an interaction.

10.9 Designs Beyond This Course

Larger factorial designs use the same logic as the 2x2. But complexity grows quickly, both in the number of effects and in how hard they are to interpret.

10.9.1 2x2x2 (Three-Factor) Designs

A 2x2x2 design includes three independent variables, each with two levels. That creates:

• three main effects (one per variable)
• three two-way interactions (A×B, A×C, B×C)
• one three-way interaction (A×B×C)

You can think of it as two 2x2 plots side-by-side; one for each level of the third variable.

A three-way interaction means that one of the two-way interactions changes across the third variable. For example, the pollution × flow interaction might look different in summer than in winter.

In short, interactions can themselves interact. The math is identical in structure, but the number of effects grows fast, and the graphs become difficult to interpret. In applied research, most studies stop at two factors unless there’s a strong reason to add more.

10.9.2 Mixed Designs

Sometimes one factor is between-subjects (e.g., different sites or treatments) and another is within-subjects (e.g., repeated measures over time). This is called a mixed design.

The interpretation of main effects and interactions is the same, but the error terms used in the F-tests differ, and the data structure must account for repeated measures. These models are important in longitudinal and environmental time-series studies, but they’re beyond our current scope.

Factorial ANOVA can handle many experimental structures, but each added factor makes the results harder to interpret, so use only the factors your question needs.

10.10 Chapter Summary

Why it matters. Single-factor designs can tell you whether something has an effect. Factorial designs tell you what changes that effect, which is usually the more interesting scientific question, and often the more realistic one for environmental systems where drivers act together. An interaction changes what every other term in the table means, so interpreting the table is part of the analysis.

Core ideas

• Fully crossed means every combination. A factorial design pairs every level of each IV with every level of the others. The notation says the structure: 2x2 is four conditions, 3x2 is six.
• A 2x2 tests three things. One main effect per factor, plus their interaction. That means three null hypotheses, and you should be able to state all three in plain language before running anything.
• An interaction is a difference between differences. It asks whether the effect of one factor changes across levels of another. The interaction sum of squares computes this difference between differences.
• The partition extends the one-way logic. \(SS_\text{Total}\) now splits into each main effect, the interaction, and error (\(SS_\text{Error}\), Chapter 8’s \(SS_\text{Within}\)). Each effect gets its own \(F\), all sharing \(MS_\text{Error}\) as the denominator. Working one table through by hand shows that the interaction term is the only new piece.
• Three tests, three effect sizes. \(\eta^2\) divides each effect’s \(SS\) by \(SS_\text{Total}\); partial \(\eta_p^2\) divides it by that effect plus error. Partial values are at least as large and do not sum to 1, so label which one you are reporting. A significant effect can still be a small one, which is exactly the case for this chapter’s interaction.
• Read the interaction first. If it’s significant, lead with simple effects: what factor A does at each level of B, as \(\Delta\) values from the cell means, tested with the pooled \(MS_\text{Error}\). A main effect can still be reported, qualified, when the simple effects all point the same way; when one is near zero or they reverse, the main effect describes no actual condition. When the interaction is not significant, the main effects stand on their own, with Tukey HSD for a factor with three or more levels.
• Learn to read the picture. In a line plot, parallel lines mean no interaction, and lines that cross, inside the plot or if extended, mean an interaction. The average gap between the lines is one main effect, and the lines’ average rise or fall is the other. There are eight possible patterns in a 2x2, and recognizing them by eye is faster than parsing the table.
• An averaged main effect can be an artifact. A factor that matters a lot in one condition and not at all in another can still produce a “significant main effect” once you average across conditions. That number is correct arithmetic, but it matches the pollution effect at neither flow rate.
• Bigger designs, same tests, harder reading. A 3x2 has the same three tests as a 2x2, but three points per line are harder to read than two. Beyond that, a three-factor design brings three main effects, three two-way interactions, and a three-way interaction, and interpretability falls off fast. Use only as many factors as your question actually needs.