The Art of Interpreting Field Trials…or, Can Good Data Lead to Bad Results?

There are a number of reasons why a field crop trial fails to show a statistically significant response. The most obvious is that the treatment really doesn't have any effect! The trial results reflect what would normally be expected in the field.

However, there are other situations where the treatment is actually having an effect that the trial is not been able to detect. These situations include:

• Large underlying field variability, so the treatment effect cannot be observed through the random variation
  
• Insufficient replication
• External effects increasing the random variability in the trial (e.g. wildlife damage)
• Conditions aren't right for the effect to be expressed (e.g. disease isn't present so there is no response to a fungicide treatment)
• Plot design did not allow the difference to be observed.

It is this final situation that I will be focusing on, as it is most relevant to many questions of nutrient use efficiency.

Consider the situation presented in Figure 1, where the response to a normal fertilizer is compared to the response to an imaginary "enhanced" fertilizer. The maximum yield for both is the same, but the enhanced fertilizer reaches the maximum yield at 100 kg of the nutrient rather than 150 kg. This would obviously mean a significant savings for a farmer who could achieve the same yield with two-thirds of the fertilizer rate.

text equivalent of graphic

Point A - On The Plateau
Comparisons of two products are set up using the least possible number of treatments, but remember that we don't know ahead of time what type of response to expect. This can lead to misleading trial results. One common trial design is to compare the usual rate of the normal fertilizer to the same rate of the enhanced fertilizer, as shown at point "A" on the figure. This trial would not show any difference between the products.

Point B - Zero Rate
In this particular example, adding a zero treatment ("B") would not make the difference any clearer, although it is certainly helpful where there is a difference in the maximum yield from each treatment. A common marketing ploy is to include the Normal fertilizer at the usual rate ("A"), with the Enhanced product at a reduced rate ("B"), and conclude that the Enhanced material is more efficient because it gave similar yield for less input. While this is correct in this example, this conclusion is accidental rather than firm proof because the same results would be seen if the rates for both products were on the yield plateau.

Point C - Expected Response Difference
The valid comparison that would show the difference between the two products would include both products at a rate where there is a difference in response, as at point "C". Since we don't know prior to the trial exactly where this point will be, the most reliable design includes multiple rates of both products so that a yield response curve can be drawn for each.

The take-home message from this is not that every trial needs multiple rates, but rather that the expected response from a given input needs to be considered in the design of the trial. The only design we can reject out-of-hand is the one where each product is used at single rates, but that are different between products, since it can never give unequivocal results. Trials where we expect an overall yield increase are valid with the zero plus high rate treatments. Where differences in nutrient efficiency are expected, however, it is important to include multiple rates of each treatment so that response curves can be drawn.