Question

What are the two types of numbers in experimental calculations?

Answer 1

The logistics of a proposed larger study

Gain familiarity with the experimental material,

Ensure that treatments are not obviously excessively mild or severe

Check that staff are sufficiently well trained in the necessary procedures

Ensure that all steps in a proposed future experiment are feasible.

Gain some information on variability, although this will not usually be sufficiently reliable to form the basis of power analysis calculations of sample size.

Exploratory experiments can be used to generate data with which to develop hypotheses for future testing. They may “work” or “not work”. They may have no clearly stated hypothesis (“let’s see what happens if..” is not a valid hypothesis on which to base an experiment).

Often they will measure many outcomes (characters). Picking out “interesting looking differences” (known as data snooping) and then doing a hypothesis test to see if the differences are statistically significant will lead to serious overestimation of the magnitude of a response and excessive numbers of false positive results. Such differences should always be tested in a controlled experiment where the hypothesis is stated a priori before the results are published.

Depending on the nature of the data, statistical analysis will often be done using an analysis of variance (ANOVA)

Confirmatory experiments are used to test some relatively simple hypothesis stated a priori. This is the type of experiment mainly considered in this web site.

The basic principles are:

Experiments involve comparisons between two or more groups

Their aim is to test a “null hypothesis” that there is no difference among the groups for the specified outcome.

If the null hypothesis is rejected at a certain level of probability (often 5%) this means that the probability of getting a result as extreme as this or more extreme in the absence of a true effect is 5% (assuming also that the experiment has been properly conducted). So it is assumed that such a difference is likely to be the result of the treatment. But, it could be a false positive resulting from sampling variation.

Failure to reject the null hypothesis does not mean that the treatment has no effect, only that if there is a real effect this experiment failed to detect it. “Absence of evidence is not evidence of absence”.

Experimental subjects need to be independently replicated because individuals (of whatever type) vary. Two subjects can normally be regarded as being independent if they can theoretically receive different treatments.

Subjects need to be assigned to groups, held in the animal house and measured at random in order to minimise the chance of bias (a systematic difference between groups)

As far as possible the experimenter should be “blind” with respect to the treatment group in order to minimise bias.

The experiments need to be powerful, i.e. they should have a high probability of detecting an effect of clinical or scientific importance if it is present.

In many cases a formal experimental designsuch as a “completely randomised”, “randomised block”, “Latin square” etc. design will be used.

In most cases it is useful if the experiment has a wide range of applicability. In other words the results should hold true under a range of different conditions (different strains, both sexes, different diets, different environments etc.). At least some of these factors should be explored using factorial and randomised block designs.

Experiments to explore relationships between variables. A typical example would be a growth curve or a dose-response relationship. In these experiments the aim is often to test whether the two variables are associated, and if so, what is the nature of that relationship. The typical statistical analysis involves correlation and/or regression.