If the populations the observations were drawn from are not normally distributed (or are not reasonably compatible with other assumptions of a parametric method, such as equal variances in all the treatment groups), parametric methods become quite unreliable because the mean and standard deviation, the key elements of parametric statistics, no longer completely describe the population. In fact, when the population substantially deviates from normality, interpreting the mean and standard deviation in terms of a normal distribution can produce a very misleading picture.

For example, recall our discussion of the distribution of heights of the entire population of Jupiter. The mean height of all Jovians is 37.6 cm in Figure 2-3A and the standard deviation is 4.5 cm. Rather than being equally distributed about the mean, the population is skewed toward taller heights. Specifically, the heights of Jovians range from 31 to 52 cm, with most heights around 35 cm. Figure 2-3B shows what the population of heights would have been if, instead of being skewed toward taller heights, they had been normally distributed with the same mean and standard deviation as the actual population (in Figure 2-3A). The heights would have ranged from 26 to 49 cm, with most heights around 37 to 38 cm. Simply looking at Figure 2-3 should convince you that envisioning a population on the basis of the mean and standard deviation can be quite misleading if the population does not, at least approximately, follow the normal distribution.

The same thing is true of statistical tests that are based on the normal distribution. When the population the samples were drawn from does not at least approximately follow the normal distribution, these tests can be quite misleading. In such cases, it is possible to use the ranks of the observations rather than the observations themselves to compute statistics that can be used to test hypotheses. By using ranks rather than the actual measurements it is possible to retain much of the information about the relative size of responses without making any assumptions about how the population the samples were drawn from is distributed. Since these tests are not based on the parameters of the underlying population, they are called nonparametric or distribution-free methods.^[1] All the methods we will discuss require only that the distributions under the different treatments have similar shapes, but there is no restriction on what those shapes are.^[2]

When the observations are drawn from normally distributed populations, the nonparametric methods in this chapter are about 95% as powerful as the analogous parametric methods. As a result, power for these tests can be estimated by computing the power of the analogous parametric test. When the observations drawn from populations that are not normally distributed, nonparametric methods are not only more reliable but also more powerful than parametric methods.

Unfortunately, you can never observe the entire population. So how can you tell whether the assumptions such as normality are met, to permit using the parametric tests such as analysis of variance? The simplest approach is to plot the observations and look at them. Do they seem compatible with the assumptions that they were drawn from normally distributed populations with roughly the same variances, that is, within a factor of 2 to 3 of each other? If so, you are probably safe in using parametric methods. If, on the other hand, the observations are heavily skewed (suggesting a population such as the Jovians in Fig. 2-3A) or appear to have more than one peak, you probably will want to use a nonparametric method. When the standard deviation is about the same size or larger than the mean and the variable can take on only positive values, this is an indication that the distribution is skewed. (A normally distributed variable would have to take on negative values.) In practice, these simple rules of thumb are often all you will need.

There are two ways to make this procedure more objective. The first is to plot the observations as a normal probability plot. A normal probability plot has a distorted vertical scale that makes normally distributed observations plot as a straight line (just as exponential functions plot as a straight line on a semilogarithmic graph). Examining how straight the line is will show how compatible the observations are with a normal distribution. One can also construct a χ² statistic to test how closely the observed data agree with those expected if the population is normally distributed with the same mean and standard deviation. Since in practice simply looking at the data is generally adequate, we will not discuss these approaches in detail.^[3]

Unfortunately, none of these methods is especially convincing one way or the other for the small sample sizes common in biomedical research, and your choice of approach (i.e., parametric versus nonparametric) often has to be based more on judgment and preference than hard evidence.

One informal approach is to do the analysis with both the applicable parametric and nonparametric methods, then compare the results. If the data are from a normal population, then the parametric method should be more sensitive (and so provide a lower P value), whereas if there is substantial nonnormality then the nonparametric method should be more sensitive (and so provide the lower P value). If the data are only slightly nonnormal, the two approaches should give similar results.

Things basically come down to the following difference of opinion: Some people think that in the absence of evidence that the data were not drawn from a normally distributed population, one should use parametric tests because they are more powerful and more widely used. These people say that you should use a nonparametric test only when there is positive evidence that the populations under study are not normally distributed. Others point out that the nonparametric methods discussed in this chapter are 95% as powerful as parametric methods when the data are from normally distributed populations and more reliable when the data are not from normally distributed populations. They also believe that investigators should assume as little as possible when analyzing their data. They therefore recommend that nonparametric methods be used except when there is positive evidence that parametric methods are suitable. At the moment, there is no definitive answer stating which attitude is preferable. And there probably never will be such an answer.

(1)

where X = 0, 1, 2,…, N; N! = N(N – 1)(N – 2)···1; and 0! = 1 by definition (see Problem 6.34).

Using EXCEL, the solution is =1-BINOMDIST(3,6,0.5,1) or 0.34375 which is the same as 11/32. Since Pr{X≥ 4} = 1 – Pr{X≤ 3}, and BINOMDIST(3,6,0.5,1) = Pr{X ≤ 3}, this computation will give the probability of at least 4 heads.

Some properties of the binomial distribution are listed in Table 7.1.

Figure 1-2 (A) Results of an experiment in which researchers administered five different doses of a drug to five different people and measured their daily urine production. Output increased as the dose of drug increased in these five people, suggesting that the drug is an effective diuretic in all people similar to those tested. (B) If the researchers had been able to administer the drug to all people and measure their daily urine output, it would have been clear that there is no relationship between the dose of drug and urine output. The five specific individuals who happened to be selected for the study in panel A are shown as shaded points. It is possible, but not likely, to obtain such an unrepresentative sample that leads one to believe that there is a relationship between the two variables when there is none. A set of statistical procedures called tests of hypotheses permits one to estimate the chance of getting such an unrepresentative sample.

The only statement that can be made with absolute certainty is that as the drug dose increased, so did urine production in the five people in the study. The real question of interest, however, is: How is the drug likely to affect all people who receive it? The assertion that the drug is effective requires a leap of faith from the limited experience, shown in Figure 1-2A, to all people.

Now, pretend that we knew how every person who would ever receive the drug would respond. Figure 1-2B shows this information. There is no systematic relationship between the drug dose and urine production! The drug is not an effective diuretic.

How could we have been led so far astray? The dark points in Figure 1-2B represent the specific individuals who happened to be studied to obtain the results shown in Figure 1-2A. While they are all members of the population of people we are interested in studying, the five specific individuals we happened to study, taken as a group, were not really representative of how the entire population of people responds to the drug.

Looking at Figure 1-2B should convince you that obtaining such an unrepresentative sample of people, though possible, is not very probable. One set of statistical procedures, called tests of hypotheses, permit you to estimate the likelihood of concluding that two things are related as Figure 1-2A suggests when the relationship is really due to bad luck in selecting people for study, and not a true effect of the drug. In this example, one can estimate that such a sample of people will turn up in a study of the drug only about 5 times in 1000 when the drug actually has no effect.

Of course it is important to realize that although statistics is a branch of mathematics, there can be honest differences of opinion about the best way to analyze a problem. This fact arises because all statistical methods are based on relatively simple mathematical models of reality, so the results of the statistical tests are accurate only to the extent that the reality and the mathematical model underlying the statistical test are in reasonable agreement.

Suppose that in a particular sample a set of possible events E₁, E₂, E₃,…, E_k (see Table 12.1) are observed to occur with frequencies o₁, o₂, o₃,…, o_k, called observed frequencies, and that according to probability rules they are expected to occur with frequencies e₁, e₂, e₃,…, e_k, called expected, or theoretical, frequencies. Often we wish to know whether the observed frequencies differ significantly from the expected frequencies.

then computed its value for the data observed in an experiment. Next, we compared the result with the value t_α that defined the most extreme 100α percent of the possible values to t that would occur (in both tails) if the two samples were drawn from a single population. If the observed value of t exceeded t_α (given in Table 4-1), we reported a “statistically significant” difference, with P <>α As Figure 4-4 showed, the distribution of possible values of t has a mean of zero and is symmetric about zero.

On the other hand, if the two samples are drawn from populations with different means, the distribution of values of t associated with all possible experiments involving two samples of a given size is not centered on zero; it does not follow the t distribution. As Figures 6-3 and 6-5 showed, the actual distribution of possible values of t has a nonzero mean that depends on the size of the treatment effect. It is possible to revise the definition of t so that it will be distributed according to the t distribution in Figure 4-4 regardless of whether or not the treatment actually has an effect. This modified definition of t is

Notice that if the hypothesis of no treatment effect is correct, the difference in population means is zero and this definition of t reduces to the one we used before. The equivalent mathematical statement is

In Chapter 4 we computed t from the observations, then compared it with the critical value for a “big” value of t with ν = n₁ + n₂ − 2 degrees of freedom to obtain a P value. Now, however, we cannot follow this approach since we do not know all the terms on the right side of the equation. Specifically, we do not know the true difference in mean values of the two populations from which the samples were drawn, μ₁ − μ₂. We can, however, use this equation to estimate the size of the treatment effect, μ₁ − μ₂.

Instead of using the equation to determine t, we will select an appropriate value of t and use the equation to estimate μ₁ − μ₂. The only problem is that of selecting an appropriate value for t.

By definition, 100α percent of all possible values of t are more negative than −t_α or more positive than +t_α. For example, only 5% of all possible t values will fall outside the interval between −t_.05 and +t_.05, where t_.05 is the critical value of t that defines the most extreme 5% of the t distribution (tabulated in Table 4-1). Therefore, 100(1 − α) percent of all possible values of t fall between −t_α and +t_α. For example, 95% of all possible values of t will fall between −t_.05 and +t_.05.

Every different pair of random samples we draw in our experiment will be associated with different values of, X¯₁ − X¯₂ and s_x¯1−x¯2 and 100(1 − α) percent of all possible experiments involving samples of a given size will yield values of t that fall between −t_α and +t_α. Therefore, for 100(1 − α) percent of all possible experiments

Solve this equation for the true difference in sample means

In other words, the actual difference of the means of the two populations from which the samples were drawn will fall within t_a standard errors of the difference of the sample means of the observed difference in the sample means. (t_a has ν = n₁ + n₂ − 2 degrees of freedom, just as when we used the t distribution in hypothesis testing). This range is called the 100(1 − α) percent confidence interval for the difference of the means. For example, the 95% confidence interval for the true difference of the population means is

This equation defines the range that will include the true difference in the means for 95% of all possible experiments that involve drawing samples from the two populations under study.

Since this procedure to compute the confidence interval for the difference of two means uses the t distribution, it is subject to the same limitations as the t test. In particular, the samples must be drawn from populations that follow a normal distribution at least approximately.^[2]

If all values of the variables satisfy an equation exactly, we say that the variables are perfectly correlated or that there is perfect correlation between them. Thus the circumferences C and radii r of all circles are perfectly correlated since C = 2πr. If two dice are tossed simultaneously 100 times, there is no relationship between corresponding points on each die (unless the dice are loaded); that is, they are uncorrelated. Such variables as the height and weight of individuals would show some correlation.

When only two variables are involved, we speak of simple correlation and simple regression. When more than two variables are involved, we speak of multiple correlation and multiple regression. This chapter considers only simple correlation. Multiple correlation and regression are considered in Chapter 15.

It is frequently desirable to express this relationship in mathematical form by determining an equation that connects the variables.