30 Randomized experiments versus observational studies
$$
$$
Here is a good place to draw a distinction between two types of studies which we shall call randomized experiments and observational studies:
- Randomized experiments place subjects under different conditions through random assignment and compare outcomes:
- Randomly assign plant clones to different drought conditions and compare C0\(_2\) uptake.
- Randomly assign tracts of a field to different fertilizer treatments and compare yields.
- Randomly assign half of a group of rats to a period of radon exposure, keeping the other half unexposed, and compare metrics of carcinogenesis between the groups.
- Observational studies compare subjects with different properties or which exist under different circumstances:
- Compare development metrics of children from different socio-economic backgrounds.
- Compare attitudes toward recycling of college students in Columbia and Greenville.
The cardinal difference between a randomized experiment and an observational study is that in an experiment the investigator randomly assigns subjects to different conditions, whereas in an observational study the investigator does not assign subjects to conditions, but observes the subjects without changing their circumstances.
Observational studies are beset with the problem of confounding variables. A confounding variable is any unrecorded property or circumstance of the subjects which is associated with the outcome of interest as well as with the any of the properties or circumstances of the subjects which are recorded in the study. For example, an observational study may attempt to relate children’s school grades (the outcome of interest) to the income of their parents, but there may be no information available about alcoholism in the children’s homes; if the rate of alcoholism in the home is related to income as well as to children’s grades in school, then alcoholism in the home is a confounding variable. If this is the case, a study which measures only children’s grades and the income of their parents cannot attribute lower grades to lower parental income, even if the data show that children whose parents earn less make poorer grades.
Confounding variables lurk in every observational study—yes, in all of them. It is therefore inappropriate to draw conclusions of causality from any observational study. You cannot say, “Circumstance A causes outcome Y,” on the basis of an observational study, because there may be an unrecorded circumstance B commonly occurring with A which is the true agent causing outcome Y. One cannot know. One can only say, “Outcome Y is associated with circumstance A,” by which we mean that the two often occur together, but A may or may not exert a causal influence upon Y.
Experiments eliminate the problem of confounding variables through the random assignment of the subjects to different conditions. The process of random assignment disassociates all unrecorded properties or circumstances of the subjects from the conditions of interest in the study. After randomization, no property or circumstance of the subjects has an association with the conditions imposed by the investigator. As a result, randomized experiments, in contrast to observational studies, are capable of yielding causal conclusions.
Ronald A.~Fisher (1890)–(1962) was an early exponent of the randomized experiment. In the next section we will study some methods developed by him.