Do not adjust for the baseline value in observational studies

Medicine and numbers
    ()

    sporsmal_grey_rgb
    Article

    In randomised trials with measurements at two or more time points, it is recommended to adjust for the baseline value, for example by using an analysis of covariance. In an observational study, such an analysis will lead to biased effect estimates and problems with interpretation.

    When we randomise patients into two different treatment groups, the expected distribution of patient characteristics will be the same in both groups at the start of the trial. Any differences between the groups will be random, not inherent. In this situation, an analysis that adjusts for the baseline value will have the highest statistical power. Adjusting for the baseline value is therefore recommended, for example by using analysis of covariance (ANCOVA) (1). It is tempting to think that the recommendation will be the same when we analyse observational data, but it is not. If there is a systematic difference between the two groups at the start of the study, an analysis that adjusts for the baseline value will answer a different research question when compared to an analysis that studies, for example, differences in a change between two groups. This is known as Lord's paradox (2).

    Two groups with different distributions

    Two groups with different distributions

    Figure 1 shows an example with simulated data where two groups, each with 200 individuals, have different distributions of a relevant variable both at baseline and at follow-up. At the start, the average value is 101 in Group A and 155 in Group B. The individuals are followed up over a given period, and the variable is measured again. The simulation has been designed in such a way that the true change over time is 0 is both groups. Because of random variations, some individuals have an increased value relative to the baseline, while others have a reduced value. The observed average change is 0.9 in Group A and −0.2 in Group B, and a 95 % confidence interval for the difference in change between the groups ranges from −0.8 to 3.1, which agrees well with the true underlying difference of 0 and the assumption that random variations have caused the difference.

    If we analyse the data with analysis of covariance, which adjusts for the baseline value, we obtain an average difference between the groups of −4.5, with a 95 % confidence interval ranging from −8.6 to −0.4. Apparently, the changes within the two groups are therefore different. This agrees poorly with what we know to be the fact, namely that the average change is 0 in both groups.

    What went wrong?

    What went wrong?

    Several explanations for this paradox have been put forward (3), p. 124–5, (4). In randomised trials we expect the distribution of the baseline values to be the same. Then, it makes sense to compare the groups adjusting for the baseline value, and estimate the difference between the groups given the same baseline value.

    The question is whether a similar comparison is meaningful when there may be systematic differences between the groups at the start of the study. We believe that it is not, and that by using an analysis of covariance we will answer a completely different research question than when comparing the average change in the two groups using, for example, a t-test. With an analysis of covariance, we attempt to answer the following question: How large is the difference between the groups at the time of follow-up, given that the baseline value is the same? It is probably of no interest to answer this question when we know that the baseline value is different.

    We believe that some of the paradox lies in the interpretation of the result, and that the problem occurs because one fails to give sufficient thought to the research question that the analysis answers. Moreover, in a randomised trial, the group allocation will not affect the baseline value, since this has been measured before the intervention (exposure). In an observational study, the participants will often also have been exposed before the start of the study, and this may have impacted on the baseline value. The baseline value will thereby be a mediator, and an analysis of covariance will lead to biased estimates (4).

    In any case, our simple advice is this: adjust for the baseline value in randomised trials, but not in observational studies.

    PDF
    Print
    Reply to article

    Recent Articles

    Made by Ramsalt Using Ramsalt Media