How do we design our trial to ensure valid results?

In this step Dr Sara Brookes discusses external and internal validity which are essential in the design and delivery of a randomised controlled trial.

External validity relates to how valid it would be to apply the findings of a trial to people or patients other than the original trial participants.

“So, it is the extent to which a trial’s results can be generalized to other similar patients, now and in the future (what we call our ‘population of interest’ or ‘target population’)[ Jüni BMJ 2001]”.

To maximise external validity, participants recruited to the trial must be representative of our target population, so carefully derived inclusion and exclusion criteria for potential participants are essential. Consideration must also be given to the intervention being evaluated and how easily it can be replicated in practice. We also need to think about our comparator – if this is a standard treatment, does it differ from one general practice, or hospital, or country to another? We also need to give attention to the setting in which the trial is carried out. For example, a beneficial effect observed in a trial conducted within a teaching hospital may be impossible to replicate in clinical practice where patients are also treated in non-teaching hospitals with different expertise and resources.

The absence of consideration of external validity has been blamed for widespread under-implementation of treatments into clinical practice that have been demonstrated to be beneficial within a randomised trial. [Rothwell Lancet 2005].

Internal validity relates to the question of whether the trial results are valid for the participants recruited into the trial.

The internal validity of a trial implies that any differences in outcomes observed between the treatment groups at the end of the trial can (apart from random error) be attributed to the treatments themselves and not any other factors. As described in an earlier session, the process of randomisation, if done properly, ensures that participants allocated to each treatment group are comparable in terms of all known and unknown factors that influence participants’ outcomes.

Figure 1: Internal and external validity of a randomised controlled trial.

Figure 1: Internal and external validity of a randomised controlled trial. However, internal validity is threatened by many potential sources of bias. Through a series of scenarios and discussions, you will be introduced to several types of bias commonly affecting randomised controlled trials. Within such a trial, bias relates to any aspect or process that occurs during its conduct (or analysis), that leads to an inaccurate estimate of the treatment effect. Such bias may exaggerate or underestimate the benefits of the new treatment compared to the control treatment. In addition, if a trial is not internally valid and the results are inaccurate (biased), then there is little rationale for generalising the results to people outside of the trial (limited external validity).

It is worth noting that different names are often used to describe the same type of bias, which can be a little confusing. What is important is to be aware of how bias might be caused and what impact it might have on a trial rather than worrying too much about the names. Following the scenarios, we will consider ways to minimise such biases in a trial.

Additional Resources

Jüni P, Altman DG, Egger M. Assessing the quality of controlled clinical trials. BMJ. 2001 Jul 7;323(7303):42-6.
Rothwell PM. External validity of randomised controlled trials: “to whom do the results of this trial apply?”. The Lancet. 2005 Jan 1;365(9453):82-93.

For Canvas related technical support please see the 24/7. The 'Help' icon on the left-hand navigation