Researchers who do not review past tests of treatments before embarking on new studies may not realise that uncertainties about treatment effects have already been convincingly addressed. This means that some patients are taking part in research unnecessarily and being denied treatment that can help them. For example, long after there was reliable evidence that giving antibiotics to patients having bowel surgery reduced their chances of dying from complications of the operation, researchers continued to do comparison studies that involved withholding antibiotics from half the patients participating in the studies (see Chapter 5). Conversely, sometimes when previous results are reviewed it soon becomes apparent that reliable evidence is lacking – so new studies are definitely needed.

And, as we pointed out in Chapter 1 (page 9), patients can also suffer when researchers have not reviewed relevant evidence from animal research systematically before beginning to test treatments in patients. In that example, had the results of animal experiments been reviewed, clinical trials of the drug nimodipine in stroke patients would never have been done.

The idea of reviewing evidence systematically is far from new. The subtitle of James Lind’s 1753 Treatise of the Scurvy, in which he reported his fair test of then favoured remedies (see Chapter 1), indicates that it contains ‘A critical and chronological view of what has been published on the subject’.

AVOIDING BIASED COMPARISONS

To ensure that comparisons are fair, several sources of bias must be identified and minimised; if they are not, a new treatment might appear better than an existing one when in fact it is not.

In considering individual research studies, this could result from:

• comparing the progress of relatively well patients given a new treatment with the progress of relatively ill patients given a standard treatment
• biased assessment of the results of treatment – for example, by comparing the opinions of patients or doctors who know that they have used an expensive new treatment, and who think this is better, with the opinions of those who know that they have had an existing standard treatment

And in reviewing several similar studies, from:

• considering only studies that show a new treatment in a favourable light, without including other ‘negative’ studies that have either failed to confirm it has benefits, or suggest that it may be harmful (‘negative’ studies are often not reported)
• biased selection from and interpretation of the available evidence.

Often people trying to decide which treatments to use simply do not recognise that these biases result in unfair tests of treatments. Sadly, however, people with vested interests sometimes exploit biases to make treatments look as if they are better than they really are. This happens when some researchers – usually but not always for commercial reasons – deliberately ignore existing evidence. They design, analyse, and report studies to paint their own results for a particular treatment in a favourable light.

UNDERSTANDING BIAS

Biases in tests of treatment are those influences and factors that can lead to conclusions about treatment effects that differ from the truth systematically, and not just by chance. Although many kinds of biases can distort the results of health research, the biases that must be minimised in fair tests of treatments are:

• biases due to differences in people compared;
• biases due to differences in the way treatment effects are assessed;
• biased reporting of the available evidence; and
• biased selection from the available evidence