Evidence about effects of interventions

Studies that do not use control groups

Uncontrolled studies are studies where the researchers describe what happens when participants are provided with an intervention, but the intervention is not compared with other interventions. Examples of uncontrolled study designs are case reports, case series and before and after studies. These study designs were explained in Chapter 2. The big problem with uncontrolled studies is that when participants are given an intervention and simply followed for a period of time with no comparison against another group, it is impossible to tell how much (if any) of the observed change is due to the effect of the intervention itself or is due to some other factor or explanation. There are some alternative explanations for effects seen in uncontrolled studies, and these need to be kept in mind if you use this type of study to guide your clinical decision making. Some of the forms of bias that commonly occur in uncontrolled studies are described below.

Controlled studies

It should now be clear that having a control group which can be compared with the intervention group in a study is the best way of making sure that bias and extraneous factors that can influence the results of a study are limited. However, it is not as simple as just having a control group as part of the study. The way in which the control group is created can make an enormous difference to how well the study design actually controls for bias.

Was the assignment of participants to groups randomised?

Randomised controlled trials, by definition, randomise participants to either the experimental or the control condition. The basic principle of randomisation is that each participant has an equal chance of being assigned to any group, such that any difference between the groups at the beginning of the trial can be assumed to be due to chance. The main benefit of randomisation is related to the idea that, this way, both known and unknown participant characteristics should be evenly distributed between the intervention and control groups. Therefore, any differences between groups that are found at the end of the study are likely to be because of the intervention.⁹

Random allocation is best done using a random numbers table which can be computer-generated. Sometimes it is done by tossing a coin or ‘pulling a number out of a hat’. Additionally, there are different randomisation designs that can be used and you should be aware of them. Researchers may choose to use some form of restriction, such as blocking or stratification, when allocating participants to groups in order to create a greater balance between the groups at baseline in known characteristics.¹⁰ Different randomisation designs are summarised below.

Was the allocation sequence concealed?

As we have seen, the big benefit of a randomised controlled trial over other study designs is the fact that participants are randomly allocated to the study groups. However, the benefits of randomisation can be undone if the allocation sequence is manipulated or interfered with in any way. As strange as this might seem, a health professional who wants their patient to receive the intervention being evaluated may swap their patient's group assignment so that their patient receives the intervention being studied. Similarly, if the person who recruits participants to a study knows which condition the participants are to be assigned to, this could influence their decision about whether or not to enrol them in the study. This is why assigning participants to study groups using alternation methods, such as every second person who comes into the clinic, or assigning participants by methods such as date of birth is problematic because the randomisation sequence is known to the people involved.⁹

Knowledge about which group a participant will be allocated to if they are recruited into a study can lead to the selective assignment of participants, and thus introduce bias into the trial. This knowledge can result in manipulation of either the sequence of groups that participants are to be allocated to or the sequence of participants to be enrolled. Either way, this is a problem. This problem can be dealt with by concealing the allocation sequence from the people who are responsible for enrolling patients into a trial or from those who assign participants to groups, until the moment of assignment.¹² Allocation can be concealed by having the randomisation sequence administered by someone who is ‘off-site’ or at a location away from where people are being enrolled into the study. Another way to conceal allocation is by having the group allocation placed in sealed, opaque envelopes. Opaque envelopes are used so that the group allocation cannot be seen if the envelope is held up to the light! The envelope is not to be opened until the patient has been enrolled into the trial (and is therefore now a participant in the study).

Hopefully, the article that you are appraising will clearly state that allocation was concealed, or that it was done by an independent or off-site person or that sealed opaque envelopes were used. Unfortunately, though, many studies do not give any indication about whether allocation was concealed,^13,14 so you are often left wondering about this, which is frustrating when you are trying to appraise a study. It is possible that some of these studies did use concealed allocation, but you cannot tell this from reading the article.

Were the groups similar at the baseline or start of the trial?

One of the principal aims of randomisation is to ensure that the groups are similar at the start of the trial in all respects, except for whether they received the experimental condition (that is, the intervention of interest) or not. However, the use of randomisation does not guarantee that the groups will have similar known baseline characteristics. This is particularly the case if there is a small sample size. Authors of a research article will usually provide data in the article about the baseline characteristics of both groups. This allows readers to make up their own minds as to whether the balance between important prognostic factors (variables that have the potential for influencing outcomes) is sufficient at the start of the trial. Consider, for example, a study about the effectiveness of acupuncture for reducing pain from migraines compared with sham acupuncture. If the participants who were allocated to the acupuncture group had less severe or less chronic pain at the start of the study than the participants who were allocated to the sham acupuncture group, any differences in pain levels that were seen at the end of the study might be the result of that initial difference and not the acupuncture that was provided.

Differences between the groups that are present at baseline after randomisation have occurred due to chance and, therefore, determining whether these differences are statistically significant by using p values is not an appropriate way of assessing such differences.¹⁵ That is, rather than using the p value that is often reported in studies, it is important to examine these differences by comparing means or proportions visually. The extent to which you might be concerned about a baseline difference between the groups depends on how large a difference it is and whether it is a key prognostic variable, both of which require some clinical judgment. The stronger the relationship between the characteristic and the outcome of interest, the more the differences between groups will weaken the strength of any inference about efficacy.⁵ For example, consider a study that is investigating the effectiveness of group therapy in improving communication for people who have chronic aphasia following stroke compared with usual care. Typically, such a study would measure and report a wide range of variables at baseline (that is, prior to the intervention) such as participants' age, gender, education level, place of residence, time since stroke, severity of aphasia, side of stroke and so on. Some of these variables are more likely to influence communication outcomes than others. The key question to consider is: are any differences in key prognostic variables between the groups large enough that they may have influenced the outcome(s)? Hopefully, if differences are evident the researchers will have corrected for this in the data analysis process.

As a reader (and critical appraiser) of research articles, it is important that you are able to see data for key characteristics that may be of prognostic value in both groups. Many articles will present such data in a table, with the data for the intervention group presented in one column and the data for the control group in another. This enables you to easily compare how similar the groups are for these variables. As well as presenting baseline data about key socio-demographic characteristics (for example, age and gender), articles should also report data about important measures of the severity of the condition (if that is relevant to the study—most times it is) so that you can see whether the groups were also similar in this respect. For example, in a study that involves participants who have had a stroke, the article may present data about the initial stroke severity of participants, as this variable has the potential to influence how participants respond to an intervention. In most cases, socio-demographic variables alone are not sufficient to determine baseline similarity.

One other area of baseline data that articles should report is the key outcome(s) of the study (that is, the pre-test measurement(s)). Let us consider the example presented earlier of people receiving group communication treatment for aphasia to illustrate why this is important. Although such an article would typically provide information about socio-demographic variables and clinical variables (such as severity of aphasia, type of stroke and side of stroke), having information about participants' initial (that is, pre-test) scores on the communication outcome measure that the study used would be helpful for considering baseline similarity. This is because, logically, participants' pre-test scores on a communication measure are likely to be a key prognostic factor for the main outcome of the study, which is communication ability.

When appraising an article, if you do conclude that there are baseline differences between the groups that are likely to be big enough to be of concern, hopefully the researchers will have statistically adjusted for these in the analysis. If they have not, you will need to try to take this into account when interpreting the study.

Were participants, health professionals and study personnel ‘blind’ to group allocation?

People involved with a trial, whether they be the participants, the treating health professionals or the study personnel, usually have a belief or expectation about what effect the experimental condition will or will not have. This conscious or unconscious expectation can influence their behaviour, which in turn can affect the results of the study. This is particularly problematic if they know which condition (experimental or control) the participant is receiving. Blinding (also known as masking) is a technique that is used to prevent participants, health professionals and study personnel from knowing which group the participant was assigned to so that they will not be influenced by that knowledge.¹⁰

In many studies, it is difficult to achieve blinding. Blinding means more than just keeping the name of the intervention hidden. The experimental and control conditions need to be indistinguishable. This is because even if they are not informed about the nature of the experimental or control conditions (which, for ethical reasons, they usually are) when they sign informed consent forms, participants can often work out which group they are in. Whereas pharmaceutical trials can use placebo medication to prevent participants and health professionals from knowing who has received the active intervention, blinding of participants and the health professionals who are providing the intervention is very difficult (and often impossible) in many non-pharmaceutical trials. We will now look a little more closely at why it is important to blind participants, health professionals and study personnel to group allocation.

A participant's knowledge of their treatment status (that is, if they know whether they are receiving the intervention that is being evaluated or not) may consciously or unconsciously influence their performance during the intervention or their reporting of outcomes. For example, if a participant was keen to receive the intervention that was being studied and they were instead randomised to the control group, they may be disappointed and their feelings about this might be reflected in their outcome assessments, particularly if the outcomes being measured are subjective in nature (for example, pain, quality of life or satisfaction). Conversely, if a participant knows or suspects that they are in the intervention group, they may be more positive about their outcomes (such as exaggerating the level of improvement that they have experienced) when they report them, as they wish to be a ‘polite patient’ and are grateful for receiving the intervention.¹⁶

The health professionals who provide the intervention often have a view about the effectiveness of interventions, and this can influence the way they interact with the study participants and the way they deliver the intervention. This in turn can influence how committed they are to providing the intervention in a reliable and enthusiastic manner, affecting participants' compliance to the intervention and participants' responses on outcome measures. For example, if a health professional believes strongly in the value of the intervention that is being studied, they may be very enthusiastic and diligent in their delivery of the intervention, which may in turn influence how participants respond to this intervention. It is easy to see how a health professional's enthusiasm (or lack of) could influence outcomes. Obviously some interventions (such as medications) are not able to be influenced easily by the way in which they are provided, but for many other interventions (such as rehabilitation techniques provided by therapists), this can be an issue.

Study personnel who are responsible for measuring outcomes (the assessors) and who are aware of whether the participant is receiving the experimental or control condition may provide different interpretations of marginal findings or differential encouragement during performance tests, either of which can distort results. For example, if an assessor knows that a participant is in the intervention group, they might be a little more generous when scoring a participant's performance on a task than they would be if they thought that the participant was in the control group. Studies should aim to use blinded assessors to prevent measurement bias from occurring. This can be done by ensuring that the assessor who measures the outcomes at baseline and at follow-up is unaware of the participant's group assignment. Sometimes this is referred to as the use of an independent assessor. The more objective the outcome that is being assessed, the less critical this issue becomes. However, there are not many truly objective outcome measures, as even measures that appear to be reasonably objective (for example, measuring muscle strength manually or functional ability) have a subjective component and, as such, can be susceptible to measurement bias. Therefore, where it is at all possible, studies should try to ensure that the people who are assessing participants' outcomes are blinded. Ideally, studies should also check and report on the success of blinding assessors and, where this information is not provided, you may wish to reasonably speculate about whether or not the outcome assessor was actually blinded as claimed.

However, there is a common situation that occurs, particularly in many trials in which non-pharmaceutical interventions are being tested, that makes assessor blinding not possible to achieve. If the participant is aware of their group assignment, then the assessment cannot be considered to be blinded. For example, consider the outcome measure of pain that is assessed using a visual analogue scale. The participant has to complete the assessment themselves due to the subjective nature of the symptom experience. In this situation, the participant is really the assessor and, if the participant is not blind to which study group they are in, then the assessor is also not blind to group allocation. Research articles often state that the outcome assessors were blind to group allocation. Most articles measure more than one outcome and often a combination of objective and subjective outcome measures are used. So, while this statement may be true for objective outcomes, if the article involved outcomes that were assessed by participant self-report and the participants were not blinded, you cannot consider that these subjective outcomes were measured by a blinded assessor.

Were all participants who entered the trial properly accounted for at its conclusion, and how complete was follow-up?

In randomised controlled trials, it is common to have missing data at follow-up. There are many reasons why data may be missing. For example, some questionnaires may not have been fully completed by participants, some participants may have decided to leave the study or some participants may have moved house and cannot be located at the time of the follow-up assessment. How much of a problem this is for the study, with respect to the bias that is consequently introduced, depends on why participants left the study and how many left the study.

It is therefore helpful to know whether all the participants who entered the trial were properly accounted for. In other words, we want to know what happened to them. Could the reason that they dropped out of the study have affected the results? This may be the case, for example, if they left the study because the intervention was making them worse or causing adverse side effects. If this was the case, this might make the intervention look more effective than it really was. Did they leave the study simply because they changed jobs and moved away, or was the reason that they dropped out related to the study or to their health? For example, it may not be possible to obtain data from participants at follow-up measurement points because they became unwell, or maybe because they improved, and no longer wanted to participate. Hopefully, you can now see why it is important to know why there are missing data for some participants.

The more participants who are ‘lost to follow-up’, the more the trial may be at risk of bias because participants that leave the study are likely to have different prognoses from those who stay in the study. It has been suggested that ‘readers can decide for themselves when the loss to follow-up is excessive by assuming, in positive trials (that is, trials that showed that the intervention was effective), that all participants lost from the treatment group did badly, and that all lost from the control group did well, and then recalculating the outcomes under these assumptions. If the conclusions of the trial do not change, then the loss to follow-up was not excessive. If the conclusions would change, the strength of inference is weakened (that is, less confidence can be placed in the study results)’.⁵

When large numbers of participants leave a study, the potential for bias is enhanced. Various authors suggest that if more than 15–20% of participants leave the study (with no data available for them), then the results should be considered with much greater caution.^16,17 Therefore, you are looking for a study to have a minimum follow-up rate of at least 80–85%. To calculate the loss to follow-up, you just need to divide the number of participants included in the analysis at the time point of interest (such as the 6-month follow-up) by the number of participants who were originally randomised into the study groups. This gives the percentage of participants who were followed up. In some articles it is straightforward to find the necessary data that you need to calculate this, particularly if the article has provided a flow diagram (see Figure 4.2). It is highly recommended that trials do this, and this is explained more fully later in the chapter when the recommended reporting of a randomised controlled trial is discussed. In other articles, this information can be obtained from the text, typically in the results section. In some articles, the only place to locate information about the number of participants who remain in the groups at a particular time point is from the column headers in a results table. And finally, in some articles, despite all of your best hunting efforts, there may be no information about the number of participants who were retained in the study. This may mean that there were no participants lost to follow-up (which is highly unlikely, as at least some participants are lost to follow-up in most studies) or the authors of the article did not report the loss of participants that occurred. Either way, as you cannot determine how complete the follow-up was, you should be suspicious of the study and consider the results to be potentially biased.

It is worth noting that where there is loss to follow-up in both the intervention and the control groups and the reasons for these losses are both known and similar between these groups, it is less likely that bias will be problematic.¹⁹ When the reasons why participants leave the study are unknown, or when they are known to be different between the groups and potentially prognostically relevant, you should be more suspicious about the validity of the study. That is why it is important to consider the reasons for loss to follow-up and whether the number of participants lost to follow-up was approximately the same for both of the study groups, as well as the actual percentage of participants who were lost to follow-up.

Were participants analysed in the groups to which they were randomised using intention-to-treat analysis?

The final criterion for assessing risk of bias that we will address in this chapter is whether or not data from all participants were analysed in the groups to which participants were initially randomised, regardless of whether they ended up receiving the treatment. This analysis principle is referred to as intention-to-treat analysis. In other words, participants should be analysed in the group that corresponds to how they were intended to be treated, not how they were actually treated.

It is important that an intention-to-treat analysis is performed, because study participants may not always receive the intervention or control condition as it was allocated (that is, intended to be received). In general, participants may not receive the intervention (even though they were allocated to the intervention group) because they are either unwell or unmotivated, or for other reasons related to prognosis.⁵ For example, in a study that is evaluating the effect of a medication, some participants in the intervention group may forget to take the medication and therefore do not actually receive the intervention as intended. In a study that is evaluating the effects of a home-based exercise program, some of the participants in the intervention group may not practise any of the exercises that are part of the intervention because they are not very motivated. Likewise, in a study that is evaluating a series of small-group education sessions for people who have had a heart attack, some participants in the intervention group may decide not to attend some or all education sessions because they feel unwell. From these examples, you can see that even though these participants were in the intervention group of these studies, they did not actually receive the intervention (either at all or only partly). It may be tempting for the researchers who are conducting these studies to analyse the data from these participants as if they were in the control group instead. However, doing this would increase the numbers in the control group who were either unmotivated or unwell. This would make the intervention appear more effective than it actually is because there would be a greater number of participants in the control group who were likely to have unfavourable outcomes. It may also be tempting for researchers to discard the results from participants who did not receive the intervention (or control condition) as was intended. This is also an unsuitable way of dealing with this issue, as these participants would then be considered as lost to follow-up and we saw in the previous criterion why it is important that as few participants as possible are lost to follow-up.

For the sake of completeness, it is important to point out that it is not only participants who are allocated to the intervention group but do not receive the intervention that we should think about. The opposite can also happen. Participants who are allocated to the control group can inadvertently end up receiving the intervention. Again, intention-to-treat analysis should be used and these participants should still be analysed as part of the control group.

The value of intention-to-treat analysis is that it preserves the value of randomisation. It helps to ensure that prognostic factors that we know about, and those that we do not know about, will still be, on average, equally distributed between the groups. Because of this, any effect that we see, such as improvement in participants' outcomes, is most likely to be because of the intervention rather than unrelated factors.

The difficulty in carrying out a true intention-to-treat analysis is that the data for all participants is needed. However, as we saw in the previous criterion about follow-up of participants, this is unrealistic to expect and most studies have missing data. There is currently no real agreement about the best way to deal with such missing data (probably because there is no ideal way), but researchers may sometimes estimate or impute data.²⁰ Data imputation is a statistical procedure that substitutes missing data in a data file with estimated data. Other studies may simply report that they have carried out an intention-to-treat analysis or that participants received the experimental or control conditions as allocated without providing details of what was actually done or how missing data were dealt with. In this case, as the reader of the article, you may choose to accept this at face value or to remain sceptical about how this issue was dealt with, depending on what other clues are available in the study report.

The role of chance

So far in this chapter we have considered the potential for bias in randomised controlled trials. Another aspect that is important to consider is the possibility that the play of chance might be an alternative explanation for the findings. So, a further question that you may wish to consider when appraising a randomised controlled trial is: did the study report a power calculation that might indicate what sample size would be necessary for the study to detect an effect if the effect actually exists? As we saw in Chapter 2, having an adequate sample size is important so that the study can avoid a type II error occurring. You may remember that a type II error is the failure to find and report a relationship when a relationship actually exists.²¹

• Variables with continuous data can take any value along a continuum within a defined range. Examples of continuous variables are age, range of motion in a joint, walking speed and score on a visual analogue scale.

• Variables with dichotomous data have only two possible values. For example, male/female, satisfied/not satisfied with treatment, and hip fracture/no hip fracture.

1. What is the size of the intervention effect?

2. What is the precision of the intervention effect?

Continuous outcomes—size of the intervention effect

When you are trying to work out how much of a difference the intervention made, you are trying to determine the size of the intervention effect. When you are dealing with continuous data, this is often quite a straightforward process.

Many articles will already have done this for you and will report the difference. In other articles, you will have to do this simple calculation yourself.

Let us consider an example. In a randomised controlled trial²² that evaluated the efficacy of a self-management program for people with knee osteoarthritis in addition to usual care, compared with usual care, one of the main outcome measures was pain, which was measured using a 0–10 cm visual analogue scale with 0 representing ‘no pain at all’ and 10 representing ‘the worst pain imaginable’. At the 3-month follow-up, the mean reduction in knee pain was 0.67 cm (standard deviation [SD] = 2.10) in the intervention group and 0.01 cm (SD = 2.00) in the control group. This difference was statistically significant (p = 0.023). You can calculate the intervention effect size (difference in mean change between the groups) as: 0.67 cm minus 0.01 cm = 0.66 cm.

Note that in this study, the authors reported the mean improvement (reduction in pain) scores at the 3-month follow-up (that is, the change in pain from baseline to the 3-month follow-up). Some studies report change scores; other studies report end scores (which are the scores at the end of the intervention period). Regardless of whether change scores or end scores are reported, the method of calculating the size of the intervention effect is the same. When dealing with change scores, it becomes the difference of the mean change between the intervention and control groups that you need.

Continuous outcomes—precision of the intervention effect

Dichotomous outcomes—size of the treatment effect

Often the outcomes that are reported in a trial will be presented as dichotomous data. As we saw at the beginning of this results section, these are data for which there are only two possible values. It is worth being aware that data measured using a continuous scale can also be categorised, using a certain cut-off point on the scale, so that the data become dichotomised. For example, data on a 10-point visual analogue pain scale can be arbitrarily dichotomised around a cut-off point of 3 (or any point that the researchers choose), so that a pain score of 3 or less is categorised as mild pain and a score of above 3 is categorised as moderate/severe pain. By doing this, the researchers have converted continuous data into data that can be analysed as dichotomous data.

Health professionals and patients are often interested in comparative results—the outcome in one group relative to the outcome in the other group. This overall (comparative) consideration is one of risk. Before getting into the details, let us briefly review the concept of risk. Risk is simply the chance, or probability, of an event occurring. A probability can be described by numbers, ranging from 0 to 1, and is a proportion or ratio. Risks and probabilities are usually expressed as a decimal, such as 0.1667, which is the same as 16.67%. Risk can be expressed in various ways (but using the same data).²⁹ We will consider each of these in turn:

Dichotomous outcomes—precision of the treatment effect

Confidence intervals are also important to consider when examining dichotomous outcomes as, again, they indicate how much uncertainty is associated with the estimate of the intervention effect (in other words, the precision or accuracy of the estimate). The principles of confidence intervals associated with dichotomous data are similar to those for continuous data, but there is one very important difference to consider—what is the appropriate no effect value to use?

Therefore, you can see that it is important to consider the type of effect measure you are interpreting in order to reliably interpret the confidence interval. The same general principle applies, though—a 95% confidence interval that does not include the no effect value indicates that the result is statistically significant. Table 4.1 presents a summary of the dichotomous effect measures, including what the no effect value is, that have been discussed in this section of the chapter.

As we could do with continuous outcomes, if the confidence interval has not been provided by the authors of the article it can be possible to calculate an approximate confidence interval. This is illustrated in Box 4.3. Again, a simplified version of the more complex equation is used, but the confidence interval that is calculated is sufficient for use by health professionals who wish to use the information to assist in clinical decision making. Once you know the confidence interval for the absolute risk reduction, if you wish you could plot it, the effect estimate and the smallest clinically worthwhile effect on a tree plot in the same way that we did for continuous outcomes in Figure 4.4.

Calculating the confidence intervals for number needed to treat is fairly straightforward, as you just use the inverse of the numbers in the confidence interval of the absolute risk reduction.³⁰ However, understanding how to interpret the confidence intervals for number needed to treat can be complicated, and the article by Altman³⁶ is recommended to understand this in detail.

When making sense of the results of a randomised controlled trial, one further issue that you should be aware of is that trials usually report results from many different outcomes. With so much information to process it may be helpful, in some cases, for you to focus your attention on the main outcome(s) of the trial, the outcome(s) of interest to the question that you initially formed and/or the outcome that is of interest to your patient. If you have got to this point and determined that the article that you have been appraising not only appears to be reasonably free of bias but also contains some clinically important results, you then proceed to look at whether you can apply the results from the study to your patient or clinical situation. This is the third and final step of the critical appraisal process that we described in Chapter 1.

• Do the results apply to your patient or situation?

• Do the benefits found outweigh any harm, costs and/or inconveniences that are involved with the intervention?

• What other factors might need to be considered when applying this evidence?

Do the results apply to your patient or situation?

When considering whether the results apply to your patient or situation, you essentially need to assess whether your patient is so different from the participants in the study that you cannot apply the results of the study that you have been reading to your patient or situation. So, rather than just thinking about the study's eligibility criteria with respect to your patient, are the differences problematic enough that the results should not be applied? Further to this, results can sometimes be individualised to a particular patient by considering the individual benefit and harm for that patient. There is some complexity involved in doing so, which is beyond the scope of this book. If you wish to learn about this further, you are advised to read the article by Glasziou and Irwig³⁹ to understand how this may be done.

Do the benefits found outweigh any harm, costs and/or inconveniences that are involved with the intervention?

Understanding whether benefits outweigh harms or costs requires using information about benefits that is given in terms of the size of the intervention effect (e.g. mean differences, relative risk reduction, number needed to treat) and comparing this against possible harms or even inconveniences. When information about harm and cost is provided in the article, this is relatively straightforward. However, this type of information is often not provided and some estimate of the costs associated with the intervention might need to be made. Information about harm is obviously more difficult to estimate, and other sources might need to be consulted to get a sense of the potential harms involved. This is a grey area of practice in which clinical experience and discussion with patients about their preferences and values become very important.

What other factors might need to be considered when applying this evidence?

A further important question to consider is: what other factors might need to be considered when applying this evidence? When you are thinking about what other factors affect the delivery of an intervention, there are a few key questions that you can ask, such as:

A central component of applying research evidence to your patient is discussing the information with them. The success of many interventions is dependent on the health professional successfully providing the patient with appropriate information.

Involving the patient in the decision-making process is important, and this is discussed further in Chapter 14. In Chapter 1 we emphasised the need for integrating information from patients, clinical experience, research and the practice context. To do so is the art of evidence-based practice. Integrating so many pieces of information from many different sources is certainly an art form and one that requires clinical reasoning and judgment. The roles of clinical reasoning and judgment are discussed further in Chapter 15.

1. Portney, L, Watkins, M. Foundations of clinical research: applications to practice. Upper Saddle River, NJ: Prentice Hall Health; 2000.

2. Deeks, J, Dinnes, J, D'Amico, R, et al. Evaluating non-randomised intervention studies. Health Technol Assess. 2003; 7:1–186.

3. Allan, L, Hays, H, Jensen, N, et al. Randomised crossover trial of transdermal fentanyl and sustained release oral morphine for treating chronic non-cancer pain. BMJ. 2001; 322:1154–1158.

4. Clemson, L, Cumming, R, Kendig, H, et al. The effectiveness of a community-based program for reducing the incidence of falls in the elderly: a randomised trial. J Am Geriatr Soc. 2004; 52:1487–1494.

5. Guyatt, GH, Sackett, DL, Cook, DJ. Users’ guides to the medical literature. II. How to use an article about therapy or prevention. JAMA. 1993; 270:2598–2601.

6. Maher, C, Sherrington, C, Herbert, R, et al. Reliability of the PEDro scale for rating quality of randomised controlled trials. Phys Ther. 2003; 83:713–721.

7. Shulz, K, Chalmers, I, Hayes, R, et al. Empirical evidence of bias: dimensions of methodological quality associated with estimates of treatment effects in controlled trials. JAMA. 1995; 273:408–412.

8. Kunz, R, Vist, G, Oxman, A. Randomisation to protect against selection bias in healthcare trials. Cochrane Database Syst Rev. (2):2007. [MR000012. doi: 10.1002/14651858.MR000012.pub2].

9. Altman, DG, Bland, JM. Treatment allocation in controlled trials: why randomise? BMJ. 1999; 318:1209.

10. Hewitt, C, Torgerson, D. Is restricted randomisation necessary? BMJ. 2006; 332:1506–1508.

11. El-Mohandes, A, Katz, K, El-Khorazaty, M, et al. The effect of a parenting education program on the use of preventive paediatric health care services among low-income, minority mothers: a randomised, controlled study. Pediatrics. 2003; 111:1324–1332.

12. Altman, D, Schulz, K, Moher, D, et al. The revised CONSORT statement for reporting randomised trials: explanation and elaboration. Ann Intern Med. 2001; 134:663–694.

13. Pildal, J, Chan, AW, Hróbjartsson, A, et al. Comparison of descriptions of allocation concealment in trial protocols and the published reports: cohort study. BMJ. 2005; 330:1049–1054.

14. Bennett, S, McKenna, K, McCluskey, A, et al. Evidence for occupational therapy interventions: status of effectiveness research indexed in the OTseeker database. Br J Occup Ther. 2007; 70:426–430.

15. Roberts, C, Torgerson, D. Understanding controlled trials: baseline imbalance in randomised controlled trials. BMJ. 1999; 319:185.

16. Herbert, R, Jamtvedt, G, Hagen, K, et al. Practical evidence-based physiotherapy, 2nd ed. Edinburgh: Elsevier; 2011.

17. Dumville, J, Torgerson, D, Hewitt, CE. Reporting attrition in randomised controlled trials. BMJ. 2006; 332:969–971.

18. Schulz, K, Altman, D, Moher, D, the CONSORT Group. CONSORT 2010 statement: updated guidelines for reporting parallel-group randomised trials. BMJ. 340, 2010. [doi: 10.1136/bmj.c332].

19. Higgins, J, Altman, D, Sterne, J, Chapter 8: Assessing risk of bias in included studiesHiggins J, Green S, eds. Cochrane Handbook for Systematic Reviews of Interventions. Version 5.1.0 (updated March 2011). The Cochrane Collaboration: 2011. Online Available www.cochrane-handbook.org

20. Hollis, S, Campbell, F. What is meant by intention to treat analysis? Survey of published randomised controlled trials. BMJ. 1999; 319:670–674.

21. Ottenbacher, K, Maas, F. How to detect effects: statistical power and evidence-based practice in occupational therapy research. Am J Occup Ther. 1999; 53:181–188.

22. Heuts, P, de Bie, R, Drietelaar, M, et al. Self-management in osteoarthritis of hip or knee: a randomised clinical trial in a primary healthcare setting. J Rheumatol. 2005; 32:543–549.

23. Cohen, J. Statistical power analysis for the behavioural sciences, 2nd ed. Hillsdale, NJ: Lawrence Erlbaum; 1988.

24. Norman, G, Sloan, J, Wyrwich, K. Interpretation of changes in health-related quality of life: the remarkable universality of half a standard deviation. Med Care. 2003; 41:582–592.

25. Guyatt, G. Making sense of quality-of-life data. Med Care. 2000; 39(II):175–179.

26. Herbert, R. How to estimate treatment effects from reports of clinical trials. I: continuous outcomes. Aust J Physiother. 2000; 46:229–235.

27. Altman, D, Bland, J. Absence of evidence is not evidence of absence. BMJ. 1995; 311:485.

28. Lahdensuo, A, Haahtela, T, Herrala, J, et al. Randomised comparison of guided self-management and traditional treatment of asthma over one year. BMJ. 1996; 312:748–752.

29. Guyatt, G, Sackett, D, Cook, D. Users' guide to the medical literature: II. How to use an article about therapy or prevention: B. what were the results and will they help me in caring for my patients? JAMA. 1994; 271:59–63.

30. Cook, R, Sackett, D. The number needed to treat: a clinically useful measure of treatment effect. BMJ. 1995; 310:452–454.

31. Sackett, D, Richardson, W, Rosenberg, W, et al. Evidence-based medicine. How to practice and teach EBM. New York: Churchill Livingstone; 1997.

32. Herbert, R. How to estimate treatment effects from reports of clinical trials. II: Dichotomous outcomes. Aust J Physiother. 2000; 46:309–313.

33. Centre for Evidence-Based Medicine, Glossary. University of Oxford, Oxford, 31 May 2012. Online Available www.cebm.net/index.aspx?o=1116

34. Laupacis, A, Sackett, D, Roberts, R. An assessment of clinically useful measures of the consequences of treatment. N Engl J Med. 1988; 318:1728–1733.

35. Straus, S, Sackett, D. Applying evidence to the individual patient. Ann Oncol. 1999; 10:29–32.

36. Altman, D. Confidence intervals for the number needed to treat. BMJ. 1998; 317:1309–1312.

37. Cumming, R. Intervention strategies and risk-factor modification for falls prevention: a review of recent intervention studies. Clin Geriatr Med. 2002; 18:175–189.

38. Campbell, A, Robertson, M, Gardner, M, et al. Falls prevention over 2 years: a randomised controlled trial in women 80 years and older. Age Ageing. 1999; 28:513–518.

39. Glasziou, P, Irwig, L. An evidence based approach to individualising treatment. BMJ. 1995; 311:1356–1359.