Questions about prognosis: examples of appraisals from different health professions
Tammy Hoffmann, Marilyn Baird, John W Bennett, Mal Boyle, Jeff Coombes, Mark R Elkins, Lisa Nissen, Sheena Reilly, Claire Rickard and Sharon Sanders
This chapter is an accompaniment to the previous chapter (Chapter 8) where the steps involved in answering a clinical question about prognosis were explained. In order to further help you learn how to deal with prognostic clinical questions when they arise and appraise the evidence, this chapter contains a number of worked examples of questions about prognosis from a range of health professions. The worked examples in this chapter follow the same format as the examples in Chapter 5. In addition, as with the worked examples that were written for Chapter 5, the authors of the worked examples in this chapter were asked not to choose a systematic review (for the reason that was explained in Chapter 5), but instead to find the next best available level of evidence to answer the prognostic question that was generated from the clinical scenario.
Occupational therapy example
Clinical scenario
You are an occupational therapist who has recently rotated into the outpatient department of a brain injury rehabilitation unit. After patients are discharged as inpatients and return home to live, they return to the hospital to receive outpatient therapy. One of your patients is Mary, a 21-year-old who sustained a mild traumatic brain injury in a motor vehicle accident 8 weeks ago. Her Glasgow Coma Scale was 14 on admission to hospital. Prior to the accident Mary worked full-time as an apprentice chef. She has been working in this job for 2 years and started after finishing high school. After discharge from hospital, Mary returned home to live with her parents and has been able to do simple tasks around the home and shopping with assistance from her mother.
Your initial outpatient assessment of Mary revealed that her current difficulties are poor endurance, poor memory and planning problems. Mary is already 2 months post-injury and her employer has just advised her that he is only able to keep her position in the apprenticeship scheme reserved for her for another 4 months. Mary has asked you how likely it is that she will be able to return to her job within that timeframe. As you are new to working in this area of clinical practice, you decide to search the literature to help you answer this question.
In adults with mild traumatic brain injury, what is the likelihood of returning to work within 6 months of the injury occurring?
Search terms and databases used to find the evidence
Database: PubMed—Clinical Queries (with ‘prognosis category’ and ‘narrow scope’ selected).
Search terms: (mild traumatic brain injury) AND (return to work*)
This search retrieves 10 results. After reading through the titles and abstracts of these articles, three appear to be relevant to your clinical question. One is primarily focused on the application of a particular outcome measure. Of the other two, you choose the article with the largest sample size and obtain its full text so that you can do a critical appraisal.
Stulemeijer M, van der Werf S, Borm G, et al. Early prediction of favourable recovery 6 months after mild traumatic brain injury. J Neurol Neurosurg Psychiatry 2008;79:936–42.
Structured abstract*
Study design: Prospective cohort study.
Setting: Level 1 trauma centre of a university medical centre in The Netherlands.
Participants: 452 patients with mild traumatic brain injury who were admitted to the emergency department and aged between 18 and 60 years (mean age 35.6 years, 78% female) were sent a questionnaire to complete (the 6-month questionnaire was returned by 201 patients). Other eligibility criteria were: able to speak and write in Dutch and did not have premorbid mental retardation or dementia.
Outcomes: Postconcussional symptoms (measured using Rivermead Post-Concussion Questionnaire) and return to work (without any negative change in work situation because of the injury).
Prognostic factors studied: Pre-injury (age, gender, education, premorbid emotional problems, physical comorbidities and prior head injury); peri-injury (Glasgow Coma Scale score; presence and duration of loss of consciousness; post-traumatic amnesia duration; brain computed tomography characteristics; dizziness, nausea/vomiting or headache in the emergency department; additional extracranial injuries); and early post-injury (postconcussional symptoms, post-traumatic stress, fatigue, pain and self-efficacy).
Follow-up period: 6 months (mean = 6.5 months, range 5.5 to 10 months).
Main results: At follow-up, 153 (76%) of the participating patients reported full return to work and 64% reported full recovery. Patients with more than 11 years of education, without nausea or vomiting on admission, with no additional extracranial injuries and only low levels of pain early after injury had a 90% chance of full return to work.
Conclusion: Early identification of patients with mild traumatic brain injury who are likely to have good 6-month recovery was feasible based on relatively simple prognostic models.
Is the evidence likely to be biased?
• Was there a representative and well-defined sample?
Yes. There was a reasonably well-defined population and representative sampling. The article states that consecutive patients who were admitted to the trauma centre with a mild traumatic brain injury were eligible to participate in the study. A definition of what was considered a mild traumatic brain injury is provided. There were only a few inclusion criteria (between 18 and 60 years of age, able to speak and write in Dutch and no premorbid intellectual disability or dementia), although it is not detailed how these latter criteria were assessed. No specific exclusion criteria are provided. The exclusion of people who could not read or write may limit the generalisability of the study results.
• Was there an inception cohort?
Yes. Participants were recruited at a similar well-described point, namely after they were admitted to hospital after the traumatic brain injury occurred.
• Was exposure determined accurately?
Yes. In this study, it would have been very unlikely that an error was made in recruiting participants who had suffered a mild traumatic brain injury as a result of a motor vehicle accident, as this exposure (or in this case, eligibility criteria) can be easily determined. The authors also explain how the severity of participants' traumatic brain injury was determined (mild traumatic brain injury was defined as having a Glasgow Coma Scale admission score of 13–15, with or without a loss of consciousness of less than 30 minutes and with or without post-traumatic amnesia).
• Were the outcomes measured accurately?
Yes. The outcomes were postconcussional symptoms and return to work. Postconcussional symptoms were measured using the Rivermead Post-Concussion Questionnaire and the definition of a ‘favourable’ outcome using this measure is provided. Return to work status: participants were considered to have full return to work when they were not on sick leave at the time of follow-up or reported no change in working status (such as to part-time or a lower level) because of their injury. There are no details provided about what is meant by a lower level of working status and whether this included modified duties, which have been useful to know about.
Participants provided the outcome measure information in a self-completed questionnaire at 6 months after recruitment to the study. It was not possible to measure outcomes in a blind fashion, as it was based on participant self-report. It was not possible to blind outcome assessors about participants' status on the prognostic factors, as the measures were based on participant self-report where the participant was technically the outcome assessor.
• Were important prognostic factors considered?
Yes. The authors appear to have identified the major factors that could influence return to work. The study examined three main categories of possible prognostic factors (refer to the structured abstract).
• Was the follow-up of participants sufficiently long and complete?
No, not sufficiently complete. The study identified 529 patients who met the inclusion criteria and of these, 452 were sent the early questionnaire (with most of the 77 missed due to logistical reasons). Complete questionnaires were returned by 280 patients and the follow-up questionnaire returned by 201 patients (which is 44% of those who were sent the early questionnaire and 72% of those who completed the first questionnaire). Compared with the total cohort, the group of participants that completed the follow-up questionnaire contained fewer men and were younger (mean difference of 2.6 years). With regards to length, the mean length of follow-up time was 6.5 months (range 5.5 to 10 months), which matches our clinical question. However, many studies that look at return to work after head injury follow participants for 12 months. Although only participants with mild traumatic brain injury were included in this study, they may have experienced impairments (physical, psychological and/or cognitive impairments) that could continue to have impacts on their ability to return to work beyond this amount of time post-injury.
Return to work: Of the 201 participants who were able to be followed up, 153 (76%) reported full return to work by the time of the follow-up assessment. You calculate the 95% confidence interval (CI) to be 71% to 81% (using the formula that was provided in Chapter 8, where 95% CI = risk ± [1 ÷ 
]). In other words, the likelihood of having returned to work by approximately 6.5 months after mild traumatic brain injury could be as low as 71% or as high as 81%.
Significant predictors of full return to work at 6 months: More than 11 years of education (odds ratio 6.4; 95% CI 2.3 to 18.3), without nausea or vomiting on admission (odds ratio 5.1, 95% CI 1.8 to 14.3), with no additional extracranial injuries (odds ratio 3.4, 95% CI 1.6 to 7.3) and only low levels of pain early after injury (odds ratio 2.3, 95% CI 0.9 to 5.9).
How might we use this evidence to inform practice?
As you have determined the internal validity of this study to be reasonably strong (although you keep in mind the bias from the incomplete follow-up) and the results useful, you proceed to assessing the applicability of the evidence by comparing your patient with the participants in the study, before deciding whether you can use the evidence to help inform your practice. Mary's mechanism and severity of injury is similar to the majority of the study participants. She is younger than the mean age of study participants, but meets the eligibility criterion for age and all of the other eligibility criteria of the study. In terms of the prognostic factors that were identified in this study, Mary has >11 years of education; did not have nausea, vomiting or extracranial injuries on admission; and had only low levels of pain early after injury. All of these factors were positively related to return to work, so this may increase the likelihood of her returning to work.
You explain to Mary that you think there is a reasonable chance that she will be able to return to her usual job by 6 months post-injury. As part of your treatment planning, you will arrange a time for Mary and yourself to meet with her employer to discuss the option of her returning to work in a modified capacity (such as shorter hours, different duties, graded return to work, etc) if this is necessary. During this meeting, you also plan to obtain more-detailed information about Mary's duties at work and then use this, in conjunction with Mary, to set her rehabilitation program and goals.
*Reproduced and adapted from the above reference with permission from BMJ Publishing Group Ltd.
Human movements example
Clinical scenario
Mr Roberts has been referred to you by his general practitioner. He is 47 years old with hypertension, high cholesterol, obesity and impaired glucose tolerance. You undertake a fitness assessment and determine that he has a low exercise capacity (lowest 20th percentile for his age). He is very worried about developing type 2 diabetes and wants to know if improving his fitness will make him less likely to develop diabetes. You want to give Mr Roberts evidence-based advice about this issue, and look for recent evidence to guide your discussion with him.
In a man with impaired glucose tolerance, does a high level of fitness lower the risk of developing type 2 diabetes?
Search terms and databases used to find the evidence
Database: PubMed—Clinical Queries (with ‘prognosis category’ and ‘narrow scope’ selected).
Search terms: (impaired glucose tolerance) AND fitness AND diabetes
This search results in nine articles, two which appear relevant from the abstracts. You obtain the full-text of each and find that both papers are describing the same cohort. You decide to use the 2009 article as it is based on more-recent data/analyses.
Lee D, Sui X, Church T, et al. Associations of cardiorespiratory fitness and obesity with risks of impaired fasting glucose and type 2 diabetes in men. Diabetes Care 2009;32:257–62.
Structured abstract (adapted from the above)
Study design: A prospective cohort study.
Setting: A medical clinic in Texas, USA.
Participants: 14,006 men in a larger study about diabetes; and of these, 7795 who had normal baseline glucose and were considered to be in an impaired fasting glucose group. Inclusion criteria were: men aged 20–79 years who had undergone at least two medical examinations at the clinic between 1974 and 2006. Exclusion criteria were: body mass index <18.5 kg/m2; abnormal resting or exercise electrocardiogram; history of heart attack, stroke, cancer or diabetes at baseline; or did not achieve at least 85% of their age-predicted maximal heart rate during the treadmill test.
Outcomes: Primary outcomes: development of impaired fasting glucose or type 2 diabetes.
Prognostic factors studied: Baseline age, examination year, waist girth, percent body fat, parental diabetes, current smoking, alcohol consumption, blood pressure, total cholesterol, baseline impaired fasting glucose, body mass index and exercise capacity (treadmill time for a maximal exercise test). Exercise was divided into fifths of treadmill time in each age group. The lowest 20% were classified as having fitness level 1, and in continuing increments of 20%, participants were classified as fitness levels 2 through 5.
Follow-up period: From baseline to first follow-up event (impaired fasting glucose or type 2 diabetes) or to the last follow-up in 2006 in participants who did not develop either condition.
Main results: 3612 men developed impaired fasting glucose and 477 men developed diabetes. Men in the highest fitness level (most fit 20%) showed a 52% lower risk of developing type 2 diabetes and a 14% lower risk of impaired glucose fasting compared with those in the lowest (least 20% fit) fitness level after accounting for body mass index.
Conclusion: Low fitness and obesity increased the risk of impaired glucose fasting and diabetes.
Is the evidence likely to be biased?
• Was there a representative and well-defined sample?
Yes. The sample was recruited from all patients who received two medical examinations at the recruiting clinic, and clear inclusion and exclusion criteria are provided in the article.
• Was there an inception cohort?
Yes. Participants in the study who had diabetes at baseline, a history of diabetes or were currently taking insulin were excluded from the study, thus leaving an inception cohort of participants without impaired fasting glucose or diabetes.
• Was the exposure determined accurately?
Yes. Fitness was determined using total duration of a treadmill test which is highly correlated (r = 0.92) with the gold-standard measure (maximal oxygen uptake). It is not clear from the study how fitness was determined on an ongoing basis. The limitations section of the paper mentions that participants revisited the clinic every 1.5 years, on average, and that fasting glucose tests were done then, but it is not clear whether that is when the fitness tests were also done.
• Were the outcomes measured accurately?
Yes. Impaired fasting glucose and diabetes were diagnosed according to the American Diabetes Association criteria.
• Were important prognostic factors considered?
Yes. Important known risk factors for developing diabetes were measured and considered in the statistical analyses.
• Was the follow-up of participants sufficiently long and complete?
Cannot tell. Participants were followed until they developed diabetes or impaired fasting glucose, or until the study ceased recruiting. The mean follow-up period was 5.1 years for the 7795 men with normal baseline glucose and 7.2 years for the entire cohort. The article does not provide any details about loss to follow-up of participants from the study. We know that some were lost to follow-up, as there is a brief mention in the limitations about comparing the main variables for men who were lost to follow-up and those who were not, but no further information is given.
Men with a body mass index ≥30.0 kg/m2, waist girth >102 cm or percent body fat ≥25% had 2.7, 1.9 and 1.3 fold higher risks for type 2 diabetes, compared with those for non-obese men. Table 9.1 shows the relative risk of developing diabetes for each fitness level category. For example, men in the highest fitness level (most fit 20%) showed a 52% lower risk of developing type 2 diabetes compared with those in the lowest (least 20% fit) fitness level after accounting for body mass index. As fitness improved, there was a lower risk of developing diabetes. The addition of body mass index into model 2 shows that the associations between fitness levels and the risks of developing diabetes were still significant irrespective of whether body weight was changed. The dose–response relationship between fitness and onset of diabetes remained in all multivariate-adjusted models.
TABLE 9.1:
Adjusted relative risk of developing type 2 diabetes according to fitness level
CI = confidence interval; SD = standard deviation.
aPer 1000 person-years adjusted for age and examination year.
bModel 1 = Adjusted for age, examination year, parental diabetes, current smoking, alcohol consumption, systolic and diastolic blood pressure, total cholesterol, and impaired fasting glucose.
cModel 2 = Adjusted for model 1 plus body mass index.
How might we use this evidence to inform practice?
You are reasonably confident of the validity of this study, although a little cautious given that details regarding the completeness of follow-up are not reported in this article (although possibly are in other articles related to this cohort of patients). This study provides evidence to support your response to Mr Roberts that he could decrease his risk of developing diabetes if he increased his fitness level, even without a change in his body weight. Mr Roberts decides that he would like to increase his fitness level and you plan to design a training program to achieve this goal, taking into account his comorbidities, medications, limitations and personal preferences.
Paramedicine example
Clinical scenario
You are a paramedic who is at a social function when a friend asks what to do if a stranger collapses with a heart attack. ‘I couldn’t bring myself to do mouth-to-mouth on a total stranger,’ he says. ‘Well, the good news is that you don’t have to. It has been shown that chest compression-only resuscitation can be used instead,’ you reply. ‘Yeah, maybe, but what’s the point—aren’t they all pretty much in a coma afterwards? I doubt any make a good recovery’ is the reply. You decide to look up the evidence so that you can provide your friend with a research-based answer to his query.
For patients who have an out-of-hospital cardiac arrest and receive chest compression only, how many people survive to hospital discharge with reasonable neurological function?
Search terms and databases used to find the evidence
Database: PubMed—Clinical Queries (with ‘prognosis category’ and ‘narrow scope’ selected).
Search terms: (cardiac arrest) AND (chest compression only) AND surviv*
This resulted in nine articles, two of which match your clinical question. They both used a similar study design, so you choose the largest and most recent study to appraise.
Bobrow B, Spaite D, Berg R, et al. Chest compression–only CPR by lay rescuers and survival from out-of-hospital cardiac arrest. JAMA 2010;304:1447–54.
Structured abstract (adapted from the above)
Study design: Prospective cohort study.
Setting: Arizona, USA (and occurred following a state-wide multifaceted promotion of chest-compression-only cardiopulmonary resuscitation).
Participants: Patients (n = 4415) ≥18 years of age who experienced an out-of-hospital cardiac arrest of a presumed cardiac origin which was not witnessed by Emergency Medical Services (EMS) personnel which occurred between 2005 and 2009.
Outcomes: The primary outcome was survival to hospital discharge. Additional outcomes were Cerebral Performance Category (CPC) on hospital discharge and the frequency and type of bystander cardiopulmonary resuscitation (CPR).
Prognostic factors studied: Age, sex, witnessed arrest, shockable rhythm, bystander CPR provision and type, location of arrest, EMS response interval, type of EMS resuscitation protocol (minimally interrupted cardiac resuscitation vs conventional basic life support/advanced cardiac life support), use of post-arrest therapeutic hypothermia and year.
Follow-up period: The patients were followed up until hospital discharge.
Main results: Of the 4415 patients, 2900 received no bystander CPR, 666 received standard CPR by a bystander, and 849 received chest compression only CPR by a bystander. Rates of survival to hospital discharge and good neurological outcome were 5.2% and 2.1% for no bystander CPR, 7.8% and 3.8% for standard CPR, and 13.3% and 6.5% for chest-compression-only CPR, respectively.
Conclusion: For patients with out-of-hospital cardiac arrest of presumed cardiac aetiology, bystander chest compression only was associated with increased survival to hospital discharge compared with conventional CPR and no bystander CPR.
Is the evidence likely to be biased?
• Was there a representative and well-defined sample?
Cannot tell, but probably. The study investigated all patients ≥18 years of age who suffered an out-of-hospital cardiac arrest, of cardiac origin, not witnessed by emergency medical service (EMS) staff, over a 5-year period for the whole state of Arizona, USA. Although it is probably implied by the authors of the article, we do not how many people were not included in this sample (that is, was it the entire state of Arizona that was potentially eligible?). However, it is hard to imagine a systematic bias towards patients with any better, or worse, prognosis.
• Was there an inception cohort?
Yes. Patients were included in the study following arrival of the ambulance.
• Was the exposure determined accurately?
Yes. All patients were assumed to be in cardiac arrest, unless they had obvious alternative explanations (such as drowning or trauma). Exposure to CPR was determined by the paramedics either by observation on their arrival or by questioning bystanders during the ongoing resuscitation attempt. Although there is some possibility of bias during collection of these data (such as misclassification of type of CPR that was performed), attempts were made to minimise this by training EMS staff in how to document the presence and type of bystander CPR.
• Were the outcomes measured accurately?
Yes. Patient survival to hospital discharge was determined by review of hospital records and the state health database. Neurological outcome was categorised, according to neurological status at discharge, using a 5-level Cerebral Performance Category (good cerebral performance, moderate cerebral disability, severe cerebral disability, coma or vegetative state and death).
• Were important prognostic factors considered?
Yes. Important prognostic factors were accounted for (see list in structured abstract).
• Was the follow-up of participants sufficiently long and complete?
Yes. Data was collected on participants until hospital discharge (which was the primary outcome measure) and few participants were lost to follow-up, mainly due to missing data (1.4% in the ‘no resuscitation’ group; 1.6% in the conventional CPR group; and 0.5% in the chest-compression-only CPR group).
Patients in each of the three groups were similar in age and gender, with the same response time for the EMS. Table 9.2 shows that patients who have a witnessed cardiac arrest and who receive some form of CPR are more likely to survive to hospital discharge with a good neurological outcome (scored 1 in the Cerebral Performance Category).
How might we use this evidence to inform practice?
This was a well-conducted study that suggests that the outcomes of patients who are given chest-compression-only CPR by bystanders is not hopeless, with about 13% surviving to hospital discharge and of those, about 6–7% with good neurological outcomes. The question about whether chest-compression-only CPR is better than conventional CPR is suggested by these results, but this is really a question that should be answered by a randomised controlled trial (if this were possible). However, you do feel empowered to tell your friend that there is reasonable evidence, from a large state-wide study, that chest-compression-only CPR from a lay person results in about 1 person in 15 surviving with a good outcome.