2

Critical care outreach and rapid response systems

John R Welch and Christian P Subbe

Hospitals around the world are increasingly deploying dedicated outreach, medical emergency or rapid response teams to provide ‘critical care without walls’.1 The objective is to ‘ensure equity of care for all critically ill patients irrespective of their location’,2 particularly focusing on those with potential or actual critical illness in general wards.

Outreach and similar services are key components of what are known as rapid response systems. These are based on multidisciplinary ‘collaboration and partnership between critical care and other departments to ensure a continuum of care for patients, and [on enhancing] the skills and understanding of all staff in the delivery of critical care’.3 However, such services are not a replacement for insufficient critical care beds or under-resourced wards.

Background

Critical care units contain a small proportion of all hospital beds and have high rates of occupancy. Hospital admission criteria have become more stringent and lengths of stay have decreased in recent years. The result is that many ward patients have serious medical problems but only the most unstable gain admission to a critical care unit. Hence many at-risk patients remain in areas with staff inexperienced in managing critical illness. The problem has been compounded by changes in nursing education that have reduced training time in acute and critical care areas. Key tasks such as measuring physiological signs are often delegated to untrained staff who may not understand the significance of abnormal values; added to this many hospitals use temporary staff less likely to provide the continuity and team working essential for effective care. Medical education is also problematic;4 training is shorter and more specialised than before, and even senior doctors may be relatively inexperienced.5

Comparisons of outcomes of patients admitted to a critical care unit from either the emergency department, operating theatre/recovery area or the wards show that those coming from wards have the highest mortality.6 Suboptimal treatment is common before transfer to critical care, and is associated with worse outcomes.7,8 Crucially, differences in mortality have been shown to be due to variations in care rather than differences between the patients themselves and the longer patients are in hospital before admission to critical care, the higher is their mortality.7,9 Management is often performed by junior teams that fail to appreciate clinical urgency and the importance of senior advice. Inadequate supervision, poor organisation, gaps in communication and continuity of care are also factors.7,8,10

Patients who experience lengthy periods of instability before there is an effective medical response are said to have suffered ‘failure to rescue’. Such failures are common.7,8,1012 In a national review of medical patients subsequently transferred to a critical care unit, many had sustained up to 72 hours of physiological instability.8 Analysis of 1000 deaths in 10 hospitals concluded that 52 deaths would have had a 50% or greater chance of being prevented; although it is noteworthy that most of these preventable deaths were in elderly, frail patients judged to have had a life expectancy of less than 12 months.13

Other groups of patients at risk are those recently discharged from the critical care unit or from the operating theatre after major surgery: about one-quarter of all ‘critical care deaths’ occur after discharge back to the ward. In particular, patients discharged prematurely suffer increased mortality.14,15

Outreach, medical emergency and rapid response teams

Medical emergency teams (METs) were introduced in Australia in the 1990s, usually comprising critical care residents and medical registrars. These teams could be directly activated by any member of staff bypassing traditional hospital hierarchies. METs expanded the role of the cardiac arrest team to include the pre-arrest period, generally using call-out criteria based on deranged physiological values or staff concern.16 In the UK, a review of critical care services in 200017 led to increased funding for critical care beds and also the creation of critical care outreach teams, largely staffed by critical care nurses. Similar services have emerged in the USA, driven by the Institute for Healthcare Improvement18 with more consideration of a whole ‘rapid response system’ (RRS). This highlights the principle that it is necessary to develop complete, coordinated systems to avoid failures to rescue reliably and consistently.

The RRS can be divided into:

• an afferent component designed to ensure timely escalation of the deteriorating patient, usually using agreed physiological values as a trigger

• an efferent component comprising an individual or team of clinicians who can promptly respond to deterioration

• governance and administrative structures to oversee and organise the service and its ways of working

• mechanisms to improve hospital processes.19

Another approach is to think of the RRS as being built on development of a ‘chain of prevention’ made up of education, monitoring, recognition, call and response.20

There are now many models and terms used.19,21 METs are usually physician led. Critical care outreach (CCO) and rapid response teams (RRTs) are typically nurse led, but may also include physiotherapists and other allied health professionals as well as doctors. Most teams respond to defined physiological triggers, although some also work proactively with known at-risk patients such as those discharged from the critical care unit.

The aim is to prevent unnecessary critical care admissions, to ensure timely transfer to the critical care unit when needed, to facilitate safe return to the ward, to share critical care skills,17 and to improve care throughout the hospital. There may also be a role in support for patients and their families after hospital discharge (Box 2.1).

Box 2.1   Functions of critical care outreach

• Identification of at-risk patients

• Support for ward staff caring for at-risk patients and those recovering from critical illness

• Referral pathways for obtaining timely, effective critical care treatments

• Immediate availability of expert critical care and resuscitation skills when required

• Facilitation of timely transfer to a critical care facility when needed

• Education for ward staff in recognition of fundamental signs of deterioration, and in understanding how to obtain appropriate help promptly

• Outpatient support to patients and their families following discharge from hospital

• Development of systems of coordinated, collaborative, continuous care of critically ill and recovering patients across the hospital and also in the community

• Audit and improvement of basic standards of acute and critical care – and of the outreach team itself – to minimise risk and optimise treatment of the critically ill throughout the hospital

Together, these elements comprise a system to deliver safe, quality care with proactive management of risk and timely treatment of critical illness.

Recognising critical illness

Patients with potential or actual critical illness can be identified by review of the history, by examination and by investigations. Higher risks are associated with extremes of age, with significant co-morbidities or with serious presenting conditions.

The timeliness of response depends largely on the quality of monitoring. Patients at risk of deterioration require either very frequent or continuous monitoring to optimise the effect of a rapid response intervention. A conference on the afferent limb of the RRS found that: ‘(1) vital sign aberrations predict risk, (2) monitoring patients more effectively may improve outcome, although some risk is random, (3) the workload implications of monitoring on the clinical workforce have not been explored, but … should be investigated, (4) the characteristics of an ideal monitoring system are identifiable, and it is possible to categorize monitoring modalities. It may also be possible to describe monitoring levels, and a system’.22 Currently, measuring and recording of vital signs on general wards are often inadequate.8,10

For early recognition of deterioration to be effective:

• the physiological values, laboratory results or other data used for patient monitoring should enable timely identification of deterioration

• there must be enough time to identify at-risk patients and then obtain expert assistance before irreversible deterioration has occurred.

Abnormal physiology and adverse outcome

There is a known association between abnormal physiology and adverse outcomes,23,24 and critical care severity scoring systems such as APACHE II25 are based upon this relationship. Patients who suffer cardiopulmonary arrest or who die in hospital generally have abnormal physiological values recorded in the preceding period, as do patients requiring transfer to the critical care unit.8,1012,23,24

The finding that abnormal physiology precedes adverse events has led to key signs being incorporated into various early warning scoring (EWS) systems. These systems use different combinations of parameters including respiratory rate, oxygen saturation, heart rate, blood pressure, temperature and level of consciousness as well as other indicators such as urine output and pain.26 The patient's measured vital signs are compared with a set of reference values, with measurements above or below designated points used as triggers for escalation. Formats vary but generally use similar approaches, awarding points for varying degrees of derangement of different functions. Improvement or further deterioration can then be tracked by changes in EWS recorded over time, so that an EWS used in this way can be described as a ‘track and trigger system’. Track and trigger systems are broadly categorised as single or multiple parameter systems, aggregate weighted scoring systems or combinations2 (Box 2.2).

Box 2.2   Classification of track and trigger warning systems2

Single-parameter systems

• Tracking: periodic observation of selected basic signs

• Trigger: one or more extreme observational values

Multiple parameter systems

• Tracking: periodic observation of selected basic vital signs

• Trigger: two or more extreme observational values

Aggregate weighted scoring systems

• Tracking: periodic observation of selected basic vital signs and the assignment of weighted scores to physiological values with calculation of a total score

• Trigger: achieving a previously agreed trigger threshold with the total score

• Combination systems

• Elements of single- or multiple-parameter systems in combination with aggregate weighted scoring

Many different systems with variable trigger thresholds have been developed.2729 This variance has led to calls for standardised systems to improve training and reliability of response, with the UK National Early Warning Score (NEWS) published in 201230 (Table 2.1) and now adopted in Wales, Ireland and England. It is based on the analysis of a large database of patients' vital signs recorded in different acute hospitals.31

Table 2.1

National early warning score (NEWS)31

image

A different approach has been taken by Australian METs, where the calling-out criteria are usually based upon single, markedly deranged physiological values, although ward staff concern is also a trigger32 (Box 2.3).

Box 2.3   Medical emergency team calling-out criteria as used in the 23-site MERIT study32

Airway  Threatened
Breathing  Respiratory rate <5 or >36 per min
   Respiratory arrest
Circulation  Systolic blood pressure <90 mmHg (11.97 kPa)
   Pulse rate <40 or >140 per min
Neurology  Sudden fall in level of consciousness (fall in GCS of >2 points)
   Repeated or extended seizures
Other  Any patient you are seriously worried about

As well as EWS systems based simply on acute physiology, there are also published methods using other data to risk stratify patients at hospital admission. These systems aim to differentiate patients who need to stay in hospital for further monitoring or treatment and those who need only minimal monitoring or may even be discharged home. Systems based on laboratory parameters alone,33 laboratory parameters in conjunction with vital sign observations,34 or indicators of acute physiology, chronic illness and functional status35 have all been validated against hospital mortality.

Another possible method of activating the RRS is for patients themselves – or their relatives – to call. This method was first used in paediatric settings but may also be useful for adults.36

Measuring outcome

The use of critical care outreach and other RRSs is based on the premise that early detection and treatment of critical illness should improve patient outcomes. The quality of these services may be evaluated against such outcomes but also other indicators including process measures (e.g. numbers of trained staff, completeness of bedside observations, timeliness of escalation and speed of response). The time from patient trigger to transfer to a critical care unit – or initiation of critical care treatments on the ward – may be a useful indicator too (i.e. the ‘Score-to-Door time’37).

Table 2.2 shows one system that can be used to evaluate outcomes of RRS interventions 24 hours after the initial event, with outcomes classified as being either positive or negative. The proportion of positive interventions provides a measure of the quality of the service.

Table 2.2

Matrix of possible outcomes of RRS intervention: the ‘Multi-disciplinary Audit EvaLuating Outcomes of Rapid response’ (MAELOR) tool

OUTCOMES POSITIVE NEGATIVE
Transfer to critical care unit, high-dependency area or operating theatre 1. Timely transfer, e.g. <4 hours after the first trigger 2. Delayed transfer, e.g. >4 hours after the first trigger
Alive on ward 3. No longer triggering 4. Still triggering
Deceased 5. On terminal care pathway/with DNAR order 6. Following cardiopulmonary arrest
Others 7. Alive with documented treatment limitations and DNAR order in place
8. a. Trigger from new pathology unrelated to previous call-out
 b. Chronic condition leading to continuous trigger (e.g. tachypnoea in advanced pulmonary fibrosis)
 c. Discharged from hospital
9. Outcome not known/lost to follow-up

Data from Morris A, Owen HM, Jones K, et al. Objective patient-related outcomes of rapid-response systems – a pilot study to demonstrate feasibility in two hospitals. Crit Care Resusc. 2013;15(1):33–9.

RRSs have highlighted shortcomings in the care of ward patients, and contributed to a significant change in attitude to at-risk patients. They have been instrumental in improving ward monitoring, and in disseminating critical care skills. There are anecdotal reports of benefit to individuals,38 and published evidence that these services improve recognition of at-risk patients, reduce length of stay, cardiac arrest rates, unplanned admissions to critical care, and morbidity and mortality.3943 However, some reports do not show significant effects. There are in fact few good quality studies, with just two randomised controlled trials published to date.32,43

Positive studies include a UK randomised trial of phased introduction of a 24-hour outreach service to 16 wards in a general acute hospital.43,44 The outreach team routinely followed up patients discharged from critical care to wards and also saw referrals generated by ward staff concern or use of an EWS system. There was a statistically significant reduction in mortality in wards where the service was operational. In contrast, a large prospective randomised trial of METs in Australia found no improvements in cardiac arrests, unplanned admissions to critical care or unexpected deaths in comparison to the control hospitals in the primary analysis.32 A secondary analysis was able to show improved outcomes in most hospitals in both the intervention and control groups, with dramatic improvements in those with the weakest baseline performance.45 This study revealed many shortcomings in identification and care of critically ill patients, with one possible conclusion being that it is essential to take a whole systems approach to early recognition of deterioration and achievement of an effective response.

Several studies have shown an inverse relation between the number of calls to the MET and the rates of cardiac arrest.46 The explanation for this is not completely clear. It may be that reductions in cardiac arrests are linked to increased proportions of patients surviving to discharge, but it is as likely that decreased cardiac arrest calls are a reflection of better patient assessment and more timely implementation of Do-Not-Attempt-Resuscitation orders and involvement of palliative care specialists in patients with terminal illness. This is not a negative: delivery of good palliative care might be one of the positive outcomes supported by a RRS.47

There has been less investigation of the follow-up of patients discharged from critical care units, although this group is known to remain at significant risk. A matched-cohort analysis of 5924 patients found follow-up by an outreach team reduced length of stay and mortality when compared with historical controls and matched patients from hospitals with no outreach.48

Setting up an outreach service

Patients with potential or actual critical care illness are found in every area of the hospital, so systems to identify and treat those patients need to be planned at an organisational level. Involvement of managerial and clinical staff is essential, especially from the wards. It is particularly important that there is agreement and clarity about how the outreach team or equivalent interacts with the parent/primary medical team.

Key steps in planning an RRS

• Appoint senior clinical and managerial leads to develop the service.

• Institute organisational needs analysis, audit and evaluation, asking:

– which patients are at risk of critical illness and where are they located?

– where do cardiopulmonary arrests and unexpected deaths occur?

– what is the source of unplanned admissions to the critical care unit?

– what is the pattern of adverse events where harm can be attributed to the process of care?

– what are the other relevant clinical governance/risk management issues (e.g. complaints), or morbidity and mortality data?

• A point prevalence study can give a snapshot view of the location of patients with physiological derangement.

• Review of unplanned admissions to the critical care unit can identify systems failings including quality of patient management and appropriateness and timeliness of escalation. Key practices can be assessed against specific, measurable standards.

• Such analyses should also highlight staff education and training needs.

Other factors to consider include:

• the patient case-mix

• existing skills of ward staff

• proposed hours of service

• size of hospital – and likely demand

• existing services such as pain teams, nutrition teams, tracheostomy specialist practitioners respiratory specialists, renal specialists, night teams, etc.

• training facilities

• outreach service location and equipment needs including information technology

• funding.

Various bodies in the UK, Australia and USA have published useful guides to setting up and developing a RRS; all are available online.2,3,49,50

The outreach team

The composition and skills of the team should be designed to meet the specific needs identified by individual organisations. At a minimum, the team should be capable of assessment, diagnosis, initiation of resuscitation, and rapid triage of the critically ill patient to a higher level of care with authority to so act. Such clinical competencies as airway management techniques, venepuncture and cannulation are essential, and so are skills in education and training, research and audit. A multiprofessional team is required for this range of skills to be available, and to enable communication with other staff across the hospital. The UK Department of Health has detailed the competencies required for care of at-risk and deteriorating patients, specifying what should be expected of junior, middle-grade and senior staff.51

A pragmatic, staged implementation could include:

1. Establishing an education programme in care of the critically ill for ward staff so that they can recognise signs of deterioration and understand the necessity and means of obtaining timely help. Staff should update their skills annually.

2. Introducing a physiological track and trigger warning system and defined referral/response protocols.

3. Developing clinical bedside support – incrementally if necessary – increasing the number of clinical areas covered by the team, and the hours of work. This might include follow-up of patients discharged from critical care and responding to patients identified through the track and trigger system or other means.2

It is essential that robust data are collected and used for audit and evaluation – and for feedback to ward managers and clinical staff. Successes should be highlighted and areas for improvement identified. Data may include:

• numbers of referrals and patient follow-ups

• date and time of each episode

• patient details (e.g. age, sex, date of hospital admission, location, emergency/elective admission, medical/surgical, resuscitation status)

• trigger event (e.g. early warning score, cardiac arrest call)

• significant problems identified

• interventions performed

• patient outcomes.

The future: technology to mitigate human factors

In an increasingly safety conscious society, ‘failure to rescue’ becomes less and less acceptable. It is clear that many of the errors that lead to ‘failure to rescue’ are caused by human factors and flaws in the design of hospital systems.5254 This was shown by the MERIT study finding that of patients needing escalation to the critical care unit – with signs that should have been referred to the MET – only 30% were actually referred.32 Hierarchical thinking, inflexible mental modelling, highly variable performance and uncoordinated, inefficient hospital organisation are all factors.5254 Even relatively simple matters such as the documentation for vital sign recording have a role: research from Australia has shown that attention to the layout of charts is likely to promote more reliable detection of deterioration.55

Automation has the potential to improve reliability of some key processes. Technological aids that automate calculation of early warning scores and communication of abnormal trigger scores are available. These systems are able to perform calculations of EWS with fewer errors and have been shown to improve outcomes.56 The development of increasingly sophisticated expert systems will enable analysis of patterns of abnormal vital signs that can produce specific alerts as well as prompts and advice about individual patients, with due consideration of their particular pathophysiology.

Conclusion

There is no doubt that there are significant numbers of patients on hospital wards with potential or actual critical illness whose care should and could be improved. The RRS represents one method of addressing these issues. In the future, it may turn out to be that the most useful contribution of RRSs is the highlighting of defects in current ways of working, and the application of what has been learned from RRS initiatives to the whole hospital.

Key features include:

• Deteriorating patients can be identified by careful monitoring of physiological signs.

• Timely escalation of appropriate patients to critical care should improve outcomes.

• Effective response to acute deterioration depends on complex human interactions that are prone to error.

• Rapid response systems standardise the response to at-risk and deteriorating patients, and improve process and clinical outcomes for critically ill patients presenting outside the critical care unit.

• Successful systems are based upon multiprofessional working, and effective communication education, data collection/audit, learning from errors, and planned improvement of whole systems of care.

References

1. Hillman, K. Critical care without walls. Curr Opin Crit Care. 2002; 8(6):594–599.

2. Department of Health and NHS Modernisation Agency, The National Outreach Report 2003/Critical Care Outreach 2003 – progress in developing services. Department of Health, London, 2003. www. dh. gov. uk/en/Publicationsandstatistics/Publications/PublicationsPolicyAndGuidance/DH_4091873

3. Intensive Care Society, Guidelines for the introduction of outreach services. Intensive Care Society, London, 2002. www. ics. ac. uk/professional/standards_and_guidelines/guidelines_for_the_introduction_of_outreach_2003

4. Tallentire, VR, Smith, SE, Skinner, J, et al. The preparedness of UK graduates in acute care: a systematic literature review. Postgrad Med J. 2012; 88(1041):365–371.

5. Chikwe, J, de Souza, AC, Pepper, JR. No time to train the surgeons. BMJ. 2004; 328(7437):418–419.

6. Goldhill, DR, Sumner, A. Outcome of intensive care patients in a group of British intensive care units. Crit Care Med. 1998; 26(8):1337–1345.

7. McQuillan, P, Pilkington, S, Allan, A, et al. Confidential inquiry into quality of care before admission to intensive care. BMJ. 1998; 316(7148):1853–1858.

8. Cullinane, M, Findlay, G, Hargraves, C, et al. An Acute Problem?. London: National Confidential Enquiry into Patient Outcome and Death; 2005.

9. Goldhill, DR, McNarry, AF, Hadjianastassiou, VG, et al. The longer patients are in hospital before intensive care admission the higher their mortality. Intensive Care Med. 2004; 30(10):1908–1913.

10. Findlay, GP, Shotton, H, Kelly, K, et al. Time to intervene? A review of patients who underwent cardiopulmonary resuscitation as a result of an in-hospital cardiorespiratory arrest. London: National Confidential Enquiry into Patient Outcome and Death; 2012.

11. Buist, MD, Jarmolowski, E, Burton, PR, et al. Recognising clinical instability in hospital patients before cardiac arrest or unplanned admission to intensive care. A pilot study in a tertiary-care hospital. Med J Aust. 1999; 171(1):22–25.

12. Berlot, G, Pangher, A, Petrucci, L, et al. Anticipating events of in-hospital cardiac arrest. Eur J Emerg Med. 2004; 11(1):24–28.

13. Hogan, H, Healey, F, Neale, G, et al. Preventable deaths due to problems in care in English acute hospitals: a retrospective case record review study. BMJ Qual Saf. 2012; 21(9):737–745.

14. Daly, K, Beale, R, Chang, RW. Reduction in mortality after inappropriate early discharge from intensive care unit: logistic regression triage model. BMJ. 2001; 322(7297):1274–1276.

15. Goldfrad, C, Rowan, K. Consequences of discharges from intensive care at night. Lancet. 2000; 355(9210):1138–1142.

16. Lee, A, Bishop, G, Hillman, KM, et al. The medical emergency team. Anaesth Intensive Care. 1995; 23(2):183–186.

17. Department of Health. Comprehensive critical care: a review of adult critical care services. London: Department of Health; 2000.

18. Berwick, DM, Calkins, DR, McCannon, CJ, et al. The 100,000 lives campaign: setting a goal and a deadline for improving health care quality. JAMA. 2006; 295(3):324–327.

19. Devita, MA, Bellomo, R, Hillman, K, et al. Findings of the first consensus conference on medical emergency teams. Crit Care Med. 2006; 34(9):2463–2478.

20. Smith, GB. In-hospital cardiac arrest: is it time for an in-hospital ‘chain of prevention’? Resuscitation. 2010; 81(9):1209–1211.

21. Esmonde, L, McDonnell, A, Ball, C, et al. Investigating the effectiveness of critical care outreach services: a systematic review. Intensive Care Med. 2006; 32(11):1713–1721.

22. DeVita, MA, Smith, GB, Adam, SK, et al. ‘Identifying the hospitalised patient in crisis’ – a consensus conference on the afferent limb of rapid response systems. Resuscitation. 2010; 81(4):375–382.

23. Kause, J, Smith, G, Prytherch, D, et al. A comparison of antecedents to cardiac arrests, deaths and emergency intensive care admissions in Australia and New Zealand, and the United Kingdom – the ACADEMIA study. Resuscitation. 2004; 62(3):275–282.

24. Harrison, GA, Jacques, T, McLaws, ML, et al. Combinations of early signs of critical illness predict in-hospital death – the SOCCER study (signs of critical conditions and emergency responses). Resuscitation. 2006; 71(3):327–334.

25. Knaus, WA, Draper, EA, Wagner, DP, et al. APACHE II: a severity of disease classification system. Crit Care Med. 1985; 13(10):818–829.

26. Bright, D, Walker, W, Bion, J. Clinical review: Outreach – a strategy for improving the care of the acutely ill hospitalized patient. Crit Care. 2004; 8(1):33–40.

27. Gao, H, McDonnell, A, Harrison, DA, et al. Systematic review and evaluation of physiological track and trigger warning systems for identifying at-risk patients on the ward. Intensive Care Med. 2007; 33(4):667–679.

28. Smith, GB, Prytherch, DR, Schmidt, PE, et al. Review and performance evaluation of aggregate weighted ‘track and trigger’ systems. Resuscitation. 2008; 77(2):170–179.

29. Smith, GB, Prytherch, DR, Schmidt, PE, et al. A review, and performance evaluation, of single-parameter ‘track and trigger’ systems. Resuscitation. 2008; 79(1):11–21.

30. Royal College of Physicians. National Early Warning Score (NEWS): standardising the assessment of acute illness severity in the NHS. Report of a working party. London: Royal College of Physicians; 2012.

31. Prytherch, DR, Smith, GB, Schmidt, PE, et al. ViEWS – towards a national early warning score for detecting adult inpatient deterioration. Resuscitation. 2010; 81(8):932–937.

32. Hillman, K, Chen, J, Cretikos, M, et al. Introduction of the medical emergency team (MET) system: a cluster-randomised controlled trial. Lancet. 2005; 365(9477):2091–2097.

33. O'Sullivan, E, Callely, E, O'Riordan, D, et al. Predicting outcomes in emergency medical admissions – role of laboratory data and co-morbidity. Acute Med. 2012; 11(2):59–65.

34. Silke, B, Kellett, J, Rooney, T, et al. An improved medical admissions risk system using multivariable fractional polynomial logistic regression modelling. QJM. 2010; 103(1):23–32.

35. Kellett, J, Deane, B. The Simple Clinical Score predicts mortality for 30 days after admission to an acute medical unit. QJM. 2006; 99(11):771–781.

36. Odell, M, Gerber, K, Gager, M. Call 4 Concern: patient and relative activated critical care outreach. Br J Nurs. 2010; 19(22):1390–1395.

37. Oglesby, KJ, Durham, L, Welch, J, et al. ‘Score to Door Time’, a benchmarking tool for rapid response systems: a pilot multi-centre service evaluation. Crit Care. 2011; 15(4):R180.

38. Park, GR, McElligot, M, Torres, C. Outreach critical care–cash for no questions? Br J Anaesth. 2003; 90(5):700–701.

39. Buist, MD, Moore, GE, Bernard, SA, et al. Effects of a medical emergency team on reduction of incidence of and mortality from unexpected cardiac arrests in hospital: preliminary study. BMJ. 2002; 324(7334):387–390.

40. Bellomo, R, Goldsmith, D, Uchino, S, et al. A prospective before-and-after trial of a medical emergency team. Med J Aust. 2003; 179(6):283–287.

41. Bellomo, R, Goldsmith, D, Uchino, S, et al. Prospective controlled trial of effect of medical emergency team on postoperative morbidity and mortality rates. Crit Care Med. 2004; 32(4):916–921.

42. Ball, C, Kirkby, M, Williams, S. Effect of the critical care outreach team on patient survival to discharge from hospital and readmission to critical care: non-randomised population based study. BMJ. 2003; 327(7422):1014–1017.

43. Priestley, G, Watson, W, Rashidian, A, et al. Introducing Critical Care Outreach: a ward-randomised trial of phased introduction in a general hospital. Intensive Care Med. 2004; 30(7):1398–1404.

44. Watson, W, Mozley, C, Cope, J, et al. Implementing a nurse-led critical care outreach service in an acute hospital. J Clin Nurs. 2006; 15(1):105–110.

45. Chen, J, Bellomo, R, Flabouris, A, et al. The relationship between early emergency team calls and serious adverse events. Crit Care Med. 2009; 37(1):148–153.

46. Buist, M, Harrison, J, Abaloz, E, et al. Six year audit of cardiac arrests and medical emergency team calls in an Australian outer metropolitan teaching hospital. BMJ. 2007; 335(7631):1210–1212.

47. Jones, DA, Bagshaw, SM, Barrett, J, et al. The role of the medical emergency team in end-of-life care: a multicenter, prospective, observational study. Crit Care Med. 2012; 40(1):98–103.

48. Harrison, DA, Gao, H, Welch, CA, et al. The effects of critical care outreach services before and after critical care: a matched-cohort analysis. J Crit Care. 2010; 25(2):196–204.

49. Australian Commission on Safety and Quality in Health Care. ‘Recognition and Response to Clinical Deterioration’ programme. Online. Available www. safetyandquality. gov. au. [(accessed 1st October 2012)].

50. 5 Million Lives Campaign, Getting started kit: rapid response teams. Institute for Healthcare Improvement, Cambridge, MA, 2008. www. ihi. org

51. Department of Health, Competencies for recognising and responding to acutely ill patients in hospital. Department of Health, London, 2009. www. dh. gov. uk/en/Publicationsandstatistics/Publications/PublicationsPolicyAndGuidance/DH_096989

52. Shearer, B, Marshall, S, Buist, MD, et al. What stops hospital clinical staff from following protocols? An analysis of the incidence and factors behind the failure of bedside clinical staff to activate the rapid response system in a multi-campus Australian metropolitan healthcare service. BMJ Qual Saf. 2012; 21(7):569–755.

53. Mackintosh, N, Rainey, H, Sandall, J. Understanding how rapid response systems may improve safety for the acutely ill patient: learning from the frontline. BMJ Qual Saf. 2012; 21(2):135–144.

54. Peebles, E, Subbe, CP, Hughes, P, et al. Timing and teamwork – an observational pilot study of patients referred to a Rapid Response Team with the aim of identifying factors amenable to re-design of a Rapid Response System. Resuscitation. 2012; 83(6):782–787.

55. Horswill, MS, Preece, MHW, Hill, A, et al, Human factors research regarding observation charts: research project overview. Report prepared for the Australian Commission on Safety and Quality in Health Care's program for Recognising and Responding to Clinical Deterioration. School of Psychology, The University of Queensland, St Lucia, Queensland, 2010. http://espace. library. uq. edu. au/eserv/UQ:220736/Horswill_et_al_2010_Obs_Chart_Research. pdf

56. Bellomo, R, Ackerman, M, Bailey, M, et al. A controlled trial of electronic automated advisory vital signs monitoring in general hospital wards. Crit Care Med. 2012; 40(8):2349–2361.