Prospective Implementation of the Ottawa Subarachnoid Hemorrhage Rule and 6-Hour Computed Tomography Rule

Introduction

Headache accounts for about 2% of all emergency department (ED) visits.^1,2 One the most serious causes is subarachnoid hemorrhage (SAH) due to a ruptured aneurysm, which accounts for about 1% of all patients with headache in ED.³ Nevertheless, SAH is often a foremost consideration among patients and physicians, who have come to rely on unenhanced computed tomography (CT) and CT angiography (CTA) as well as lumbar puncture (LP) to exclude this immediately life-threatening diagnosis.² Patients who are alert and neurologically intact at presentation represent a difficult diagnostic challenge. Unfortunately, some patients with SAH are misdiagnosed on the initial visit when investigations are not performed, while countless others at near zero risk are subjected to unnecessary testing.⁴ Patients treated early while still clinically well (ie, alert with no neurological deficits) have the best prognosis, while misdiagnosed patients may worsen catastrophically (due to re-bleeding or vasospasm) and fare poorly.⁵ Hence, early diagnosis is of paramount importance, yet overinvestigation is also harmful due to pain, exposure to radiation and contrast, delays to diagnosis, and the cascade of unnecessary testing and intervention for small, incidental aneurysms.

Our group has rigorously developed 2 evidence-based decision rules to optimize testing in alert and neurologically intact patients with acute headache. We previously found that current CT technology has near perfect sensitivity for SAH when performed within 6 hours of headache onset, obviating the need for LP (the 6-hour-CT rule).⁶ We also created the Ottawa SAH rule (Figure 1), which also has near perfect sensitivity for SAH.^7–9 Conducting an implementation study is an essential step in the development of any clinical decision rule. No prior prospective validation has been done for the 6-hour-CT rule, although 3 retrospective studies have demonstrated nearly perfect sensitivity.^10–12

• Download figure
• Download PowerPoint

The goal of this study was to determine the impact of the Ottawa SAH rule and 6-hour-CT rule when fully implemented versus current practice. Our specific objectives were to: (1) estimate the clinical impact of the Ottawa SAH rule and the 6-hour-CT rule on testing rates (ie, CT, LP, CTA) and on ED length of stay; (2) validate the accuracy of the 6-hour-CT rule for identifying patients with SAH when applied prospectively in a new cohort of patients.

Methods

Data Availability

Data will be made available to all interested researchers upon request. All requests will be reviewed by the Steering Committee, and the data will be available from the corresponding author on reasonable request.

Design

We conducted a prospective multicenter before-after controlled study comparing outcomes during 2 consecutive study periods in 6 hospital EDs. Sites were urban Canadian tertiary care teaching hospitals. We collected control phase data from January 2010 to June 2013. During this period, one site dropped out of the study in March 2011 because of high department clinical pressures. Following an education/training period, we commenced the intervention phase at the 5 remaining sites from June 2013 to January 2016. Data are fully anonymized in the database. The Research Ethics Boards waived the requirement for informed consent.

Study Population

We included consecutive alert patients in ED over 16 years of age presenting with a chief complaint of a nontraumatic, acute headache, or syncope associated with a headache who were assessed by an emergency physician. To be eligible, patients on arrival to the ED must have been alert (defined as Glasgow Coma Scale of 15/15, or awake and fully oriented), and presented within 14 days of a nontraumatic headache (defined as no known falls or direct trauma to the head within 7 days), which was acute in onset (defined as reaching maximal intensity in <1 hour). The patient must have presented to the ED within 14 days of the headache’s onset.

Patients were excluded for: (1) 3 or more previous similar headaches (ie, same intensity/character as their current headache) over a period of >6 months (eg, established migraines), (2) confirmed SAH before arrival at study ED, (3) previously investigated with CT and LP for the same headache, (4) papilledema, (5) new focal neurological deficit, (6) previous diagnosis of intracranial aneurysm or SAH, (7) known brain neoplasm, (8) cerebroventricular shunt, (9) headache within 72 hours following a LP, and (10) headache described as gradual or peak intensity beyond 1 hour.

In the control phase, there was no change in patient management. Physicians were instructed not to use the decision rules as a basis for patient care and the decision to order CT and perform a LP or CTA was left at their discretion. In the intervention phase, physicians were actively encouraged to use the Ottawa SAH rule and the 6-hour-CT rule but had the option to override the proposed rules. Each participating hospital obtained local ethics approval before starting data collection.

Setting

Our sites included: The Ottawa Hospital Civic Campus and General Campuses, Kingston General Hospital, Vancouver General Hospital, Alberta Health Sciences Centre, and Hôpital de l’Enfant Jésus in Québec City. The combined annual census of the 6 study sites was 365 000 ED visits. The data management and analysis were conducted at the Clinical Epidemiology Program at the Ottawa Hospital Research Institute.

Patient Assessment

In the control phase, patients were assessed by staff physicians certified in emergency medicine or supervised residents. Physicians were asked to record their findings on data collection forms attached to the patient’s record of treatment (attached by clerk, nurse or physician). Daily patient census was reviewed by the local site study nurse. Patients without a physician form had a data collection form completed by the site study nurse. The control phase was comprised of both patients enrolled by physicians and eligible patients identified during the census reviews. Similarly, eligible patients were identified in the same manner during the intervention period at all hospitals by having the site study nurse review daily patient census logs. Our central coordinator subsequently confirmed patient eligibility based on the inclusion and exclusion criteria. The central coordinator also verified data quality.

The sources of data reviewed included the ED health record (including nursing and physician notes), hospital electronic records, CT requisitions, clinic records, follow-up computer checks at neurosurgical hospitals, and provincial coroner’s records.

Intervention

During the intervention period, we asked physicians to utilize the Ottawa SAH rule and the 6-hour-CT rule. The training consisted of education with reminders to actively implement the Ottawa SAH rule and the 6-hour-CT rule with: (1) physicians were also given a one-hour group presentation, (2) targeted individual physician training, (3) additional educational initiatives (lecture, posters, pocket cards), and (4) a process-of-care modification (either paper or electronic) with a change in the CT requisition (paper or electronic depending on site) required physicians to check off the individual components of the rules or the reason they were overriding the rules before the technologist processed the requisition. The decision to investigate remained at the discretion of the treating physician.

Outcome Measures

The primary outcome, SAH, was defined as: (1) subarachnoid blood on CT, (2) xanthochromia in the cerebrospinal fluid, or (3) red blood cells (>1×10⁶/L) in the final tube of cerebrospinal fluid with an aneurysm demonstrated on cerebral angiography, CTA, or magnetic resonance imaging angiography.

Patients underwent unenhanced head CT and LP at the discretion of the treating physician. Certified radiologists or neuroradiologists, unaware of this study, interpreted all advanced imaging by their usual standard. CT scans were performed without contrast with third generation or better CT scanners using thin slices. The LPs were done according to current practice, and the red blood cell count and presence/absence of visible xanthochromia was recorded by laboratory technicians unaware of this study.

Proxy Outcome SAH

For every patient who underwent no testing, who had only CT but no LP, or who had an LP with red blood cells >1×10⁶/L in the final tube, we performed an electronic health records review at 6 months and at study end at every hospital with neurosurgical capacity in the same city as the index ED visit. In prior studies involving over 4000 subjects, we had also performed telephone follow-up to further exclude a subsequent SAH, but given a yield of zero and our Research Ethics Boards’ requirement for explicit consent before attempting telephone contact, we did not use telephone follow-up in the current study.^7–9,13 The need for consent would have precluded enrolling consecutive eligible patients with headache; including those who had not had a study form completed by a physician during the control phase and all patients in the intervention phase.

Analysis

We included in the primary analysis of diagnostic accuracy every consecutive eligible patient with headache during both the control and intervention periods, at all participating sites. Patients were not excluded in the intervention period if the physician failed to use the modified CT requisition, or elected not to follow the Ottawa SAH rule or the 6-hour-CT rule.

A χ² analysis stratified by site compared the proportion of patients with headache undergoing testing (CT, LP, CTA, or any testing post-CT) in the control versus intervention periods. We also measured the time from emergency physician assessment to discharge/referral (termed physician decision interval) as well as the total length of stay in ED in minutes from registration to discharge/referral. Both time intervals were tested using a 2-sided Student T-test. The accuracy of the Ottawa SAH rule and the 6-hour-CT rule for identifying SAH was assessed by sensitivity and specificity with 95% CIs.

We estimated a total sample size of 3000 patients (1500 in control period and 1500 in intervention period), with 180 positive SAH cases, based on enrollment windows and previous studies in which approximately 6% of enrolled cases experienced SAH.

Results

We enrolled 3672 patients with acute headache, including 188 (5.1%) with subarachnoid hemorrhage (Figure 2). Of these patients, 1743 were within the control phase and 1929 were following implementation of the Ottawa SAH rule and the 6-hour-CT rule. All patients in both the control and intervention periods with SAH were identified during their initial ED visit.

• Download figure
• Download PowerPoint

Table 1 depicts the characteristics of all patients stratified by phase. The characteristics of the patients were similar between the control and intervention phases for most characteristics. The patients in the intervention phase were recorded to have more: thunderclap headache (56.8% versus 40.2%, loss of consciousness [5.1% versus 3.5%], neck pain [33.6% versus 27.5%], and neck stiffness with flexion [6.9% versus 4.2%]). The patients in the intervention phase had less frequent headache onset with exertion (13.7% versus 16.5%).

The rate of CT use remained constant (88.0% in control versus 87.5% in the intervention; P=0.643; Table 2). The LP rate decreased from 38.9% to 25.9% in the intervention group (P<0.0001). While the CTA rate increased from 18.8% in the control phase to 21.7% in the intervention phase (P=0.029), the rate of additional testing beyond CT (ie, LP or CTA) decreased from 51.3% to 42.2% (P<0.0001). Admission rates declined from 9.8% in the control phase to 7.4% (P=0.011). The mean length of stay in the ED from arrival to discharge/referral was 6.3±4.0 hours in the control phase versus 6.4±4.2 hours in the intervention phase (P=0.685). The mean physician decision interval was 4.9±4.1 hours in the control phase versus 5.2±4.4 hours in the intervention phase (P=0.053). Physicians followed the Ottawa SAH rules in 77.5% of patients in the control phase versus 85.6% of cases in the intervention phase (P<0.0001).

For the 1204 patients who had CT within 6 hours of headache onset, there were 530 patients in the control phase and 674 in the intervention phase (Table 3). These patients had an absolute decrease in additional testing (ie, LP or CTA) rates of 14.8% (46.6% control phase versus 31.8% intervention phase; P<0.0001). Physicians adherence to the Ottawa SAH rule increased from 80.9% in the control phase to 92.5% in the intervention phase (P<0.0001). Likewise, the adherence to the 6-hour CT rule progressed from 60.7% to 85.5% (P<0.0001).

The sensitivity of the 6-hour-CT rule was 95.5% (95% CI, 89.8–98.5) for subarachnoid hemorrhage (Table 4). Five patients with early CT had SAH with CT reported as normal: 2 unruptured aneurysms on CTA and presumed traumatic LP (as deemed by the treating neurosurgeon); 1 missed by the radiologist on the initial interpretation; 1 dural vein fistula (ie, nonaneurysmal); and 1 patient with sickle cell anemia with profound anemia (Hgb, 63 g/L) with a 3 mm aneurysm. The sensitivity of the Ottawa SAH rule was 100% (95% CI, 98.1%–100%).

Discussion

Implementation studies are an essential step in the development of clinical decision rules, so that their impact can be assessed.^14–17 This before-after controlled implementation study of the Ottawa SAH rule and the 6-hour-CT rule found that overall investigation rates remained steady following implementation. We found that physicians were enthusiastic to adopt the 6-hour-CT rule. We found a clinically significant absolute reduction of 13.0% in LPs performed following the implementation of the Ottawa SAH rule and the 6-hour-CT rule. We found the Ottawa SAH rule to be 100% sensitive for SAH, further validating this rule in a new patient population. The 6-hour-CT rule remained highly sensitive at 95.5% for subarachnoid hemorrhage. While not perfectly sensitive, all 5 cases without a positive early CT were easily reconciled; one scan was misread initially as negative with blood present on subsequent blinded neuroradiologist re-assessment. Two were incidental aneurysms with traumatic tap, which underwent treatment, as determined by their respective treating physicians. These false positive cases were included in the main analyses as SAH, as they met our conservative study definition of SAH. One patient had a dural vein fistula, a rare cause of nonaneurysmal SAH. The final case was in a high-risk patient with sickle cell anemia, a hemoglobin of 63 g/L and a 3-mm posterior circulation aneurysm, which was coiled. This case highlights that blood is not reliably detectable in fluid-filled cavities (eg, cerebrospinal fluid) when the concentration of hemoglobin is <100 g/L. As blood becomes less dense on CT imaging,¹⁸

physicians should not apply the 6-hour-CT rule in patients known to be severely anemic. We also wish to emphasize that an experienced radiologist interprets the CT to exclude SAH. Omitting an unnecessary LP benefits a busy clinician, but is also beneficial from a patient perspective to avoid a painful procedure and the risk of developing a post lumbar headache which could last for days to weeks. Notwithstanding these rules, physicians need to remain vigilant for the other serious causes of headache, not all of which are identifiable on CT or LP.

We recommend the Ottawa SAH rule and the 6-hour-CT rule be implemented to standardize which neurologically intact patients with a new rapidly peaking patients with headache require investigation to rule out SAH. These rules provides clinicians with validated tools to standardize which patients with headache do not require investigation to rule out subarachnoid hemorrhage. Careful application of the Ottawa SAH rule will correctly identify almost all patients with subarachnoid hemorrhage and recommend no testing for low-risk patients. The 6-hour-CT rule will decrease the number of unnecessary LPs and CTAs for patients presenting within 6 hours of headache onset.

Given that headaches are frequent, yet few are due to SAH, physicians are often looking for the proverbial needle in a haystack.¹⁹ SAHs commonly occur in middle age, with a median age of 50 years.²⁰ Mortality irrespective of treatment is also high, around 50% by 6 months.²⁰ Of survivors, 2 in 5 have residual neurological deficits.^20,21 It has been estimated that 40% of patients with SAH have a warning headache (or sentinel leak) a short time before a larger, catastrophic bleed. These patients may seem relatively well at initial presentation and may be misdiagnosed at a time when the SAH is best treated.⁴ A previous population-based study found that nearly half of patients with SAH from rural/remote settings had a previous ED visit <2 weeks before their SAH diagnosis for a symptom compatible with a warning headache, as compared with 1 in 16 in urban community hospitals and 1 in 60 in tertiary care EDs.²² Misdiagnosis and delayed diagnosis account for most of the delay in referral to a neurosurgeon.^4,20,23–35 Patients with an undiagnosed small SAH that subsequently rehemorrhages have a worse prognosis.³³ Early treatment (surgery to clip aneurysms or intravascular coiling) can be life-saving for patients with SAH.^5,36–38 Hence, without standardizing investigations, patients suffer from a missed opportunity for early diagnosis of their SAH.

Our results have comparable sensitivity for the Ottawa SAH rule to a previous retrospective study of 454 patients with headache (including 9 subarachnoid hemorrhages). They found the Ottawa SAH rule had 100% sensitivity (95% CI, 62.9%–100%).³⁹ Their wide confidence bands were due to their relatively small sample size.

Limitations

During the control phase, physicians were explicitly told not to follow the Ottawa SAH rule and the 6-hour-CT rule, but rather use their usual care. Nevertheless, since physicians completed study forms, this may have taught physicians to consider SAH more carefully and made them more aware of the Ottawa SAH rule and the 6-hour-CT rule. We think the magnitude of this effect was small but may have attenuated the difference between the control and intervention phases.

We were unable to control for the temporal trends in the study EDs, including the global phenomenon of ever increasing ED congestion due to access block to in-hospital beds, such that patients who are admitted spend many hours to days in the ED. This, or other temporal factors not controlled for in this study, may explain why the length of stay did not decrease substantially despite less testing in our intervention phase.

The Ottawa SAH rule and 6-hour-CT rule only apply to patients with headache at risk for SAH. They are not designed to assess for other serious etiologies of headache (eg, bacterial meningitis, cerebral venous thrombosis, stroke, etc). Most of these conditions present with neurological deficits, altered mentation, or gradual onset headache, which exclude the use of the rules and suggest alternate diagnoses.

Conclusions

This implementation study validates the accuracy of the Ottawa SAH rule and 6-hour-CT rule for SAH. Both the Ottawa SAH rule and the 6-hour-CT rule are now fully validated and ready to use clinically. Using the Ottawa SAH rule did not increase or decrease the number of investigations performed. The 6-hour-CT rule resulted in a modest decrease in testing following a normal early CT. Utilizing the Ottawa SAH rule and the 6-hour-CT rule allows clinicians in ED to safely standardize care for alert, patients with acute headache.

Acknowledgments

We thank the physicians who completed data collection forms and the nurses and clerks at all our study sites. We thank the Office of the Coroner of Ontario for verifying outcomes. We thank Vancouver General Hospital, Vancouver, British Columbia; Ottawa Hospital, Ottawa, ON (Juanita Wilzer, Rebecca Briscoe); Kingston General Hospital, Kingston, Ontario (Jane Reid, Vlad Latiu); CHU de Québec—Hôpital de L’Enfant-Jésus, Quebec City, Quebec (Marilyne Dufresne, Suzy Lavoie). We thank our Ottawa Hospital Research Institute colleagues (Katherine Madill, Sheryl Domingo, My-Linh Tran, Catherine Clement, and Angela Marcantonio). Dr Perry conceived the idea and prepared the manuscript. Jane Sutherland coordinated the study and Dr Wells provided statistical assistance and revised the manuscript. Drs Sivilotti, Émond, Hohl, Lesiuk, M. Khan, K. Abdulaziz, and Dr Stiell assisted with design and manuscript writing. Dr Perry is the Guarantor and takes full responsibility for the manuscript and study.

Footnotes