Study setting
We performed a single center retrospective study in a tertiary care freestanding children’s hospital ED with an annual volume of over 80,000 patients. Ours is the only children’s hospital in the region and is part of a large, integrated healthcare delivery system that includes 42 hospitals with a shared electronic medical record, including the radiology Picture Archiving and Communication System. Since November 2017, our ED implemented widespread ED availability of 4 distinct rMRI neuroimaging protocols (Table 1): ventricular shunt evaluation, abusive head trauma screen, stroke, and nonspecific neurologic complaints (e.g. headache, seizure, altered mental status, vomiting). Prior to this time, although some protocols were available, they were not yet utilized as part of routine ED practice. rMRI are performed using five MRI scanners: 1) three GE Signa 1.5 T, 2) one GE Signa 3 T (GE Healthcare, Chicago, IL), and 3) one Siemens Skyra 3 T (Siemens Medical Solutions, Malvern, PA). The protocols performed are the same for all patients irrespective of age.
Neuroimaging studies performed as part of an ED encounter at our institution are not done with sedation; and, if sedation is needed, imaging is performed once the patient is admitted (or scheduled as an outpatient.) It is possible, however, for patients to receive oral, intranasal, or intravenous anxiolysis (e.g. midazolam, dexmedetomidine) to facilitate neuroimaging. We did not specifically identify use of sedative agents utilized for the purposes of neuroimaging in the context of this study.
Study cohort
We included neuroimaging studies (rMRI brain, HCT, and full MRI) performed as part of an ED encounter during two periods: November 1, 2016 to July 31, 2017 (control period) and November 1, 2017 to July 31, 2018 (rMRI period). We excluded studies performed for patients > 18 years-old. HCT remains the modality of choice for acute head trauma [2], therefore, we excluded imaging studies in patients > 12 months of age if the imaging was for trauma as documented as part of the “indication” section of the radiology report. We included studies in patients with possible trauma ≤12 months of age because of a dedicated rMRI protocol to evaluate for abusive head trauma in this age group, which may or may not be associated with a corroborating history or physical examination for an acute traumatic injury.
Neuroimaging studies
For each encounter, we defined the “index” neuroimaging study as the first study (non-contrast rMRI, non-contrast HCT, or full MRI with and/or without contrast) performed in the ED. We included only one ED encounter per patient per study period, and, a priori, determined that if a patient had multiple encounters during one of the two study periods, only the last encounter for that period would be included.
We assessed for follow-up imaging after the index study, defined as any neuroimaging (including outpatient, inpatient, and ED) performed within 14 days of the index study. For cases when more than one study was performed within this period, the earliest was considered the follow-up study. We further evaluated within this 14-day period if a full MRI was performed, even if it was not the first follow-up imaging study performed.
For index and follow-up neuroimaging studies, two pediatric emergency medicine physicians (S.R. and J.R.M), blinded to the clinical history and examination and to the type of imaging performed, reviewed all attending radiology interpretations and classified them into one of four categories: a) positive, in which the findings from neuroimaging would require further testing, admission, or subspecialist consultation; b) negative, defined as no acute pathology identified or would typically not require further investigation or follow up, c) unknown, in which the results could not be classified as positive or negative, and d) unsuccessful, defined as a study which was sufficiently limited to preclude radiology interpretation. For this process, the investigators both reviewed approximately 50% of all of the included imaging studies in order to determine if there was sufficient agreement, defined, a priori, as Kappa ≥0.70 [17], before reviewing the remainder independently. For discordances between the two raters as well as interpretations deemed unknown, an attending radiologist (also blinded as described above) reviewed the interpretation and classified the study as positive or negative.
For patients with a follow-up full MRI within 14 days of the index study and for whom the index scan was either a HCT or rMRI during the rMRI study period, two attending radiologists, blinded to the index study modality, independently reviewed the full MRI interpretations and compared them to HCT or rMRI index exam interpretations to evaluate for undetected pathology in these studies. Specifically, they categorized index studies as: true positive (including possible progression of disease), true negative, false positive, and false negative.
Data acquisition
We used Centricity RIS-IC (version 6.0; GE Healthcare, Chicago, IL) to determine which patients had neuroimaging during the two study periods. These data were then linked to radiology results obtained using mPower Clinical Analytics (version 3.2.1; Burlington, Massachusetts) using unique accession numbers. Patient-related data were obtained from data harbored by the electronic health record using the business intelligence platform SAP BusinessObjects (SAP, Waldorf, Germany). We extracted the following for each encounter: patient age, sex, race, ED chief complaint, reason for neuroimaging, emergency severity index (ESI) score, time of arrival, time of final disposition, ED disposition (admitted, discharged, transferred to another institution, or deceased), time and duration of index neuroimaging, and follow-up neuroimaging. The ESI is assigned by a triage nurse and is an ED triage algorithm that stratifies patients into 5 groups on the basis of acuity and resource needs, with 1 representing the most acute [18]. ED chief complaint is also assigned by a triage nurse at the time of arrival and is based on a standardized list of 109 complaints.
Outcomes
The primary outcome was rates of neuroimaging (rMRI, HCT, and full MRI) between both periods. Secondary outcomes were time to index neuroimaging study, ED LOS, rates of unsuccessful index imaging, follow-up imaging, and undetected pathology on initial imaging for patients with rMRI or HCT as the index study and for whom a subsequent MRI was performed within 14 days of initial evaluation.
We also assessed imaging patterns across the entire ED population (not exclusively those with neuroimaging) and evaluated use of any neuroimaging as well as modality-specific rates for each time period. We applied the same exclusion criteria as in our primary cohort and identified those with trauma using the International Classification of Diseases, revision-10 codes, S00-T88.
Data analysis
We summarized demographics between the two time periods using proportions. We calculated interrater reliability using Cohen’s Kappa statistic [17]. All continuous data were nonparametric. For our primary outcome, we compared the difference in proportions for each type of neuroimaging (HCT, rMRI, and full MRI) between the two periods and analyzed changes in neuroimaging by chief complaint and assessed for differences using chi-squared tests. For secondary outcomes we compared results using chi-squared and Wilcoxon rank-sum tests. For the evaluation of undetected pathology on index imaging during the rMRI period, we calculated the false negative rate of rMRI or HCT using full MRI performed within 14 days as the reference standard. We further assessed the accuracy of these index studies by calculating the sensitivity, specificity, positive predictive value, negative predictive value, positive likelihood ratio and negative likelihood ratio with 95% confidence intervals. Analyses were performed using the epiR (version 0.9.99) package for R, version 3.5.1 (R Foundation for Statistical Computing, Vienna, Austria). This study was approved with a waiver of informed consent by the University of Pittsburgh Human Research Protection Office.
Exploratory analysis
In order to assess the effect of our assumptions regarding non-accidental trauma (i.e. including patients < 12 months with any trauma) on the primary outcome, we performed an additional analysis excluding all patients with trauma regardless of age. Additionally, we reported the number of rMRI performed and the percent of neuroimaging performed as rMRI during each week of the rMRI period.