Skip to main content

The performance of the EMS triage (RETTS-p) and the agreement between the field assessment and final hospital diagnosis: a prospective observational study among children < 16 years



The rapid triage and treatment system for paediatrics (RETTS-p) has been used by the emergency medical services (EMS) in the west of Sweden since 2014. The performance of the RETTS-p in the pre-hospital setting and the agreement between the EMS nurse’s field assessment and the hospital diagnosis is unknown. The aim of this study was to evaluate the performance of the RETTS-p in the EMS and the agreement between the EMS field assessment and the hospital diagnosis.


A prospective observational study was conducted among 454 patients < 16 years of age who were assessed and transported to the PED. Two instruments were used for comparison: 1) Classification of an emergent patient according to predefined criteria as compared to the RETTS-p and 2) Agreement between the EMS nurse’s field assessment and the hospital diagnosis.


Among all children, 11% were identified as having vital signs associated with an increased risk of death and 7% were diagnosed in hospital with a potentially life-threatening condition. Of the children triaged with RETTS-p (85.9%), 149 of 390 children (38.2%) were triaged to RETTS-p red or orange (life-threatening, potentially life-threatening), of which 40 (26.8%) children were classified as emergent. The hospitalised children were triaged with the highest frequency to level yellow (can wait; 41.5%). In children with RETTS-p red or orange, the sensitivity for a defined emergent patient was 66.7%, with a corresponding specificity of 67.0%. The EMS field assessment was in agreement with the final hospital diagnosis in 80% of the cases.


The RETTS-p sensitivity in this study is considered moderate. Two thirds of the children triaged to life threatening or potentially life threatening were later identified as non-emergent. Of those, one in six was discharged from the PED without any intervention. Further, one third of the children were under triaged, the majority were found in the yellow triage level (can wait). The highest proportion of hospitalised patients was found in the yellow triage level. Our result is in agreement with previous studies using other triage instruments. A computerised decision support system might help the EMS triage to increase sensitivity and specificity.

Peer Review reports


Increased demand for paediatric emergency department (PED) resources has led to the necessity to utilise triage systems in the PED when the demand exceeds the resources that are available. The requirements relating to a triage system are that it has to be safe, reproducible and efficient [1]. Furthermore, the triage system should demonstrate validity and reliability and also be of relevance for the assessment of the patient. The aim of a triage system is to rapidly aid in the assessment of the patient and categorise patients based on the severity of the condition [2]. Patients with critical conditions are prioritised to reduce the waiting time for a physician evaluation [3]. Triage systems are constructed based on expert opinion and, as such, there is no gold standard for what constitutes a “true” emergency [4, 5], but five-level priority scales have shown better accuracy compared with three-level scales in order to identify the most critically ill patients [2, 6]. Internationally, the dominant triage systems are the emergency severity index (ESI), the Manchester triage system (MTS), the Australasian triage scale (ATS), the South African triage scale (SATS) and the paediatric Canadian triage scale (pCTAS). Previous studies have reported moderate to good inter-rater reliability and internal validity [5, 7]. The ESI has achieved a better performance in predicting hospital admission compared with the MTS, ATS and pCTAS [8]. Furthermore, the ESI, MTS and pCTAS have also achieved a higher performance when studied in the country/region where they were developed compared with being exported and implemented in other countries [9]. In Sweden, the majority of the PEDs at university hospitals use the rapid triage and treatment system for paediatrics (RETTS-p), a five-level triage system which was developed in Sweden and initially implemented in 2010. In 2014, the RETTS-p was introduced in the emergency medical services (EMS) in the western part of Sweden to start the triage process and assess the level of severity at an earlier stage. There is a lack of knowledge relating to the accuracy of pre-hospital triage and the performance of the RETTS-p in the EMS. The aim of this study was therefore to evaluate the accuracy of the pre-hospital RETTS-p and the agreement between the EMS assessment and the final hospital diagnosis among patients < 16 years of age allocated to the PED by the EMS nurse.



This study is an observational prospective study with a retrospective analysis of paediatric patients assessed by an EMS nurse and transported to the PED. Prospectively, we informed the EMS staff about the study and educated the staff at several workplace meetings at all nine of the ambulance stations in the EMS organisation. The education was made up of a repetition of the triage process with the RETTS-p triage protocol and also repetition of patient assessment according to local guidelines. This was further emphasised in official weekly newsletters sent out to all the EMS staff before and during the study. The purpose was to minimise bias and to achieve a uniform patient assessment and also to increase the data quality of the prehospital patient records.

Study population

Patients were eligible to be included in the study if they had had contact with dispatch and had been assessed as being in need of prehospital emergency care. The first thousand EMS assignments a month were collected consecutively in 2016 (n = 12,000) as a convenient sample. All children below 16 years of age in this sample were included if the inclusion criteria were met. The inclusion criteria were: 1) < 16 years of age and thus eligible for paediatric care at the children’s hospital and 2) primary assessment by an EMS nurse at the scene and assessed at the PED. The exclusion criteria were: 1) inter-hospital transport, 2) assistance to other ambulance, 3) assignments with no patient contact. A total of 716 children were initially identified in the sample and 65 children were excluded as they did not fulfil the inclusion criteria. Informed consent was not required for this EMS record registry based study. Informed consent is mostly not recommended by Ethical Review Boards in Sweden in this type of study for the following reasons. 1) Individual patients could never be identified in the analysis, since their identification number was translated to a code. Their integrity thus remained unaffected. 2) Some of the most severe cases could never be contacted in retrospect as they had either died or were in a very poor clinical condition. Furthermore logistical reasons and difficulties to understand our language would prevent communication with a number of patients. Thus, a requirement of informed consent would increase the risk of selection bias, thereby hampering the reliability of the data. 3) Approaching patients and/or relatives about these types of issue may create more anxiety than satisfaction and may therefore be regarded as unethical.


The study was conducted in a single-site urban setting in the western part of Sweden. The EMS organisation covers 900 km2, with a population of 660,000 inhabitants. The EMS carries out more than 80,000 assignments every year, of which 58,575 assignments are identified as primary assessments and approximately 3100 are children < 16 years of age.

All health care in Sweden is tax funded and free to residents of Sweden, regardless of the type of disease. In Sweden, the twenty-one county councils are responsible for providing health care for the residents within the county, including the EMS. The EMS can be organised within the body of a university hospital, county or contracted to a private company. In the study organisation, the EMS is organised under the university hospital. The EMS has 22 units, of which 18 are ambulances all with advanced life-saving (ALS) capacity and four special units; two single responders manned by one specialised trained registered nurse, one emergency physician-manned unit and a scene-command unit. Within the urban area, there is a designated children’s hospital with full capacity to handle all kinds of paediatric ailment, resulting in short transportation times from the scene to the PED. In total, the EMS carries out approximately 5.4% runs annually with children < 16 years of age. The retrospective patient data were collected from the EMS organisation patient records and from the hospital medical records for the specific in-hospital care event. Each record was reviewed manually and unidentified data such as vital signs, triage level, EMS nurse assessment and ICD code were then entered into a registry of which statistical analyses were conducted in a aggregated form.

RETTS-p triage system

The RETTS-p triage system is similar to the MTS triage system in its organisational structure, with flowcharts for each presentation. In order to identify patients at risk of deteriorating, the RETTS-p has included vital signs (VS) in each flowchart. The RETTS-p was developed and is maintained and licensed by a Swedish company (Predicare AB). The RETTS-p consists of two components, VS and emergency symptoms and signs (ESS). VS (SaO2, respiratory rate, pulse rate, temperature and level of consciousness) are recorded for each patient, together with an ESS stating the level of severity. The highest triage level of the VS or ESS constitutes the final triage level. The patient is triaged to a triage level colour ranging from red, defined as life threatening, orange potentially life threatening, yellow, green and blue. Red and orange are considered to be in need of a direct medical evaluation by a physician to what is defined as an acute process. The triage levels of yellow and green can wait without jeopardising the individual medical risk, but yellow is regarded as a higher level due to deviating VS or signs and symptoms (ESS) that should be evaluated by a physician before green-triaged patients. The blue level is the lowest triage colour and alternatives other than the PED may be more appropriate. At the time of this study, only triage levels green to red were used in the EMS nurse triage.

Prehospital management of the paediatric patient

When a patient dials the Swedish emergency number (112), a dispatcher assesses the patient according to a medical index and assigns the patient a priority of 1 to 4. Priority 1 is life threatening and one or two ALS units respond to the patient with blue lights and sirens. Priority 2 is considered urgent and a unit should respond within 30 min. Priority 3 can wait without medical risk. Priority 4 is assigned when there is a need for transport only and it is carried out by a non-emergency patient transport manned by one emergency medical technician. For priority 1–3, it is mandatory according to legislation in Sweden that a registered nurse conducts the assessment at the scene and is responsible for the administration of drugs. All ambulances in Sweden are therefore manned by a registered nurse, often with a postgraduate education in prehospital emergency care. Ambulance crew set-ups can be two registered nurses or one nurse and one EMT. The EMS nurse has been given the responsibility independently to decide upon the level of care which includes 1. Transport to hospital, 2. Arrange an appointment at a primary care centre and 3. Treat and release and/or give advice on self-care. To aid in the assessment, the EMS nurse uses the triage system (RETTS-p) to help discriminate between degrees of severity and a telephone consultation with a paediatrician at the PED, especially in cases where the patient is non-conveyed, is encouraged. The EMS nurse’s on-the-scene triage level becomes the triage level at the PED. To increase patient safety, the handover to the PED nurse always takes place together with the patient/ significant other.

Evaluation of the RETTS-p triage accuracy

Defining a true emergency patient is problematic and, as a result, no gold standard exists [10]. Predominantly, two options are available when aiming to test the internal validity of triage systems: 1. by proxy variables, for example, interventions performed and/or the need for hospitalisation and 2. physiological factors such as vital signs or blood samples. We have used a mixture of these in this study and a similar approach has been used when evaluating the MTS [10,11,12]. An “emergent” patient is defined as having abnormal VS according to the definition or a potentially life-threatening condition according to the final diagnosis. Reference VS that are compared against the RETTS-p VS are based on the paediatric increased risk of mortality acute physiology score III (PRISM III-APS) [13]. An increased risk of mortality in the PRISM-APS has been regarded as a qualifier for an “emergent” patient. Furthermore, SaO2 is part of the RETTS-p VS, but, in the PRISM III-APS, PaO2 is reported. SaO2 has been converted to PaO2 using a formula presented by Severinghaus, defining SaO2 of < 91% as the outside reference with an increased risk of mortality [14] Additional file 1. Potentially life-threatening conditions are defined according to conditions presented in previous studies [10, 11]. Examples of these conditions are sepsis, meningitis, high-energy trauma, substantial blood loss and intoxication Additional file 2. We have used a sequential step comparison of 1. qualification against VS and 2. comparison against defined potentially life-threatening conditions. This is an approximation of criterion validation (true emergency) and the RETTS-p triage levels of red or orange fulfilling the criteria are regarded as a true positive.

The agreement of the EMS field assessment at the scene compared with hospital diagnosis

The EMS assessment, in addition to the triage level, has been collected from EMS records, where the EMS nurse registers the assessed condition and/or symptoms. In order to compare the EMS field assessment with the final physician diagnosis in hospital, a diagnosis classification instrument has been used. This instrument categorises life-threatening diagnoses and non-life-threatening diagnoses and has previously been used in another study [15]. It has also been further explained in a study protocol [16]. The instrument comprises five categories (A-E) and is described in Table 4. In cases where it was difficult to determine the category, the case was discussed in a group to reach consensus about the classification.

Statistical analysis

The results in this study are presented as numbers and percentages or as the median with 25th and 75th percentiles. Missing data have been denoted in each table and, in Table 3, only patients recorded with a final triage colour in the ambulance were included. In Table 1, group comparisons were performed using Fisher’s exact test for categorical variables and the Mann-Whitney U test for continuous variables. To calculate sensitivity, specificity, positive and negative predicted values and likelihood ratios, data from Table 3 were used, where RETTS-p levels red and orange were regarded as a positive test and an emergent patient was defined as having either life-threatening VS or a potentially life-threatening condition. Under-triage and over-triage was defined as 1-Sensitivity (proportion of yellow/green triaged among all emergent patients) and 1-Specificity (proportion of red/orange triaged among all non-emergent patients) respectively. All tests are two-sided and p-values below 0.01 are considered significant due to the number of tests in Table 1. SPSS version 22 was used to perform data processing and statistical analysis.

Table 1 Patient characteristics of EMS RETTS-p triaged patients vs. No triage colour


A total of 651 children < 16 years of age were identified in the population. Of these, 197 patients were treated at the scene or referred to primary care. A total of 454 patients were assessed and transported to the PED. From now on, the results will only deal with the patients transported to the PED.

Patients assessed to a triage colour according to RETTS-p vs. not assessed to a triage colour

Of the 454 assessed patients, 390 (85.9%) patients received a triage colour according to RETTS-p from the EMS nurse. The group associated with a triage colour was significantly older. The majority of the patients in both groups were males, had no previous medical history, arrived at the PED between 4 pm and 12 pm, visited the PED for a medical ailment and were discharged from the PED with an intervention by a physician such as X-ray, ultrasound, suturing, blood sample, medical treatment or prescription. Of all the children, 11% were identified with VS associated with an increased risk of death and 7% were diagnosed with a potentially life-threatening condition. A total of 31.7% of all children were discharged from the PED without any intervention (Table 1).

Accuracy of the RETTS-p

A total of 390 patients transported to the PED were assessed with a triage colour according to RETTS-p by the EMS nurse. Of them, 149 patients (38.2%) were triaged to a level red (life threatening) or orange (potentially life threatening) condition, according to RETTS-p guidelines. Of them, 55% of the red- and 31% of the orange- triaged patients were hospitalised (Table 2). According to the definition of an emergent patient, based on the classification of VS and a potentially life-threatening condition, only 40 patients (26.8, 95% CI [22.5,31.7]) at the red and orange triage level qualified as emergent patients. We found that the majority of the patients admitted to in-patient care were not classified as emergent patients according to VS or a potentially life-threatening condition (Table 3). Eight patients with life-threatening VS according to the definition had more than one VS outside the reference interval. An abnormal heart rate (n = 24) and an abnormal temperature (n = 12) were the most frequent presentations among the patients identified with VS according to the emergent patient category. If triaged to yellow or green, the predictive value of not being life threatening or potentially life threatening was 91.7, 95% CI [88.5,94.1]. The RETTS-p triage levels of red and orange had a sensitivity of 66.7% for detecting an emergent patient, 95% CI [53.3,78.3], i.e. an under-triage of 33.3%. The corresponding specificity was 67.0, 95% CI [61.6,72.0], i.e. an over-triage of 33.0%. The positive likelihood ratio was 2.02, 95% CI [1.59, 2.56], and the negative likelihood ratio was 0.47, 95% CI [0.33, 0.67].

Table 2 EMS triage level according to RETTS-p and hospitalisation
Table 3 Pre-hospital triage level according to RETTS-P in comparison with emergency definition

Agreement between EMS field assessment and hospital diagnosis

A total of 412 of 454 patients transported to the PED received a hospital diagnosis (ICD-10-SE). The remainder were treated by a nurse practitioner in the PED or the patient left the PED before being examined by a physician. Thirteen patients (3.2%) were classified with a life-threatening specified hospital diagnosis and 320 patients (77.6%) were classified with a specified hospital diagnosis but not life threatening. Another 79 patients had a final diagnosis described as a symptom (for example, dyspnoea) or a diffuse assessment such as observation or evaluation of a non-defined incident. The EMS nurse’s field assessment was in agreement with the hospital diagnosis in 329 (80%) of the cases. Among the remaining 83 cases, the field assessment was not in agreement with the hospital diagnosis or the symptoms reported by the EMS nurse were atypical or uncommon for the hospital diagnosis. In 33 patients (8%), the EMS field assessment diverged from the hospital diagnosis. One of these patients had a life-threatening hospital diagnosis with a fatal outcome in the PED (Table 4).

Table 4 Agreement between the EMS field assessment and the final diagnosis


In the present study, we evaluated the accuracy of the RETTS-p triage system in an EMS setting and the EMS nurse’s assessment compared with hospital diagnosis. To our knowledge, this is the first study to evaluate a PED triage system in the prehospital setting and the EMS nurse’s assessment in comparison with the physician’s assessment and outcome in hospital. The findings in this study showed a sensitivity, when red- or orange-triaged patients were later identified as emergent cases, of 68.2% and a specificity of 67.0%. The EMS nurse’s field assessment was in agreement with the physician’s hospital diagnosis in 80% of cases.

The triage sensitivity in this study is consistent with other studies of the MTS in the PED, with a reported sensitivity of 58–74% [10, 11, 17,18,19]. This has previously been reported as moderate sensitivity [5]. The RETTS-p specificity of 67.0% in our study was lower than that in previous MTS studies in the PED, reporting rates of 78–87% [10, 11, 17,18,19], but higher compared with a study from Norway, a Scandinavian country similar to Sweden with a modified paediatric SATS triage in the PED, which reported a sensitivity of 74% and a specificity of 48% [20].

Furthermore, we found an over-triage of 33.0% and an under-triage of 33.3%. Other studies of paediatric triage systems in the ED have reported an over-triage of 16–54% and an under-triage of 2–28.7% [10,11,12, 17, 18, 21,22,23]. In this study, the majority of the patients found as under-triaged were triaged to yellow level, were one year of age or below with fever or pulse rate as a deviating vital sign compared with the reference standard. Of the 20 patients who were under-triaged, three patients were hospitalised, five patients were treated in the PED and a total of 12 patients were administered drugs by the EMS nurse. The majority of the EMS under-triaged children were later diagnosed with infectious diseases and febrile seizures with a viral origin.

However, the aim of a triage system is not to identify a certain diagnosis but to identify patients at risk. Children may run an increased risk of under-triage since they have more unspecific complaints and deviations in vital signs occurring later in the process [24]. It may therefore be useful to conduct a triage at the scene combining both ESS and VP. Even though a built-in over-triage is needed, over-triaged patients consume resources. In a simulation study of mass casualty trauma care, over-triaging patients increased the overall mortality when the number of non-urgent patients increased [25]. An incorrect triage level can also be explained by misclassification. However, previous studies of the RETTS-p have reported good to very good agreement between nurses in the PED triaging children [26, 27]. On the other hand, using memory and written manual guidelines during transport may further complicate the EMS nurse triage process.

In this study, the under- and over-triaged cases were found with an even distribution over the year, which suggests that increasing triage knowledge is dependent on patient frequency, which was similar across the year. On the other hand, compared with the above-mentioned PED triage studies with high exposure of triage cases, the findings are similar. In the latest version (2019) of the RETTS-p, changes have been made to reduce under-triage, specifically for patients presenting with fever above 38.5 C (VS) and current or past symptoms of chills (ESS). They should now be triaged to orange level (potentially life threatening). In the future, this may lead to a decrease in under-triaged children but also an increase in over-triage. Designing a triage system with low under-triage, while, at the same time, not increasing over-triage, is essential and problematic. Acceptable rates of over- and under-triage have not been agreed upon. However, for trauma patients, the American College of Surgeons states an under-triage of below 5 % and 25–35% over-triage, but as much as 50% has also been mentioned [28].

Regarding admission to in-patient care, the highest frequency of patients in all categories admitted to in-patient care were triaged to yellow level. Conditions such as seizures, infections, commotio (concussion) and fractures which required surgery were found at this triage level. This may indicate a weak association between the triage level and the severity of the condition when classified according to the need for hospitalisation. However, the hospital admission rate per triage level increases with a higher urgency level from 9.8% at green level to 55% at the red triage level. The association between triage level and the need for hospitalisation has also been addressed in other studies of triage in the PED. The ESI has shown an association of 84–100% predicting hospital admission at the highest triage levels [7, 23, 29]. The proportion of patients at the red triage level in this study is relatively high (9.8%) compared with studies of the ESI, pCTAS and the MTS (0.1–2.9%) [7, 11, 29, 30]. This suggests that greater over-triage is built into the RETTS-p system compared with other triage studies in the PED.

EMS field assessment in agreement with hospital diagnosis

Of the total number of children that were transported to the PED in this study, 13 (3.2%) were given a specified hospital diagnosis identified as life threatening, such as toxic shock syndrome, intracerebral haemorrhage or anaphylaxis. The prevalence of critical conditions in the PED has been reported in studies, with rates of 0.28–4.2%, depending on the region, patient cohort and the definition of a critically ill child [31,32,33,34]. The fact that this study only involved the patients initially in contact with the EMS may have increased the prevalence, albeit low overall. In the majority of patients, the field assessment was in agreement with the final hospital diagnosis, with only one patient classified as having a life-threatening final diagnosis which was not in agreement with the field assessment. However, the diagnosis for another 32 patients of those classified with a non-life-threatening final diagnosis was not in agreement with the EMS field assessment. Paramedics’ ability to predict hospital admission and the aetiology of the condition in adults has been shown to be moderate in several studies [35,36,37]. In one study, 19% of the paediatric transports with lights and sirens were later identified as unnecessary [38]. In a study of adults from Ireland, advanced paramedics showed agreement in 70% of the cases when the field diagnosis and the ED physicians’ working diagnosis were compared. Areas of less agreement included patients with no abnormalities [39]. These results suggest that it is particularly difficult to exclude a life-threatening condition, as further tests and examinations are required after arrival at hospital. Overall limited exposure and lack of training may be one reason for a less appropriate field assessment, which has been previously reported [40, 41]. Furthermore, environmental characteristics also play a role in the assessment, where the EMS is confronted by various challenges, such as worried parents, language barriers with no support in interpretation on the scene or no observable abnormality.

Children not assessed to a triage colour according to RETTS-p

In this study, 14.1% of the patients were not triaged to a colour, which can be explained in several ways. Firstly, short transportation times may have influenced the triage before arrival at the PED. Secondly, patients were significantly younger in the non-triage group, indicating difficulties either in the EMS nurse’s competence assessing infants according to the triage system or in obtaining VS in these patients. A lack of VS in paediatric patients in the pre-hospital setting has also been previously reported, with an increasing rate of fewer recorded VS in younger children [42, 43]. This is worrying, as the youngest children run a higher risk of serious bacterial infections [10, 44]. Thirdly, some of these patients were transported directly to the emergency room and the EMS nurse may have been occupied with patient interventions before arrival at the PED. As a result, the patient was assessed to the highest triage level, even though this was not recorded. However, the number of patients not triaged to a colour in this study appears to be similar to studies of other triage systems, such as the MTS in the PED, which reported a missing triage level in 5–16% of the patients [10,11,12, 45]. On average, the EMS nurses in the study organisation assess approximately 50 paediatric patients every year in an unselected patient population. This may indicate that the lack of exposure has an impact on triage knowledge and assessment. On the other hand, the use of a triage system by the EMS clinician at the scene in every case may help to identify patients at risk in a systematic approach when assessing patient severity. A triage system with both VS and ESS may contribute to increased patient safety when utilised in a setting and used by nurses with limited exposure to paediatric conditions, especially since the ESS part of the system acts as a checklist based on severity for each condition within the triage system. However, adherence to guidelines and protocols has been reported with large variability and suboptimal adherence for certain patient groups in the EMS (7.8–95%) [46]. One survey of EMS nurses reported adherence to guidelines of 84% and one factor explaining the variation in adherence was the characteristics of the guidelines [47]. Using manual triage systems, where the EMS nurse has to recall from memory when documenting the triage process, can be problematic. A study of the adult CTAS revealed improved performance when implementing a computerised triage system compared with traditional manual triage [48]. Introducing an electronic version of the RETTS-p adapted for pre-hospital use and incorporated in a decision support system, in combination with regular training on paediatric assessment, may increase adherence to and confidence in the system. Furthermore, the introduction of dedicated units for paediatric cases with staff rotation into the PED would increase exposure and competence in the at-the-scene assessments.

Strengths and limitations

The data in this study were collected consecutively from a relatively large group from all the eligible patients in contact with the EMS where the data were reported prospectively. Another strength is that the EMS nurse was not aware of the situation in the PED regarding flow time and triage levels. The PED nurse triaging a patient is constantly aware of this. It is defined as triage drift, where the PED nurse may overrule the triage system for a specific patient for logistical reasons. There are good reasons to discuss several limitations. This is a relatively small study. Despite being collected from a relatively large sample, the frequency of children in the EMS is limited. Furthermore, patient assessment is a dynamic process and triage should be seen as a sequential process over time. For this reason, any deterioration in a condition or improvement due to an intervention by the EMS nurse may have an impact on the patient’s condition and triage at a later stage of the chain of care. However, in the study, short transportation times and the fact that the final triage colour and the last recorded VS have been taken into consideration have minimised the risk of this. Furthermore, the potential for patient records being recorded with inaccurate levels and VS must be considered. This study was conducted at a single site and it may therefore be problematic to extrapolate the results of this study. However, information about the use of a triage system in a pre-hospital context could be generalised. The definition of an emergent patient is constructed and should be interpreted as such. Using the PRISM III-APS instead of PRISM III, the mortality risk reference interval for VS is increased, detecting even smaller changes. This may have included more children in the emergent VS category. On the other hand, including these children who may be at risk of death according to the PRISM III-APS appears valid.


When used by the EMS, the RETTS-p performed at the same level as other triage systems in the PED. The sensitivity of the RETTS-p in this study is considered to be moderate. The majority of under-triaged patients were found at the yellow triage level, which may be a threat to patient safety later in the chain of care. Even though it may be better to err on the safe side, it seems that the relatively large number of non-emergent red and orange patients could have yielded less under-triage than reported in this study. This indicates that the triage of certain subgroups of patients needs to be further addressed. A computerised decision support system with an adapted pre-hospital triage version could be proposed, aiming at increasing sensitivity and specificity. More studies are needed to further study the use of hospital triage systems in the EMS.

Availability of data and materials

The datasets analysed during the current study are available from the corresponding author on reasonable request.



Advanced life support


Australasian triage scale


Emergency medical services


Emergency severity index


Emergency signs and symptoms


International statistical classification of diseases and related health problems – tenth revision


Manchester triage system


Pediatric Canadian triage and acuity scale


Paediatric emergency department


Paediatric increased risk of mortality III – acute physiology score


Rapid emergency triage and treatment system for paediatrics


Vital signs


  1. Nina W, Ortenwall P. Triage--a method for the best possible care in the emergency department. Lakartidningen. 2005;102(39):2751–3.

    PubMed  Google Scholar 

  2. Fernandes CM, Tanabe P, Gilboy N, Johnson LA, McNair RS, Rosenau AM, et al. Five-level triage: a report from the ACEP/ENA five-level triage task force. J Emerg Nurs. 2005;31(1):39–50 quiz 118.

    Article  PubMed  Google Scholar 

  3. Farrohknia N, Castren M, Ehrenberg A, Lind L, Oredsson S, Jonsson H, et al. Emergency department triage scales and their components: a systematic review of the scientific evidence. Scand J Trauma Resusc Emerg Med. 2011;19:42.

    Article  PubMed  PubMed Central  Google Scholar 

  4. Moll HA. Challenges in the validation of triage systems at emergency departments. J Clin Epidemiol. 2010;63(4):384–8.

    Article  PubMed  Google Scholar 

  5. van Veen M, Moll HA. Reliability and validity of triage systems in paediatric emergency care. Scand J Trauma Resusc Emerg Med. 2009;17:38.

    Article  PubMed  PubMed Central  Google Scholar 

  6. Travers DA, Waller AE, Bowling JM, Flowers D, Tintinalli J. Five-level triage system more effective than three-level in tertiary emergency department. J Emerg Nurs. 2002;28(5):395–400.

    Article  PubMed  Google Scholar 

  7. Green NA, Durani Y, Brecher D, DePiero A, Loiselle J, Attia M. Emergency severity index version 4: a valid and reliable tool in pediatric emergency department triage. Pediatr Emerg Care. 2012;28(8):753–7.

    Article  PubMed  Google Scholar 

  8. Aeimchanbanjong K, Pandee U. Validation of different pediatric triage systems in the emergency department. World J Emerg Med. 2017;8(3):223–7.

    Article  PubMed  PubMed Central  Google Scholar 

  9. de Magalhaes-Barbosa MC, Robaina JR, Prata-Barbosa A, Lopes CS. Validity of triage systems for paediatric emergency care: a systematic review. Emerg Med J. 2017;34(11):711–9.

    Article  PubMed  Google Scholar 

  10. van Veen M, Steyerberg EW, Ruige M, van Meurs AH, Roukema J, van der Lei J, et al. Manchester triage system in paediatric emergency care: prospective observational study. BMJ. 2008;337:a1501.

    Article  PubMed  PubMed Central  Google Scholar 

  11. Roukema J, Steyerberg EW, van Meurs A, Ruige M, van der Lei J, Moll HA. Validity of the Manchester triage system in paediatric emergency care. Emerg Med J. 2006;23(12):906–10.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  12. Zachariasse JM, Kuiper JW, de Hoog M, Moll HA, van Veen M. Safety of the Manchester triage system to detect critically ill children at the emergency department. J Pediatr. 2016;177:232–7 e1.

    Article  PubMed  Google Scholar 

  13. Pollack MM, Patel KM, Ruttimann UE. The pediatric risk of mortality III--acute physiology score (PRISM III-APS): a method of assessing physiologic instability for pediatric intensive care unit patients. J Pediatr. 1997;131(4):575–81.

    Article  CAS  PubMed  Google Scholar 

  14. Severinghaus JW. Simple, accurate equations for human blood O2 dissociation computations. J Appl Physiol Respir Environ Exerc Physiol. 1979;46(3):599–602.

    CAS  PubMed  Google Scholar 

  15. Tarnqvist J, Dahlen E, Norberg G, Magnusson C, Herlitz J, Stromsoe A, et al. On-scene and final assessments and their interrelationship among patients who use the EMS on multiple occasions. Prehosp Disaster Med. 2017;32(5):528–35.

    Article  PubMed  Google Scholar 

  16. Hagiwara MA, Nilsson L, Stromsoe A, Axelsson C, Kangstrom A, Herlitz J. Patient safety and patient assessment in pre-hospital care: a study protocol. Scand J Trauma Resusc Emerg Med. 2016;24:14.

    Article  PubMed  PubMed Central  Google Scholar 

  17. Seiger N, van Veen M, Steyerberg EW, Ruige M, van Meurs AH, Moll HA. Undertriage in the Manchester triage system: an assessment of severity and options for improvement. Arch Dis Child. 2011;96(7):653–7.

    Article  CAS  PubMed  Google Scholar 

  18. van Veen M, Steyerberg EW, Van't Klooster M, Ruige M, van Meurs AH, van der Lei J, et al. The Manchester triage system: improvements for paediatric emergency care. Emerg Med J. 2012;29(8):654–9.

    Article  PubMed  Google Scholar 

  19. Seiger N, van Veen M, Steyerberg EW, van der Lei J, Moll HA. Accuracy of triage for children with chronic illness and infectious symptoms. Pediatrics. 2013;132(6):e1602–8.

    Article  PubMed  Google Scholar 

  20. Engan M, Hirth A, Trønnes H. Validation of a modified triage scale in a Norweigan pediatric emergency department. Int J Pediatr. 2018;2018:4676758.

    Article  PubMed  PubMed Central  Google Scholar 

  21. Twomey M, Cheema B, Buys H, Cohen K, de Sa A, Louw P, et al. Vital signs for children at triage: a multicentre validation of the revised south African triage scale (SATS) for children. S Afr Med J. 2013;103(5):304–8.

    Article  CAS  PubMed  Google Scholar 

  22. Mullan PC, Torrey SB, Chandra A, Caruso N, Kestler A. Reduced overtriage and undertriage with a new triage system in an urban accident and emergency department in Botswana: a cohort study. Emerg Med J. 2014;31(5):356–60.

    Article  PubMed  Google Scholar 

  23. Travers DA, Waller AE, Katznelson J, Agans R. Reliability and validity of the emergency severity index for pediatric triage. Acad Emerg Med. 2009;16(9):843–9.

    Article  PubMed  Google Scholar 

  24. Athey J, Dean JM, Ball J, Wiebe R, Melese-d'Hospital I. Ability of hospitals to care for pediatric emergency patients. Pediatr Emerg Care. 2001;17(3):170–4.

    Article  CAS  PubMed  Google Scholar 

  25. Hupert N, Hollingsworth E, Xiong W. Is overtriage associated with increased mortality? Insights from a simulation model of mass casualty trauma care. Disaster Med Public Health Prep. 2007;1(1 Suppl):S14–24.

    Article  PubMed  Google Scholar 

  26. Henning B, Lydersen S, Dollner H. A reliability study of the rapid emergency triage and treatment system for children. Scand J Trauma Resusc Emerg Med. 2016;24:19.

    Article  PubMed  PubMed Central  Google Scholar 

  27. Westergren H, Ferm M, Haggstrom P. First evaluation of the paediatric version of the Swedish rapid emergency triage and treatment system shows good reliability. Acta Paediatr. 2014;103(3):305–8.

    Article  PubMed  Google Scholar 

  28. Tignanelli CJ, Vander Kolk WE, Mikhail JN, Delano MJ, Hemmila MR. Noncompliance with American College of Surgeons Committee on trauma recommended criteria for full trauma team activation is associated with undertriage deaths. J Trauma Acute Care Surg. 2018;84(2):287–94.

    Article  PubMed  Google Scholar 

  29. Jafari-Rouhi AH, Sardashti S, Taghizadieh A, Soleimanpour H, Barzegar M. The emergency severity index, version 4, for pediatric triage: a reliability study in Tabriz Children's hospital, Tabriz. Iran Int J Emerg Med. 2013;6(1):36.

    Article  PubMed  Google Scholar 

  30. Al-Hindi AA, Al-Akhfash AA, Fareed AM, Alhusainan KS, Algasomy SF, Althowainy IR. Efficacy of implementation of a 5 scale pediatric triage and acuity scale in pediatric emergency. Saudi Arabia Saudi Med J. 2014;35(9):999–1004.

    PubMed  Google Scholar 

  31. Chavez H, Garcia CT, Sakers C, Darko R, Hannan J. Epidemiology of the critically ill child in the Resuscitation Bay. Pediatr Emerg Care. 2018;34(1):6–9.

    Article  PubMed  Google Scholar 

  32. Chen EH, Cho CS, Shofer FS, Mills AM, Baren JM. Resident exposure to critical patients in a pediatric emergency department. Pediatr Emerg Care. 2007;23(11):774–8.

    Article  PubMed  Google Scholar 

  33. Kwak YH, Kim DK, Jang HY. Utilization of emergency department by children in Korea. J Korean Med Sci. 2012;27(10):1222–8.

    Article  PubMed  PubMed Central  Google Scholar 

  34. Moneret-Vautrin DA, Morisset M, Flabbee J, Beaudouin E, Kanny G. Epidemiology of life-threatening and lethal anaphylaxis: a review. Allergy. 2005;60(4):443–51.

    Article  CAS  PubMed  Google Scholar 

  35. Christie A, Costa-Scorse B, Nicholls M, Jones P, Howie G. Accuracy of working diagnosis by paramedics for patients presenting with dyspnoea. Emerg Med Australas. 2016;28(5):525–30.

    Article  PubMed  Google Scholar 

  36. Hauswald M. Can paramedics safely decide which patients do not need ambulance transport or emergency department care? Prehosp Emerg Care. 2002;6(4):383–6.

    Article  PubMed  Google Scholar 

  37. Pointer JE, Levitt MA, Young JC, Promes SB, Messana BJ, Ader ME. Can paramedics using guidelines accurately triage patients? Ann Emerg Med. 2001;38(3):268–77.

    Article  CAS  PubMed  Google Scholar 

  38. Burns B, Hansen ML, Valenzuela S, Summers C, Van Otterloo J, Skarica B, et al. Unnecessary use of red lights and sirens in pediatric transport. Prehosp Emerg Care. 2016;20(3):354–61.

    Article  PubMed  PubMed Central  Google Scholar 

  39. Cummins NM, Dixon M, Garavan C, Landymore E, Mulligan N, O'Donnell C. Can advanced paramedics in the field diagnose patients and predict hospital admission? Emerg Med J. 2013;30(12):1043–7.

    Article  PubMed  Google Scholar 

  40. Guise JM, Meckler G, O'Brien K, Curry M, Engle P, Dickinson C, et al. Patient safety perceptions in pediatric out-of-hospital emergency care: Children's safety initiative. J Pediatr. 2015;167(5):1143–8 e1.

    Article  PubMed  PubMed Central  Google Scholar 

  41. Fleischman RJ, Yarris LM, Curry MT, Yuen SC, Breon AR, Meckler GD. Pediatric educational needs assessment for urban and rural emergency medical technicians. Pediatr Emerg Care. 2011;27(12):1130–5.

    Article  PubMed  PubMed Central  Google Scholar 

  42. Ramgopal S, Elmer J, Escajeda J, Martin-Gill C. Differences in Prehospital patient assessments for pediatric versus adult patients. J Pediatr. 2018;199:200–5 e6.

    Article  PubMed  PubMed Central  Google Scholar 

  43. Magnusson C, Herlitz J, Karlsson T, Axelsson C. Initial assessment, level of care and outcome among children who were seen by emergency medical services: a prospective observational study. Scand J Trauma Resusc Emerg Med. 2018;26(1):88.

  44. Nijman RG, Vergouwe Y, Thompson M, van Veen M, van Meurs AH, van der Lei J, et al. Clinical prediction model to aid emergency doctors managing febrile children at risk of serious bacterial infections: diagnostic study. BMJ. 2013;346:f1706.

    Article  PubMed  PubMed Central  Google Scholar 

  45. Seiger N, van Veen M, Almeida H, Steyerberg EW, van Meurs AH, Carneiro R, et al. Improving the Manchester triage system for pediatric emergency care: an international multicenter study. PLoS One. 2014;9(1):e83267.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  46. Ebben RH, Vloet LC, Verhofstad MH, Meijer S, Mintjes-de Groot JA, van Achterberg T. Adherence to guidelines and protocols in the prehospital and emergency care setting: a systematic review. Scand J Trauma Resusc Emerg Med. 2013;21:9.

    Article  PubMed  PubMed Central  Google Scholar 

  47. Ebben RH, Vloet LC, Schalk DM, Mintjes-de Groot JA, van Achterberg T. An exploration of factors influencing ambulance and emergency nurses' protocol adherence in the Netherlands. J Emerg Nurs. 2014;40(2):124–30.

    Article  PubMed  Google Scholar 

  48. Dong SL, Bullard MJ, Meurer DP, Colman I, Blitz S, Holroyd BR, et al. Emergency triage: comparing a novel computer triage program with standard triage. Acad Emerg Med. 2005;12(6):502–7.

    Article  PubMed  Google Scholar 

Download references


CM wishes to acknowledge Ambulance and Prehospital Emergency Care at the Sahlgrenska University Hospital for support and access to patient data.

Ethical approval and consent to participate

The study was approved by the central ethical review board, Gothenburg, Sweden, Dnr: 970–15. This study was exempted from informed consent due to absence of an intervention (the triage system has been in use since 2014), study protocol and the extracted data was anonymised. However, standard protocols requires asking parent/guardian for confidentiality upon admission. None of the parent/guardian that asked to remain confidential has been part of the retrospective analysis.


CM received funding from LÖF, a nationwide Swedish insurance company which insures all publicly financed healthcare providers, and the OLA Foundation (the foundation for prolonged and life-saving activities).

for the patient safety PhD project of which this study is part. The amount of funds received for this study have covered the work (50%) done by CM. It is in the funders’ interest to obtain knowledge about patient safety in pre-hospital care. However, the funders have not had any influence over the design, aim or the process in this study. Open access funding provided by University of Gothenburg.

Author information

Authors and Affiliations



CM, CA and JH developed the design and the research protocol. CM created the data forms and collected the data. TK supervised the statistical analysis. CM wrote the first draft. CA, JH, MJH and TK supervised and analysed the data and their interpretation. All the authors have participated in the final draft of the manuscript and have read and approved the final manuscript.

Corresponding author

Correspondence to Carl Magnusson.

Ethics declarations

Consent for publication

Not applicable.

Competing interests

The authors declare that they have no competing interests.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary information

Additional file 1.

The paediatric risk of mortality III – acute physiology score (PRISM III-APS): Life-threatening vital signs for paediatrics according to the PRISM III-APS.

Additional file 2.

Paediatric life-threatening conditions: a list of conditions considered to be time-sensitive.

Rights and permissions

Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (, which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver ( applies to the data made available in this article, unless otherwise stated.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Magnusson, C., Herlitz, J., Karlsson, T. et al. The performance of the EMS triage (RETTS-p) and the agreement between the field assessment and final hospital diagnosis: a prospective observational study among children < 16 years. BMC Pediatr 19, 500 (2019).

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: